Vehicle positioning method, equipment, device, medium and product

Abstract

The present application provides a vehicle positioning method, equipment, device, medium and product. According to an example of the present application, the method may include: determining the characteristic position of the parking space at different moments based on a time-series image of the target parking space; determining the visual position of the vehicle at the first moment based on the characteristic position of the parking space at the first moment, the characteristic position of the parking space at the second moment and the target position of the vehicle at the second moment, the second moment being before the first moment in time sequence; determining the wheel speed position of the vehicle at the first moment based on the wheel speed pulse data of the vehicle; fusing the visual position and the wheel speed position to obtain the target position of the vehicle at the first moment, and only requiring an image sensor and a wheel speed sensor to determine and fuse the visual position obtained based on the change of the characteristic position of the parking space and the wheel speed position calculated from the wheel speed pulse data, thereby effectively suppressing the error of a single sensor, improving positioning accuracy while reducing cost and computing power requirements.

Description

Technical Field

[0001] The present application relates to the field of positioning technology, and in particular to a vehicle positioning method, equipment, device, medium and product. Background Art

[0002] With the rapid development of intelligent driving technology, automated parking systems (APS) have become a research hotspot due to their ability to significantly improve parking efficiency and reduce driving difficulty. The core of this technology lies in high-precision, real-time positioning of the vehicle. Positioning errors can lead to collision risks or parking failures.

[0003] Current positioning methods typically rely on high-precision sensors like ultrasonic radar and millimeter-wave radar, along with complex positioning algorithms, resulting in high positioning costs. Therefore, reducing hardware costs and computing power requirements while ensuring positioning accuracy has become a challenge for automated parking. Summary of the Invention

[0004] To overcome the problems existing in the related art, the present application provides a vehicle positioning method, equipment, device, medium and product.

[0005] According to a first aspect of any embodiment of the present application, a vehicle positioning method is provided, the method comprising:

[0006] Determine the characteristic positions of the parking space at different times based on the time-series images of the target parking space;

[0007] determining a visual pose of the vehicle at the first moment based on a parking space feature position at a first moment, a parking space feature position at a second moment, and a target pose of the vehicle at the second moment, wherein the second moment is chronologically prior to the first moment;

[0008] determining a wheel speed posture of the vehicle at the first moment based on the wheel speed pulse data of the vehicle;

[0009] The visual pose and the wheel speed pose are fused to obtain a target pose of the vehicle at the first moment.

[0010] According to a second aspect of any embodiment of the present application, a vehicle positioning device is provided, the device comprising:

[0011] A position determination module is used to determine the characteristic positions of the parking space at different times based on the time-series images of the target parking space;

[0012] a visual pose determination module, configured to determine a visual pose of the vehicle at the first moment based on a parking space feature position at the first moment, a parking space feature position at the second moment, and a target pose of the vehicle at the second moment, wherein the second moment is chronologically prior to the first moment;

[0013] a wheel speed posture determination module, configured to determine the wheel speed posture of the vehicle at the first moment based on the wheel speed pulse data of the vehicle;

[0014] A posture fusion module is used to fuse the visual posture and the wheel speed posture to obtain the target posture of the vehicle at the first moment.

[0015] According to a third aspect of any embodiment of the present application, an electronic device is provided, including:

[0016] processor;

[0017] a memory for storing processor-executable instructions;

[0018] The processor implements the method described in any embodiment of the present application by running the executable instructions.

[0019] According to a fourth aspect of any embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the method described in any embodiment of the present application is implemented.

[0020] According to a fifth aspect of any embodiment of the present application, a computer program product is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the method described in any embodiment of the present application is implemented.

[0021] The technical solution provided by this application may have the following beneficial effects:

[0022] According to the above embodiments, the characteristic positions of the parking spaces at different moments are determined based on the time-series images of the target parking space; the visual posture of the vehicle at the first moment is determined based on the characteristic position of the parking space at the first moment, the characteristic position of the parking space at the second moment and the target posture of the vehicle at the second moment; the wheel speed posture of the vehicle at the first moment is determined based on the wheel speed pulse data of the vehicle; the visual posture and the wheel speed posture are fused to obtain the target posture of the vehicle at the first moment; only image sensors and wheel speed sensors are required to determine and fuse the visual posture obtained based on the change of the characteristic position of the parking space and the wheel speed posture inferred from the wheel speed pulse data, thereby effectively suppressing the error of a single sensor, and reducing the cost and computing power requirements while improving positioning accuracy.

[0023] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0025] Figure 1 This is a flow chart of a vehicle positioning method according to an exemplary embodiment of the present application;

[0026] Figure 2 is a schematic diagram illustrating the position of a corner point of a parking space according to an exemplary embodiment of the present application;

[0027] Figure 3 is a flow chart of another vehicle positioning method according to an exemplary embodiment of the present application;

[0028] Figure 4 is a structural diagram of an electronic device according to an exemplary embodiment of the present application;

[0029] Figure 5 This is a block diagram of a vehicle positioning device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0031] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0033] Automatic parking technology uses environmental perception, path planning, and motion control algorithms to enable vehicles to autonomously park in and out of parking spaces without human intervention. However, current positioning methods typically rely on high-precision sensors and complex positioning algorithms, resulting in excessively high vehicle positioning costs and computing power requirements.

[0034] In order to solve the above problems, the present application proposes a vehicle positioning method. To further illustrate the present application, the following embodiments are provided:

[0035] See also Figure 1 , Figure 1 This is a flow chart illustrating a vehicle positioning method according to an exemplary embodiment of the present application. This vehicle positioning method can be executed by an automatic parking system and applied to automatic parking scenarios. The automatic parking system can be applied to vehicles or to server-side services such as single servers, cluster servers, and cloud servers. This vehicle positioning method can also be executed by other systems or devices in different application scenarios, and this embodiment of the present application does not limit this.

[0036] like Figure 1 As shown, the vehicle positioning method may include the following steps:

[0037] Step 101: Determine characteristic positions of the parking space at different times based on the time-series images of the target parking space.

[0038] In this step, the automatic parking system can collect images of the vehicle's environment in real time through the image sensors installed in the vehicle during cruising and parking.

[0039] The image sensor is used to provide a complete view of the vehicle's surroundings. The image sensor can be installed anywhere on the vehicle, and the images captured by multiple image sensors can be stitched together to form a 360-degree bird's-eye view.

[0040] For example, to expand the field of view and reduce hardware costs, the image sensors can be multiple fisheye cameras located in front, behind, and on both sides of the vehicle. An inverse perspective mapping (IPM) algorithm can be used to stitch the images captured by these multiple fisheye cameras into a bird's-eye view (BEV) to provide complete information about the vehicle's surroundings.

[0041] The collected time-series images can be used to detect the target parking space within the vehicle's environment and determine the relative position of the target parking space and the vehicle. If the target parking space in the time-series images meets the conditions of a clear camera field of view and complete visibility of the parking space features, the target parking space can be determined to be within the vehicle's parking observation area.

[0042] When the target parking space is in the parking space observation area, based on the time-series images of the target parking space, algorithms such as semantic segmentation based on deep learning (such as U-Net, Mask R-CNN) or computer vision (such as edge detection, Hough transform) can be used to extract the characteristic positions of the parking space at different times from the time-series images and determine the characteristic positions of the parking space at different times.

[0043] The target parking space is the space where the vehicle is to be parked, such as a mechanical parking space or a fixed-plane parking space. The parking space observation area is a specific distance range to the side, front, or rear of the vehicle. It ensures that parking space features, such as parking space lines and corners, are clearly visible and unobstructed in the image. A time-series image is a multi-frame image of the environment captured continuously in chronological order. The parking space feature position is the location of the parking space feature in the vehicle coordinate system.

[0044] Step 102: Based on the parking space feature position at the first moment, the parking space feature position at the second moment, and the target posture of the vehicle at the second moment, the visual posture of the vehicle at the first moment is determined, and the second moment is located before the first moment in time sequence.

[0045] In this step, the vehicle's visual pose in the world coordinate system can be calculated based on the parking space feature positions at the previous and next moments. The vehicle's visual pose at the first moment is determined by solving a rigid body transformation model, etc., based on the parking space feature positions at the first moment, the parking space feature positions at the second moment, and the vehicle's target pose at the second moment.

[0046] The second moment is before the first moment in time. The first moment can be the current moment (time t+1), and the second moment can be the previous moment adjacent to the first moment (time t). The visual pose is the vehicle pose calculated based on the parking space features in the time series and can provide an absolute pose reference.

[0047] The target pose is the final positioning pose of the vehicle in the world coordinate system. The target pose can include the vehicle position and vehicle heading angle, which reflect the vehicle position and vehicle orientation respectively.

[0048] Step 103: Determine the wheel speed posture of the vehicle at the first moment based on the wheel speed pulse data of the vehicle.

[0049] In this step, wheel speed pulse data from the vehicle's wheel speed sensor can be obtained via the Controller Area Network (CAN) bus. Based on this wheel speed pulse data, the change in the vehicle's wheel from the second moment to the first moment is determined. This change is used to update the wheel position of the vehicle at the second moment, thereby obtaining the wheel speed position of the vehicle at the first moment.

[0050] Among them, the wheel speed posture is the vehicle posture calculated through wheel speed pulse data, which can provide high-frequency short-time relative posture.

[0051] Step 104: Fuse the visual pose and the wheel speed pose to obtain the target pose of the vehicle at the first moment.

[0052] In this step, the visual pose is determined based on image features and has high absolute accuracy, but may be affected by factors such as shooting distance, ambient lighting, and occlusion. The wheel speed pose is determined based on wheel speed pulse data and has high short-term relative accuracy but has cumulative errors.

[0053] Fusion algorithms such as Kalman filtering and particle filtering can be used to fuse visual pose and wheel speed pose. This can correct for both visual influencing factors and accumulated wheel speed errors, yielding the vehicle's target pose at the first moment. This fused target pose is then used for parking path planning and control, ensuring the vehicle accurately parks in the target space.

[0054] In one embodiment, a planned parking path may be generated based on the target position at the first moment, and the vehicle may be controlled to park in the target parking space according to the planned parking path.

[0055] For example, a path planning algorithm can be used to plan the optimal parking path from the vehicle's current position to the target parking space, based on kinematic constraints such as the vehicle's minimum turning radius and maximum steering angle. Once planned, the path can be discretized into a series of dense path points. Based on the lateral and heading deviations between the target position and the parking path, the front wheel steering angle and vehicle speed control are calculated in real time, ensuring the vehicle accurately tracks the parking path.

[0056] During the parking process, the system continuously monitors environmental changes, integrates and locates the target position at the current moment in real time, updates the parking path, and uses the updated parking path to park until the vehicle is completely parked in the target parking space.

[0057] As mentioned above, by generating a planned path based on a high-precision target pose, the geometric rationality and feasibility of the parking trajectory can be significantly improved. By linking the target pose with the path planning, it can be ensured that the vehicle can accurately track the predetermined trajectory, ultimately achieving safe and smooth automatic parking.

[0058] The vehicle positioning method of this embodiment determines the characteristic positions of parking spaces at different moments based on time-series images of the target parking space, determines the visual pose of the vehicle at the first moment based on the characteristic position of the parking space at the first moment, the characteristic position of the parking space at the second moment and the target pose of the vehicle at the second moment, determines the wheel speed pose of the vehicle at the first moment based on the wheel speed pulse data of the vehicle, and fuses the visual pose and the wheel speed pose to obtain the target pose of the vehicle at the first moment. Only image sensors and wheel speed sensors are required to determine and fuse the visual pose obtained based on the change of the characteristic position of the parking space and the wheel speed pose inferred from the wheel speed pulse data, thereby effectively suppressing the error of a single sensor, improving positioning accuracy while reducing costs and computing power requirements.

[0059] The aforementioned embodiments describe how to achieve high-precision positioning of a vehicle during automated parking using only image sensors and wheel speed sensors by combining a visual pose derived from changes in parking space feature positions with a wheel speed pose inferred from wheel speed pulse data. The following embodiments provide a more detailed description of the visual pose determination process, which is applicable to any of the aforementioned embodiments.

[0060] In one embodiment, the parking space feature position may include a corner position of a parking space corner in the target parking space. The image position of the parking space corner in the image coordinate system is detected from the time-series image. The image position is converted from the image coordinate system to the vehicle coordinate system to obtain the corner position.

[0061] See also Figure 2 , Figure 2 A schematic diagram illustrating the corner point locations of parking spaces is shown. Taking the four corner points of a target parking space as an example, algorithms such as the Shi-Tomasi corner point detection algorithm can be used to detect the image positions of the four corner points of the target parking space 20 in the image coordinate system from a time-series image. Coordinate conversion algorithms such as the Perspective-n-Point (PnP) algorithm are used to convert the image positions from the image coordinate system to the vehicle coordinate system to obtain the corner point positions.

[0062] By detecting the image positions of the four parking space corner points in the image coordinate system, the image positions are converted to the vehicle coordinate system to obtain the corner point positions. . Indicates the X-axis coordinate of the parking space corner point in the vehicle coordinate system at the second moment, Indicates the Y-axis coordinate of the parking space corner point in the vehicle coordinate system at the second moment.

[0063] As mentioned above, by detecting the image positions of parking space corners in the image coordinate system from the time-series images, the image positions are converted from the image coordinate system to the vehicle coordinate system to obtain the corner positions, providing accurate geometric constraints for visual pose calculation and avoiding the problem of decreased accuracy of visual pose due to perspective distortion.

[0064] In one embodiment, a feature position pair for the same parking space feature is constructed based on the parking space feature positions at the first and second moments. Based on the feature position pair, the vehicle's posture change from the second moment to the first moment is determined. The visual posture at the first moment is determined based on the posture change and the target posture at the second moment.

[0065] A feature position pair is a matching pair of the same parking space feature at consecutive moments. A pose change describes the change in motion of the vehicle from the second moment to the first moment. The pose change can include both translational and rotational components.

[0066] For example, when determining the characteristic position of the parking space at the first moment and the parking space feature position at the second moment , the parking space features at the first moment and the second moment can be matched one by one to form a feature position pair of the same parking space feature For example, the parking space corner point located in the left front of the target parking space at time t is , at time t+1 , the characteristic position pair of the corner point of the parking space is .

[0067] Using residual constraints, geometric consistency constraints, and deep learning-based methods, the vehicle's pose change from the second moment to the first moment is determined based on feature position pairs. This pose change is superimposed on the target pose at the second moment to obtain the visual pose at the first moment.

[0068] As mentioned above, by constructing feature position pairs and calculating the pose change, the spatiotemporal continuity of parking space features can be effectively utilized to improve the accuracy of visual pose. At the same time, by combining the known target pose for inference, the reliability of visual pose estimation can be enhanced.

[0069] In one embodiment, for each feature position pair, a residual constraint reflecting the change in vehicle posture is constructed, and the posture change is determined with the optimization goal of minimizing the sum of the residual constraints of all feature position pairs.

[0070] The residual constraint can describe the mathematical relationship between the change in vehicle posture and the change in parking space feature position. The feature position alignment can include the parking space feature position of the same parking space feature at the first moment and the second moment.

[0071] For example, for each successfully matched feature position pair, a residual constraint equation can be established. This equation reflects that, when the vehicle's position changes, the parking space features at the previous moment should theoretically coincide with the observed feature points at the current moment after undergoing motion transformation. If the position change is accurately estimated, this coincidence error (i.e., residual) should approach zero.

[0072] Due to factors such as sensor noise and feature matching errors in practice, the residuals of all feature position pairs are usually not zero at the same time. Therefore, the three degrees of freedom of the pose change (longitudinal displacement, lateral displacement, and heading angle change) can be adjusted through nonlinear optimization methods to minimize the sum of squared residuals of all feature position pairs.

[0073] For example, for each feature position pair, a residual constraint may be constructed according to the following formula 1:

[0074] Formula 1

[0075] in, Indicates the The residual vector of feature position pairs, and represents the coordinates of the parking space feature at the second moment in the vehicle coordinate system, and Indicates the coordinates of the parking space feature at the first moment in the vehicle coordinate system, Indicates the change in the vehicle's X-axis from the second moment to the first moment. Indicates the change in the vehicle's Y axis from the second moment to the first moment. Indicates the change in the vehicle's heading angle from the second moment to the first moment.

[0076] The optimization target can be determined according to the following formula 2:

[0077] Formula 2

[0078] in, represents the residual sum of squares for all feature position pairs.

[0079] The optimal pose change can be solved by nonlinear optimization (such as Levenberg-Marquardt algorithm) or closed-form solution (such as SVD decomposition) , so that the projection error of the parking space feature before and after the movement is minimized.

[0080] The visual pose can include the visual position and visual heading angle , the visual pose at the first moment can be determined according to the following formula 3:

[0081] Formula 3

[0082] in, Indicates the second moment, Indicates the first moment, Indicates the X-axis coordinate of the target pose in the world coordinate system at the second moment, Indicates the Y-axis coordinate of the target pose in the world coordinate system at the second moment, represents the vehicle heading angle at the second moment, Represents the coordinate of the X-axis of the visual pose in the world coordinate system at the first moment, Represents the Y-axis coordinate of the visual pose in the world coordinate system at the first moment, Indicates the vehicle heading angle at the first moment.

[0083] As described above, by constructing residual constraints and minimizing the residual sum of squares, the accuracy of parking space feature matching can be ensured, and the stability of posture changes in complex parking environments such as lighting changes or occlusion scenes can be improved, thereby enhancing the positioning reliability of visual posture.

[0084] In the preceding embodiments, we described how to accurately calculate the vehicle's visual pose by combining feature matching and residual optimization of parking space corners with the conversion from the image coordinate system to the vehicle coordinate system. The following embodiments provide a more detailed description of the wheel speed pose determination process, which is applicable to any of the above embodiments.

[0085] In one embodiment, a driving state of a vehicle at a first moment may be obtained. The rear wheel posture of the rear wheel of the vehicle at the first moment is determined based on the driving state and the rear wheel posture of the rear wheel of the vehicle at the second moment. The wheel speed posture at the first moment is determined based on the rear wheel posture at the first moment.

[0086] The rear wheels of a vehicle can include the left rear wheel and the right rear wheel. The rear wheel posture is the position and orientation information of the rear wheels of the vehicle at a certain moment, which is used to calculate the vehicle's motion trajectory. The rear wheel posture can include the rear wheel position and the rear wheel heading angle.

[0087] For example, the vehicle's driving state at the first moment can be obtained through methods such as analyzing wheel speed sensor pulse signals, detecting angular velocity using an inertial measurement unit (IMU), and parsing steering angle sensor data. For example, the difference in pulse counts between the left and right rear wheels can be used to determine whether the vehicle is turning, or the vehicle's motion trend can be verified by combining the yaw angular velocity from the IMU.

[0088] Based on the driving state and the rear wheel posture at the second moment, the rear wheel posture of the vehicle's rear wheels at the first moment is determined through kinematic inverse calculation and trajectory backtracking optimization. For example, if the vehicle is turning at the second moment, the rear wheel position and orientation at the first moment are inferred based on the turning radius and heading angle change.

[0089] The wheel speed pose at the first moment can be determined based on the rear wheel pose at the first moment through methods such as smooth interpolation of heading angles and pose optimization based on kinematic constraints. For example, the average of the left and right rear wheel positions can be used as the vehicle center coordinates, and the wheel speed pose at the first moment can be determined in combination with historical vehicle heading angle data to improve the continuity of pose estimation.

[0090] As described above, by combining the vehicle's driving state and rear wheel posture, the cumulative error of the wheel speed pulse data can be effectively suppressed, the long-term stability of the track calculation can be improved, and the accuracy of the wheel speed pulse data positioning can be improved.

[0091] In one embodiment, a change in the number of pulses of the rear wheels of the vehicle from the second moment to the first moment is obtained. An incremental distance traveled by the rear wheels of the vehicle is determined based on a preset distance traveled per unit pulse and the change in the number of pulses. A driving state is determined based on the incremental distance traveled by the rear wheels of the vehicle.

[0092] The pulse count change is the change in the number of pulses output by the wheel speed sensor from the second moment to the first moment, and is used to calculate the wheel's rotational distance. The incremental travel distance is the actual distance the wheel travels from the second moment to the first moment, reflecting the change in wheel motion from the second moment to the first moment.

[0093] For example, pulse signals from wheel speed sensors installed on the left and right rear wheels can be collected in real time, and the change in the number of pulses for each rear wheel from the second moment to the first moment can be calculated. The incremental distance traveled by each rear wheel from the second moment to the first moment can be calculated by combining the change in the number of pulses with a pre-calibrated preset distance traveled per unit pulse.

[0094] If the incremental travel distances of the left and right rear wheels are equal, it can be determined that the vehicle is in a straight-line driving state; if the incremental travel distances of the left and right rear wheels are not equal, it can be determined that the vehicle is in a curved driving state.

[0095] For example, the incremental travel distance of the rear wheels of the vehicle can be determined according to the following formula 4:

[0096] Formula 4

[0097] in, represents the incremental distance traveled by the vehicle's rear wheels, Indicates the preset driving distance per unit pulse. Indicates the change in the number of pulses at the rear wheels of the vehicle.

[0098] The incremental travel distances of the vehicle's left and right rear wheels can be calculated based on the above formula 4. If the incremental travel distances of the left and right rear wheels are different, it indicates that differential motion exists and the vehicle is turning. For example, the driving state can be determined according to the following formula 5:

[0099] Formula 5

[0100] in, Indicates the incremental distance traveled by the right rear wheel, Indicates the incremental distance traveled by the left rear wheel.

[0101] As described above, by determining the driving state based on the incremental driving distance of the rear wheels of the vehicle, low-cost vehicle motion state recognition can be achieved without the need for additional sensors such as IMU, thereby further simplifying the system architecture.

[0102] In one embodiment, when the driving state is a straight-line driving state, the rear wheel posture at the first moment is determined based on the incremental driving distance of the rear wheels of the vehicle and the rear wheel posture at the second moment.

[0103] For example, when driving in a straight line, the rear wheel position at the first moment can be calculated using the planar kinematics formula based on the rear wheel position and rear wheel heading angle at the second moment, combined with the incremental travel distance. Specifically, the rear wheel positions of the left and right rear wheels at the first moment are translated along the original heading angle by the corresponding travel distance, while the heading angle remains unchanged.

[0104] When the driving state is a straight-line driving state, the rear wheel position of the left rear wheel at the first moment can be determined according to the following formula 6: and the rear wheel heading angle ( ):

[0105] Formula 6

[0106] in, Indicates the second moment, Indicates the first moment, represents the X-axis coordinate of the left rear wheel in the vehicle coordinate system at the second moment, represents the Y-axis coordinate of the left rear wheel in the vehicle coordinate system at the second moment, represents the vehicle heading angle of the left rear wheel at the second moment, represents the X-axis coordinate of the left rear wheel in the vehicle coordinate system at the first moment, represents the Y-axis coordinate of the left rear wheel in the vehicle coordinate system at the first moment, represents the vehicle heading angle of the left rear wheel at the first moment, Indicates the change in the number of pulses of the left rear wheel.

[0107] The rear wheel position of the right rear wheel at the first moment can be determined according to the following formula 7: and the right wheel heading angle ( ):

[0108] Formula 7

[0109] in, represents the X-axis coordinate of the right rear wheel in the vehicle coordinate system at the second moment, represents the Y-axis coordinate of the right rear wheel in the vehicle coordinate system at the second moment, represents the vehicle heading angle of the right rear wheel at the second moment, represents the X-axis coordinate of the right rear wheel in the vehicle coordinate system at the first moment, represents the Y-axis coordinate of the right rear wheel in the vehicle coordinate system at the first moment, represents the vehicle heading angle of the right rear wheel at the first moment, Indicates the change in the number of pulses of the right rear wheel.

[0110] When the driving state is a curved driving state, the vehicle's heading angle change is determined based on the vehicle wheelbase and the incremental travel distance of the vehicle's rear wheels, and the rear wheel posture at the first moment is determined based on the heading angle change, the incremental travel distance of the vehicle's rear wheels, and the rear wheel posture at the second moment. The heading angle change is the change in the vehicle's heading angle from the second moment to the first moment.

[0111] For example, when driving on a curve, the difference between the incremental driving distance of the left rear wheel and the incremental driving distance of the right rear wheel can be obtained, and the vehicle's heading angle change can be calculated based on the difference between the incremental driving distances of the left and right rear wheels and the vehicle wheelbase.

[0112] After obtaining the difference in incremental distance traveled by the left and right rear wheels, the trajectory of each rear wheel can be recalculated based on the turning circle center. The arc length formula can be used to update the rear wheel position at the second moment. The rear wheel heading angle at the first moment can be obtained by adding the heading angle change to the rear wheel heading angle at the second moment.

[0113] For example, the heading angle change can be determined according to the following formula 8:

[0114] Formula 8

[0115] in, Indicates the change in heading angle, Indicates the vehicle's wheelbase.

[0116] When the driving state is a curved driving state, the rear wheel position of the left rear wheel at the first moment can be determined according to the following formula 9: and the rear wheel heading angle ( ):

[0117] Formula 9

[0118] in, Represents the normalization term related to the turning radius, which is used to approximate the turning arc length. The instantaneous heading angle indicating the direction of motion of the left rear wheel, Represents the displacement component of the left rear wheel along the steering arc.

[0119] The rear wheel position of the right rear wheel at the first moment can be determined according to the following formula 10: and the right wheel heading angle ( ):

[0120] Formula 10

[0121] in, The instantaneous heading angle indicating the direction of motion of the right rear wheel, Represents the displacement component of the right rear wheel along the steering arc.

[0122] Wheel speed pose can include wheel speed position and wheel speed and heading angle After determining the rear wheel posture of the left and right rear wheels, the center points of the left and right rear wheels (the center of the vehicle's rear axle) can be used as representative points of the vehicle. Based on the rear wheel positions of the left and right rear wheels at the first moment, the wheel speed position of the vehicle at the first moment is calculated by taking the average, and the heading angle change is superimposed on the wheel speed and heading angle of the previous moment to obtain the wheel speed posture at the first moment.

[0123] For example, the wheel speed posture at the first moment can be determined according to the following formula 11:

[0124] Formula 11

[0125] in, Indicates the X-axis coordinate of the wheel speed position at the first moment in the world coordinate system, Indicates the Y-axis coordinate of the wheel speed position at the first moment in the world coordinate system, represents the vehicle heading angle at the first moment, Represents the vehicle heading angle at the second moment.

[0126] As mentioned above, the wheel speed and posture calculation strategy is dynamically adjusted by classifying the driving state. In the straight-line driving state, the rear wheel posture at the first moment can be determined based on the incremental driving distance of the vehicle's rear wheels and the rear wheel posture at the second moment, which can improve the calculation efficiency. In the curved driving state, the heading angle change compensation mechanism is introduced to correct the steering deviation during curved driving and improve the posture positioning accuracy under complex trajectories.

[0127] In one embodiment, wheel speed and posture can be fused with visual posture based on a Kalman filter algorithm. A state transition model can be used to generate a predicted state at a first moment based on the wheel speed and posture. The covariance matrix of the predicted state can be updated based on the Jacobian matrix of the state transition model.

[0128] Based on the visual pose, an observation model is established. Based on the observation model and the predicted state, the residuals and covariances are determined. Based on the residuals and covariances, the Kalman gain is determined. Using the Kalman gain, the predicted state and covariance matrix are modified to obtain the target pose.

[0129] The state transition model describes the dynamic evolution of the vehicle's state variables under the influence of control variables. The state transition model can include state variables and control variables. State variables describe the vehicle's current position and orientation, and can include vehicle position and heading angle. Control variables can include distance increments and heading angle changes derived from wheel speed pulse data.

[0130] The Jacobian matrix is ​​the matrix of partial derivatives of the state transition model with respect to the state variables. It is used to linearize nonlinear motion models to support predictive updates of the covariance matrix. The covariance matrix combines the covariance of the preceding state, the Jacobian matrix of the state transition model, and the characteristics of process noise to quantify the propagation of uncertainty in the predicted state.

[0131] The observation model represents the actual observed vehicle pose and is used to map the visual pose to the state vector. The observation model can include both the observation vector and the observation matrix. The residual represents the difference between the visual pose and the predicted state. The covariance incorporates the uncertainty of the predicted state and the characteristics of visual measurement noise to assess the confidence level of the residual.

[0132] For example, state variables and control variables can be defined. The state variables are determined according to the following formula 12: :

[0133] Formula 12

[0134] in, and Indicates the vehicle position, Indicates the vehicle heading angle.

[0135] The control variable can be determined according to the following formula 13 :

[0136] Formula 13

[0137] in, Indicates the vehicle's moving distance increment, Indicates the change in the vehicle's heading angle.

[0138] Based on the state variables and control variables, a state transition model can be established according to the vehicle's kinematic characteristics to describe the vehicle's motion under the influence of the control variables. The state transition model is determined according to the following formula 14:

[0139] Formula 14

[0140] You can initialize the state based on the state transition model and the covariance matrix , forming the initial state parameters of the filtering algorithm. Based on the determined wheel speed and posture, the state transition model is used to determine the predicted state according to the following formula 15:

[0141] Formula 15

[0142] The Jacobian matrix is ​​determined according to the following formula 16: :

[0143] Formula 16

[0144] The Jacobian matrix and the process noise covariance can be used to update the covariance matrix according to the following formula 17:

[0145] Formula 17

[0146] in, It represents the process noise covariance and can reflect the error of wheel speed pulse.

[0147] Based on the determined visual pose, the observation model can be determined according to the following formula 18:

[0148] Formula 18

[0149] in, represents the observation vector, and The vehicle position representing the visual pose, The vehicle heading angle representing the visual pose, represents the observation matrix, Represents a three-dimensional identity matrix, which is used to reflect the consistency of the visual pose and state variable dimensions.

[0150] The residual can be determined based on the observation model and the predicted state according to the following formula 19 and covariance :

[0151] Formula 19

[0152] in, It represents the visual noise measurement covariance, which is used to reflect the error of visual observation.

[0153] By weighing the predicted covariance and the measured covariance, the optimal fusion weight is calculated to determine the correction strength of the visual pose to the predicted state. The Kalman gain can be determined according to the following formula 20 :

[0154] Formula 20

[0155] The predicted state and residual are fused using the Kalman gain to obtain the revised optimal state estimate, and the covariance matrix is ​​updated to reflect the fused uncertainty. The predicted state and covariance matrix are modified according to the following formula 21:

[0156] Formula 21

[0157] As mentioned above, by introducing the Kalman filter algorithm, using the state transition model to predict the wheel speed and posture, and updating the covariance matrix through the Jacobian matrix, the uncertainty of the predicted state can be reasonably quantified. In the observation update stage, the weights of prediction and observation are adaptively adjusted through the Kalman gain, which significantly reduces the impact of wheel speed cumulative error and visual instantaneous noise.

[0158] To further introduce the vehicle positioning process, Figure 3 A flow chart of another vehicle positioning method is shown. The vehicle positioning method may include the following steps:

[0159] Step 301: Detect the image positions of parking space corner points from the time-series images.

[0160] In this step, the surround view camera may be used to collect time-series images, and the parking space corner points of the target parking space in the time-series images may be extracted using an image processing algorithm. After extraction, a two-dimensional image position is formed in the image coordinate system.

[0161] Step 302: Convert the image position from the image coordinate system to the vehicle coordinate system to obtain the corner point position.

[0162] In this step, the camera calibration parameters and installation position information can be used to convert the image position into the vehicle coordinate system through perspective transformation to obtain the corner point position of the parking space in three-dimensional space.

[0163] Step 303: construct a feature position pair of the corner point of the same parking space based on the corner point positions at the first moment and the second moment.

[0164] In this step, the corner point positions of two consecutive frames (the first time and the second time) are obtained. The corner point positions of the same parking space corner point at different times are matched to construct feature position pairs. Each feature position pair contains the corner point positions of the same parking space corner point at the first time and the second time.

[0165] Step 304: For each feature position pair, construct a residual constraint that reflects the change in vehicle posture.

[0166] In this step, a residual constraint equation is established for each matched feature position pair.

[0167] Step 305: Determine the pose change amount with the optimization goal of minimizing the sum of the residual constraints of all feature position pairs.

[0168] In this step, the optimal pose change can be solved by minimizing the sum of the residual constraints of all feature position pairs.

[0169] Step 306: Determine the visual posture at the first moment based on the posture change and the target posture at the second moment.

[0170] In this step, the posture change can be added to the target posture at the second moment to calculate the visual posture at the first moment.

[0171] Step 307: Obtain the driving state of the vehicle at the first moment.

[0172] In this step, the wheel speed pulse data of the left and right rear wheels can be obtained to calculate the change in the number of pulses for each rear wheel. The incremental distance traveled by each rear wheel is calculated by combining the change in the number of pulses and the preset travel distance per unit pulse.

[0173] If the incremental travel distances of the left and right rear wheels are equal, it can be determined that the vehicle is in a straight-line driving state; if the incremental travel distances of the left and right rear wheels are not equal, it can be determined that the vehicle is in a curved driving state.

[0174] Step 308: Determine the rear wheel posture at the first moment according to the driving state and the rear wheel posture of the vehicle's rear wheels at the second moment.

[0175] In this step, when the driving state is a straight-line driving state, the rear wheel posture at the first moment can be determined based on the incremental driving distance of the rear wheels of the vehicle and the rear wheel posture at the second moment.

[0176] When the vehicle is traveling along a curve, the difference between the incremental distances traveled by the left and right rear wheels is obtained. The vehicle's heading angle change is calculated based on the difference in incremental distances traveled by the left and right rear wheels and the vehicle's wheelbase. The rear wheel posture at the first moment is determined based on the heading angle change, the incremental distances traveled by the vehicle's rear wheels, and the rear wheel posture at the second moment.

[0177] Step 309: Based on the rear wheel posture at the first moment, determine the wheel speed posture at the first moment.

[0178] In this step, the average value of the rear wheel positions of the left and right rear wheels at the first moment can be used as the wheel speed position of the vehicle at the first moment, and the heading angle change can be superimposed on the wheel speed heading angle of the vehicle at the previous moment to obtain the wheel speed posture at the first moment.

[0179] Step 310: Generate a predicted state based on the wheel speed and posture using a state transition model.

[0180] In this step, the current vehicle posture is predicted based on the wheel speed and posture using the state transition model to obtain a preliminary predicted state. The covariance matrix of the predicted posture is calculated based on the Jacobian matrix to quantify the error in the predicted state.

[0181] Step 311: Use the visual pose to update the predicted pose to obtain the target pose at the first moment.

[0182] In this step, an observation model is built based on the visual pose to describe the observed vehicle pose. The observed model is compared with the predicted state to determine the residual and covariance, reflecting the degree of inconsistency and uncertainty between the observed and predicted information. The Kalman gain is calculated based on the residual and covariance matrices, dynamically adjusting the weights between the predicted state and the observed visual pose, ultimately outputting the fused target pose.

[0183] Step 312: Based on the target posture at the first moment, control the vehicle to park in the target parking space.

[0184] In this step, a path planning algorithm can be used to generate a planned parking path based on the target position at the first moment. Based on the planned parking path, steering and speed are dynamically adjusted to steer the vehicle into the target parking space while continuously monitoring environmental changes and position accuracy.

[0185] Figure 4This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. The electronic device may be, for example, a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a personal digital assistant, a server, a smart home appliance, a car computer, etc. Figure 4 At the hardware level, the electronic device includes a processor 401, an internal bus 402, a network interface 403, a memory 404, and a non-volatile memory 405. Of course, it may also include hardware required for other services. The processor 401 reads the corresponding computer program from the non-volatile memory 405 into the memory 404 and then runs it, forming a vehicle positioning device at the logical level. Of course, in addition to software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0186] Figure 5 This is a block diagram of a vehicle positioning device according to an exemplary embodiment of the present application. Figure 5 The device may include: a position determination module 501, a visual posture determination module 502, a wheel speed posture determination module 503 and a posture fusion module 504, wherein:

[0187] The position determination module 501 is used to determine the characteristic positions of the parking space at different times based on the time-series images of the target parking space;

[0188] The visual pose determination module 502 is configured to determine the visual pose of the vehicle at the first moment based on a parking space feature position at the first moment, a parking space feature position at the second moment, and a target pose of the vehicle at the second moment, where the second moment is chronologically prior to the first moment;

[0189] The wheel speed and posture determination module 503 is used to determine the wheel speed and posture of the vehicle at the first moment based on the wheel speed pulse data of the vehicle;

[0190] The posture fusion module 504 is configured to fuse the visual posture and the wheel speed posture to obtain a target posture of the vehicle at the first moment.

[0191] In one example, the visual pose determination module 502, when used to determine the visual pose of the vehicle at the first moment based on the parking space feature position at the first moment, the parking space feature position at the second moment and the target pose of the vehicle at the second moment, includes: constructing a feature position pair of the same parking space feature based on the parking space feature positions at the first moment and the second moment; determining the pose change of the vehicle from the second moment to the first moment based on the feature position pair; and determining the visual pose at the first moment based on the pose change and the target pose at the second moment.

[0192] In one example, the visual posture determination module 502, when used to determine the posture change of the vehicle from the second moment to the first moment based on the feature position pair, includes: constructing a residual constraint reflecting the vehicle posture change for each feature position pair; and determining the posture change with the optimization goal of minimizing the sum of the residual constraints of all feature position pairs.

[0193] In one example, the wheel speed posture determination module 503, when used to determine the wheel speed posture of the vehicle at the first moment based on the wheel speed pulse data of the vehicle, includes: obtaining the driving state of the vehicle at the first moment; determining the rear wheel posture of the rear wheels of the vehicle at the first moment based on the driving state and the rear wheel posture of the rear wheels of the vehicle at the second moment; and determining the wheel speed posture at the first moment based on the rear wheel posture at the first moment.

[0194] In one example, the wheel speed and posture determination module 503, when used to obtain the driving state of the vehicle at the first moment, includes: obtaining the change in the number of pulses of the rear wheels of the vehicle from the second moment to the first moment; determining the incremental driving distance of the rear wheels of the vehicle based on the preset driving distance of the unit pulse and the change in the number of pulses; and determining the driving state according to the incremental driving distance of the rear wheels of the vehicle.

[0195] In one example, the wheel speed and posture determination module 503, when used to determine the rear wheel posture of the vehicle's rear wheels at the first moment based on the driving state and the rear wheel posture of the vehicle's rear wheels at the second moment, includes: when the driving state is a straight driving state, determining the rear wheel posture at the first moment based on the incremental driving distance of the vehicle's rear wheels and the rear wheel posture at the second moment; when the driving state is a curved driving state, determining the vehicle's heading angle change based on the vehicle wheelbase and the incremental driving distance of the vehicle's rear wheels, and determining the rear wheel posture at the first moment based on the heading angle change, the incremental driving distance of the vehicle's rear wheels and the rear wheel posture at the second moment.

[0196] In one example, the parking space feature position includes the corner point position of the parking space corner point in the target parking space; the position determination module 501, when used to determine the parking space feature position at different times based on the time-series image of the target parking space, includes: detecting the image position of the parking space corner point in the image coordinate system from the time-series image; converting the image position from the image coordinate system to the vehicle coordinate system to obtain the corner point position.

[0197] In one example, the posture fusion module 504, when used to fuse the visual posture and the wheel speed posture, includes: based on the wheel speed posture, using a state transition model to generate a predicted state at the first moment, the state transition model including a state vector and a control vector; according to the Jacobian matrix of the state transition model, updating the covariance matrix of the predicted state; according to the visual posture, establishing an observation model, the observation model is used to reflect the mapping relationship between the visual posture and the state vector; based on the observation model and the predicted state, determining the residual and covariance, and determining the Kalman gain based on the residual and covariance; using the Kalman gain, correcting the predicted state and the covariance matrix to obtain the target posture.

[0198] In one example, the posture fusion module 504 is further configured to generate a planned parking path based on the target posture at the first moment; and control the vehicle to park in the target parking space according to the planned parking path.

[0199] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0200] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present application scheme. Those of ordinary skill in the art can understand and implement it without paying any creative work.

[0201] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is further provided, such as a memory including instructions. The instructions can be executed by a processor of a vehicle positioning device to implement any of the methods described in the above embodiments.

[0202] The non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., and this application does not limit this.

[0203] In an exemplary embodiment, a computer program product including a computer program / instruction is further provided. The computer program / instruction can be executed by a processor of a vehicle positioning device to implement any of the methods described in the above embodiments.

[0204] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0205] Other embodiments of the present invention will readily occur to those skilled in the art after consideration of the specification and practice of the invention claimed herein. The present application is not limited to the precise construction described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present application is limited solely by the appended claims.

[0206] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A vehicle positioning method, characterized in that: The method comprises: Determining characteristic positions of the parking space at different times based on the time-series images of the target parking space, wherein the characteristic positions of the parking space include corner positions of parking space corners in the target parking space in a vehicle coordinate system; constructing a feature position pair of the same parking space feature based on parking space feature positions at a first moment and a second moment, wherein the second moment is before the first moment in time sequence; For each feature position pair, a residual constraint is constructed to reflect the change in vehicle posture. By adjusting the longitudinal displacement, lateral displacement, and heading angle change of the vehicle from the second moment to the first moment, minimizing the sum of residual constraints of all feature position pairs as an optimization goal, a posture change is determined, wherein the posture change includes a translation change and a rotation component; Superimposing the posture change onto the target posture at the second moment to determine the visual posture of the vehicle at the first moment; determining a wheel speed posture of the vehicle at the first moment based on the wheel speed pulse data of the vehicle; The visual pose and the wheel speed pose are fused to obtain a target pose of the vehicle at the first moment.

2. The method according to claim 1, characterized in that Determining the wheel speed posture of the vehicle at the first moment based on the wheel speed pulse data of the vehicle includes: Acquiring the driving state of the vehicle at the first moment; determining the rear wheel posture of the vehicle's rear wheel at the first moment according to the driving state and the rear wheel posture of the vehicle's rear wheel at the second moment; Based on the rear wheel posture at the first moment, the wheel speed posture at the first moment is determined.

3. The method according to claim 2, characterized in that The obtaining of the driving state of the vehicle at the first moment includes: Obtaining a change in the number of pulses of the rear wheel of the vehicle from the second moment to the first moment; determining an incremental travel distance of the rear wheels of the vehicle based on a preset travel distance per unit pulse and a change in the number of pulses; The driving state is determined based on the incremental driving distance of the rear wheels of the vehicle.

4. The method according to claim 2, characterized in that The determining the rear wheel posture of the vehicle rear wheel at the first moment based on the driving state and the rear wheel posture of the vehicle rear wheel at the second moment includes: When the driving state is a straight-line driving state, determining the rear wheel posture at the first moment based on the incremental driving distance of the rear wheels of the vehicle and the rear wheel posture at the second moment; When the driving state is a curved driving state, the heading angle change of the vehicle is determined based on the vehicle wheelbase and the incremental driving distance of the rear wheels of the vehicle, and the rear wheel posture at the first moment is determined based on the heading angle change, the incremental driving distance of the rear wheels of the vehicle and the rear wheel posture at the second moment.

5. The method according to claim 1, wherein Determining characteristic positions of the parking space at different times based on the time-series images of the target parking space includes: Detecting the image position of the parking space corner point in the image coordinate system from the time-series image; The image position is converted from the image coordinate system to the vehicle coordinate system to obtain the corner point position.

6. The method according to claim 1, characterized in that The fusing of the visual posture and the wheel speed posture comprises: Based on the wheel speed and posture, generating a predicted state at the first moment using a state transition model, the state transition model including a state vector and a control vector; updating the covariance matrix of the predicted state according to the Jacobian matrix of the state transition model; Establishing an observation model according to the visual pose, wherein the observation model is used to reflect the mapping relationship between the visual pose and the state vector; Determining a residual and a covariance based on the observation model and the predicted state, and determining a Kalman gain based on the residual and the covariance; The predicted state and the covariance matrix are corrected using the Kalman gain to obtain the target pose.

7. The method according to claim 1, characterized in that The method further comprises: generating a planned parking path based on the target posture at the first moment; According to the planned parking path, the vehicle is controlled to park in the target parking space.

8. A vehicle positioning device, characterized in that: The device comprises: a position determination module, configured to determine characteristic positions of parking spaces at different moments based on the time-series images of the target parking space, wherein the characteristic positions of the parking spaces include corner positions of parking space corners in the target parking space in a vehicle coordinate system; A visual pose determination module is configured to construct a feature position pair for the same parking space feature based on parking space feature positions at a first moment and a second moment, where the second moment is temporally prior to the first moment; construct a residual constraint reflecting a change in vehicle pose for each feature position pair; determine a pose change by adjusting the longitudinal displacement, lateral displacement, and heading angle change of the vehicle from the second moment to the first moment, with minimizing the sum of the residual constraints of all feature position pairs as the optimization goal, the pose change comprising a translation change and a rotational component; and superimpose the pose change on the target pose at the second moment to determine the visual pose of the vehicle at the first moment. a wheel speed posture determination module, configured to determine the wheel speed posture of the vehicle at the first moment based on the wheel speed pulse data of the vehicle; A posture fusion module is used to fuse the visual posture and the wheel speed posture to obtain the target posture of the vehicle at the first moment.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 7 by running the executable instructions.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product having a computer program / instructions stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images