Vehicle positioning method, equipment, device, medium and product

Abstract

The invention provides a vehicle positioning method, equipment, a device, a medium and a product. According to one embodiment of the invention, the method comprises the following steps: determining parking space feature positions at different moments according to a time sequence image for a target parking space; based on the parking space feature position at the first moment, the parking space feature position at the second moment and the target pose of the vehicle at the second moment, the visual pose of the vehicle at the first moment is determined, and the second moment is located before the first moment in the time sequence; determining a wheel speed pose of the vehicle at the first moment according to the wheel speed pulse data of the vehicle; according to the method, the visual pose and the wheel speed pose are fused to obtain the target pose of the vehicle at the first moment, the visual pose obtained based on the parking space feature position change and the wheel speed pose calculated by the wheel speed pulse data can be determined and fused only through the image sensor and the wheel speed sensor, and therefore the error of a single sensor is effectively inhibited, and the accuracy of the vehicle is improved. The positioning precision is improved, and meanwhile, the cost and computing power requirements are reduced.

Description

Technical Field

[0001] This application relates to the field of positioning technology, and in particular, to a vehicle positioning method, device, apparatus, medium, and product. Background Art

[0002] With the rapid development of intelligent driving technology, the Automated Parking System (APS) has become a research hotspot because it can significantly improve parking efficiency and reduce the difficulty of driving operations. The core of this technology lies in the high-precision real-time positioning of vehicles. If positioning errors occur, it may lead to collision risks or parking failures.

[0003] Current positioning methods usually rely on high-precision sensors such as ultrasonic radars and millimeter-wave radars and complex positioning algorithms, resulting in an increase in positioning costs. Therefore, how to reduce hardware costs and computing power requirements while ensuring positioning accuracy has become a problem to be solved in automatic parking. Summary of the Invention

[0004] To overcome the problems existing in the related art, this application provides a vehicle positioning method, device, apparatus, medium, and product.

[0005] According to the first aspect of any embodiment of this application, a vehicle positioning method is provided. The method includes: Determining the parking space feature positions at different times according to the sequential images of the target parking space; Based on the parking space feature position at the first time, the parking space feature position at the second time, and the target pose of the vehicle at the second time, determining the visual pose of the vehicle at the first time, where the second time is temporally before the first time; Determining the wheel speed pose of the vehicle at the first time according to the wheel speed pulse data of the vehicle; Fusing the visual pose and the wheel speed pose to obtain the target pose of the vehicle at the first time.

[0006] According to the second aspect of any embodiment of this application, a vehicle positioning device is provided. The device includes: A position determination module for determining the parking space feature positions at different times according to the sequential images of the target parking space; A visual pose determination module for determining the visual pose of the vehicle at the first time based on the parking space feature position at the first time, the parking space feature position at the second time, and the target pose of the vehicle at the second time, where the second time is temporally before the first time; A wheel speed pose determination module, configured to determine the wheel speed pose of the vehicle at the first moment according to the wheel speed pulse data of the vehicle; A pose fusion module, configured to fuse the visual pose and the wheel speed pose to obtain the target pose of the vehicle at the first moment.

[0007] According to a third aspect of any embodiment of the present application, there is provided an electronic device, including: A processor; A memory for storing instructions executable by the processor; Wherein, the processor realizes the method described in any embodiment of the present application by running the executable instructions.

[0008] According to a fourth aspect of any embodiment of the present application, there is provided a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the method described in any embodiment of the present application as above is realized.

[0009] According to a fifth aspect of any embodiment of the present application, there is provided a computer program product, on which a computer program / instruction is stored, and when the computer program / instruction is executed by a processor, the method described in any embodiment of the present application as above is realized.

[0010] The technical solution provided by the present application may include the following beneficial effects: According to the above embodiments, by determining the parking space feature positions at different moments according to the sequential images of the target parking space, and based on the parking space feature position at the first moment, the parking space feature position at the second moment, and the target pose of the vehicle at the second moment, the visual pose of the vehicle at the first moment is determined. According to the wheel speed pulse data of the vehicle, the wheel speed pose of the vehicle at the first moment is determined, and the visual pose and the wheel speed pose are fused to obtain the target pose of the vehicle at the first moment. Only an image sensor and a wheel speed sensor are required to determine and fuse the visual pose obtained based on the change of the parking space feature position and the wheel speed pose calculated from the wheel speed pulse data, thereby effectively suppressing the error of a single sensor, and reducing the cost and computing power requirements while improving the positioning accuracy.

[0011] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings herein are incorporated into the specification and form a part of the present application, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0013] Figure 1 is a flowchart of a vehicle positioning method shown according to an exemplary embodiment of the present application; Figure 2 It is a schematic diagram of the corner position of a parking space corner shown according to an exemplary embodiment of the present application; Figure 3 It is a flowchart of another vehicle positioning method shown according to an exemplary embodiment of the present application; Figure 4 It is a schematic structural diagram of an electronic device shown according to an exemplary embodiment of the present application; Figure 5 It is a block diagram of a vehicle positioning device shown according to an exemplary embodiment of the present application. Detailed implementation manners

[0014] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0015] The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "the", and "said" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0016] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0017] Automatic parking technology realizes the autonomous parking or driving out of a vehicle from a parking space under the condition of no human intervention through environment perception, path planning, and motion control algorithms. However, the current positioning methods usually rely on high-precision sensors and complex positioning algorithms, resulting in too high positioning costs and computing power requirements for the vehicle.

[0018] To solve the above problems, the present application proposes a vehicle positioning method. To further illustrate the present application, the following embodiments are provided: Please refer to Figure 1 ,Figure 1 FIG. Figure 1 is a flowchart of a vehicle positioning method according to an exemplary embodiment of the present application. The vehicle positioning method can be executed by an automatic parking system and applied to an automatic parking scenario. The automatic parking system can be applied to a vehicle or a server side such as a single server, a cluster server, or a cloud server. The vehicle positioning method can also be executed by other systems or devices in different application scenarios, and the embodiments of the present application do not limit this.

[0019] As Figure 1 shown, the vehicle positioning method may include the following steps: Step 101: Determine the parking space feature positions at different times according to the sequential images of the target parking space.

[0020] In this step, the automatic parking system can collect images of the vehicle's surrounding environment in real time through an image sensor mounted on the vehicle during the cruise and parking processes.

[0021] Among them, the image sensor is used to provide a complete view of the surrounding environment of the current vehicle. The image sensor can be set at any position on the vehicle, and the images captured by multiple image sensors can be stitched into a 360° top view.

[0022] Exemplarily, in order to expand the field of view and reduce the hardware cost, the image sensor can be multiple fisheye cameras respectively located in the front, rear, left, and right sides of the vehicle. The inverse perspective mapping (IPM) algorithm can be used to stitch the images collected by the multiple fisheye cameras into a bird's eye view (BEV) to provide complete information about the vehicle's surrounding environment.

[0023] The target parking space in the vehicle's surrounding environment can be detected according to the collected sequential images, and the relative position between the target parking space and the vehicle can be determined. If the target parking space in the sequential images meets conditions such as clear camera vision and complete visibility of parking space features, it can be determined that the target parking space is in the parking space observation area of the vehicle.

[0024] In the case where the target parking space is in the parking space observation area, according to the sequential images of the target parking space, algorithms such as semantic segmentation based on deep learning (such as U-Net, Mask R-CNN) or computer vision (such as edge detection, Hough transform) can be used to extract the parking space feature positions at different times from the sequential images and determine the parking space feature positions at different times.

[0025] Among them, the target parking space is the parking space where the vehicle is to be parked in the garage, such as: mechanical parking space, flat fixed parking space, etc. The parking space observation area is a specific distance range located on the side or front and rear of the vehicle, used to ensure that parking space features such as parking space lines and parking space corner points are clearly visible in the image and have no obvious occlusion. The sequential images are multiple frames of environmental images continuously acquired in chronological order. The parking space feature position is the position information of the parking space feature in the vehicle coordinate system.

[0026] Step 102: Based on the parking space feature position at the first moment, the parking space feature position at the second moment, and the target pose of the vehicle at the second moment, determine the visual pose of the vehicle at the first moment, where the second moment is chronologically before the first moment.

[0027] In this step, the visual pose of the vehicle in the world coordinate system can be calculated based on the parking space feature positions at the front and rear moments. By solving the rigid body transformation model and other methods, based on the parking space feature position at the first moment, the parking space feature position at the second moment, and the target pose of the vehicle at the second moment, determine the visual pose of the vehicle at the first moment.

[0028] Among them, the second moment is chronologically before the first moment. The first moment can be the current moment (t + 1 moment), and the second moment can be the previous moment adjacent to the first moment (t moment). The visual pose is the vehicle pose calculated based on the parking space features in the sequence, which can provide an absolute pose reference.

[0029] The target pose is the final positioning pose of the vehicle in the world coordinate system. The target pose can include the vehicle position and the vehicle heading angle, which respectively reflect the vehicle position and the vehicle orientation.

[0030] Step 103: Determine the wheel speed pose of the vehicle at the first moment according to the wheel speed pulse data of the vehicle.

[0031] In this step, the wheel speed pulse data of the vehicle's wheel speed sensor can be obtained through the Controller Area Network (CAN) bus. Based on the wheel speed pulse data, determine the change amount of the vehicle wheels from the second moment to the first moment, and use this change amount to update the wheel pose of the vehicle wheels at the second moment to obtain the wheel speed pose of the vehicle at the first moment.

[0032] Among them, the wheel speed pose is the vehicle pose calculated through the wheel speed pulse data, which can provide a high-frequency short-term relative pose.

[0033] Step 104: Fuse the visual pose and the wheel speed pose to obtain the target pose of the vehicle at the first moment.

[0034] In this step, since the visual pose is determined based on image features, it has high absolute accuracy but may be affected by factors such as shooting distance, ambient light, and occlusion; while the wheel speed pose is determined based on wheel speed pulse data, it has high short-term relative accuracy but has cumulative errors.

[0035] Fusion algorithms such as the Kalman filter algorithm and particle filter can be used to fuse the visual pose and the wheel speed pose. This can correct the visual influencing factors while correcting the wheel speed cumulative error, obtaining the target pose of the vehicle at the first moment. The fused target pose will be used for parking path planning and control to ensure that the vehicle accurately parks in the target parking space.

[0036] In one embodiment, a planned parking path can be generated based on the target pose at the first moment. According to the planned parking path, the vehicle is controlled to park in the target parking space.

[0037] Exemplarily, according to the kinematic constraints such as the minimum turning radius and maximum steering angle of the vehicle, a path planning algorithm can be used to plan the optimal parking path from the current pose to the target parking space. After the planning is completed, the path can be discretized into a series of dense path points. According to the lateral deviation and heading deviation between the target pose and the parking path, the front wheel steering angle and vehicle speed control amount are calculated in real time to make the vehicle accurately track the parking path.

[0038] During the parking process, the system continuously monitors the environmental changes, fuses and locates the target pose at the current moment in real time, updates the parking path, and uses the updated parking path for parking until the vehicle completely parks in the target parking space.

[0039] As described above, by generating a planned path based on the high-precision target pose, the geometric rationality and executability of the parking trajectory can be significantly improved. Through the linkage between the target pose and path planning, it can be ensured that the vehicle can accurately track the predetermined trajectory, and finally achieve safe and smooth automatic parking in the parking space.

[0040] The vehicle positioning method of this embodiment determines the characteristic positions of the parking space at different moments by according to the sequential images of the target parking space. Based on the characteristic position of the parking space at the first moment, the characteristic position of the parking space at the second moment, and the target pose of the vehicle at the second moment, the visual pose of the vehicle at the first moment is determined. According to the wheel speed pulse data of the vehicle, the wheel speed pose of the vehicle at the first moment is determined. The visual pose and the wheel speed pose are fused to obtain the target pose of the vehicle at the first moment. Only an image sensor and a wheel speed sensor are required to determine the visual pose obtained based on the change of the characteristic position of the parking space and the wheel speed pose deduced from the wheel speed pulse data and fuse them, thereby effectively suppressing the errors of a single sensor, reducing the cost and computing power requirements while improving the positioning accuracy.

[0041] In the foregoing embodiments, it is introduced that by combining the visual pose obtained based on the change of the characteristic position of the parking space with the wheel speed pose calculated from the wheel speed pulse data, high-precision positioning of the vehicle during the automatic parking process is achieved only by using an image sensor and a wheel speed sensor. In the following embodiments, the determination process of the visual pose will be described in more detail and can be applied to any of the foregoing embodiments.

[0042] In one embodiment, the characteristic position of the parking space may include the corner positions of the corner points in the target parking space. From the sequential images, the image positions of the corner points of the parking space in the image coordinate system are detected. The image positions are converted from the image coordinate system to the vehicle coordinate system to obtain the corner positions.

[0043] Please refer to Figure 2 , Figure 2 which shows a schematic diagram of the corner positions of a corner point of a parking space. Taking the four corner points of the target parking space as the parking space features as an example, algorithms such as the Shi-Tomasi corner detection algorithm can be used to detect the image positions of the four corner points of the target parking space 20 in the image coordinate system from the sequential images. Coordinate conversion algorithms such as the Perspective-n-Point (PnP) algorithm are used to convert the image positions from the image coordinate system to the vehicle coordinate system to obtain the corner positions.

[0044] By detecting the image positions of the four corner points in the image coordinate system and converting the image positions to the vehicle coordinate system, the corner positions are obtained. . represents the coordinate of the corner point of the parking space on the X-axis in the vehicle coordinate system at the second moment, represents the coordinate of the corner point of the parking space on the Y-axis in the vehicle coordinate system at the second moment.

[0045] As described above, by detecting the image positions of the corner points of the parking space in the image coordinate system from the sequential images and converting the image positions from the image coordinate system to the vehicle coordinate system to obtain the corner positions, accurate geometric constraints are provided for visual pose calculation, avoiding the problem of accuracy degradation of the visual pose caused by perspective distortion.

[0046] In one embodiment, according to the characteristic positions of the parking space at the first moment and the second moment, a pair of characteristic positions of the same parking space feature is constructed. Based on the pair of characteristic positions, the pose change amount of the vehicle from the second moment to the first moment is determined. According to the pose change amount and the target pose at the second moment, the visual pose at the first moment is determined.

[0047] Among them, the pair of characteristic positions is a matching pair composed of the characteristic positions of the same parking space feature at consecutive moments. The pose change amount is used to describe the motion change amount of the vehicle from the second moment to the first moment, and the pose change amount may include a translational change amount and a rotational component.

[0048] Exemplarily, when determining the parking space feature position at the first moment and the parking space feature position at the second moment , the parking space features at the first moment and the second moment can be matched one by one to form a feature position pair of the same parking space feature . For example: The parking space corner point located at the left front of the target parking space has a parking space feature position at time t of , and at time t + 1 is , and the feature position pair of this parking space corner point is .

[0049] Based on the feature position pair, by means of residual constraint, geometric consistency constraint, deep learning-based method, etc., determine the pose change amount of the vehicle from the second moment to the first moment. Superimpose the pose change amount on the target pose at the second moment to obtain the visual pose at the first moment.

[0050] As described above, by constructing the feature position pair and calculating the pose change amount, the spatio-temporal continuity of the parking space feature can be effectively utilized to improve the accuracy of the visual pose. At the same time, by combining the known target pose for calculation, the reliability of the visual pose estimation can be enhanced.

[0051] In one embodiment, for each feature position pair, construct a residual constraint reflecting the vehicle pose change. With the sum of the residual constraints of all feature position pairs minimized as the optimization objective, determine the pose change amount.

[0052] Among them, the residual constraint can describe the mathematical relationship between the vehicle pose change and the parking space feature position change. The feature position pair can include the parking space feature positions of the same parking space feature at the first moment and the second moment.

[0053] Exemplarily, for each pair of successfully matched feature position pairs, a residual constraint equation can be established, which can reflect that when assuming the vehicle has a pose change, the theoretical parking space feature at the previous moment should coincide with the observed feature point at the current moment after motion transformation. If the pose change amount is estimated accurately, this coincidence error (i.e., the residual) should approach zero.

[0054] Due to factors such as sensor noise and feature matching error in practice, the residuals of all feature position pairs usually will not be zero at the same time. Therefore, through the method of nonlinear optimization, adjust the three degrees of freedom (longitudinal displacement, lateral displacement, and heading angle change) of the pose change amount to minimize the sum of the squared residuals of all feature position pairs.

[0055] Exemplarily, for each feature position pair, a residual constraint can be constructed according to the following formula 1 Formula 1 Among them, represents the residual vector of the th feature position pair, and represents the coordinates of the parking space feature at the second moment in the vehicle coordinate system, and represents the coordinates of the parking space feature at the first moment in the vehicle coordinate system, represents the change in the X-axis of the vehicle from the second moment to the first moment, represents the change in the Y-axis of the vehicle from the second moment to the first moment, represents the change in the heading angle of the vehicle from the second moment to the first moment.

[0056] The optimization objective can be determined according to the following formula 2: Formula 2 Among them, represents the sum of the squared residuals of all feature position pairs.

[0057] The optimal pose change can be solved by solution methods such as non-linear optimization (such as the Levenberg-Marquardt algorithm) or closed-form solution (such as SVD decomposition),

[0058] The visual pose can include the visual position and the visual heading angle The visual pose at the first moment can be determined according to the following formula 3: Formula 3 Among them, represents the second moment, represents the first moment, represents the X-axis coordinate of the target pose in the world coordinate system at the second moment, represents the Y-axis coordinate of the target pose in the world coordinate system at the second moment, represents the vehicle heading angle at the second moment, represents the X-axis coordinate of the visual pose in the world coordinate system at the first moment, represents the Y-axis coordinate of the visual pose in the world coordinate system at the first moment, represents the vehicle heading angle at the first moment.

[0059] As described above, by constructing a residual constraint and an optimization method for minimizing the sum of squared residuals, the accuracy of parking space feature matching can be ensured, the stability of the pose change in complex parking environments such as lighting changes or occlusion scenarios can be improved, and thus the positioning reliability of the visual pose can be enhanced.

[0060] In the foregoing embodiments, it is introduced that by feature matching and residual optimization of the parking space corner points, combined with the conversion from the image coordinate system to the vehicle coordinate system, the accurate calculation of the vehicle visual pose is realized. In the following embodiments, the determination process of the wheel speed pose will be described in more detail and can be applied to any of the above embodiments.

[0061] In one embodiment, the driving state of the vehicle at the first moment can be obtained. According to the driving state and the rear-wheel pose of the vehicle rear wheels at the second moment, the rear-wheel pose of the vehicle rear wheels at the first moment is determined. Based on the rear-wheel pose at the first moment, the wheel speed pose at the first moment is determined.

[0062] Among them, the vehicle rear wheels can include the left rear wheel and the right rear wheel. The rear-wheel pose is the position and orientation information of the vehicle rear wheels at a certain moment, which is used to deduce the motion trajectory of the vehicle. The rear-wheel pose can include the rear-wheel position and the rear-wheel heading angle.

[0063] Exemplarily, the driving state of the vehicle at the first moment can be obtained by analyzing the wheel speed sensor pulse signal, detecting the angular velocity of the Inertial Measurement Unit (IMU), parsing the steering angle sensor data, etc. For example, by comparing the pulse count differences between the left and right rear wheels to determine whether the vehicle is turning, or by combining the yaw angular velocity of the IMU to verify the motion trend of the vehicle.

[0064] According to the driving state and the rear-wheel pose of the vehicle rear wheels at the second moment, the rear-wheel pose of the vehicle rear wheels at the first moment is determined by means of kinematic inverse calculation, track backtracking optimization, etc. For example, if the vehicle is in a turning state at the second moment, the rear-wheel position and rear-wheel orientation at the first moment are deduced based on the turning radius and the change amount of the heading angle.

[0065] The wheel speed pose at the first moment can be determined based on the rear-wheel pose at the first moment by means of heading angle smooth interpolation, pose optimization based on kinematic constraints, etc. For example, the average value of the left and right rear-wheel positions is taken as the vehicle center point coordinate, and the wheel speed pose at the first moment is determined in combination with the historical vehicle heading angle data to improve the continuity of pose estimation.

[0066] As described above, by combining the driving state of the vehicle and the rear-wheel pose, the cumulative error of the wheel speed pulse data can be effectively suppressed, the long-term stability of dead reckoning can be improved, and the accuracy of wheel speed pulse data positioning can be improved.

[0067] In one embodiment, the change amount of the pulse number of the vehicle rear wheels from the second moment to the first moment is obtained. Based on the preset driving distance per unit pulse and the change amount of the pulse number, the incremental driving distance of the vehicle rear wheels is determined. According to the incremental driving distance of the vehicle rear wheels, the driving state is determined.

[0068] Among them, the change in the number of pulses is the change in the number of pulses output by the wheel speed sensor from the second moment to the first moment, and is used to calculate the rotational distance of the wheel. The incremental driving distance is the actual distance traveled by the wheel from the second moment to the first moment, and reflects the change in the movement of the wheel from the second moment to the first moment.

[0069] Exemplarily, the pulse signals of the wheel speed sensors installed on the left and right rear wheels can be collected in real time, and the change in the number of pulses of each rear wheel from the second moment to the first moment can be calculated. Combining the change in the number of pulses and the preset driving distance per unit pulse, the incremental driving distance of each rear wheel from the second moment to the first moment can be calculated.

[0070] If the incremental driving distances of the left and right rear wheels are equal, it can be determined that the vehicle is in a straight driving state; if the incremental driving distances of the left and right rear wheels are not equal, it can be determined that the vehicle is in a curve driving state.

[0071] Exemplarily, the incremental driving distance of the rear wheels of the vehicle can be determined according to the following formula 4: Formula 4 Among them, represents the incremental driving distance of the rear wheels of the vehicle, represents the preset driving distance per unit pulse, represents the change in the number of pulses of the rear wheels of the vehicle.

[0072] Based on the above formula 4, the incremental driving distances of the left and right rear wheels of the vehicle can be calculated. If the incremental driving distances of the left and right rear wheels are not equal, it indicates that there is differential motion and the vehicle is in a steering driving situation. Exemplarily, the driving state can be determined according to the following formula 5: Formula 5 Among them, represents the incremental driving distance of the right rear wheel, represents the incremental driving distance of the left rear wheel.

[0073] As described above, by judging the driving state based on the incremental driving distance of the rear wheels of the vehicle, low-cost identification of the vehicle motion state can be achieved, without additional sensors such as IMUs, thereby further simplifying the system architecture.

[0074] In one embodiment, in the case where the driving state is a straight driving state, based on the incremental driving distance of the rear wheels of the vehicle and the pose of the rear wheels at the second moment, the pose of the rear wheels at the first moment is determined.

[0075] Exemplarily, in the straight driving state, the rear wheel pose at the first moment can be deduced based on the rear wheel position and the rear wheel heading angle at the second moment, combined with the incremental driving distance, through the planar kinematic formula. That is, the rear wheel positions of the left and right rear wheels at the first moment are respectively translated along the original heading angle direction by the corresponding driving distance, while the heading angle remains unchanged.

[0076] In the case where the driving state is the straight driving state, the rear wheel position of the left rear wheel at the first moment can be determined according to the following formula 6 and the rear wheel heading angle ( ): Formula 6 where represents the second moment, represents the first moment, represents the coordinate of the left rear wheel on the X-axis of the vehicle coordinate system at the second moment, represents the coordinate of the left rear wheel on the Y-axis of the vehicle coordinate system at the second moment, represents the vehicle heading angle of the left rear wheel at the second moment, represents the coordinate of the left rear wheel on the X-axis of the vehicle coordinate system at the first moment, represents the coordinate of the left rear wheel on the Y-axis of the vehicle coordinate system at the first moment, represents the vehicle heading angle of the left rear wheel at the first moment, represents the change amount of the pulse number of the left rear wheel.

[0077] The rear wheel position of the right rear wheel at the first moment can be determined according to the following formula 7 and the right wheel heading angle ( ): Formula 7 where represents the coordinate of the right rear wheel on the X-axis of the vehicle coordinate system at the second moment, represents the coordinate of the right rear wheel on the Y-axis of the vehicle coordinate system at the second moment, represents the vehicle heading angle of the right rear wheel at the second moment, represents the coordinate of the right rear wheel on the X-axis of the vehicle coordinate system at the first moment, represents the coordinate of the right rear wheel on the Y-axis of the vehicle coordinate system at the first moment, represents the vehicle heading angle of the right rear wheel at the first moment, represents the change amount of the pulse number of the right rear wheel.

[0078] In the case where the driving state is a curved driving state, based on the vehicle wheelbase and the incremental driving distance of the vehicle's rear wheels, determine the change amount of the vehicle's heading angle, and based on the change amount of the heading angle, the incremental driving distance of the vehicle's rear wheels, and the rear wheel pose at the second moment, determine the rear wheel pose at the first moment. Wherein, the change amount of the heading angle is the change amount of the vehicle's heading angle from the second moment to the first moment.

[0079] Exemplarily, in the curved driving state, the difference between the incremental driving distance of the left rear wheel and the incremental driving distance of the right rear wheel can be obtained, and based on the difference between the incremental driving distances of the left and right rear wheels and the vehicle wheelbase, the change amount of the vehicle's heading angle can be calculated.

[0080] After obtaining the difference between the incremental driving distances of the left and right rear wheels, the movement trajectory of each rear wheel can be recalculated with the steering center as the reference. The arc length formula can be used to update the rear wheel pose at the second moment, and the heading angle of the rear wheel at the second moment is superimposed with the change amount of the heading angle to obtain the heading angle of the rear wheel at the first moment.

[0081] Exemplarily, the change amount of the heading angle can be determined according to the following formula 8: Formula 8 Wherein, represents the change amount of the heading angle, represents the vehicle wheelbase of the vehicle.

[0082] In the case where the driving state is a curved driving state, the rear wheel position and the rear wheel heading angle ( ) of the left rear wheel at the first moment can be determined according to the following formula 9: Formula 9 Wherein, represents a normalization term related to the turning radius, which is used for approximate processing of the turning arc length, represents the instantaneous heading angle of the movement direction of the left rear wheel, represents the displacement component of the left rear wheel along the turning arc.

[0083] The rear wheel position and the right wheel heading angle ( ) of the right rear wheel at the first moment can be determined according to the following formula 10: Formula 10 Wherein, represents the instantaneous heading angle of the movement direction of the right rear wheel, represents the displacement component of the right rear wheel along the turning arc.

[0084] The wheel speed pose can include the wheel speed position and the wheel speed heading angle , after determining the rear-wheel poses of the left and right rear wheels, the center points of the left and right rear wheels (the center of the vehicle's rear axle) can be used as the representative points of the vehicle. Based on the rear-wheel positions of the left and right rear wheels at the first moment, the wheel speed position of the vehicle at the first moment can be calculated by taking the average, and the heading angle change amount can be superimposed on the wheel speed heading angle of the previous moment, so as to obtain the wheel speed pose at the first moment.

[0085] Exemplarily, the wheel speed pose at the first moment can be determined according to Formula 11 as follows: Formula 11 Wherein, represents the coordinate of the wheel speed position on the X-axis of the world coordinate system at the first moment, represents the coordinate of the wheel speed position on the Y-axis of the world coordinate system at the first moment, represents the vehicle heading angle of the vehicle at the first moment, represents the vehicle heading angle of the vehicle at the second moment.

[0086] As described above, by dynamically adjusting the wheel speed pose calculation strategy through driving state classification, in the straight driving state, the rear-wheel pose at the first moment can be determined only based on the incremental driving distance of the vehicle's rear wheels and the rear-wheel pose at the second moment, which can improve the calculation efficiency; in the curve driving state, a heading angle change amount compensation mechanism is introduced, which can correct the steering deviation during curve driving and improve the pose positioning accuracy under complex trajectories.

[0087] In one embodiment, the wheel speed pose and the visual pose can be fused based on the Kalman filter algorithm. The predicted state at the first moment can be generated based on the wheel speed pose using the state transition model. According to the Jacobian matrix of the state transition model, the covariance matrix of the predicted state is updated.

[0088] According to the visual pose, an observation model is established. Based on the observation model and the predicted state, the residual and covariance are determined, and according to the residual and covariance, the Kalman gain is determined. The predicted state and the covariance matrix are corrected using the Kalman gain to obtain the target pose.

[0089] Among them, the state transition model is used to describe the dynamic evolution law of the state variables of the vehicle under the action of the control variables. The state transition model can include state variables and control variables. The state variables are used to describe the vehicle position and orientation of the vehicle at present, and can include the vehicle position and the vehicle heading angle. The control variables can include the moving distance increment and the heading angle change amount derived from the wheel speed pulse data.

[0090] The Jacobian matrix is the partial derivative matrix of the state transition model with respect to the state variables, which is used to linearize the non-linear motion model to support the prediction update of the covariance matrix. The covariance matrix is used to combine the previous state covariance, the Jacobian matrix of the state transition model, and the process noise characteristics, and can quantify the uncertainty propagation of the predicted state.

[0091] The observation model can represent the actually observed vehicle pose information, which is used to reflect the mapping relationship between the visual pose and the state vector. The observation model can include an observation vector and an observation matrix. The residual can represent the difference between the visual pose and the predicted state, and the covariance can fuse the uncertainty of the predicted state and the visual measurement noise characteristics to evaluate the confidence of the residual.

[0092] Exemplarily, state variables and control variables can be defined. The state variables are determined according to Equation 12 below : Equation 12 where, and represent the vehicle position, represents the vehicle heading angle.

[0093] The control variables can be determined according to Equation 13 below : Equation 13 where, represents the incremental movement distance of the vehicle, represents the change in the vehicle heading angle.

[0094] Based on the state variables and control variables, a state transition model can be established according to the vehicle kinematic characteristics to describe the motion law of the vehicle under the action of the control variables. The state transition model is determined according to Equation 14 below: Equation 14 Based on the state transition model, the state and the covariance matrix can be initialized to form the initial state parameters of the filtering algorithm. Based on the determined wheel speed pose, using the state transition model, the predicted state is determined according to Equation 15 below: Equation 15 The Jacobian matrix is determined according to Equation 16 below : Equation 16 The covariance matrix can be updated according to Equation 17 using the Jacobian matrix and the process noise covariance: Equation 17 Among them, represents the process noise covariance, which can reflect the error of the wheel speed pulse.

[0095] Based on the determined visual pose, the observation model can be determined according to the following formula 18: Formula 18 Among them, represents the observation vector, and represent the vehicle position of the visual pose, represents the vehicle heading angle of the visual pose, represents the observation matrix, represents the three-dimensional identity matrix, which is used to reflect the consistency between the visual pose and the dimension of the state variable.

[0096] Based on the observation model and the predicted state, the residual and covariance can be determined according to the following formula 19: Formula 19 Among them, represents the visual noise measurement covariance, which is used to reflect the error situation of visual observation.

[0097] By weighing the prediction covariance and the measurement covariance, the optimal fusion weight is calculated to determine the correction intensity of the visual pose on the predicted state. The Kalman gain can be determined according to the following formula 20 : Formula 20 The predicted state and the residual are fused using the Kalman gain to obtain the corrected optimal state estimate, and the covariance matrix is updated simultaneously to reflect the uncertainty after fusion. The predicted state and the covariance matrix are corrected according to the following formula 21: Formula 21 As described above, by introducing the Kalman filtering algorithm, the wheel speed pose is predicted using the state transition model, and the covariance matrix is updated through the Jacobian matrix, which can reasonably quantify the uncertainty of the predicted state. In the observation update stage, the weights of prediction and observation are adaptively adjusted through the Kalman gain, significantly reducing the influence of wheel speed cumulative error and visual instantaneous noise.

[0098] To further introduce the vehicle positioning process, Figure 3 a flowchart of another vehicle positioning method is shown. This vehicle positioning method may include the following steps: Step 301: Detect the image position of the parking space corner points from the sequential images.

[0099] In this step, the time-series images can be collected by the surround-view camera, and the parking space corner points of the target parking space in the time-series images can be extracted by the image processing algorithm, and after the extraction, a two-dimensional image position is formed in the image coordinate system.

[0100] Step 302: Convert the image position from the image coordinate system to the vehicle coordinate system to obtain the corner point position.

[0101] In this step, the camera calibration parameters and installation position information can be used to convert the image position into the vehicle coordinate system through perspective transformation to obtain the corner point position of the parking space in three-dimensional space.

[0102] Step 303: construct a feature position pair of the corner point of the same parking space according to the corner point positions at the first moment and the second moment.

[0103] In this step, the corner point positions of two consecutive frames (the first moment and the second moment) can be obtained. The corner point positions of the same parking space corner point at different moments are matched to construct feature position pairs, each of which contains the corner point positions of the same parking space corner point at the first moment and the second moment.

[0104] Step 304: For each feature position pair, construct a residual constraint that reflects the change in vehicle posture.

[0105] In this step, a residual constraint equation is established for each matched feature position pair.

[0106] Step 305: Determine the change in posture by minimizing the sum of the residual constraints of all feature position pairs as the optimization goal.

[0107] In this step, the optimal pose change can be solved by minimizing the sum of the residual constraints of all feature position pairs.

[0108] Step 306: Determine the visual posture at the first moment according to the posture change and the target posture at the second moment.

[0109] In this step, the posture change can be superimposed on the target posture at the second moment to infer the visual posture at the first moment.

[0110] Step 307: Obtain the driving state of the vehicle at the first moment.

[0111] In this step, the wheel speed pulse data of the left and right rear wheels can be obtained to calculate the pulse number change of each rear wheel. The incremental driving distance of each rear wheel is calculated by combining the pulse number change and the preset driving distance per unit pulse.

[0112] If the incremental driving distances of the left and right rear wheels are equal, it can be determined that the vehicle is in a straight-line driving state; if the incremental driving distances of the left and right rear wheels are not equal, it can be determined that the vehicle is in a curved driving state.

[0113] Step 308: Determine the rear wheel pose at the first moment according to the driving state and the rear wheel pose of the vehicle at the rear wheels at the second moment.

[0114] In this step, in the case where the driving state is a straight driving state, the rear wheel pose at the first moment can be determined based on the incremental driving distance of the vehicle's rear wheels and the rear wheel pose at the second moment.

[0115] In the case where the driving state is a curved driving state, obtain the difference between the incremental driving distance of the left rear wheel and the incremental driving distance of the right rear wheel, and calculate the change amount of the vehicle's heading angle according to the difference between the incremental driving distances of the left and right rear wheels and the vehicle wheelbase. Based on the change amount of the heading angle, the incremental driving distance of the vehicle's rear wheels, and the rear wheel pose at the second moment, determine the rear wheel pose at the first moment.

[0116] Step 309: Determine the wheel speed pose at the first moment based on the rear wheel pose at the first moment.

[0117] In this step, the average value of the rear wheel positions of the left and right rear wheels at the first moment can be used as the wheel speed position of the vehicle at the first moment, and the change amount of the heading angle is superimposed on the wheel speed heading angle of the vehicle at the previous moment to obtain the wheel speed pose at the first moment.

[0118] Step 310: Generate a predicted state using the state transition model based on the wheel speed pose.

[0119] In this step, based on the wheel speed pose, use the state transition model to predict the vehicle pose at the current moment to obtain a preliminary predicted state. Calculate the covariance matrix of the predicted pose based on the Jacobian matrix to quantify the error of the predicted state.

[0120] Step 311: Update the predicted pose using the visual pose to obtain the target pose at the first moment.

[0121] In this step, according to the visual pose, establish an observation model to describe the actually observed vehicle pose. Compare the difference between the observation model and the predicted state to determine the residual and covariance, which reflect the degree of inconsistency and uncertainty between the observed information and the predicted information. Calculate the Kalman gain based on the residual and covariance matrix, dynamically adjust the weights of the predicted state and the observed visual pose, and finally output the fused target pose.

[0122] Step 312: Control the vehicle to park in the target parking space based on the target pose at the first moment.

[0123] In this step, a path planning algorithm can be used to generate a planned parking path based on the target pose at the first moment. According to the planned parking path, dynamically adjust the steering and vehicle speed, and while continuously monitoring the environmental changes and pose accuracy, control the vehicle to park in the target parking space.

[0124] Figure 4 This is a schematic structural diagram of an electronic device shown according to an exemplary embodiment of the present application. The electronic device may be, for example, a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a personal digital assistant, a server, a smart home appliance, a vehicle-mounted computer, etc. Referring to Figure 4 , at the hardware level, the electronic device includes a processor 401, an internal bus 402, a network interface 403, a memory 404, and a non-volatile memory 405. Of course, it may also include other hardware required for other services. The processor 401 reads the corresponding computer program from the non-volatile memory 405 into the memory 404 and then runs it, forming a vehicle positioning device at the logical level. Of course, in addition to the software implementation method, the present application does not exclude other implementation methods, such as logical devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logical unit, and may also be hardware or a logical device.

[0125] Figure 5 This is a block diagram of a vehicle positioning device shown according to an exemplary embodiment of the present application. Referring to Figure 5 , the device may include: a position determination module 501, a visual pose determination module 502, a wheel speed pose determination module 503, and a pose fusion module 504, where: The position determination module 501 is configured to determine the parking space feature positions at different times according to the sequential images of the target parking space; The visual pose determination module 502 is configured to determine the visual pose of the vehicle at the first time based on the parking space feature position at the first time, the parking space feature position at the second time, and the target pose of the vehicle at the second time, where the second time is before the first time in time sequence; The wheel speed pose determination module 503 is configured to determine the wheel speed pose of the vehicle at the first time according to the wheel speed pulse data of the vehicle; The pose fusion module 504 is configured to fuse the visual pose and the wheel speed pose to obtain the target pose of the vehicle at the first time.

[0126] In one example, when the visual pose determination module 502 is used to determine the visual pose of the vehicle at the first moment based on the parking space feature position at the first moment, the parking space feature position at the second moment, and the target pose of the vehicle at the second moment, it includes: constructing a feature position pair of the same parking space feature according to the parking space feature positions at the first moment and the second moment; determining the pose change amount of the vehicle from the second moment to the first moment based on the feature position pair; and determining the visual pose at the first moment according to the pose change amount and the target pose at the second moment.

[0127] In one example, when the visual pose determination module 502 is used to determine the pose change amount of the vehicle from the second moment to the first moment based on the feature position pair, it includes: constructing a residual constraint reflecting the vehicle pose change for each feature position pair; and determining the pose change amount with the optimization goal of minimizing the sum of the residual constraints of all feature position pairs.

[0128] In one example, when the wheel speed pose determination module 503 is used to determine the wheel speed pose of the vehicle at the first moment according to the wheel speed pulse data of the vehicle, it includes: obtaining the driving state of the vehicle at the first moment; determining the rear wheel pose of the vehicle at the first moment according to the driving state and the rear wheel pose of the vehicle rear wheel at the second moment; and determining the wheel speed pose at the first moment based on the rear wheel pose at the first moment.

[0129] In one example, when the wheel speed pose determination module 503 is used to obtain the driving state of the vehicle at the first moment, it includes: obtaining the change amount of the number of pulses of the vehicle rear wheel from the second moment to the first moment; determining the incremental driving distance of the vehicle rear wheel based on the preset driving distance per unit pulse and the change amount of the number of pulses; and determining the driving state according to the incremental driving distance of the vehicle rear wheel.

[0130] In one example, when the wheel speed pose determination module 503 is used to determine the rear wheel pose of the vehicle at the first moment according to the driving state and the rear wheel pose of the vehicle rear wheel at the second moment, it includes: in the case where the driving state is a straight driving state, determining the rear wheel pose at the first moment based on the incremental driving distance of the vehicle rear wheel and the rear wheel pose at the second moment; in the case where the driving state is a curve driving state, determining the change amount of the heading angle of the vehicle based on the vehicle wheelbase and the incremental driving distance of the vehicle rear wheel, and determining the rear wheel pose at the first moment based on the change amount of the heading angle, the incremental driving distance of the vehicle rear wheel, and the rear wheel pose at the second moment.

[0131] In one example, the parking space feature position includes the corner positions of the corner points in the target parking space; when the position determination module 501 is configured to determine the parking space feature positions at different times according to the sequential images of the target parking space, it includes: detecting the image positions of the corner points of the parking space in the image coordinate system from the sequential images; and converting the image positions from the image coordinate system to the vehicle coordinate system to obtain the corner positions.

[0132] In one example, when the pose fusion module 504 is configured to fuse the visual pose and the wheel speed pose, it includes: generating a predicted state at the first moment based on the wheel speed pose by using a state transition model, where the state transition model includes a state vector and a control vector; updating the covariance matrix of the predicted state according to the Jacobian matrix of the state transition model; establishing an observation model according to the visual pose, where the observation model is used to reflect the mapping relationship between the visual pose and the state vector; determining a residual and a covariance based on the observation model and the predicted state, and determining a Kalman gain according to the residual and the covariance; and correcting the predicted state and the covariance matrix by using the Kalman gain to obtain the target pose.

[0133] In one example, the pose fusion module 504 is further configured to generate a planned parking path based on the target pose at the first moment; and control the vehicle to park into the target parking space according to the planned parking path.

[0134] The implementation processes of the functions and roles of each unit in the above device are specifically described in detail in the implementation processes of the corresponding steps in the above method, and will not be elaborated here.

[0135] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0136] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is further provided, such as a memory including instructions, and the above instructions can be executed by a processor of the vehicle positioning device to implement the method described in any one of the above embodiments.

[0137] Among them, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., and the present application does not limit this.

[0138] In an exemplary embodiment, a computer program product including computer programs / instructions is further provided, and the above computer programs / instructions can be executed by a processor of the vehicle positioning device to implement the method described in any one of the above embodiments.

[0139] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0140] Those skilled in the art will readily conceive of other implementations of the present application after considering the specification and practicing the invention herein. The present application is not limited to the exact structures described above and shown in the figures, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

[0141] The foregoing is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the scope of protection of the present application.

Claims

1. A vehicle positioning method, characterized in that, The method includes: Determining the characteristic positions of the parking space at different times according to the sequential images of the target parking space; Based on the characteristic position of the parking space at the first time, the characteristic position of the parking space at the second time, and the target pose of the vehicle at the second time, determining the visual pose of the vehicle at the first time, where the second time is temporally before the first time; Determining the wheel speed pose of the vehicle at the first time according to the wheel speed pulse data of the vehicle; Fusing the visual pose and the wheel speed pose to obtain the target pose of the vehicle at the first time.

2. The method according to claim 1, characterized in that, The determining the visual pose of the vehicle at the first time based on the characteristic position of the parking space at the first time, the characteristic position of the parking space at the second time, and the target pose of the vehicle at the second time includes: Constructing a pair of characteristic positions of the same parking space characteristic according to the characteristic positions of the parking space at the first time and the second time; Based on the pair of characteristic positions, determining the pose change amount of the vehicle from the second time to the first time; Determining the visual pose of the first time according to the pose change amount and the target pose of the second time.

3. The method according to claim 2, wherein The determining the pose change amount of the vehicle from the second time to the first time based on the pair of characteristic positions includes: For each pair of characteristic positions, constructing a residual constraint reflecting the vehicle pose change; Taking the minimization of the sum of the residual constraints of all pairs of characteristic positions as the optimization objective to determine the pose change amount.

4. The method according to claim 1, characterized in that, The determining the wheel speed pose of the vehicle at the first time according to the wheel speed pulse data of the vehicle includes: Obtaining the driving state of the vehicle at the first time; According to the driving state and the rear wheel pose of the vehicle at the second time, determining the rear wheel pose of the vehicle at the first time; Based on the rear wheel pose at the first time, determining the wheel speed pose at the first time.

5. The method according to claim 4, characterized in that The obtaining the driving state of the vehicle at the first time includes: Obtaining the change amount of the number of pulses of the vehicle rear wheel from the second time to the first time; Based on the preset driving distance per unit pulse and the change amount of the number of pulses, determining the incremental driving distance of the vehicle rear wheel; Determining the driving state according to the incremental driving distance of the vehicle rear wheel.

6. The method according to claim 4, characterized in that The determining the rear wheel pose of the vehicle at the first time according to the driving state and the rear wheel pose of the vehicle at the second time includes: In the case where the driving state is a straight driving state, based on the incremental driving distance of the vehicle rear wheel and the rear wheel pose at the second time, determining the rear wheel pose at the first time; In the case where the driving state is a curve driving state, based on the wheelbase of the vehicle and the incremental driving distance of the vehicle rear wheel, determining the change amount of the heading angle of the vehicle, and based on the change amount of the heading angle, the incremental driving distance of the vehicle rear wheel, and the rear wheel pose at the second time, determining the rear wheel pose at the first time.

7. The method according to claim 1, wherein The characteristic position of the parking space includes the corner position of the corner point of the target parking space; The determining the characteristic positions of the parking space at different times according to the sequential images of the target parking space includes: Detect the image position of the parking space corner points in the image coordinate system from the sequential images; Convert the image position from the image coordinate system to the vehicle coordinate system to obtain the corner point position.

8. The method according to claim 1, wherein The fusion of the visual pose and the wheel speed pose includes: Based on the wheel speed pose, generate a predicted state at the first moment using a state transition model, where the state transition model includes a state vector and a control vector; Update the covariance matrix of the predicted state according to the Jacobian matrix of the state transition model; Based on the visual pose, establish an observation model, where the observation model is used to reflect the mapping relationship between the visual pose and the state vector; Based on the observation model and the predicted state, determine the residual and covariance, and determine the Kalman gain according to the residual and covariance; Use the Kalman gain to correct the predicted state and the covariance matrix to obtain the target pose.

9. The method according to claim 1, characterized in that, The method further includes: Generate a planned parking path based on the target pose at the first moment; Control the vehicle to park into the target parking space according to the planned parking path.

10. A vehicle positioning device, characterized in that, The device includes: A position determination module for determining the parking space feature positions at different moments according to the sequential images of the target parking space; A visual pose determination module for determining the visual pose of the vehicle at the first moment based on the parking space feature position at the first moment, the parking space feature position at the second moment, and the target pose of the vehicle at the second moment, where the second moment is before the first moment in time sequence; A wheel speed pose determination module for determining the wheel speed pose of the vehicle at the first moment according to the wheel speed pulse data of the vehicle; A pose fusion module for fusing the visual pose and the wheel speed pose to obtain the target pose of the vehicle at the first moment.

11. An electronic device, characterized in that, Includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor realizes the method according to any one of claims 1-9 by running the executable instructions.

12. A computer-readable storage medium having computer instructions stored thereon, characterized in that, When the instruction is executed by the processor, it realizes the method according to any one of claims 1-9.

13. A computer program product, on which a computer program / instructions are stored, characterized in that, When the computer program / instruction is executed by the processor, it realizes the method according to any one of claims 1-9.

Images