A parking space tracking method, device, equipment and storage medium

Abstract

The present application relates to the field of autonomous driving technology, and in particular to a method, device, equipment and storage medium for tracking available parking spaces, the method comprising: obtaining vehicle wheel speed information, historical perceived parking space information, historical available parking space information, and currently perceived parking space information at historical moments; determining first predicted parking space information of the historical perceived parking space information at the current moment based on the vehicle wheel speed information and the historical perceived parking space information; determining second predicted parking space information at the current moment based on the historical available parking space information; fusing the first predicted parking space information and the second predicted parking space information to obtain fused predicted parking space information; determining current available parking space information at the current moment based on the fused predicted parking space information and the currently perceived parking space information. The method increases the accuracy and stability of parking space information prediction by fusing the first predicted parking space information and the second predicted parking space information, and can improve the accuracy of available parking space tracking.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a parking space tracking method, device, equipment and storage medium. Background Art

[0002] The automatic parking system is divided into four parts: parking space sensing, parking space tracking, parking space selection, and parking. Among them, the parking space tracking technology tracks the perceived parking spaces based on the parking space sensing, obtains the continuous tracking of the same parking space position by the vehicle during the automatic parking process, and then outputs it to the decision module for selection, and then the decision module selects the parking space and then parks.

[0003] In the related art, parking space tracking technology generally adopts image target tracking technology. However, the tracking accuracy of image target tracking technology is low, and tracking failure is likely to occur when the vehicle speed is too fast or the vehicle turns. Summary of the invention

[0004] The present application provides a parking space tracking method, device, equipment and storage medium, which can improve the accuracy of parking space tracking and effectively prevent the failure of parking space tracking at a curve.

[0005] In a first aspect, an embodiment of the present application discloses a method for tracking available parking spaces, the method comprising:

[0006] Obtain vehicle wheel speed information, historical perceived parking space information, and historical available parking space information at historical moments, as well as current perceived parking space information at the current moment;

[0007] Determine the first predicted parking space information of the historically sensed parking space information at the current moment according to the vehicle wheel speed information and the historically sensed parking space information;

[0008] Determine the second predicted parking space information at the current moment according to the historical available parking space information;

[0009] Fusing the first predicted parking space information and the second predicted parking space information to obtain fused predicted parking space information;

[0010] Based on the fusion of predicted parking space information and currently sensed parking space information, the current available parking space information at the current moment is determined.

[0011] Further, the vehicle wheel speed information includes vehicle speed information and steering angle velocity information; determining the first predicted parking space information of the historically sensed parking space information at the current moment according to the vehicle wheel speed information and the historically sensed parking space information includes:

[0012] Determine the vehicle deflection angle information from the historical moment to the current moment according to the steering angular velocity information;

[0013] Determine the vehicle deflection displacement information from the historical moment to the current moment according to the vehicle speed information and the vehicle deflection angle information;

[0014] According to the vehicle deflection displacement information and the historically sensed parking space information, first predicted parking space information of the historically sensed parking space information at the current moment is determined.

[0015] Further, according to the historical available parking space information, the second predicted parking space information at the current moment is determined, including:

[0016] The historical available parking space information is predicted based on the Kalman filter to obtain the second predicted parking space information at the current moment.

[0017] Further, the first predicted parking space information includes first coordinate information of the target available parking space; the second predicted parking space information includes second coordinate information of the target available parking space; and the first predicted parking space information and the second predicted parking space information are fused to obtain fused predicted parking space information, including:

[0018] Determining deviation correction information according to the first coordinate information and the second coordinate information;

[0019] Based on the first coordinate information and the deviation correction information, the fused predicted parking space information of the target available parking space is obtained.

[0020] Furthermore, according to the fusion prediction parking space information and the current sensed parking space information, the current available parking space information at the current moment is determined, including:

[0021] Determine the parking space prediction area at the current moment based on the fused predicted parking space information;

[0022] According to the currently sensed parking space information, determine the parking space sensing area at the current moment;

[0023] Determine the matching information between the parking space prediction area and the parking space perception area;

[0024] When the matching degree information is preset matching degree information, the current available parking space information at the current moment is determined based on the fusion of the predicted parking space information and the currently sensed parking space information; the preset matching degree information is used to characterize that the matching degree between the parking space prediction area and the parking space perception area is greater than or equal to the matching degree threshold.

[0025] Furthermore, the matching information between the parking space prediction area and the parking space perception area is determined, including:

[0026] Determine the similarity cost matrix between the parking space prediction area and the parking space perception area;

[0027] The matching information between the parking space prediction area and the parking space perception area is determined based on the similarity cost matrix.

[0028] Furthermore, according to the fusion prediction parking space information and the current sensed parking space information, the current available parking space information at the current moment is determined, including:

[0029] According to the fusion of predicted parking space information and currently perceived parking space information, the current available parking space information at the current moment is determined based on the Kalman filter.

[0030] In a second aspect, an embodiment of the present application discloses a parking space tracking device, the device comprising:

[0031] An information acquisition module is used to acquire vehicle wheel speed information, historical perceived parking space information and historical available parking space information at historical moments, as well as current perceived parking space information at the current moment;

[0032] A first predicted parking space information determination module, used to determine first predicted parking space information of the historically sensed parking space information at a current moment according to the vehicle wheel speed information and the historically sensed parking space information;

[0033] A second predicted parking space information determination module, used to determine the second predicted parking space information at the current moment according to the historical available parking space information;

[0034] A fused predicted parking space information determination module is used to fuse the first predicted parking space information and the second predicted parking space information to obtain fused predicted parking space information;

[0035] The current available parking space information determination module is used to determine the current available parking space information at the current moment according to the fusion predicted parking space information and the current perceived parking space information.

[0036] As an optional implementation manner, the vehicle wheel speed information includes vehicle speed information and steering angle velocity information; the first predicted parking space information determination module includes:

[0037] A vehicle deflection angle information determination unit, used to determine the vehicle deflection angle information from a historical moment to a current moment according to the steering angular velocity information;

[0038] A vehicle deflection displacement information determination unit, used to determine the vehicle deflection displacement information from a historical moment to a current moment according to the vehicle speed information and the vehicle deflection angle information;

[0039] The first predicted parking space information determining unit is used to determine the first predicted parking space information of the historically sensed parking space information at the current moment according to the vehicle deflection displacement information and the historically sensed parking space information.

[0040] As an optional implementation, the second predicted parking space information determination module includes:

[0041] The second predicted parking space information determining unit is used to predict the historical available parking space information based on the Kalman filter to obtain the second predicted parking space information at the current moment.

[0042] As an optional implementation, the first predicted parking space information includes first coordinate information of the target available parking space; the second predicted parking space information includes second coordinate information of the target available parking space; and the fusion predicted parking space information determination module includes:

[0043] a deviation correction information determining unit, configured to determine the deviation correction information according to the first coordinate information and the second coordinate information;

[0044] The fused predicted parking space information determining unit is used to obtain the fused predicted parking space information of the target available parking space based on the first coordinate information and the deviation correction information.

[0045] As an optional implementation, the current available parking space information determination module includes:

[0046] A parking space prediction area determination unit, used to determine the parking space prediction area at the current moment according to the fused prediction parking space information;

[0047] A parking space sensing area determination unit, used to determine the parking space sensing area at the current moment according to the currently sensed parking space information;

[0048] A matching degree information determination unit, used to determine matching degree information between the parking space prediction area and the parking space perception area;

[0049] The current available parking space information determination unit is used to determine the current available parking space information at the current moment based on the fusion of predicted parking space information and currently sensed parking space information when the matching degree information is preset matching degree information; the preset matching degree information is used to characterize that the matching degree between the parking space prediction area and the parking space perception area is greater than or equal to the matching degree threshold.

[0050] As an optional implementation, the matching degree information determination unit includes:

[0051] A similarity cost matrix determination subunit, used to determine a similarity cost matrix between a parking space prediction area and a parking space perception area;

[0052] The matching degree information determination subunit is used to determine the matching degree information of the parking space prediction area and the parking space perception area based on the similarity cost matrix.

[0053] As an optional implementation manner, the current available parking space information determining unit is further configured to determine the current available parking space information at the current moment based on a Kalman filter according to the fusion predicted parking space information and the current perceived parking space information.

[0054] In a third aspect, an embodiment of the present application discloses an electronic device capable of tracking parking spaces, the device comprising a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded by the processor and executing the parking space tracking method as described above.

[0055] In a fourth aspect, an embodiment of the present application discloses a computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the parking space tracking method as described above.

[0056] The technical solution provided by the embodiment of the present application has the following technical effects:

[0057] The parking space tracking method predicts the position of the parking space based on the vehicle wheel speed information to obtain the first predicted parking space information, predicts the position of the parking space based on the historical parking space information to obtain the second predicted parking space information, and then fuses the first predicted parking space information and the second predicted parking space information, and determines the current parking space information based on the fused parking space prediction information and the current sensed parking space information, thereby realizing the tracking of the parking space. The method increases the accuracy and stability of the parking space information prediction by fusing the first predicted parking space information and the second predicted parking space information, can improve the accuracy of the parking space tracking, and effectively prevent the parking space tracking failure at the bend. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 It is a schematic diagram of an application environment of a parking space tracking method provided in an embodiment of the present application;

[0060] Figure 2 It is a flowchart of a parking space tracking method provided in an embodiment of the present application;

[0061] Figure 3 is a schematic diagram of a surround fisheye image provided by an embodiment of the present application;

[0062] Figure 4 is a schematic diagram of a vehicle body coordinate system provided in an embodiment of the present application;

[0063] Figure 5is a flow chart of a method for determining first predicted parking space information provided by an embodiment of the present application;

[0064] Figure 6 is a schematic diagram of a curve obtained by integrating wheel speed information provided in an embodiment of the present application;

[0065] Figure 7 is a schematic diagram of a method for determining current available parking space information at the current moment provided by an embodiment of the present application;

[0066] Figure 8 It is a schematic diagram of the intersection and union ratio area of ​​two quadrilaterals provided in an embodiment of the present application;

[0067] Fig. 9 This is a parking space tracking effect diagram provided by an embodiment of the present application;

[0068] Fig.10 It is a schematic diagram of the structure of a parking space tracking device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0070] It should be noted that the terms "first", "second", etc. in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0071] In order to make the purpose, technical solution and advantages disclosed in the embodiments of the present application more clearly understood, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present application and are not used to limit the embodiments of the present application.

[0072] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0073] When a vehicle is parking automatically, it usually uses on-board sensing sensors, such as fisheye cameras, millimeter-wave radars, and ultrasonic radars, to sense and identify parking spaces. However, due to the characteristics of the sensors themselves, any sensing results are subject to error. If the sensor's sensing results are used directly, when the vehicle is traveling at a high speed or on a curve, the parking space sensing results may change suddenly, which is unacceptable for vehicles that are parking automatically. Therefore, it is necessary to track the parking space based on the sensor's sensing results to ensure that the parking space's position, speed, and other information do not change suddenly.

[0074] In view of this, an embodiment of the present application provides a method for tracking available parking spaces, which predicts the positions of available parking spaces based on the historical perceived parking space information of on-board sensors, vehicle wheel speed information, etc., and estimates the current perceived parking space information based on the prediction results, thereby improving the accuracy of available parking space tracking.

[0075] See also Figure 1 , Figure 1 is a schematic diagram of an application environment of a parking space tracking method provided in an embodiment of the present application, such as Figure 1 As shown, the application environment may include a vehicle.

[0076] In the embodiment of the present application, the vehicle is provided with a data acquisition sensor, and the vehicle may be a vehicle with an automatic driving function or an assisted driving function. Optionally, the vehicle is equipped with an automated valet parking (AVP) system. The user issues a parking command through the vehicle controller or mobile terminal, and the vehicle can automatically drive to the parking space in the parking lot after receiving the command, and the process does not require user operation and monitoring.

[0077] As an optional implementation, the vehicle-mounted perception sensor may include a radar sensor and a camera sensor. The camera sensor may include but is not limited to a monocular vision sensor, a binocular stereo vision sensor, a panoramic vision sensor, an infrared camera sensor, a fisheye camera sensor, etc. The radar sensor may include but is not limited to a laser radar sensor, a millimeter wave radar sensor, an ultrasonic radar sensor, etc. The vehicle-mounted perception sensor is arranged around or inside the vehicle body according to their respective working characteristics. As an example, the vehicle-mounted perception sensor includes a left camera, a right camera, a front camera, and a rear camera, which are arranged on the left side, right side, front, and rear of the vehicle body, respectively. Optionally, the left camera includes one to multiple sub-cameras, the right camera includes one to multiple sub-cameras, and the front camera includes one to multiple sub-cameras. Since the field of view detected by the camera sensor is small, in order to improve the accuracy of detection, multiple cameras are used to locate and identify the target. The vehicle-mounted perception sensor may also include a left radar, a right radar, a left front radar, and a right front radar, which are arranged on the left side, right side, left side of the front of the vehicle body, and right side of the front of the vehicle body, respectively.

[0078] As an optional implementation, during the automatic parking process, the vehicle senses and identifies the parking space through the on-board sensing sensors. The automatic parking system then tracks the available parking spaces sensed by the on-board sensing sensors and selects the available parking spaces for parking path planning. Finally, the automatic driving vehicle parks into the garage, thus completing the automatic parking.

[0079] The following describes a specific embodiment of a parking space tracking method of the present application. Figure 2 It is a flowchart of a parking space tracking method provided in an embodiment of the present application. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in the order of the method shown in the embodiment or the figure or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 2 As shown, the method may include:

[0080] S201: Obtain vehicle wheel speed information, historical perceived parking space information, and historical available parking space information at historical moments, as well as current perceived parking space information at the current moment.

[0081] In an embodiment of the present application, when tracking available parking spaces, vehicle information at historical moments is first obtained, and based on the vehicle information at historical moments, available parking spaces at the current moment are predicted. Then, currently perceived parking space information at the current moment is obtained. Based on the predictions of available parking spaces at historical moments and currently perceived parking space information, available parking space information at the current moment is determined, thereby achieving tracking of available parking spaces from historical moments to the current moment.

[0082] In the embodiment of the present application, the historical moment and the current moment are two consecutive sampling moments of the vehicle-mounted perception sensor. The vehicle wheel speed information is the information collected by the wheel speed sensor in the vehicle at each sampling moment. The perceived parking space information is a frame of parking space perception image acquired by the vehicle-mounted perception sensor at the sampling moment, and the perceived parking space information at the sampling moment is obtained by processing the parking space perception image. The perceived parking space information obtained at the historical moment of the historical perception parking space information. The current perception parking space information is the perception parking space information obtained at the current moment. The historical available parking space information is the available parking space information determined at the historical moment according to the predicted available parking space information and the historical perception parking space information. The available parking space information can be understood as the estimated value obtained after comprehensively predicting the errors of the available parking space information and the perception parking space information. The estimated value can more accurately represent the actual position of the available parking space at that moment.

[0083] As an optional implementation, the parking space perception image is a surround fisheye image collected by a vehicle-mounted surround fisheye camera. The perceived parking space information is the parking space coordinates of the parking space in the vehicle body coordinate system. Figure 3 is a schematic diagram of a surround fisheye image provided by an embodiment of the present application, such as Figure 3 As shown, after the on-board surround fisheye camera collects the surround fisheye image, the preset algorithm model is used to extract and identify the parking spaces in the surround fisheye image, and the perceived parking space coordinates are output. As an example, after the on-board surround fisheye camera collects the surround fisheye image, the parking space perception model is trained through the Center Net open source algorithm, and then the perceived parking space coordinates of the four corner points of each parking space in the surround fisheye image in the sensor coordinate system are output. Then, through coordinate transformation, the perceived parking space coordinates in the sensor coordinate system are converted into the parking space coordinates in the vehicle body coordinate system. Figure 4 is a schematic diagram of a vehicle body coordinate system provided in an embodiment of the present application, such as Figure 4As shown, the x direction is parallel to the vehicle's direction of travel, and the y direction is perpendicular to the vehicle's direction of travel. Let the perceived parking space coordinate in the sensor coordinate system be P. According to the fisheye camera model, the perceived parking space P is first dedistorted to obtain P', and then P' is transformed to the vehicle body coordinate system through projection matrix transformation to obtain the parking space coordinate Pw in the vehicle body coordinate system. The specific formula for transforming the perceived parking space coordinate P in the sensor coordinate system into the parking space coordinate Pw in the vehicle body coordinate system is as follows:

[0084] P w =HP′ formula (1);

[0085] Wherein, H represents the projection matrix, which can be obtained through vehicle body calibration technology.

[0086] S203: Determine first predicted parking space information of the historically sensed parking space information at the current moment according to the vehicle wheel speed information and the historically sensed parking space information.

[0087] In the embodiment of the present application, the historically perceived parking space information is the perceived parking space coordinates of the available parking space perceived by the vehicle-mounted sensing system at the historical moment in the vehicle body coordinate system. Optionally, the perceived parking space coordinates may be the coordinates of the four corner points of the available parking space, or may be the coordinates of a point in the available parking space, for example, the coordinates of the intersection of the diagonal lines of the four corner points of the available parking space. The first predicted parking space information is the predicted parking space coordinates of the available parking space perceived at the historical moment in the vehicle body coordinate system at the current moment.

[0088] In an embodiment of the present application, when tracking the available parking spaces sensed at historical moments, predictions can be made based on the vehicle wheel speed information and the historically sensed parking space information, thereby obtaining the first predicted parking space information of the historically sensed parking space information at the current moment. That is to say, the available parking spaces sensed at historical moments are predicted based on the vehicle wheel speed information, and the positions of the available parking spaces at the current moment are predicted. Optionally, when determining the first predicted parking space information, the prediction can be made at the sampling moment when the vehicle wheel speed information and the sensed parking space information are obtained, or at a sampling moment after the sampling moment when the vehicle wheel speed information and the sensed parking space information are obtained. In other words, when predicting the first predicted parking space information, it can be predicted at a historical moment, or it can be predicted at the current moment.

[0089] In the embodiment of the present application, the vehicle wheel speed information includes vehicle speed information and steering angle velocity information. According to the vehicle speed information and steering angle velocity information, the position change of the available parking space for the vehicle within the time interval between two sampling times can be predicted. Specifically. Figure 5 is a flow chart of a method for determining first predicted parking space information provided by an embodiment of the present application, such as Figure 5 As shown, the method may include:

[0090] S501: Determine the vehicle deflection angle information from the historical moment to the current moment according to the steering angular velocity information.

[0091] In an embodiment of the present application, the time interval from the historical moment to the current moment can be determined based on the historical moment and the current moment, and then the vehicle deflection angle information from the historical moment to the current moment can be determined based on the time interval and the steering angular velocity information.

[0092] S503: Determine the vehicle deflection displacement information from the historical moment to the current moment according to the vehicle speed information and the vehicle deflection angle information.

[0093] In an embodiment of the present application, after determining the vehicle deflection angle information from the historical moment to the current moment, the vehicle deflection displacement information from the historical moment to the current moment can be determined based on the vehicle deflection angle information, vehicle speed information and time interval.

[0094] S505: Determine first predicted parking space information of the historically sensed parking space information at the current moment according to the vehicle deflection displacement information and the historically sensed parking space information.

[0095] In an embodiment of the present application, the deflection displacement of a certain parking space relative to the vehicle from the historical moment to the current moment can be determined based on the vehicle's deflection displacement information, and the first predicted parking space information of the historically perceived parking space information at the current moment can be determined based on the deflection displacement of the parking space and the historically perceived parking space information.

[0096] As an optional implementation, the current time is k, the historical time is k-1, and the time interval from time k-1 to time k is: dt = t k -t k-1 .

[0097] Assume that the wheel speed information at time k-1 is obtained, and the vehicle speed is v t0 , the steering angular velocity is a t0 , the deflection angle of the vehicle from time k-1 to time k is Δθ, the lateral displacement of the vehicle from time k-1 to time k is Δx, and the longitudinal displacement of the vehicle from time k-1 to time k is Δy. Then

[0098] Δθ=a t0 *dt

[0099] Δx=v t0 *dt*cos(Δθ)

[0100] Δy=v t0 *dt*sin(Δθ) formula (2);

[0101] After calculating the lateral and longitudinal offsets of the vehicle from time k-1 to time k, the predicted position of the parking space perceived at time k-1 in the vehicle coordinate system at time k can be obtained according to the two-dimensional plane transformation formula.

[0102] Specifically, let the coordinates of a point in a parking space at time k-1 in the vehicle coordinate system be P k-1 (x,y), then let P k-1 At time k, the coordinate in the vehicle coordinate system is P k Then P k The coordinates can be calculated by the following formula:

[0103]

[0104] The above formula can be used to calculate P k , then P k That is, the position of the available parking spaces perceived at the historical moment is predicted at the current moment based on the wheel speed information.

[0105] Generally speaking, the wheel speed information usually has a small error when driving in a straight line, but a large error when turning a curve. Figure 6 is a schematic diagram of a curve obtained by integrating wheel speed information provided in an embodiment of the present application, such as Figure 6 As shown in the figure, by integrating the wheel speed information for a period of time, the curve shown in the figure is obtained. The curve in the figure shows the trajectory of the vehicle, where the horizontal and vertical coordinates in the figure represent the displacement of the vehicle in two directions. It can be seen that when the vehicle is traveling in a straight line, the displacement is equal to the distance traveled, and the error of predicting the available parking space based on the wheel speed information is small. However, the displacement of the vehicle at a curve cannot accurately represent the distance traveled by the vehicle, so at a curve, the error of predicting the available parking space based on the wheel speed information is large.

[0106] S205: Determine the second predicted parking space information at the current moment according to the historical available parking space information.

[0107] In an embodiment of the present application, the available parking spaces at the current moment can be predicted based on the historical available parking space information, thereby avoiding the prediction error caused by a single prediction method. Optionally, the second predicted parking space information is predicted by a Kalman filter. That is, the historical available parking space information is predicted based on the Kalman filter to obtain the second predicted parking space information at the current moment. Specifically, the historical available parking space information is an estimated value of the available parking space coordinates at the historical moment. The estimated value can accurately reflect the real position of the available parking space at the historical moment. Optionally, the parking space coordinates can be the coordinates of the four corner points of the available parking space, or the coordinates of a point in the available parking space, such as the coordinates of the intersection of the diagonals of the four corner points of the available parking space. The second predicted parking space information is the estimated value of the available parking space coordinates predicted at the historical moment, and the parking space coordinates in the vehicle body coordinate system at the current moment are predicted by the prediction algorithm in the Kalman filter.

[0108] In some embodiments, if the historical moment is the initial moment, the predicted value of the available parking space at the historical moment can be obtained based on the first-order Markov chain, and then the predicted value of the available parking space at the historical moment can be used as the historical available parking space information, and the Kalman filter is used to predict the historical available parking space to obtain the predicted position of the available parking space at the current moment.

[0109] As an optional implementation, assuming that there are n target available parking spaces at time k-1, n is greater than or equal to 0, and the positions of these n target available parking spaces at time k are predicted by a Kalman filter. The Kalman filter prediction formula is as follows:

[0110] x′(k)=A*x(k-1)+B*u(k)

[0111] P′(k)=A*P(k-1)*A T +*Q formula (4);

[0112] in, is the predicted state vector at time k, i.e., the coordinates of the available parking spaces at time k predicted by the Kalman filter;

[0113] A is the state transfer matrix;

[0114] is the state vector at time k-1, i.e., the coordinates of the target parking space at time k-1;

[0115] B is the control matrix;

[0116] is the control vector at time k;

[0117] P′(k) is the covariance matrix of the state vector at time k;

[0118] P(k-1) is the covariance matrix of the state vector at time k-1;

[0119] A T is the transposed matrix of the state transfer matrix A;

[0120] Q is the process noise matrix.

[0121] In this implementation, when performing prediction calculation, let the state quantity x = [x, y, vx, vy], where (x, y) is the coordinates of the intersection of the parking space in the vehicle body coordinate system, and (vx, vy) is the rate of change of the intersection of the parking space in the x direction and the y direction. The current moment is the k moment, and the previous moment is the k-1 moment. Assuming that the k-1 moment is the initial moment, then B = 0, That is, there is no disturbance. dt is the interval from time k-1 to time k. The state transfer matrix A can be expressed as:

[0122]

[0123] P changes with time, and the state covariance matrix P at the initial moment is:

[0124]

[0125] The process noise matrix Q can be set to the identity matrix, that is, Q can be expressed as:

[0126]

[0127] In this implementation, the intersection coordinates of the target parking space updated at time k-1 are known, and the intersection coordinates of the target parking space at time k can be predicted using the Kalman filter.

[0128] S207: Fusing the first predicted parking space information and the second predicted parking space information to obtain fused predicted parking space information.

[0129] In an embodiment of the present application, the first predicted parking space information and the second predicted parking space information are fused to obtain fused predicted parking space information. The fused predicted parking space information is the fused predicted coordinates of the parking space obtained by fusing the predicted coordinates of the parking space obtained based on the wheel speed information prediction and the predicted coordinates of the parking space obtained based on the Kalman filter prediction. Specifically, the first predicted parking space information includes the first coordinate information of the target parking space. The second predicted parking space information includes the second coordinate information of the target parking space. There may be multiple predicted parking spaces, and the target parking space is any one of the predicted parking spaces. The fused predicted parking space information and the second predicted parking space information are fused to obtain the fused predicted parking space information: according to the first coordinate information and the second coordinate information, the deviation correction information is determined. Then, based on the first coordinate information and the deviation correction information, the fused predicted parking space information of the target parking space is obtained. Since the fused predicted parking space information fuses the coordinates of the parking space obtained by the two prediction methods, the prediction error can be further reduced and the process noise can be reduced.

[0130] As an optional implementation, let the coordinates of the four corner points of the target parking space at time k predicted based on the wheel speed information be P 1 , P 2 , P 3 and P 4 Based on the coordinates of the four corner points, the coordinates of the diagonal intersection point of the target parking space can be calculated as P. The coordinates of the diagonal intersection point of the target parking space at time k predicted by the Kalman filter are P', and P' is the state quantity predicted by the Kalman filter. Calculate the relative displacement of P and P', that is, determine the deviation correction information. The relative displacement calculation formula of P and P' is as follows:

[0131] Δp x =p x -p x '

[0132] Δp y =p y -p y ′ Formula (5).

[0133] According to the calculated relative displacement, the predicted parking space coordinates obtained based on the wheel speed information are corrected to obtain the fused predicted parking space coordinates. Specifically, the four corner point coordinates obtained based on the wheel speed information are translated according to the relative displacement calculated above to obtain the fused predicted parking space coordinates. The specific calculation formula for translating the four corner point coordinates obtained based on the wheel speed information is as follows:

[0134] p' ix =p x +Δp x

[0135] p' iy =p y +Δp y Formula (6);

[0136] Wherein, i is a natural number.

[0137] S209: Determine the current available parking space information at the current moment according to the fusion predicted parking space information and the current perceived parking space information.

[0138] In an embodiment of the present application, after obtaining the fused predicted parking space information, the current available parking space information at the current moment can be determined based on the fused predicted parking space information and the currently perceived parking space information. Specifically, the current available parking space information at the current moment is determined based on the Kalman filter according to the fused predicted parking space information and the currently perceived parking space information. The fused predicted parking space information is the fused predicted coordinates of the available parking space. The currently perceived parking space information is the currently perceived parking space coordinates in the vehicle body coordinate system of the available parking space perceived by the on-board perception system at the current moment. The available parking space coordinates are the estimated values ​​of the available parking space coordinates at the current moment, and the estimated value can accurately reflect the actual position of the available parking space at the current moment. The available parking space coordinates at the current moment can be calculated based on the fused predicted coordinates of the available parking space and the currently perceived parking space coordinates.

[0139] In the embodiment of the present application, since there may be multiple available parking spaces represented by the fused predicted parking space information, there may also be multiple available parking spaces represented by the currently sensed parking space information. Therefore, when calculating the available parking space coordinates at the current moment, it is first necessary to match the available parking spaces represented by the fused predicted parking space information with the available parking spaces represented by the currently sensed parking space information one by one. Figure 7 is a schematic diagram of a method for determining the current available parking space information at the current moment provided by an embodiment of the present application, such as Figure 7 As shown, according to the fusion prediction parking space information and the current perception parking space information, determining the current available parking space information at the current moment may include:

[0140] S701: Determine the parking space prediction area at the current moment according to the fused predicted parking space information.

[0141] In the embodiment of the present application, the parking space prediction area may be a parking space prediction frame. The fused prediction parking space information is the parking space fused prediction coordinates, and the parking space contour frame of each predicted parking space may be determined according to the parking space fused prediction coordinates, that is, the parking space prediction frame.

[0142] S703: Determine the parking space sensing area at the current moment according to the currently sensed parking space information.

[0143] In the embodiment of the present application, the parking space sensing area may be a parking space sensing frame. The current sensing parking space information is the current parking space sensing coordinates, and the parking space contour frame line of each sensing parking space may be determined according to the current parking space sensing coordinates, that is, the parking space sensing frame.

[0144] S705: Determine the matching degree information between the parking space prediction area and the parking space perception area.

[0145] In the embodiment of the present application, similarity calculation is performed based on the parking space prediction area and the parking space perception area to determine the matching information of the parking space prediction area and the parking space perception area. Specifically, the similarity cost matrix of the parking space prediction area and the parking space perception area is first determined, and then the matching information of the parking space prediction area and the parking space perception area is determined based on the similarity cost matrix.

[0146] As an optional implementation, assume that there are n available parking spaces represented by the fused predicted parking space information, and there may be m available parking spaces represented by the current perceived parking space information. That is to say, at the current moment, there are n predicted available parking spaces, and the on-board perception sensor perceives m perceived available parking spaces. That is, at the current moment, there are n available parking space prediction frames and m available parking space perception frames. The similarity cost matrix cost(m,n) of the available parking space prediction frames and the available parking space perception frames is calculated. Optionally, the similarity cost matrix is ​​calculated using the Intersection-over-Union (IOU) formula, and the IOU calculation formula is specifically: Cost(m,n)=IOU m,n Specifically, the available parking spaces can be represented by four corner points to form a quadrilateral. The n predicted available parking spaces and the m perceived parking spaces constitute a matrix of m rows and n columns. Each element in the matrix is ​​the IOU value, and then the intersection calculation and union calculation are performed. Figure 8 is a schematic diagram of the intersection and union ratio area of ​​two quadrilaterals provided in an embodiment of the present application, such as Figure 8 As shown, when calculating the union of quadrilaterals, the areas of the two quadrilaterals are added together to find the number of pixels in each quadrilateral. When calculating the intersection of quadrilaterals, the area of ​​the intersection of the two quadrilaterals is found. The specific calculation formula is as follows:

[0147]

[0148] Among them, Area i Represents the quadrilateral area i, i.e., the parking space perception box. Area j Represents the quadrilateral area j, i.e., the parking space prediction box. Area represents the quadrilateral area Area i and Area j The intersection area is the overlapping area between the parking space perception box and the parking space prediction box.

[0149] In the embodiment of the present application, after calculating the similarity cost matrix of the n parking space prediction frames and the m parking space perception frames, the n parking space prediction frames and the m parking space perception frames can be matched according to the similarity cost matrix through a matching algorithm, so as to match the predicted parking spaces with the perceived parking spaces one by one and obtain matching degree information. In other words, the purpose of the calculation process is to obtain a perceived parking space A1 corresponding to a certain parking space A0, which is sensed by the vehicle-mounted perception sensor at a historical moment, and the predicted parking space A2 is obtained by predicting the position of A0 at the current moment based on A1, and the parking space A is re-perceived at the current moment to obtain the perceived parking space A3. Since there may be multiple perceived parking spaces perceived at the current moment, it is necessary to determine the perceived parking space A3 from the multiple perceived parking spaces, so that the predicted parking space A2 and the perceived parking space A3 are matched and corresponded. Optionally, the matching algorithm may be a KM (KuhnMunkras) matching algorithm, which uses the KM matching algorithm to match the n available parking space prediction frames and the m available parking space perception frames according to a similarity cost matrix cost(m, n).

[0150] S707: When the matching degree information is the preset matching degree information, the current available parking space information at the current moment is determined according to the fusion predicted parking space information and the current perceived parking space information.

[0151] In the embodiment of the present application, the preset matching degree information is used to indicate that the matching degree between the parking space prediction area and the parking space perception area is greater than or equal to a matching degree threshold. Fig. 9 This is a parking space tracking effect diagram provided by an embodiment of the present application, such as Fig. 9 As shown in FIG. 1 , when the predicted available parking space coincides with the perceived available parking space, it indicates that the predicted available parking space matches the perceived available parking space. When the parking space prediction area matches the parking space perception area successfully, the position of the available parking space can be updated based on the Kalman filter according to the fusion of the predicted parking space information and the current perceived parking space information.

[0152] As a possible implementation method, for the predicted parking space and the perceived parking space that are successfully matched, the estimated value of the parking space coordinates is calculated based on the Kalman filter according to the coordinates of the predicted parking space and the coordinates of the perceived parking space, so as to update the position of the parking space coordinates. The Kalman filter update formula is as follows:

[0153] K(k)=P′(k)*H T *(H*P′(k)*H T +R) -1

[0154] x(k)=x′(k)+K(k)*(z(k)-H*x′(k))

[0155] P(k)=(IK(k)*H)*P′(k) Formula (8).

[0156] Where K is the Kalman gain;

[0157] P' is the predicted state covariance matrix;

[0158] z(k) is the coordinate of the perceived parking space at the current time k, that is, Z(k) is the observation value of the vehicle-mounted perception sensor at the current time, which can be equal to the coordinate of the center point of the parking space perception box;

[0159] x(k) corresponds to the updated state quantity, which is the estimated value of the coordinates of the available parking spaces at the current moment.

[0160] H is the observation matrix, which can be expressed as:

[0161]

[0162] R is the observation noise matrix, which can be expressed as:

[0163] noise=|Δθ|*100+0.02

[0164]

[0165] Among them, θ is the rotation angle between the parking space perception image obtained at the historical moment and the parking space perception image obtained at the current moment. When the rotation angle is large, the observation noise is larger, and the weight of the observation value is smaller, thereby reducing the influence of wheel speed information noise.

[0166] In the embodiment of the present application, for the predicted parking spaces that are successfully matched, the position is updated according to the above process. For the predicted parking spaces that are not successfully matched, the Kalman filter is not updated, only the position is predicted, and then the next match is performed. The predicted parking spaces that are not matched within the preset time period are deleted. Optionally, the preset time period can be a soft number of sampling time intervals, such as two sampling time intervals. For the perceived parking spaces that are not successfully matched, they can be used as target parking spaces for position prediction and tracking.

[0167] The parking space tracking method described in the embodiment of the present application predicts the position of the parking space based on the vehicle wheel speed information to obtain the first predicted parking space information, predicts the position of the parking space based on the historical parking space information to obtain the second predicted parking space information, and then fuses the first predicted parking space information and the second predicted parking space information, and determines the current parking space information based on the fused parking space prediction information and the current sensed parking space information, thereby realizing the tracking of the parking space. By fusing the first predicted parking space information and the second predicted parking space information, the method increases the accuracy and stability of the parking space information prediction, can improve the accuracy of the parking space tracking, and effectively prevents the failure of the parking space tracking at the bend.

[0168] The present application also provides a parking space tracking device. Fig.10 is a schematic diagram of the structure of a parking space tracking device provided in an embodiment of the present application, such as Fig.10 As shown, the device comprises:

[0169] The information acquisition module 1001 is used to acquire the vehicle wheel speed information, historical sensed parking space information and historical available parking space information at historical moments, as well as the current sensed parking space information at the current moment.

[0170] The first predicted parking space information determining module 1003 is used to determine the first predicted parking space information of the historically sensed parking space information at the current moment according to the vehicle wheel speed information and the historically sensed parking space information.

[0171] The second predicted parking space information determination module 1005 is used to determine the second predicted parking space information at the current moment according to the historical available parking space information.

[0172] The fused predicted parking space information determination module 1007 is used to fuse the first predicted parking space information and the second predicted parking space information to obtain fused predicted parking space information.

[0173] The current available parking space information determination module 1009 is used to determine the current available parking space information at the current moment according to the fused predicted parking space information and the currently sensed parking space information.

[0174] As an optional implementation, the vehicle wheel speed information includes vehicle speed information and steering angle velocity information. The first predicted parking space information determination module includes:

[0175] The vehicle deflection angle information determination unit is used to determine the vehicle deflection angle information from the historical moment to the current moment according to the steering angular velocity information.

[0176] The vehicle deflection displacement information determination unit is used to determine the vehicle deflection displacement information from a historical moment to a current moment according to the vehicle speed information and the vehicle deflection angle information.

[0177] The first predicted parking space information determining unit is used to determine the first predicted parking space information of the historically sensed parking space information at the current moment according to the vehicle deflection displacement information and the historically sensed parking space information.

[0178] As an optional implementation, the second predicted parking space information determination module includes:

[0179] The second predicted parking space information determining unit is used to predict the historical available parking space information based on the Kalman filter to obtain the second predicted parking space information at the current moment.

[0180] As an optional implementation, the first predicted parking space information includes first coordinate information of the target available parking space. The second predicted parking space information includes second coordinate information of the target available parking space. The fusion predicted parking space information determination module includes:

[0181] The deviation correction information determining unit is used to determine the deviation correction information according to the first coordinate information and the second coordinate information.

[0182] The fused predicted parking space information determining unit is used to obtain the fused predicted parking space information of the target available parking space based on the first coordinate information and the deviation correction information.

[0183] As an optional implementation, the current available parking space information determination module includes:

[0184] The parking space prediction area determination unit is used to determine the parking space prediction area at the current moment according to the fused prediction parking space information.

[0185] The parking space sensing area determining unit is used to determine the parking space sensing area at the current moment according to the currently sensed parking space information.

[0186] The matching degree information determining unit is used to determine the matching degree information between the parking space prediction area and the parking space perception area.

[0187] The current available parking space information determination unit is used to determine the current available parking space information at the current moment according to the fusion prediction parking space information and the current sensed parking space information when the matching degree information is the preset matching degree information. The preset matching degree information is used to indicate that the matching degree between the parking space prediction area and the parking space sense area is greater than or equal to the matching degree threshold.

[0188] As an optional implementation, the matching degree information determination unit includes:

[0189] The similarity cost matrix determination subunit is used to determine the similarity cost matrix of the parking space prediction area and the parking space perception area.

[0190] The matching degree information determination subunit is used to determine the matching degree information of the parking space prediction area and the parking space perception area based on the similarity cost matrix.

[0191] As an optional implementation manner, the current available parking space information determining unit is further configured to determine the current available parking space information at the current moment based on a Kalman filter according to the fusion predicted parking space information and the current perceived parking space information.

[0192] The device in the embodiment of the present application and the parking space tracking method embodiment are based on the same application concept. Regarding the implementation of the device, please refer to the specific implementation of the method, which will not be repeated here.

[0193] An embodiment of the present application discloses an electronic device capable of tracking parking spaces. The device includes a processor and a memory. The memory stores at least one instruction or at least one program. The at least one instruction or at least one program is loaded by the processor and executes the parking space tracking method described above.

[0194] In an embodiment of the present application, the memory can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, application programs required for functions, etc.; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory. As an example, the device is an electronic control unit (ECU).

[0195] An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the parking space tracking method as described above.

[0196] In an embodiment of the present application, the above-mentioned computer storage medium may be located in at least one of a plurality of network servers of a computer network. Optionally, the computer readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid state drive (SSD) or an optical disk, etc. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM).

[0197] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of this specification are described. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0198] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0199] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0200] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A parking space tracking method, It is characterized in that The method comprises: Obtaining vehicle wheel speed information, historical perceived parking space information, and historical available parking space information at historical moments, as well as current perceived parking space information at the current moment; the vehicle wheel speed information at historical moments is information collected by a wheel speed sensor in the vehicle at historical moments; the historical available parking space information is available parking space information determined at historical moments based on predicted available parking space information and historical perceived parking space information; the historical perceived parking space information is the perceived parking space coordinates of the available parking space perceived by the vehicle-mounted perception system at historical moments in the vehicle body coordinate system; Determine, according to the vehicle wheel speed information and the historically sensed parking space information, first predicted parking space information of the historically sensed parking space information at the current moment; Determining the second predicted parking space information at the current moment according to the historical available parking space information; fusing the first predicted parking space information and the second predicted parking space information to obtain fused predicted parking space information; The current available parking space information at the current moment is determined according to the fused predicted parking space information and the currently sensed parking space information.

2. The method according to claim 1, It is characterized in that The vehicle wheel speed information includes vehicle speed information and steering angle velocity information; and determining the first predicted parking space information of the historically sensed parking space information at the current moment according to the vehicle wheel speed information and the historically sensed parking space information includes: Determining the vehicle deflection angle information from the historical moment to the current moment according to the steering angular velocity information; Determining the vehicle deflection displacement information from the historical moment to the current moment according to the vehicle speed information and the vehicle deflection angle information; The first predicted parking space information of the historically sensed parking space information at the current moment is determined according to the vehicle deflection displacement information and the historically sensed parking space information.

3. The method according to claim 2, It is characterized in that The determining, based on the historical available parking space information, the second predicted parking space information at the current moment includes: The historical available parking space information is predicted based on a Kalman filter to obtain the second predicted parking space information at the current moment.

4. The method according to claim 3, It is characterized in that The first predicted parking space information includes first coordinate information of a target available parking space; the second predicted parking space information includes second coordinate information of the target available parking space; The fusing the first predicted parking space information and the second predicted parking space information to obtain the fused predicted parking space information includes: Determining deviation correction information according to the first coordinate information and the second coordinate information; Based on the first coordinate information and the deviation correction information, the fused predicted parking space information of the target parking space is obtained.

5. The method according to claim 1, It is characterized in that The determining, according to the fused predicted parking space information and the currently sensed parking space information, the current available parking space information at the current moment includes: Determining the parking space prediction area at the current moment according to the fused predicted parking space information; Determining a parking space sensing area at the current moment according to the currently sensed parking space information; Determining matching information between the parking space prediction area and the parking space sensing area; In the case where the matching degree information is preset matching degree information, the current available parking space information at the current moment is determined according to the fused predicted parking space information and the currently perceived parking space information; the preset matching degree information is used to characterize that the matching degree between the parking space prediction area and the parking space perception area is greater than or equal to a matching degree threshold.

6. The method according to claim 5, It is characterized in that The determining of the matching degree information between the parking space prediction area and the parking space sensing area includes: Determining a similarity cost matrix between the parking space prediction area and the parking space perception area; The matching degree information between the parking space prediction area and the parking space perception area is determined based on the similarity cost matrix.

7. The method according to claim 1, It is characterized in that The determining, according to the fused predicted parking space information and the currently sensed parking space information, the current available parking space information at the current moment includes: According to the fused predicted parking space information and the currently sensed parking space information, the currently available parking space information at the current moment is determined based on a Kalman filter.

8. A parking space tracking device, It is characterized in that The device comprises: An information acquisition module, used to acquire vehicle wheel speed information, historical perceived parking space information and historical available parking space information at historical moments, as well as current perceived parking space information at the current moment; the vehicle wheel speed information at historical moments is information collected by a wheel speed sensor in the vehicle at historical moments; the historical available parking space information is available parking space information determined at historical moments based on predicted available parking space information and historical perceived parking space information; the historical perceived parking space information is the perceived parking space coordinates of the available parking space perceived by the vehicle-mounted perception system at historical moments in the vehicle body coordinate system; A first predicted parking space information determination module, configured to determine first predicted parking space information of the historically sensed parking space information at the current moment according to the vehicle wheel speed information and the historically sensed parking space information; A second predicted parking space information determination module, configured to determine the second predicted parking space information at the current moment according to the historical available parking space information; A fused predicted parking space information determination module, configured to fuse the first predicted parking space information and the second predicted parking space information to obtain fused predicted parking space information; The current available parking space information determination module is used to determine the current available parking space information at the current moment according to the fused predicted parking space information and the currently sensed parking space information.

9. An electronic device capable of tracking parking spaces. It is characterized in that The device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded by the processor and executes the parking space tracking method according to any one of claims 1 to 7.

10. A computer-readable storage medium, It is characterized in that At least one instruction or at least one program segment is stored in the storage medium, and the at least one instruction or at least one program segment is loaded and executed by a processor to implement the parkable space tracking method according to any one of claims 1-7.

Images