Parking method and device, vehicle and storage medium

Abstract

The invention provides a parking method and device, a vehicle and a storage medium, and the method comprises the steps: determining an aerial view BEV image at a t moment, and the t moment is any moment of the vehicle in a parking process; if a parking space angular point is detected in the BEV image, the detected parking space angular point is matched with a parking space angular point in a map, the map is created after the vehicle starts a parking function, and the parking space angular point in the map is a parking space angular point created or updated in the map after the parking space angular point is detected in the BEV image; the total matching number is determined, the fusion pose at the moment t is determined based on the total matching number, the fusion pose is used for parking positioning of the vehicle, and the total matching number is the total number of successful matching. The method provided by the invention is helpful for improving the accuracy of vehicle parking positioning.

Description

Technical Field

[0001] This application relates to the field of automobiles, and particularly to a parking method, device, vehicle, and storage medium. Background Art

[0002] In related technologies, a wheel odometer or a wheel odometer integrated with an IMU is usually used as a vehicle parking positioning solution. The solution of calculating the vehicle pose based on motion sensor data such as a wheel speedometer or an IMU inevitably has a positioning cumulative error, which may cause problems such as multiple parking maneuvers or the vehicle parking skew.

[0003] Although some current parking positioning methods based on graph optimization have relatively accurate results, they consume a large amount of computing power and have a high output delay, and it is difficult to deploy this solution on in-vehicle chips. Therefore, there is an urgent need for a parking solution to achieve accurate and reliable parking positioning on in-vehicle chips with limited computing power. Summary of the Invention

[0004] Embodiments of this application provide a parking method, device, vehicle, and storage medium, which helps to improve the accuracy of vehicle parking positioning.

[0005] In a first aspect, embodiments of this application provide a parking method, including: determining a bird's-eye view (BEV) image at time t, where t is any moment during the vehicle's parking process; if a parking space corner point is detected in the BEV image, matching the detected parking space corner point with the parking space corner point in a map, where the map is created after the vehicle starts the parking function, and the parking space corner point in the map is the parking space corner point created or updated in the map after the parking space corner point is detected in the BEV image; determining the total number of matches, and determining the fusion pose at time t based on the total number of matches, where the fusion pose is used for the vehicle to perform parking positioning, and the total number of matches is the total number of successful matches.

[0006] In one possible implementation, determining the fusion pose at time t based on the total number of matches includes: if the total number of matches is greater than or equal to a preset total number, determining the fusion pose at time t based on the first vehicle body pose at time t and the second vehicle body pose at time t; where the first vehicle body pose at time t is determined by the vehicle body poses at the first sampling moment and the second sampling moment; the first sampling moment and the second sampling moment are the sampling moments of the four-wheel pulse data of the vehicle, the first sampling moment is the previous sampling moment adjacent to time t, and the second sampling moment is the next sampling moment adjacent to time t; the second vehicle body pose at time t is obtained by calculating using the iterative closest point (ICP) algorithm.

[0007] In one possible implementation, the method further includes: if the total number of matches is less than the preset total number, determining the first vehicle body pose at time t as the fused pose at time t.

[0008] In one possible implementation, the determination of the first vehicle body pose at time t from the vehicle body poses at the first sampling time and the second sampling time includes: the first vehicle body pose at time t is determined by interpolating the vehicle body poses between the first sampling time and the second sampling time.

[0009] In one possible implementation, the vehicle body pose at the first sampling time is determined from the fused pose at time t2 and the movement of the vehicle between time t2 and the first sampling time, and the vehicle body pose at the second sampling time is determined from the fused pose at time t2 and the movement of the vehicle between time t2 and the second sampling time, where time t2 is the previous moment of time t.

[0010] In one possible implementation, the matching of the detected parking space corner points with the parking space corner points in the map includes: comparing the coordinates and orientations of the detected parking space corner points with the coordinates and orientations of the parking space corner points in the map respectively; if the coordinate error between the detected parking space corner points and the parking space corner points in the map is within a first preset error, and if the orientation error between the detected parking space corner points and the parking space corner points in the map is within a second preset error, determining that the matching is successful.

[0011] In one possible implementation, the method further includes: if the ratio of the actual observation times to the theoretical observation times of the target parking space corner points in the map is less than or equal to a preset threshold, deleting the target parking space corner points; where the target parking space corner points are any parking space corner points within the BEV image field of view of the vehicle, the actual observation times are used to represent the cumulative number of times the target parking space corner points are actually detected within the BEV image field of view of the vehicle, and the theoretical observation times are used to represent the cumulative number of times the target parking space corner points are theoretically detected within the BEV image field of view of the vehicle.

[0012] In one possible implementation, the parking space corner points in the map are the parking space corner points updated in the map after detecting the parking space corner points in the BEV image, including: if the detected parking space corner points are successfully matched with the parking space corner points in the map, fusing the detected parking space corner points with the parking space corner points in the map to update the parking space corner points in the map.

[0013] In one possible implementation, the method further includes: if no parking space corner points are detected in the BEV image, determining the fusion pose at time t1, where time t1 is the next moment after time t.

[0014] In one possible implementation, determining the fusion pose at time t based on the first vehicle body pose at time t and the second vehicle body pose at time t includes: if the error between the first vehicle body pose at time t and the second vehicle body pose at time t is greater than or equal to a third preset error, determining the first vehicle body pose at time t as the fusion pose at time t. Alternatively, if the error between the first vehicle body pose at time t and the second vehicle body pose at time t is less than the third preset error, fusing the first vehicle body pose at time t and the second vehicle body pose at time t to obtain the fusion pose at time t.

[0015] In a second aspect, an embodiment of the present application provides a parking device, including one or more functional modules, and the one or more functional modules are used to execute the parking method as described in the first aspect.

[0016] In a third aspect, an embodiment of the present application provides a vehicle, including: a processor and a memory, where the memory is used to store a computer program; the processor is used to run the computer program to implement the parking method as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a program is stored, and when the program runs on a vehicle, the vehicle is enabled to implement the parking method as described in the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a program, and when the program runs on a processor of a vehicle, the vehicle is enabled to execute the parking method as described in the first aspect.

[0019] In a possible design, the program in the fifth aspect can be stored in whole or in part on a storage medium packaged together with the processor, or can be stored in whole or in part on a memory not packaged together with the processor. Description of the Drawings

[0020] Figure 1 It is a schematic flowchart of an embodiment of the parking method provided by the present application; Figure 2 It is a schematic diagram of the interpolation method provided by an embodiment of the present application; Figure 3 It is a schematic structural diagram of the parking device provided by an embodiment of the present application; Figure 4 It is a schematic structural diagram of the vehicle provided by an embodiment of the present application. Detailed implementation manners

[0021] In the embodiments of the present application, unless otherwise specified, the character " / " indicates that the related objects before and after are in an "or" relationship. For example, A / B may represent A or B. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone, these three situations.

[0022] It should be noted that the terms "first", "second", etc. involved in the embodiments of the present application are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features, nor can they be understood as indicating or implying an order.

[0023] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. In addition, "at least one (item)" or its similar expression means any combination of these items, which can include any combination of single item (item) or plural items (items). For example, at least one (item) of A, B, or C may represent: A, B, C, A and B, A and C, B and C, or A, B, and C. Each of A, B, and C itself can be an element or a set containing one or more elements.

[0024] In the embodiments of the present application, "exemplary", "in some embodiments", "in another embodiment", etc. are used to give examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" in the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of the word "exemplary" is intended to present concepts in a specific way.

[0025] The "of", "corresponding", and "corresponding" in the embodiments of the present application can sometimes be used interchangeably. It should be noted that when their differences are not emphasized, the meanings to be expressed are the same. In the embodiments of the present application, communication and transmission can sometimes be used interchangeably. It should be noted that when their differences are not emphasized, the meanings they express are the same. For example, transmission can include sending and / or receiving, and can be a noun or a verb.

[0026] In the embodiments of the present application, the equality involved can be used in combination with greater than, applicable to the technical solutions adopted when it is greater than, or can also be used in combination with less than, applicable to the technical solutions adopted when it is less than. It should be noted that when equality is used in combination with greater than, it cannot be used in combination with less than; when equality is used in combination with less than, it is not used in combination with greater than.

[0027] In the related art, a wheel odometer or a wheel odometer integrated with an IMU is usually used as a vehicle parking positioning solution. The solution of calculating the vehicle pose based on motion sensor data such as a wheel speed sensor or an IMU inevitably has a cumulative positioning error, which may lead to problems such as multiple parking maneuvers or the vehicle being parked obliquely.

[0028] Although some current parking positioning methods based on graph optimization have relatively accurate results, they consume a large amount of computing power and have a high output delay, making it difficult to deploy this solution on in-vehicle chips. Therefore, there is an urgent need for a parking solution to achieve accurate and reliable parking positioning on in-vehicle chips with limited computing power.

[0029] Based on the above problems, the present application provides a parking method, which helps to improve the accuracy of vehicle parking positioning.

[0030] Figure 1 The flowchart of an embodiment of the parking method provided by the present application includes the following steps: Step 101, during the parking process of the vehicle, the vehicle obtains a BEV image at time t.

[0031] Specifically, the user can start the parking function on the vehicle, and this parking function can achieve automatic parking of the vehicle.

[0032] It can be understood that when the user activates the above parking function, the brake pedal of the vehicle has been depressed by the user and the vehicle is in a stationary state.

[0033] When the parking function of the vehicle is activated by the user, the vehicle can enter the automatic parking mode, and in this automatic parking mode, the vehicle can perform automatic parking.

[0034] During the parking process of the vehicle, the vehicle can continuously capture multiple images of the surrounding scene of the vehicle body through the surround cameras arranged on its vehicle body to obtain a BEV image at time t.

[0035] Among them, time t can be any moment during the parking process of the vehicle.

[0036] Exemplarily, if the vehicle is equipped with 4 surround cameras on its vehicle body, the vehicle can capture 4 images of the surrounding scene of the vehicle body at time t.

[0037] Then, the vehicle can synthesize multiple images of the surrounding scene of the vehicle body obtained by shooting into 1 bird's-eye view (BEV).

[0038] Exemplarily, taking the 4 surround cameras on the vehicle body as an example, when the vehicle captures 4 images of the surrounding scene of the vehicle body through these 4 surround cameras, it can synthesize 1 BEV image based on these 4 images.

[0039] Step 102, the vehicle determines its body pose at time t.

[0040] Specifically, during the parking process of the vehicle, the vehicle can determine its body pose at time t.

[0041] Among them, the body pose can be obtained by the wheel odometry calculation method, and the body pose can include information such as the abscissa, ordinate, and heading angle.

[0042] The way for the vehicle to determine its body pose at time t can include: obtaining the four-wheel pulse data of the vehicle, and calculating the body pose of the vehicle at time t based on the four-wheel pulse data of the vehicle.

[0043] In some alternative embodiments, the sampling frequency of the four-wheel pulse data of the vehicle is higher than the shooting frequency of the BEV image, and there may not necessarily be corresponding four-wheel pulse data at the shooting moment (i.e., time t). In this case, the body pose at time t can be estimated by the linear interpolation method.

[0044] Exemplarily, the four-wheel pulse data of the previous sampling moment adjacent to time t and the four-wheel pulse data of the next sampling moment can be found. Based on the four-wheel pulse data of the previous sampling moment adjacent to time t, the body pose of the previous sampling moment adjacent to time t is calculated. Based on the four-wheel pulse data of the next sampling moment adjacent to time t, the body pose of the next sampling moment adjacent to time t is calculated. Then, interpolation is performed between the body pose of the previous sampling moment adjacent to time t and the body pose of the next sampling moment adjacent to time t to obtain the body pose at time t.

[0045] Among them, the way to calculate the body pose of the previous sampling moment adjacent to time t based on the four-wheel pulse data of the previous sampling moment adjacent to time t can include: determining the movement of the vehicle between time t2 and the previous sampling moment adjacent to time t based on the four-wheel pulse data between time t2 and the previous sampling moment adjacent to time t. Based on the fused pose of the vehicle at time t2 and the movement of the vehicle between time t2 and the previous sampling moment adjacent to time t, the body pose of the previous sampling moment adjacent to time t is determined. Here, time t2 is the previous moment of time t, and the way to obtain the fused pose of the vehicle at time t2 can specifically refer to the way to obtain the fused pose of the vehicle at time t in the following steps, which will not be elaborated here.

[0046] The method for calculating the vehicle body pose at the next sampling moment adjacent to the t moment based on the four-wheel pulse data at the next sampling moment adjacent to the t moment may include: determining the movement of the vehicle between the t2 moment and the next sampling moment adjacent to the t moment based on the four-wheel pulse data between the t2 moment and the next sampling moment adjacent to the t moment, and determining the vehicle body pose at the next sampling moment adjacent to the t moment based on the fused pose of the vehicle at the t2 moment and the movement of the vehicle between the t2 moment and the next sampling moment adjacent to the t moment, where the t2 moment is the previous moment of the t moment, and the method for obtaining the fused pose of the vehicle at the t2 moment may specifically refer to the method for obtaining the fused pose of the vehicle at the t moment in the following steps, which will not be elaborated here.

[0047] Now, an exemplary description of the linear interpolation of the vehicle body pose will be given in combination with Figure 2

[0048] Refer to Figure 2 , where ta and tb are the sampling moments of the four-wheel pulse data of the vehicle, and t is the shooting moment of the BEV image of the vehicle. Among them, ta is the previous sampling moment adjacent to t, and tb is the next sampling moment adjacent to t. At the ta moment, the four-wheel pulse data between the t2 moment and the ta moment can be obtained, and the movement of the vehicle between the t2 moment and the ta moment can be calculated based on the four-wheel pulse data between the t2 moment and the ta moment. The vehicle body pose at the ta moment can be determined based on the fused pose of the vehicle at the t2 moment and the movement of the vehicle between the t2 moment and the ta moment; at the tb moment, the four-wheel pulse data between the t2 moment and the tb moment can be obtained, and the movement of the vehicle between the t2 moment and the tb moment can be calculated based on the four-wheel pulse data between the t2 moment and the tb moment. The vehicle body pose at the tb moment can be determined based on the fused pose of the vehicle at the t2 moment and the movement of the vehicle between the t2 moment and the tb moment. Then, linear interpolation can be performed between the vehicle body pose at the ta moment and the vehicle body pose at the tb moment, and thus the vehicle body pose at the t moment can be determined.

[0049] Exemplarily, the vehicle body pose T at the ta moment ta can be calculated by the following formula: ; where T t2-merge is the fused pose of the vehicle at the t2 moment, and △T t2-ta is the movement of the vehicle between the t2 moment and the ta moment.

[0050] The vehicle body pose T at the tb moment tb can be calculated by the following formula: ; where T t2-merge is the fused pose of the vehicle at the t2 moment, and △Tt2-tb The movement of the vehicle between time t2 and time tb.

[0051] Step 103, the vehicle detects the parking space corner points in the BEV image at time t.

[0052] Specifically, the parking space corner points refer to the corner points of the vehicle entrance in the parking space.

[0053] Among them, the method of detecting the parking space corner points in the BEV image can be through a preset parking space detection model, or the method of detecting the parking space corner points in the BEV image can be through other image recognition methods. The embodiments of the present application do not make special limitations on this.

[0054] If the vehicle detects parking space corner points in the BEV image at time t, execute Step 104.

[0055] If the vehicle does not detect parking space corner points in the BEV image at time t, execute Step 112.

[0056] Step 104, the vehicle matches the detection data with the map data, where the detection data includes the parking space corner point information detected in the BEV image, and the map data includes the parking space corner point information created in the map.

[0057] Specifically, the parking space corner point information can include coordinates and orientation.

[0058] Among them, the parking space corner point information in the detection data can include the coordinates and orientation in the vehicle body coordinate system, and the parking space corner point information in the map data can include the coordinates and orientation in the world coordinate system.

[0059] The map can be created by the vehicle after the parking function is started, and the parking space corner points in the map can be created or updated by the vehicle during the parking process.

[0060] Among them, the method for the vehicle to match the detection data with the map data can include: comparing the coordinates and orientation of the parking space corner points in the detection data with the coordinates and orientation of the parking space corner points in the map data respectively.

[0061] Exemplarily, if the coordinate error between the parking space corner points in the detection data and the parking space corner points in the map data is within the first preset error, and if the orientation error between the parking space corner points in the detection data and the parking space corner points in the map data is within the second preset error, it can be considered that the detection data and the map data are successfully matched. Otherwise, it can be considered that the detection data and the map data are mismatched.

[0062] It should be noted that there may be multiple parking space corner points in the map data, and there may be multiple parking space corner points in the detection data. By matching the detection data with the map data, it can be determined whether the multiple parking space corner points in the detection data are successfully matched.

[0063] Among them, the ways for the vehicle to match the detection data with the map data may include: the vehicle sequentially matches the parking space corner points in the detection data with all the parking space corner points in the map data to determine whether each parking space corner point in the detection data is successfully matched.

[0064] Exemplarily, assume that the detection data contains 2 parking space corner points P1 and P2, and the map data contains n parking space corner points such as Q1, Q2, …, Qn. The way to match the parking space corner points in the detection data with the parking space corner points in the map data can be to match P1 with the n parking space corner points Q1, Q2, …, Qn to determine whether there is a parking space corner point in the map data that matches P1, and to match P2 with the n parking space corner points Q1, Q2, …, Qn to determine whether there is a parking space corner point in the map data that matches P2.

[0065] It can be understood that there are no parking space corner points in the map in the initial state. After the vehicle obtains the detection data, it can create parking space corner points in the map based on the parking space corner points in the detection data to create map data. As the number of parking space corner points in the map continuously increases, after the vehicle obtains the detection data, it can match the parking space corner points in the detection data with the parking space corner points in the map data. If the parking space corner points in the map data do not match the parking space corner points in the detection data, new parking space corner points can be created in the map based on the parking space corner points in the detection data. If the parking space corner points in the map data match the parking space corner points in the detection data, the parking space corner points in the map can be updated based on the parking space corner points in the map data and the parking space corner points in the detection data.

[0066] It should be noted that the coordinate system adopted by the parking space corner points in the map data can be the world coordinate system, and the coordinate system adopted by the parking space corner points in the detection data can be the vehicle body coordinate system centered on the vehicle body. When the vehicle matches the detection data with the map data, the parking space corner points in the detection data can be converted to the world coordinate system and then compared with the parking space corner points in the map data; or, the parking space corner points in the map data can be converted to the vehicle body coordinate system and then the parking space corner points in the detection data can be compared with the parking space corner points in the map data. The embodiments of the present application do not make special limitations on this.

[0067] Among them, the ways to convert the parking space corner points in the detection data to the world coordinate system may include: based on the vehicle body pose of the vehicle, convert the parking space corner points in the detection data to the world coordinate system.

[0068] The method of converting the parking space corner points in the map data into the vehicle body coordinate system may include: based on the vehicle body pose, converting the parking space corner points in the map data into the vehicle body coordinate system.

[0069] If any parking space corner point in the detection data cannot find a matching parking space corner point in the map data, step 105 is executed. Or, If any parking space corner point in the detection data finds a matching parking space corner point in the map data, step 106 is executed.

[0070] In some alternative embodiments, the parking space corner point information in the map data may further include the actual observation times m and the theoretical observation times n. Wherein, m is used to represent the cumulative number of times that the parking space corner points are actually detected by the parking space detection model or related recognition algorithms within the BEV field of view of the vehicle, and n is used to represent the cumulative number of times that the existing parking space corner points in the map data are theoretically detected within the BEV field of view of the vehicle.

[0071] Exemplarily, assume that there is a detected parking space corner point S in the map data. In the initial state, m = 1 and n = 1. After 2 observations within the BEV field of view of the vehicle, wherein, through the first observation, the parking space corner point successfully detected in the BEV image matches the parking space corner point S, and through the second observation, no parking space corner point matching the parking space corner point S is successfully detected in the BEV image. In this case, the actual observation times m = 1 + 1 = 2, and the theoretical observation times = 1 + 2 = 3.

[0072] It can be understood that during the matching process, there may be false detection situations. For example, non-parking space corner points in the out-of-vehicle scene are wrongly detected as parking space corner points, which will bring noise interference to the vehicle's parking positioning. In the embodiments of the present application, during the matching process, by judging the value of m / n, the falsely detected parking space corner points in the map data can be deleted. For example, if the m / n of the target parking space corner point is less than or equal to the preset threshold, the target parking space corner point can be deleted. The target parking space corner point can be any parking space corner point within the BEV image field of view of the vehicle in the map, thereby reducing noise interference and improving the accuracy of parking positioning.

[0073] Step 105, the vehicle creates a parking space corner point in the map based on the parking space corner points that have not been successfully matched in the detection data.

[0074] Step 106, the vehicle judges the total number of parking space corner points that are successfully matched in the detection data and the map data.

[0075] If the total number of parking space corner points that are successfully matched in the detection data and the map data is less than the preset total number (for example, 2), step 107 is executed.

[0076] If the total number of parking space corner points that match successfully between the detected data and the map data is greater than or equal to the preset total number, step 108 is executed.

[0077] Step 107, the vehicle takes the body pose obtained at time t as the fused pose at time t.

[0078] It can be understood that the fused pose at time t can be used for the vehicle to perform parking positioning.

[0079] Step 108, the vehicle compares the first body pose with the second body pose.

[0080] Specifically, the first body pose can be the body pose calculated by the wheeled odometry method in step 102, and the second body pose can be the body pose calculated by the Iterative Closest Point (ICP) algorithm.

[0081] It can be understood that there may be an error between the first body pose and the second body pose.

[0082] If the error between the first body pose and the second body pose is greater than or equal to the third preset error, it can be considered that the second body pose is incorrect, and step 109 is executed. Or, If the error between the first body pose and the second body pose is less than the third preset error, step 110 is executed.

[0083] Step 109, the vehicle takes the first body pose as the fused pose at time t.

[0084] Step 110, the vehicle fuses the first body pose and the second body pose to obtain the fused pose at time t.

[0085] Specifically, the first body pose and the second body pose can be fused by the Extended Kalman Filter (EKF) algorithm.

[0086] Exemplarily, the EKF algorithm can use the Constant Turn Rate and Velocity (CTRV) model and a 12-dimensional observation variable z. In some embodiments, the EKF algorithm can also use other models and variables, and the embodiments of the present application do not make special limitations on this.

[0087] Taking the 12-dimensional observation variable z as an example, z can be represented by the following formula: ; where x vis is the abscissa in the second body pose, y visis the ordinate in the second vehicle body pose, θ vis is the heading angle in the second vehicle body pose, x wo is the abscissa in the first vehicle body pose, y wo is the ordinate in the first vehicle body pose, θ wo is the heading angle in the first vehicle body pose, v fl is the linear velocity of the left front wheel, v fr is the linear velocity of the right front wheel, v rl is the linear velocity of the left rear wheel, v rr is the linear velocity of the right rear wheel, Ψ is the steering wheel angle, a n is the centripetal acceleration.

[0088] Based on the state variable X, the theoretical observation h can be calculated, where X can include the abscissa x, ordinate y, linear velocity v, heading angle θ, and angular velocity ω of the vehicle in the world coordinate system.

[0089] h can be characterized by the following formula: ; where w is the vehicle width, l is the wheelbase between the front and rear axles of the vehicle, and k is the transmission ratio from the steering wheel to the front wheels.

[0090] The Jacobian matrix J of the theoretical observation h h can be characterized by the following formula: ; ; ; ; ; Next, based on the prediction of the EKF, (x, y, θ) can be solved, and the solved (x, y, θ) can be used as the fused pose at time t.

[0091] Step 111, the vehicle updates the parking space corner points in the map data based on the first parking space corner points and the second parking space corner points, where the first parking space corner points are the parking space corner points in the detection data, the second parking space corner points are the parking space corner points in the map data, and the first parking space corner points and the second parking space corner points have a matching relationship.

[0092] Specifically, the way for the vehicle to update the parking space corner points in the map data based on the first parking space corner points and the second parking space corner points can be achieved through the following formula: ; ; ; Among them, x m is the abscissa in the fusion pose, y m is the ordinate in the fusion pose, θ m is the heading angle in the fusion pose, p v is the coordinate of the first parking space corner point in the vehicle body coordinate system, β v is the orientation of the first parking space corner point in the vehicle body coordinate system, p is the coordinate of the second parking space corner point in the world coordinate system, β is the orientation of the second parking space corner point in the world coordinate system, p update is the coordinate of the updated parking space corner point in the world coordinate system, β update is the orientation of the updated parking space corner point in the world coordinate system, and Tmerge is the matrix after the fusion pose is transformed.

[0093] It can be understood that this step 111 can be executed before step 106, or this step 111 can be executed at any step between step 106 and step 110. The embodiments of the present application do not make special limitations on this.

[0094] Step 112, execute the steps at time t1.

[0095] Specifically, for the specific steps at time t1, the relevant descriptions of steps 101 - 111 at time t1 are not repeated here. Among them, time t1 can be the next moment of time t, and the relationship between time t1 and time t can be specifically referred to Figure 2 .

[0096] The embodiments of the present application model the parking space map points through the parking space corner point observation data, so as to reduce the cumulative error of the positioning system and can achieve reliable and accurate positioning on a computing platform with limited computing power in the scenario of parking in a marked parking space.

[0097] Figure 3 is the structural schematic diagram of the parking device provided by the embodiments of the present application. As Figure 3 shown, the above parking device 30 may include: a determination module 31, a matching module 32, and a fusion module 33; among them, The determination module 31 is used to determine the bird's-eye view BEV image at time t, where time t is any moment during the vehicle's parking process; The matching module 32 is used to match the detected parking space corner points with the parking space corner points in the map if the parking space corner points are detected in the BEV image. The map is created after the vehicle starts the parking function, and the parking space corner points in the map are the parking space corner points created or updated in the map after the parking space corner points are detected in the BEV image; The fusion module 33 is used to determine the total number of matches, and based on the total number of matches, determine the fusion pose at time t, where the fusion pose is used for the vehicle to perform parking positioning, and the total number of matches is the total number of successful matches.

[0098] In one possible implementation, the fusion module 33 is further configured to, if the total number of matches is greater than or equal to a preset total number, determine the fusion pose at time t based on the first body pose at time t and the second body pose at time t; wherein, the first body pose at time t is determined by the body pose at the first sampling time and the body pose at the second sampling time; the first sampling time and the second sampling time are the sampling times of the four-wheel pulse data of the vehicle, the first sampling time is the previous sampling time adjacent to time t, and the second sampling time is the next sampling time adjacent to time t; the second body pose at time t is obtained by calculating using the Iterative Closest Point (ICP) algorithm.

[0099] In one possible implementation, the determination module 31 is further configured to, if the total number of matches is less than the preset total number, determine the first body pose at time t as the fusion pose at time t.

[0100] In one possible implementation, the first body pose at time t is determined by interpolating the body pose between the first sampling time and the second sampling time.

[0101] In one possible implementation, the body pose at the first sampling time is determined by the fusion pose at time t2 and the movement of the vehicle between time t2 and the first sampling time, and the body pose at the second sampling time is determined by the fusion pose at time t2 and the movement of the vehicle between time t2 and the second sampling time, where time t2 is the previous moment of time t.

[0102] In one possible implementation, the matching module 32 is further configured to compare the coordinates and orientations of the detected parking space corner points with the coordinates and orientations of the parking space corner points in the map respectively; If the coordinate error between the detected parking space corner point and the parking space corner point in the map is within a first preset error, and if the orientation error between the detected parking space corner point and the parking space corner point in the map is within a second preset error, it is determined that the matching is successful.

[0103] In one possible implementation, the parking device 30 further includes: A deletion module, configured to delete the target parking space corner point if the ratio of the actual observation times to the theoretical observation times of the target parking space corner point in the map is less than or equal to a preset threshold. Wherein, the target parking space corner point is any parking space corner point within the field of view of the BEV image of the vehicle in the map. The actual observation times are used to represent the cumulative number of times the target parking space corner point is actually detected within the field of view of the BEV image of the vehicle, and the theoretical observation times are used to represent the cumulative number of times the target parking space corner point is theoretically detected within the field of view of the BEV image of the vehicle.

[0104] In one possible implementation, the parking device 30 further includes: An update module, configured to, if the detected parking space corner point successfully matches the parking space corner point in the map, fuse the detected parking space corner point with the parking space corner point in the map to update the parking space corner point in the map.

[0105] In one possible implementation, the determination module 31 is further configured to, if no parking space corner point is detected in the BEV image, determine the fusion pose at time t1, where t1 is the next moment after time t.

[0106] In one possible implementation, the fusion module 33 is further configured to, if the error between the first vehicle body pose at time t and the second vehicle body pose at time t is greater than or equal to a third preset error, determine the first vehicle body pose at time t as the fusion pose at time t. Or, if the error between the first vehicle body pose at time t and the second vehicle body pose at time t is less than the third preset error, fuse the first vehicle body pose at time t and the second vehicle body pose at time t to obtain the fusion pose at time t.

[0107] Figure 3 The parking device 30 provided in the illustrated embodiment can be used to execute the technical solution of the method embodiment shown in the present application, and its implementation principle and technical effects can be further referred to the relevant descriptions in the method embodiment.

[0108] It should be understood that the division of the various modules of the above parking device 30 is only a division of logical functions. In actual implementation, they can be fully or partially integrated into a physical entity, or physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also all be implemented in the form of hardware; or some modules can be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or can be integrated in a certain chip of the terminal device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together or can be independently implemented. In the implementation process, each step of the above method or each of the above modules can be completed by the integrated logic circuit of the hardware in the processor element or the instructions in the form of software.

[0109] For example, the above-mentioned modules can be one or more integrated circuits configured to implement the above method, such as: one or more application specific integrated circuits (ASIC for short), or, one or more digital signal processors (DSP for short), or, one or more field programmable gate arrays (FPGA for short), etc. Again, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC for short).

[0110] Figure 4 A schematic structural diagram of a vehicle 400 provided by an embodiment of the present application. The above vehicle 400 may include: at least one processor; and at least one memory communicatively connected to the above processor. The above memory stores program instructions executable by the above processor, and the processor in the vehicle 400 can execute the actions performed in the storage access method provided by the embodiment of the present application by calling the above program instructions.

[0111] Such as Figure 4 As shown, the vehicle 400 is presented in the form of a general computing device. The components of the vehicle 400 may include but are not limited to: one or more processors 410, a memory 420, a communication bus 440 connecting different system components (including the memory 420 and the processor 410), and a communication interface 430.

[0112] The communication bus 440 represents one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, an Accelerated Graphics Port, a processor, or a local bus using any of the various bus architectures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnection (PCI) bus.

[0113] Vehicle 400 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the terminal device, including volatile and non-volatile media, removable and non-removable media.

[0114] Memory 420 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) and / or cache memory. The terminal device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Although Figure 4 not shown, a disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., Compact Disc Read Only Memory (CD-ROM), Digital Video Disc Read Only Memory (DVD-ROM), or other optical media) may be provided. In these instances, each drive may be connected to the communication bus 440 by one or more data media interfaces. Memory 420 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the embodiments of the present application.

[0115] A program / util utility having a set (at least one) of program modules can be stored in the memory 420. Such program modules include—but are not limited to—an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment. The program modules generally execute the functions and / or methods in the embodiments described in this application.

[0116] The vehicle 400 can also communicate with one or more external devices (such as a keyboard, a pointing device, a display, etc.), and can also communicate with one or more devices that enable a user to interact with the terminal device, and / or communicate with any device that enables the terminal device to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through the communication interface 430. And, the vehicle 400 can also communicate with one or more networks (such as a Local Area Network (LAN), a Wide Area Network (WAN), and / or a public network, such as the Internet) through a network adapter ( Figure 4 not shown in the figure). The above network adapter can communicate with other modules of the terminal device through the communication bus 440. It should be understood that although Figure 4 not shown in the figure, other hardware and / or software modules can be used in combination with the vehicle 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, Redundant Arrays of Independent Drives (RAID) systems, tape drives, and data backup storage systems, etc.

[0117] The processor 410 executes various functional applications and data processing by running the programs stored in the memory 420, such as implementing the methods provided in the embodiments of this application.

[0118] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of this application is only for illustrative purposes and does not constitute a structural limitation on the vehicle 400. In other embodiments of this application, the vehicle 400 can also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0119] In the above embodiments, the processors involved may include, for example, a CPU, a DSP, a microcontroller, or a digital signal processor. They may also include a GPU, an embedded neural-network processor (hereinafter referred to as: NPU), and an image signal processor (hereinafter referred to as: ISP). The processor may further include necessary hardware accelerators or logic processing hardware circuits, such as an ASIC, or one or more integrated circuits for controlling the execution of the technical solution programs of the present application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.

[0120] The embodiments of the present application further provide a readable storage medium, in which a program is stored. When it runs on a system, it causes the system to execute the method provided by the embodiments shown in the present application.

[0121] The embodiments of the present application further provide a program product, which includes a program. When it runs on a system, it causes the system to execute the method provided by the embodiments shown in the present application.

[0122] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent the situation where A exists alone, A and B exist simultaneously, or B exists alone. Where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0123] Those of ordinary skill in the art can realize that the units and algorithm steps described in the embodiments disclosed herein can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0124] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0125] In several embodiments provided by the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs.

[0126] As described above, the foregoing is only the specific implementation manner of the present application. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. The protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A parking method, characterized in that, The method includes: Determine the bird's-eye view (BEV) image at time t, where t is any moment during the vehicle's parking process; If parking space corner points are detected in the BEV image, match the detected parking space corner points with the parking space corner points in the map. The map is created after the vehicle activates the parking function, and the parking space corner points in the map are the parking space corner points created or updated in the map after the parking space corner points are detected in the BEV image; Determine the total number of matches, and based on the total number of matches, determine the fusion pose at time t. The fusion pose is used for the vehicle to perform parking positioning, and the total number of matches is the total number of successful matches.

2. The method according to claim 1, wherein The determining the fusion pose at time t based on the total number of matches includes: If the total number of matches is greater than or equal to a preset total number, determine the fusion pose at time t based on the first vehicle body pose at time t and the second vehicle body pose at time t; Wherein, the first vehicle body pose at time t is determined by the vehicle body pose at the first sampling moment and the vehicle body pose at the second sampling moment; the first sampling moment and the second sampling moment are the sampling moments of the four-wheel pulse data of the vehicle, the first sampling moment is the previous sampling moment adjacent to time t, and the second sampling moment is the next sampling moment adjacent to time t; the second vehicle body pose at time t is obtained by calculating using the iterative closest point (ICP) algorithm.

3. The method according to claim 2, wherein The method further includes: If the total number of matches is less than the preset total number, determine the first vehicle body pose at time t as the fusion pose at time t.

4. The method according to claim 2, wherein The determining the first vehicle body pose at time t by the vehicle body pose at the first sampling moment and the vehicle body pose at the second sampling moment includes: The first vehicle body pose at time t is determined by interpolating the vehicle body pose between the first sampling moment and the second sampling moment.

5. The method according to claim 4, characterized in that, The vehicle body pose at the first sampling moment is determined by the fusion pose at time t2 and the movement of the vehicle between time t2 and the first sampling moment, and the vehicle body pose at the second sampling moment is determined by the fusion pose at time t2 and the movement of the vehicle between time t2 and the second sampling moment. Time t2 is the previous moment of time t.

6. The method according to claim 1, wherein The matching the detected parking space corner points with the parking space corner points in the map includes: Compare the coordinates and orientations of the detected parking space corner points with the coordinates and orientations of the parking space corner points in the map respectively; If the coordinate error between the detected parking space corner points and the parking space corner points in the map is within a first preset error, and if the orientation error between the detected parking space corner points and the parking space corner points in the map is within a second preset error, determine that the match is successful.

7. The method according to claim 1, characterized in that The method further includes: If the ratio of the actual observation times to the theoretical observation times of the target parking space corner point in the map is less than or equal to a preset threshold, delete the target parking space corner point. Wherein, the target parking space corner point is any parking space corner point within the field of view of the BEV image of the vehicle in the map. The actual observation times are used to represent the cumulative number of times the target parking space corner point is actually detected within the field of view of the BEV image of the vehicle, and the theoretical observation times are used to represent the cumulative number of times the target parking space corner point is theoretically detected within the field of view of the BEV image of the vehicle.

8. The method according to claim 1, wherein The parking space corner points in the map are the parking space corner points updated in the map after the parking space corner points are detected in the BEV image, including: If the detected parking space corner point successfully matches the parking space corner point in the map, the detected parking space corner point and the parking space corner point in the map are fused to update the parking space corner point in the map.

9. The method according to claim 1, characterized in that, The method further includes: If no parking space corner point is detected in the BEV image, determine the fusion pose at time t1, where t1 is the next moment after time t.

10. The method according to claim 2, characterized in that, Determining the fusion pose at time t based on the first vehicle body pose at time t and the second vehicle body pose at time t includes: If the error between the first vehicle body pose at time t and the second vehicle body pose at time t is greater than or equal to a third preset error, determine the first vehicle body pose at time t as the fusion pose at time t. Or, If the error between the first vehicle body pose at time t and the second vehicle body pose at time t is less than the third preset error, fuse the first vehicle body pose at time t and the second vehicle body pose at time t to obtain the fusion pose at time t.

11. A parking device, characterized in that, The device includes: A determination module for determining a bird's-eye view (BEV) image at time t, where t is any moment during the vehicle's parking process; A matching module for, if a parking space corner point is detected in the BEV image, matching the detected parking space corner point with the parking space corner points in the map. The map is created after the vehicle starts the parking function, and the parking space corner points in the map are the parking space corner points created or updated in the map after the parking space corner points are detected in the BEV image; A fusion module for determining the total number of matches and determining the fusion pose at time t based on the total number of matches. The fusion pose is used for the vehicle to perform parking positioning, and the total number of matches is the total number of successful matches.

12. A vehicle, characterized in that, Including: A processor and a memory. The memory is used to store a program; the processor is used to run the program to implement the parking method according to any one of claims 1-10.

13. A readable storage medium, characterized in that, The readable storage medium stores a program, and when the program runs on the vehicle, it implements the parking method according to any one of claims 1-10.

Images