Parking planning method, electronic equipment, vehicle and computer readable storage medium

Abstract

The invention relates to the field of vehicle control, in particular to a parking planning method, electronic equipment, a vehicle and a computer readable storage medium. The parking planning method is used for performing at least one round of trajectory planning according to a time sequence in a vehicle parking process. The method comprises the following steps: determining parking reference data; determining a first planning track and first occupation prediction data based on the parking reference data and a preset parking planning model; and taking the first occupation prediction data, the first planning track, the driving environment data acquired in the next round and the target parking space as parking reference data for planning the track in the next round, and continuing to execute the step of determining the first planning track and predicting the space occupation state based on the parking reference data and a preset parking planning model. And the parking planning is finished. According to the invention, the problems of delayed information transmission, high processing complexity, poor real-time performance and the like of the existing memory parking technology can be solved.

Description

Technical Field

[0001] The present application relates to the field of vehicle control, and particularly to a parking planning method, an electronic device, a vehicle, and a computer-readable storage medium. Background Art

[0002] Memory parking is an intelligent driving technology based on simultaneous localization and mapping (SLAM). When parking for the first time, the vehicle will record and store data such as the complete parking path from the starting point to the parking space, the surrounding environment information of the vehicle, and the driving habits of the driver. When returning to the same parking scenario again, the vehicle can call the previously stored data and automatically cruise to the vicinity of the parking space and park in the space according to the memorized route.

[0003] However, during the parking process, the vehicle generally operates in complex and changeable driving scenarios. To enable the vehicle to successfully park in the space, existing memory parking technologies often need to use high-precision sensors such as lasers and radars, satellites, or deploy monitoring systems or sensors inside the parking lot to identify the environment where the vehicle is located and locate the position of the vehicle. However, in this method, there is excessive interaction between various sensors and systems, resulting in problems such as information transfer delay, high processing complexity, and poor real-time performance. Summary of the Invention

[0004] In view of the above, embodiments of the present application provide a parking planning method, an electronic device, a vehicle, and a computer-readable storage medium, aiming to solve problems such as information transfer delay, high processing complexity, and poor real-time performance existing in existing memory parking technologies.

[0005] In a first aspect, an embodiment of the present application provides a parking planning method for performing at least one round of trajectory planning in chronological order during the vehicle parking process. The parking planning method includes: determining parking reference data, where, in the case of performing trajectory planning in the first round, the parking reference data includes the driving environment data collected in the first round and the target parking space to be parked; determining a first planned trajectory and first occupancy prediction data based on the parking reference data and a preset parking planning model, where the first occupancy prediction data includes the predicted probability that each spatial unit in the future driving environment of the vehicle is occupied by an object; if the parking planning is not completed, using the first occupancy prediction data, the first planned trajectory, the driving environment data collected in the next round, and the target parking space as the parking reference data for performing the next round of trajectory planning, and continuing to execute the step of determining the first planned trajectory and predicting the spatial occupancy state based on the parking reference data and the preset parking planning model until the parking planning is completed.

[0006] In the embodiments of the present application, at least one round of trajectory planning is performed during the parking process. The first round of trajectory planning is based on the collected driving environment data and the target parking space. Each subsequent round of trajectory planning combines the current latest driving environment dynamics, the trajectory planning result of the previous round (i.e., the first planned trajectory), and the space occupancy situation of the previous round (i.e., the first occupancy prediction data), enabling the vehicle to anticipate the obstacle occupancy risk in the driving environment in advance, timely capture the changes of obstacles during the parking process, and thus timely and flexibly adjust the parking trajectory to avoid obstacles, making the parking path always fit the actual driving scenario, improving the planning accuracy, ensuring a steady approach to the target parking space in various scenarios, and enhancing the parking safety.

[0007] In some embodiments, the preset parking planning model includes an encoder, a self-vehicle trajectory decoder, and an occupancy network decoder; the determining of the first planned trajectory and the first occupancy prediction data based on the parking reference data and the preset parking planning model includes: encoding the parking reference data based on the encoder to obtain encoded features; performing trajectory planning on the vehicle based on the encoded features and the self-vehicle trajectory decoder to obtain the first planned trajectory; predicting the probability that each space unit in the future driving environment of the vehicle is occupied by an object based on the encoded features and the occupancy network decoder to obtain the first occupancy prediction data.

[0008] By adopting the above technical solution, the parking planning model can output scene understanding (i.e., the first occupancy prediction data) and the first planned trajectory together, thereby improving the response speed of trajectory planning.

[0009] In some embodiments, encoding the parking reference data based on the encoder to obtain encoded features includes: extracting the bird's-eye view features and trajectory features in the parking reference data based on the encoder; performing feature fusion on the trajectory features and the bird's-eye view features based on the encoder to obtain the encoded features.

[0010] In some embodiments, the encoder includes a self-attention layer, and performing feature fusion on the trajectory features and the bird's-eye view features based on the encoder to obtain the encoded features includes: determining a query vector based on the bird's-eye view features; determining a key vector and a value vector based on the trajectory features; inputting the query vector, the key vector, and the value vector into the self-attention layer to obtain the encoded features.

[0011] In some embodiments, the training steps of the preset parking planning model include: obtaining a first data set, where the first data set includes driving environment samples and corresponding actual spatial occupancy data, and the actual spatial occupancy data includes the actual probability that each spatial unit in the future driving environment of the vehicle is occupied by an object; determining second occupancy prediction data based on the driving environment samples and an occupancy network module, where the occupancy network module includes an encoder and an occupancy network decoder in the parking planning model to be trained; inputting the actual spatial occupancy data and the second occupancy prediction data into a preset first loss function to obtain a first loss value; if it is determined that the first loss function converges based on the first loss value, training the parking planning model to be trained to obtain the preset parking planning model.

[0012] In some embodiments, training the parking planning model to be trained to obtain the preset parking planning model includes: obtaining a second data set, where the second data set includes the first data set and the actual driving trajectory corresponding to the driving environment sample; inputting the driving environment sample into the parking planning model to be trained to obtain a second planned trajectory and third occupancy prediction data; inputting the second planned trajectory, the actual driving trajectory, the third occupancy prediction data, and the actual spatial occupancy data into a preset second loss function to obtain a second loss value; updating the model parameters of the parking planning model to be trained based on the second loss value to obtain the preset parking planning model.

[0013] In some embodiments, before determining the parking reference data, it further includes: collecting driving environment data at preset intervals; if the driving environment data collected in the current period matches the pre-stored parking lot mapping data, activating the memory parking function of the vehicle; determining the parking reference data, including: determining the parking reference data when the memory parking function of the vehicle is activated.

[0014] In a second aspect, an embodiment of the present application further provides an electronic device, where the electronic device includes a processor and a memory, the memory is used to store instructions, and the processor is used to call the instructions in the memory so that the electronic device executes the parking planning method as described in the first aspect.

[0015] In a third aspect, an embodiment of the present application further provides a vehicle, where the vehicle includes the electronic device described in the second aspect.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and when the computer instructions run on an electronic device, the electronic device is enabled to execute the parking planning method as described in the first aspect. Description of the Drawings

[0017] Figure 1 Schematic diagram of the implementation environment of the parking planning method provided by the embodiment of the present application.

[0018] Figure 2 Flowchart of the steps of the parking planning method provided by an embodiment of the present application.

[0019] Figure 3 Schematic diagram of the structure of the parking planning model provided by an embodiment of the present application.

[0020] Figure 4 Schematic diagram of the structure of the parking planning model provided by another embodiment of the present application.

[0021] Figure 5 Schematic diagram of the scenario for performing multi-round trajectory planning provided by an embodiment of the present application.

[0022] Figure 6 Flowchart of the training method of the parking planning model provided by an embodiment of the present application.

[0023] Figure 7 Schematic diagram of the structure of the electronic device provided by an embodiment of the present application.

[0024] Figure 8 Schematic diagram of the structure of the vehicle provided by an embodiment of the present application. Detailed implementation manners

[0025] In order to more clearly understand the above objects, features and advantages of the present application, the present application will be described in detail below with reference to the accompanying drawings and specific implementation manners. It should be noted that, without conflict, the implementation manners of the present application and the features in the implementation manners may be combined with each other.

[0026] Many specific details are set forth in the following description in order to provide a thorough understanding of the present application. The described implementation manners are only a part of the implementation manners of the present application, rather than all of the implementation manners.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs. The terms used in the specification of the present application herein are only for the purpose of describing specific implementation manners, and are not intended to limit the present application.

[0028] It should be further noted that in this text, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising that element.

[0029] In this application, "at least one" means one or more, and "a plurality" means two or more than two. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0030] In the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0031] The embodiments of this application provide a parking planning method, an electronic device, a vehicle and a computer-readable storage medium.

[0032] The parking planning method of this application can be applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a processor, a microprogrammed control unit (MCU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc. The electronic device can be an in-vehicle processor, such as a vehicle controller, etc., but is not limited thereto.

[0033] The electronic device can be configured in a vehicle and communicatively connected to in-vehicle sensors in the vehicle. The in-vehicle sensors can be, but are not limited to, image sensors, radar sensors, etc. The sensors can be used to collect driving environment data of the vehicle and transmit the driving environment data to the electronic device.

[0034] Among them, the driving environment data can reflect the driving environment where the vehicle is located. The driving environment data can include, but is not limited to, road images and laser point cloud data collected by in-vehicle sensors.

[0035] After the sensors collect the driving environment data, the electronic device can execute a parking planning method during the parking process based on the driving environment data.

[0036] Such as Figure 1 shown, it is a schematic diagram of the implementation environment of the parking planning method provided by the embodiments of the present application. The parking planning method of the embodiments of the present application can perform at least one round of trajectory planning in sequence during the vehicle parking process, so as to automatically park the vehicle.

[0037] The parking planning method can include: determining parking reference data. Among them, in the case of performing trajectory planning in the first round, the parking reference data includes the driving environment data collected in the first round and the target parking space. Based on the parking reference data and a preset parking planning model, determine a first planning trajectory and first occupancy prediction data. The first occupancy prediction data includes the predicted probability that each spatial unit in the future driving environment of the vehicle is occupied by an object. Then, if the parking planning is not completed, use the first occupancy prediction data, the first planning trajectory, the driving environment data collected in the next round, and the target parking space as the parking reference data for the next round of trajectory planning, and continue to execute the step of determining the first planning trajectory and predicting the spatial occupancy state based on the parking reference data and the preset parking planning model until the parking planning is completed.

[0038] In the embodiments of the present application, at least one round of trajectory planning is performed during the parking process. The first-round planning is started based on the collected driving environment data and the target parking space. Each subsequent round of trajectory planning combines the current latest driving environment dynamics, as well as the trajectory planning result of the previous round (i.e., the first planning trajectory) and the spatial occupancy situation of the previous round (i.e., the first occupancy prediction data), enabling the vehicle to anticipate the obstacle occupancy risk in the driving environment in advance, timely capture the changes of obstacles during the parking process, thereby flexibly adjusting the parking trajectory, avoiding obstacles, making the parking path always fit the actual driving scenario, improving the planning accuracy, ensuring that it can steadily move towards the target parking space in various scenarios, and enhancing the parking safety.

[0039] Refer to Figure 2 shown Figure 2It is a flowchart of the steps of an embodiment of the parking planning method of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The parking planning method may include the following steps.

[0040] Step 201, determine parking reference data.

[0041] During the parking process of the vehicle (from the start of parking to the end of parking), at least one round of trajectory planning can be performed, and each round of trajectory planning has corresponding parking reference data.

[0042] In the case of performing trajectory planning for the first time, the parking reference data may include the driving environment data collected for the first time and the target parking space to be parked.

[0043] In the case of performing trajectory planning subsequently, the parking reference data may include the driving environment data collected in the most recent preset time period, the target parking space to be parked, and the trajectory planning result of the previous round.

[0044] Among them, the trajectory planning result of the previous round includes: the first planned trajectory output by the parking planning model during the previous round of trajectory planning and the first occupancy prediction data.

[0045] In some embodiments, the target parking space to be parked may be a preset parking space pre-set by the user. For example, if the user's common parking space is parking space A, the user can set parking space A as the target parking space to be parked.

[0046] In other embodiments, the target parking space to be parked may be an empty parking space in the parking lot. For example, the electronic device may determine the empty parking space in the parking lot based on the collected driving environment data, and use the empty parking space as the target parking space to be parked.

[0047] In other embodiments, the electronic device may detect whether the preset parking space to be parked is occupied by a vehicle. If the preset parking space to be parked is not occupied by a vehicle, the preset parking space to be parked is used as the target parking space to be parked. If the preset parking space to be parked is occupied by a vehicle, an empty parking space is searched for in the parking lot, and the empty parking space is used as the target parking space to be parked.

[0048] Among them, the step of searching for an empty parking space in the parking lot may include: inputting the driving environment data into the occupancy network module to obtain an empty parking space. The parking space attribute of the empty parking space is parkable, and the parking space attribute of the occupied parking space is non-parkable.

[0049] Reference Figure 3 As shown, the occupancy network (Occupancy Network, OCC) module may include an encoder and an occupancy network decoder in a preset parking planning model.

[0050] The preset parking space to be parked can be marked in the parking lot mapping data.

[0051] The steps for obtaining parking lot mapping data may include: after the vehicle arrives at the parking lot, in response to a mapping activation request, the memory parking mapping function is activated. Then, the electronic device can map the parking lot to obtain the parking lot mapping data.

[0052] For example, the electronic device can collect parking lot environment data such as camera images and lidar point cloud data from on-vehicle sensors. Then, the parking lot environment data is input into the occupancy network module to obtain the parking lot mapping data.

[0053] The parking lot mapping data may include the occupancy status of each spatial unit in the parking lot. This occupancy status can be used to indicate whether the spatial unit (e.g., voxel) is occupied by an object and the object type corresponding to the object occupying the spatial unit.

[0054] For example, the object types in the parking lot mapping data may include elements such as driving areas, walls, curbs, gates, ground markings, columns, people, motor vehicles, non-motor vehicles, etc. in the parking lot.

[0055] After the electronic device obtains the parking lot mapping data, the parking lot mapping data can be stored.

[0056] In some embodiments, during the driving process of the vehicle, the electronic device can collect driving environment data at preset intervals; if the driving environment data collected in the current period matches the pre-stored parking lot mapping data, it indicates that the vehicle has entered or is about to enter the parking lot, and the electronic device can activate the memory parking function of the vehicle.

[0057] Among them, the steps for the electronic device to detect whether the driving environment data collected in the current period matches the pre-stored parking lot mapping data may include: detecting that the scene similarity between the driving environment data and the parking lot entrance in the parking lot mapping data exceeds a preset threshold, then determining that the driving environment data collected in the current period matches the pre-stored parking lot mapping data.

[0058] In some other embodiments, it is possible to determine whether the distance between the vehicle and the parking lot is less than a preset distance according to the current positioning of the vehicle. For example, if it is less than 50 meters, the memory parking function can be activated.

[0059] After the electronic device activates the memory parking function, it can perform the first-round trajectory planning, that is, the electronic device can start to determine the parking reference data for the first-round parking trajectory planning, and then perform the following step 202.

[0060] Step 202: Determine the first planned trajectory and the first occupancy prediction data based on the parking reference data and the preset parking planning model.

[0061] Among them, the first occupancy prediction data includes the predicted probability that each spatial unit in the future driving environment of the vehicle is occupied by an object. For example, the first occupancy prediction data includes the predicted probability that each spatial unit in the driving environment is occupied by an object within the next T1 seconds. The first occupancy prediction data may also include the type of object occupying the spatial unit.

[0062] For example, the object types include A, B, and C, and the first occupancy prediction data may include the probability that the spatial unit is occupied by A, the probability that the spatial unit is occupied by B, and the probability that the spatial unit is occupied by C.

[0063] The first planned trajectory is the planned trajectory in the future driving environment. For example, the first planned trajectory is the planned trajectory of the vehicle within the next T1 seconds.

[0064] The specific value of the above T1 can be set according to actual application requirements, and the embodiments of the present application do not limit this. For example, T1 can be set to 4 seconds. Then, after each round of trajectory planning, the trajectory planning result includes the planned trajectory for the next 4 seconds and the occupancy prediction data for the next 4 seconds.

[0065] Refer again to Figure 3 , Figure 3 which is a schematic structural diagram of a parking planning model provided by an embodiment of the present application. The parking planning model may include: an encoder, a self-vehicle trajectory decoder, and an occupancy network decoder (i.e., an OCC decoder). In some embodiments, step 2021 may be implemented in the following manner: Step 1: Encode the parking reference data based on the encoder to obtain encoded features.

[0066] In some embodiments, the encoder may be implemented based on a transformer. The electronic device may input the parking reference data into the encoder to obtain encoded features.

[0067] Among them, Transformer is a deep learning model architecture based on the attention mechanism. Transformer adopts an encoder-decoder architecture. The encoder is responsible for converting the input sequence into a series of feature representations. The encoder is composed of multiple identical encoder layers stacked together, and each encoder layer contains two sub-layers: the multi-head self-attention mechanism and the feed-forward neural network. The multi-head self-attention mechanism repeats the attention mechanism multiple times. Each time, the input is projected using a different linear transformation to obtain multiple different attention representations. Finally, these representations are concatenated and fused through a linear layer, so that the model can learn richer information from different subspaces. The feed-forward neural network is a simple two-layer fully-connected neural network with an activation function in the middle. The feed-forward neural network can perform further feature transformation and non-linear processing on the output of the multi-head self-attention mechanism to enhance the expression ability of the model. The decoder generates the target sequence based on the output of the encoder and the previously generated partial output sequence.

[0068] In some other embodiments, step 2021 can be implemented in the following manner: extracting the bird's-eye view feature and the trajectory feature in the parking reference data based on the encoder; performing feature fusion on the trajectory feature and the bird's-eye view feature based on the encoder to obtain the encoded feature.

[0069] For example, referring to Figure 4 as shown, the encoder can further include: a Lift-Splat-Shoot (LSS) model, a fully-connected layer, and a self-attention layer.

[0070] The step of the electronic device extracting the bird's-eye view feature (i.e., bev feature) and the trajectory feature in the parking reference data based on the encoder may include: Step 1. Extracting the bird's-eye view (Bird-Eye-View, BEV) feature in the parking reference data based on the LSS model. For example, the electronic device can input the driving environment data (such as sensor data) in the parking reference data into the LSS model to obtain the bird's-eye view feature.

[0071] The dimension of the bird's-eye view feature can be denoted as , where H is the number of spatial units in the horizontal direction, W is the number of spatial units in the vertical direction, D is the number of grids in the height direction, and E is the encoding dimension of each spatial unit. The spatial unit can be a voxel.

[0072] For example, H = W = 500, D = 10, E = 64, and the size of each grid is 0.2m, indicating that the field of view range is a space with a horizontal and vertical range of 100m 100m and a height of 2m.

[0073] Step 2. Extract the trajectory features from the parking reference data based on the fully connected layer.

[0074] For example, during the first-round trajectory prediction, the electronic device can input the target parking space in the parking reference data, etc. into the fully connected layer to obtain the trajectory features.

[0075] The trajectory features can be denoted as T E, where T represents the number of trajectory points. Assume that the driving environment data in the parking reference data is the driving environment data collected in the most recent T2 seconds (e.g., 4 seconds), and two frames of driving environment data are collected per second. Based on each frame of driving environment data, 1 trajectory feature corresponding to a trajectory point can be obtained. In this case, the size of T can be equal to twice T2, and T E can represent the trajectory in the previous T2 seconds.

[0076] During the trajectory prediction in subsequent rounds, the electronic device can input the target parking space in the parking reference data and the trajectory prediction result of the previous round, etc. into the fully connected layer to obtain the trajectory features.

[0077] After obtaining the trajectory features and the bird's-eye view features, the electronic device can perform feature fusion on the trajectory features and the bird's-eye view features based on the encoder to obtain the encoded features. For example, the following Step 3 can be executed: Step 3. Perform feature fusion on the trajectory features and the bird's-eye view features based on the self-attention layer to obtain the encoded features.

[0078] For example, the electronic device can determine the query vector based on the bird's-eye view features; determine the key vector and the value vector based on the trajectory features, and then input the query vector, the key vector, and the value vector into the self-attention layer to obtain the encoded features.

[0079] For example, the electronic device can reshape the dimension of the bird's-eye view features into two-dimensional features and regard it as the query vector q, regard the trajectory features as the key vector k and the value vector v, then input q, k, v into the self-attention layer for calculation, and reshape the calculation result into four-dimensional features This four-dimensional feature is the encoded feature.

[0080] The calculation formula of the self-attention layer is as follows: ; Among them, the encoded feature is denoted as attention, softmax is an activation function, q is the query vector, k is the key vector, and k T is the transpose of the key vector, d = E, and v is the value vector.

[0081] In this embodiment, feature fusion can reflect the relationship between trajectory features and bird's-eye view features, which is beneficial to enabling the encoded features to accurately reflect the driving environment, further improving the accuracy of subsequent occupancy space prediction, and enabling the vehicle to smoothly avoid obstacles when driving along the subsequently planned trajectory.

[0082] After obtaining the encoded features, the following steps 4 and 5 can be further executed: Step 4: The electronic device can perform trajectory planning for the vehicle based on the encoded features and the ego-vehicle trajectory decoder to obtain the first planned trajectory. Among them, the ego-vehicle trajectory encoder can adopt a transformer structure.

[0083] The electronic device can input the encoded features into the ego-vehicle trajectory encoder to obtain the first planned trajectory. The first planned trajectory can be denoted as T 2, representing T two-dimensional points in the world coordinate system. For example, T = 8, predicting one trajectory point every 0.5 seconds, T 2 represents predicting the ego-vehicle trajectory within the next 4s.

[0084] Step 5: The electronic device can also predict the probability that each spatial unit in the future driving environment of the vehicle is occupied by an object based on the encoded features and the occupancy network decoder to obtain the first occupancy prediction data.

[0085] The dimension of the first occupancy prediction data is , where is used to characterize the spatial units in the driving environment, and C represents the number of object types occupying the spatial units. For example, C = 9, and the object types can include drivable areas, walls, curbs, turnstiles, ground markings, columns, people, motor vehicles, non-motor vehicles and other various types of objects.

[0086] In some embodiments, after generating the first planned trajectory, the electronic device can control the driving trajectory of the vehicle based on the first planned trajectory so that the vehicle can smoothly drive into the target parking space.

[0087] The electronic device can also execute step 203.

[0088] Step 203: Detect whether the parking plan is completed.

[0089] For example, when the vehicle successfully drives into the target parking space, it can be determined that the parking plan is completed.

[0090] For another example, when the driver intervenes in parking, it can be determined that the parking plan is completed.

[0091] The above methods for determining the end of the parking plan are only examples. In actual application, it can be set according to requirements, and the embodiments of the present application do not limit this.

[0092] If the parking planning is not completed, execute step 204.

[0093] Step 204: Use the first occupancy prediction data, the first planned trajectory, the driving environment data collected in the next round, and the target parking space as the parking reference data for the next round of trajectory planning.

[0094] After step 204, step 202 can be continued, that is, perform the next round of parking trajectory planning.

[0095] That is, referring to Figure 5 As shown, in the embodiment of the present application, the trajectory planning results output in the previous round of trajectory planning (that is, the first planned trajectory and the first occupancy prediction data) can be combined with the driving environment data and the target parking space collected in the current acquisition cycle as the parking reference data in the next round of trajectory planning, and the parking reference data is input into the parking planning model until the parking planning is completed.

[0096] For example, assume that the electronic device performs one-second trajectory planning every A seconds. The time of the first round of trajectory planning is recorded as the 0th second. In the first round of trajectory planning, the electronic device can input the driving environment data newly collected in the most recent T2 seconds and the target parking space into the parking planning model to obtain the trajectory planning result 1. The trajectory planning result 1 is the first planned trajectory and the first occupancy prediction data corresponding to the next T1 seconds. Then, the electronic device can control the vehicle to drive according to the first planned trajectory.

[0097] When reaching the A-th second, the electronic device can enter the second round of trajectory planning. For example, input the driving environment data newly collected in the most recent T2 seconds, the target parking space, and the trajectory planning result 1 into the parking planning model to obtain the trajectory planning result 2. Then, the electronic device can control the vehicle to drive according to the first planned trajectory in the trajectory planning result 2.

[0098] When reaching the 2A-th second, the electronic device can enter the third round of trajectory planning. For example, the electronic device inputs the driving environment data newly collected in the most recent T2 seconds, the target parking space, and the trajectory planning result 2 into the parking planning model to obtain the trajectory planning result 3. Then, the electronic device can control the vehicle to drive according to the first planned trajectory in the trajectory planning result 3.

[0099] After the parking planning is completed, the following step 205 can be executed.

[0100] Step 205: Exit the memory parking function.

[0101] In some embodiments, the training method of the above preset parking planning model may include two-stage training.

[0102] Specifically, referring toFigure 6 As shown in Figure 6 , the training method of the parking planning model may include: Step 601: Train the occupancy network module.

[0103] Among them, the occupancy network module includes an encoder and an occupancy network decoder in the parking planning model to be trained.

[0104] In some embodiments, step 601 may be implemented through the following steps a1 to a3: Step a1: The training device may obtain a first data set, where the first data set includes driving environment samples and corresponding actual spatial occupancy data.

[0105] The actual spatial occupancy data includes the actual probability that each spatial unit, such as a voxel, is occupied by an object in the future driving environment of the vehicle. The actual spatial occupancy data may also include the object type corresponding to the object occupying the spatial unit, such as drivable area, wall, curb, gate, ground marking, column, person, motor vehicle, non-motor vehicle, etc.

[0106] For example, the training device may collect in-vehicle sensor data to obtain driving environment samples and label the actual spatial occupancy data in the driving environment samples. For example, when the object type displayed by voxel 1 is a wall, the actual probability that voxel 1 is occupied by the wall may be labeled as 1, and the actual probability that voxel 1 is occupied by other objects, such as motor vehicles, columns, etc., may be labeled as 0.

[0107] In some embodiments, the first data set may further include a parking space label in the driving environment sample, and the parking space label may include the four corner points of the parking space in the driving environment sample, whether the parking space can be parked in, but is not limited thereto.

[0108] Step a2: Determine second occupancy prediction data based on the driving environment samples and the occupancy network module, where the occupancy network module includes an encoder and an occupancy network decoder in the parking planning model to be trained.

[0109] In some embodiments, the occupancy network module may further detect parking space data based on the driving environment samples to obtain predicted parking space data, and the predicted parking space data may include the four corner points of the parking space in the predicted driving environment sample, whether the parking space can be parked in, but is not limited thereto.

[0110] Step a3: Input the actual spatial occupancy data and the second occupancy prediction data into a preset first loss function to obtain a first loss value.

[0111] In some embodiments, the first loss function may be the following occupancy network loss function L OCC : Among them, y i represents the actual probability of occupancy of each spatial unit by an object and the object type (i.e., the actual spatial occupancy data of the spatial unit), and y i can be encoded using one-hot encoding. That is, if the object occupying the spatial unit is of category i, then yi = 1; if the object occupying the spatial unit is not of category i, then yi = 0, and pi is the predicted probability that the object occupying the spatial unit is of category i (i.e., the second occupancy prediction data of the spatial unit).

[0112] In some other embodiments, the first loss function can be the following L total loss function.

[0113] Among them, L lot is the parking space detection loss function, and L occ is the occupancy network loss function as described above.

[0114] The parking space detection loss function L lot is as follows: Among them, p i is the true three-dimensional coordinate (x, y, z) of each parking space corner point in the world coordinate system, is the coordinate of the parking space corner point predicted by the occupancy network module in the world coordinate system. is the attribute of each parking space predicted by the occupancy network module, including the parkable or non-parkable attribute; h k represents the true attribute of each parking space, which is encoded using one-hot encoding. That is, when it is of category k (such as parkable), h k = 1, and when the parking space is non-parkable, otherwise h k = 0.

[0115] In some embodiments, if it is determined based on the first loss value that the first loss function does not converge, then the model parameters of the occupancy network module can be updated, and then the occupancy network module is trained again until the first loss function converges, and then the following step 602 can be executed.

[0116] Step 602: Train the parking planning model to be trained to obtain the preset parking planning model.

[0117] In some embodiments, step 602 can be implemented in the following manner: Step b1: Obtain a second data set, where the second data set includes the first data set and the actual driving trajectory corresponding to the driving environment sample.

[0118] For example, the driving trajectory of the host vehicle can be marked in the driving environment sample. Suppose the driving trajectory is N coordinate points, and the value of N can be set according to the actual application data requirements. For example, N can be set to 60. If one trajectory point is marked every 0.5 s, the actual driving trajectory can include the trajectory within 30 seconds.

[0119] Step b2: Input the driving environment sample into the parking planning model to be trained to obtain a second planned trajectory and third occupancy prediction data.

[0120] Step b3: Input the second planned trajectory, the actual driving trajectory, the third occupancy prediction data, and the actual space occupancy data into a preset second loss function to obtain a second loss value.

[0121] The second loss function can refer to the following L total as shown: ; where L cse is the cross-entropy loss function, which can represent the cross-entropy loss between the third occupancy prediction data and the actual space occupancy data. L 2 refers to the L 2 loss function. tragt is the actual driving trajectory point at each moment, trapred is the predicted driving trajectory point at each moment (i.e., in the second planned trajectory, the planned trajectory point corresponding to this moment), T represents the number of driving trajectory points in the driving trajectory, and i represents the i-th trajectory point in the driving trajectory.

[0122] Step b4: Update the model parameters of the parking planning model to be trained based on the second loss value to obtain the preset parking planning model.

[0123] Specifically, if it is determined based on the second loss value that the second loss function has not converged, update the model parameters of the parking planning model to be trained until the second loss function converges, and use the current parking planning trajectory model as the preset parking planning trajectory model.

[0124] In the embodiment of the present application, at least one round of trajectory planning is performed during parking. The first round of trajectory planning is based on the collected driving environment data and the target parking space. Each subsequent round of trajectory planning combines the current latest driving environment dynamics, as well as the trajectory planning result of the previous round (i.e., the first planned trajectory) and the space occupancy situation of the previous round (i.e., the first occupancy prediction data), so that the vehicle can anticipate the obstacle occupancy risk in the driving environment in advance, timely capture the changes of obstacles during parking, and thus timely and flexibly adjust the parking trajectory to avoid obstacles, making the parking path always fit the actual driving scenario, improving the planning accuracy, ensuring that the vehicle can steadily move towards the target parking space in various scenarios, and enhancing the parking safety.

[0125] In addition, the embodiments of the present application can plan the parking trajectory based on the driving environment data collected by vehicle-mounted sensors, reduce the dependence on external facilities configured in the parking lot, and can reduce the cost of transforming the parking lot infrastructure.

[0126] The parking planning model of the embodiments of the present application adopts an end-to-end training method, mapping from data input to control output, and outputting scene perception (i.e., spatial occupancy status) and planning trajectory together, which is beneficial to improving the response speed of parking planning.

[0127] That is, the embodiments of the present application can perform accurate parking trajectory planning in a complex driving environment and ensure the response speed of trajectory planning.

[0128] Figure 7 It is a schematic diagram of an embodiment of an electronic device of the present application.

[0129] The electronic device 100 includes a memory 20, a processor 30, and a computer program 40 stored in the memory 20 and executable on the processor 30. When the processor 30 executes the computer program 40, it implements the steps in the above-mentioned embodiments of the parking planning method, such as Figure 2 the steps 201 to 205 shown.

[0130] Exemplarily, the computer program 40 can also be divided into one or more modules / units, and the one or more modules / units are stored in the memory 20 and executed by the processor 30. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 40 in the electronic device 100.

[0131] Those skilled in the art can understand that the schematic diagram is only an example of the electronic device 100, does not constitute a limitation on the electronic device 100, and may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device 100 may further include input / output devices, network access devices, buses, etc.

[0132] The processor 30 may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, a single-chip microcomputer, or the processor 30 may also be any conventional processor, etc.

[0133] The memory 20 can be used to store computer programs 40 and / or modules / units. The processor 30 realizes various functions of the electronic device 100 by running or executing the computer programs and / or modules / units stored in the memory 20, and by calling the data stored in the memory 20. The memory 20 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created according to the use of the electronic device 100 (such as audio data). In addition, the memory 20 may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0134] If the modules / units integrated in the electronic device 100 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0135] Reference Figure 8 As shown, an embodiment of this application also provides a vehicle, and the vehicle may include the above-described electronic device.

[0136] In some embodiments, the vehicle may also be configured with sensors, such as image sensors and radar sensors, to obtain driving environment data.

[0137] In several embodiments provided by this application, it should be understood that the disclosed electronic device and method can be implemented in other ways. For example, the above-described electronic device embodiments are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation.

[0138] In addition, each functional unit in various embodiments of this application can be integrated in the same processing unit, or each unit can exist physically alone, or two or more units can be integrated in the same unit. The above integrated unit can be implemented in the form of hardware or in the form of a combination of hardware and software functional modules.

[0139] For those skilled in the art, it is obvious that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or electronic devices stated in the claims of the electronic device can also be implemented by the same unit or electronic device through software or hardware. The words such as first and second are used to represent names and do not represent any specific order.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A parking planning method, characterized in that: The parking planning method is used to perform at least one round of trajectory planning in a time sequence during a vehicle parking process, and the parking planning method includes: Determining parking reference data, wherein, in the case of performing trajectory planning in the first round, the parking reference data includes driving environment data collected in the first round and a target parking space; Determining a first planning trajectory and first occupancy prediction data based on the parking reference data and a preset parking planning model, wherein the first occupancy prediction data includes a predicted probability of each spatial unit being occupied by an object in a future driving environment of the vehicle; If the parking planning is not completed, the first occupancy prediction data, the first planned trajectory, the driving environment data collected in the next round, and the target parking space are used as parking reference data for the next round of trajectory planning, and the steps of determining the first planned trajectory based on the parking reference data and the preset parking planning model and predicting the space occupancy status are continued until the parking planning is completed.

2. The parking planning method according to claim 1, characterized in that: The preset parking planning model includes an encoder, an ego vehicle trajectory decoder and an occupancy network decoder; The determining the first planning trajectory and the first occupancy prediction data based on the parking reference data and a preset parking planning model includes: encoding the parking reference data based on the encoder to obtain a coding feature; Performing trajectory planning for the vehicle based on the encoding feature and the ego-vehicle trajectory decoder to obtain the first planned trajectory; The probability of each space unit in the future driving environment of the vehicle being occupied by an object is predicted based on the coding features and the occupancy network decoder to obtain the first occupancy prediction data.

3. The parking planning method according to claim 2, characterized in that: The step of encoding the parking reference data based on the encoder to obtain a coding feature includes: extracting bird's-eye view features and trajectory features in the parking reference data based on the encoder; The trajectory feature and the bird's-eye view feature are fused based on the encoder to obtain the encoding feature.

4. The parking planning method according to claim 3, characterized in that: The encoder includes a self-attention layer, and the feature fusion of the trajectory feature and the bird's-eye view feature based on the encoder to obtain the encoded feature includes: Determining a query vector based on the bird's-eye view feature; Determining a key vector and a value vector based on the trajectory features; The query vector, the key vector, and the value vector are input into the self-attention layer to obtain the encoded features.

5. The parking planning method according to claim 2, characterized in that: The training steps of the preset parking planning model include: Acquire a first data set, the first data set including a driving environment sample and actual space occupancy data corresponding to the driving environment sample, the actual space occupancy data including an actual probability of each space unit being occupied by an object in the future driving environment of the vehicle; Determining second occupancy prediction data based on the driving environment sample and an occupancy network module, wherein the occupancy network module includes an encoder and an occupancy network decoder in a parking planning model to be trained; Inputting the actual space occupancy data and the second occupancy prediction data into a preset first loss function to obtain a first loss value; If it is determined based on the first loss value that the first loss function converges, the parking planning model to be trained is trained to obtain the preset parking planning model.

6. The parking planning method according to claim 5, characterized in that: The training of the parking planning model to be trained to obtain the preset parking planning model includes: Acquire a second data set, where the second data set includes the first data set and an actual driving trajectory corresponding to the driving environment sample; Inputting the driving environment sample into the parking planning model to be trained to obtain a second planning trajectory and third occupancy prediction data; Inputting the second planned trajectory, the actual driving trajectory, the third occupancy prediction data and the actual space occupancy data into a preset second loss function to obtain a second loss value; The model parameters of the parking planning model to be trained are updated based on the second loss value to obtain the preset parking planning model.

7. The parking planning method according to any one of claims 1 to 5, characterized in that: Before determining the parking reference data, the method further includes: Collecting driving environment data at preset intervals; If the driving environment data collected in the current cycle matches the pre-stored parking lot mapping data, the memory parking function of the vehicle is activated; The determining of parking reference data includes: The parking reference data are determined when a memory parking function of the vehicle is activated.

8. An electronic device, comprising a processor and a memory, characterized in that: The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the parking planning method according to any one of claims 1 to 7.

9. A vehicle, characterized in that: The vehicle comprises the electronic device as claimed in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the parking planning method according to any one of claims 1 to 7.

Images