Training a path-planning agent for automatically parking a motor vehicle, planning a path for automatically parking a motor vehicle, and automatically parking a motor vehicle
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- VALEO SCHALTER & SENSOREN GMBH
- Filing Date
- 2026-01-21
- Publication Date
- 2026-07-30
Smart Images
Figure EP2026051384_30072026_PF_FP_ABST
Abstract
Description
[0001] 2024PF01881
[0002] 1
[0003] Training a path planning agent for the automatic parking of a motor vehicle, planning a path for the automatic parking of a motor vehicle, and automatic parking of a motor vehicle.
[0004] The present invention relates to a computer-implemented training method for training a path planning agent for the automatic parking of a motor vehicle by reinforcement learning, a computer-implemented method for planning a path for the automatic parking of a motor vehicle, wherein a path planning agent is trained and used using such a computer-implemented training method, and a method for the automatic parking of a motor vehicle, wherein such a computer-implemented method is carried out. The invention further relates to a data processing system for carrying out such a computer-implemented method or training method, an arrangement comprising a data processing system and an electronic vehicle guidance system, and corresponding computer program products.
[0005] In automated driving systems, numerous well-known algorithms exist for path planning, also known as trajectory planning. One class of such path planning algorithms is based on so-called agents, which have been trained using reinforcement learning.
[0006] The publication Y. Chen et al.: “Deep Reinforcement Learning in Autonomous Car Path Planning and Control: A Survey” (arXiv:2404.00340v1) provides an overview of literature on Deep Reinforcement Learning (DRL) in the field of path planning and control for vehicles.
[0007] Known path planning algorithms for automatic parking may exhibit reduced accuracy, particularly in difficult or complex parking scenarios, due to inaccuracies or noise in the underlying environmental data provided by the vehicle's environmental sensor systems, which may have been preprocessed, and so on. 2024PF01881
[0008] 2
[0009] It is an object of the present invention to increase the accuracy of path planning in the automatic parking of a motor vehicle.
[0010] This problem is solved by the subject matter of the independent claim. Further embodiments and preferred developments are the subject matter of the dependent claims.
[0011] The invention is based on the idea of providing an orientation term in the reward function when training a path planning agent for the automatic parking of a motor vehicle by reinforcement learning, which corresponds to a deviation of a vehicle end orientation simulated by the path planning agent from a predetermined vehicle target orientation.
[0012] According to one aspect of the invention, a computer-implemented training method for training a path planning agent for the automatic parking of a motor vehicle by reinforcement learning is provided, based on a plurality of training scenarios, in particular predefined training scenarios. Each training scenario of the plurality of training scenarios is characterized by a respective vehicle starting position, a respective vehicle starting orientation, a respective vehicle target position, and a respective vehicle target orientation.For each of the numerous training scenarios, the path planning agent generates a simulated path from the respective vehicle starting position with the respective vehicle starting orientation to a simulated vehicle ending position with a simulated vehicle ending orientation, based on a reinforcement learning strategy. At least one value of a reward function for the simulated path is calculated, and the reinforcement learning strategy is updated depending on this at least one value of the reward function. The reward function depends on an orientation term that corresponds to a deviation of the simulated vehicle ending orientation from the respective vehicle target orientation.
[0013] Unless otherwise specified, all steps of the computer-implemented training procedure can be performed by a data processing system comprising at least one data processing device. In particular, the at least one data processing device is configured or adapted to perform the steps of the computer-implemented training procedure. For this purpose, the at least one data processing device may, for example, store a computer program containing commands which, when executed by the at least one data processing device,
[0014] 3
[0015] at least one data processing device must be made to execute the computer-implemented training procedure. The terms "data processing system" and "at least one data processing device" can be used interchangeably.
[0016] In the event that the at least one data processing device comprises two or more data processing devices, certain steps performed by the at least one data processing device can be understood, for example, as different data processing devices performing different steps or different parts of a step. In particular, it is not necessary for each data processing device to perform the steps completely. In other words, the execution of the steps can be distributed among the two or more data processing devices.
[0017] In reinforcement learning, the so-called environment comprises a set of states, particularly for the vehicle, and a set of possible actions, also particularly for the vehicle. The path-planning agent can interact with the environment at discrete time points. At each time point, one of the environment's states exists, and the path-planning agent selects one of the actions.
[0018] The reinforcement learning strategy, also known as a policy, is a collection of probability measures, each representing a preference of the path-planning agent to select a specific action at a given time and under a given state. Subsequently, the environment outputs a follow-up state depending on the selected action, and these steps are repeated until the entire path for a training scenario has been simulated. The reward function is evaluated, in particular, after the entire path has been simulated. However, depending on the specific implementation of the reinforcement learning strategy, the reward function, or a portion thereof, can also be evaluated after each repetition.
[0019] The reward function can be positive or negative, or it can include positive and / or negative terms. Positive terms can be understood as positive rewards or rewards in the narrower sense, whereas negative terms can be understood as negative rewards or punishments.
[0020] The states each include, in particular, a position and an orientation of the vehicle. Furthermore, the states can include the vehicle's target position and the 2024PF01881
[0021] 4
[0022] The vehicle's objective orientation for the respective training scenario must be included. Furthermore, the states can also include the positions and, where applicable, the shapes of obstacles and / or drivable areas for the vehicle. Additionally, the states can include, for example, the steering position or wheel position of the vehicle and / or the direction of travel of the vehicle, in particular whether the vehicle is traveling forwards or backwards.
[0023] The actions may include, for example, maintaining or changing the direction of travel of the motor vehicle and / or maintaining or changing the steering position or wheel position of the motor vehicle.
[0024] A position for the motor vehicle can be understood here and in the following as the position of a predefined point, in particular on the motor vehicle.
[0025] For example, the center point of a front or rear axle of the motor vehicle can serve as a reference point for the position.
[0026] An orientation for the vehicle can be understood here and in the following as the direction of a vector originating from the point that defines the vehicle's position, or from another predefined point. The deviation of the simulated vehicle end orientation from the respective vehicle target orientation thus corresponds, in particular, to an angle between the two corresponding vectors.
[0027] By using a large number of training scenarios, the reward function can be optimized by varying the reinforcement learning strategy across these scenarios. This allows the trained path planning agent, acting according to the trained reinforcement learning strategy, to then be used for productive path planning. Specifically, the path planning agent generates an action corresponding to a segment of a path for the vehicle. It can then generate another action corresponding to another segment of the path, and so on, until the vehicle's target position and orientation are reached. The resulting path is essentially a sequence of all the segments generated in this way. Known reinforcement learning methods can be used to optimize or update the reinforcement learning strategy based on the value of the reward function.
[0028] 5
[0029] In some embodiments, a training scenario may include additional content beyond that mentioned, in particular the positions, orientations, shapes, and / or dimensions of obstacles, other vehicles, walls, curbs, and so on. A training scenario may, in particular, represent or model the environment perceptible to a motor vehicle, especially as perceived by at least one environmental sensor system.
[0030] Since the path planning agent is to be trained for the automatic parking of the vehicle, the training scenarios are parking scenarios. These are characterized in particular by the fact that the vehicle's starting position, starting orientation, destination position, and destination orientation are predefined, as well as by the positions and, if applicable, shapes of obstacles and / or a drivable area for the vehicle. The various training scenarios differ primarily in one or more of these aspects. Therefore, a large number of training scenarios can be generated and used without significant computational effort, thus achieving high reliability and accuracy in path planning by the trained path planning agent. In practice, the scenario may vary over time, for example, due to errors in environmental perception and / or odometry.Because the path planning agent was confronted with a large number of training scenarios during training, it can still reliably handle the current scenario.
[0031] According to the invention, the reward function includes the orientation term. This ensures that, through repeated updates of the reinforcement learning strategy, the trained path planning agent prioritizes actions that ultimately result in the smallest possible deviation between the actual vehicle orientation and the predetermined target orientation, depending on the value of the reward function. This reduces the risk of collisions between the vehicle and obstacles during productive use of the path planning agent and increases the overall accuracy of path planning during automatic vehicle parking.
[0032] It should be noted that the reward function may also include other terms, such as the number of changes of direction along the simulated path and / or the total length of the simulated path and / or the total number of time steps required for the simulated path, from 2024PF01881
[0033] 6
[0034] The reward function can depend on a minimum or average distance of the vehicle from obstacles along the simulated path, and so on. For example, the reward function can include a weighted sum of the terms considered.
[0035] According to at least one embodiment, each training scenario is characterized by a vehicle anchor position and a vehicle anchor orientation. The vehicle anchor position is shifted relative to the vehicle target position by a predefined positional offset parallel to the respective vehicle target orientation. The vehicle anchor orientation is the same as the vehicle target orientation. The respective simulated path has an intermediate vehicle state characterized by an intermediate vehicle position with minimal deviation from the respective vehicle anchor position and a corresponding intermediate vehicle orientation. The reward function depends on an anchor term that depends on the minimal deviation of the intermediate vehicle position from the respective vehicle anchor position and / or on a deviation of the intermediate vehicle orientation from the respective vehicle anchor orientation.
[0036] For example, the positional offset depends on the specific training scenario. In particular, the positional offset can differ for different training scenarios. However, it is also possible that the positional offset is identical for some or all of the training scenarios.
[0037] The intermediate vehicle position corresponds, for example, to a position of the vehicle along the simulated path whose deviation from the respective vehicle anchor position is smaller than all other positions of the vehicle along the simulated path. The corresponding intermediate vehicle orientation corresponds to the orientation of the vehicle at the intermediate vehicle position.
[0038] In such embodiments, the path planning agent is therefore rewarded if the simulated path exhibits a particularly small minimum deviation of the vehicle's intermediate position from its respective anchor position and / or if the simulated path exhibits a particularly small deviation of the vehicle's intermediate orientation from its anchor orientation, i.e., from its destination orientation. In particular, the reward can be greater the smaller the corresponding deviations are, and conversely, the penalty can be less severe the smaller the corresponding deviations are. 2024PF01881
[0039] 7
[0040] In the productive application of the path planning agent, paths are therefore preferentially generated where the vehicle already has an orientation as close as possible to the target orientation before reaching its destination position. This increases the flexibility of the path planning with regard to possible variations in environmental conditions or environmental data acquired by the vehicle's sensors, and any necessary planning corrections, etc. Overall, the reliability and / or accuracy of the path planning are further increased.
[0041] According to at least one embodiment, the reward function depends on a target position term that corresponds to a deviation of the simulated vehicle end position from the respective vehicle target position, i.e., in particular, a distance of the simulated vehicle end position from the respective vehicle target position.
[0042] Accordingly, simulated paths that end as close as possible to the respective vehicle target position are preferred. This further increases the accuracy of path planning.
[0043] According to at least one embodiment, the reward function depends on a path length term that corresponds to the path length of the respective simulated path.
[0044] The path length can correspond to a geometric length of the simulated path, or to a number of segments from which the simulated path is composed, i.e., in particular, a number of discrete time steps required to generate the simulated path, or a combination of the geometric path length and the number of segments.
[0045] In particular, this prioritizes simulated paths with shorter path lengths. This increases the efficiency of path planning and the resulting paths for parking the vehicle.
[0046] According to at least one embodiment, the reward function depends on a robustness term that corresponds to a minimum distance of the simulated path from at least one obstacle according to the respective training scenario and / or an average distance of the simulated path from the at least one obstacle according to the respective training scenario. 2024PF01881
[0047] 8
[0048] This prioritizes paths where a collision with obstacles is particularly unlikely, further increasing the safety of path planning.
[0049] According to at least one embodiment, the path planning agent incorporates an artificial neural network.
[0050] This enables the path planning agent to process particularly complex training scenarios and to handle particularly complex environmental scenarios after training.
[0051] According to a further aspect of the invention, a computer-implemented method for planning a path for the automatic parking of a motor vehicle is provided. A path planning agent is trained using a computer-implemented training method according to the invention. Environmental data representing the current environment of the motor vehicle are received, in particular from at least one environmental sensor system of the motor vehicle. A further vehicle target position and a further vehicle target orientation are determined based on the environmental data. The trained path planning agent determines at least one path for the motor vehicle from a current vehicle position with a current vehicle orientation to the further vehicle target position with a further vehicle target orientation.A path for use in the automatic parking of the motor vehicle is selected from the at least one path, in particular according to a predefined rule.
[0052] Unless otherwise specified, all steps of the computer-implemented method can be performed by another data processing system comprising at least one further data processing device, in particular by another data processing system of the motor vehicle. Specifically, the at least one further data processing device is configured or adapted to perform the steps of the computer-implemented method. For this purpose, the at least one further data processing device can, for example, store another computer program containing instructions which, when executed by the at least one further data processing device, cause the at least one further data processing device to execute the computer-implemented method. 2024PF01881
[0053] 9
[0054] All further data processing devices of the at least one additional data processing device can be part of the motor vehicle. However, it is also possible that all further data processing devices of the at least one additional data processing device are part of an external computing system outside the motor vehicle, for example, a mobile electronic device, a backend server, or a cloud computing system. It is also possible that the at least one additional data processing device comprises both at least one vehicle data processing device of the motor vehicle and at least one external data processing device of the external computing system.The at least one vehicle data processing device may, for example, comprise one or more electronic control units (ECUs), and / or one or more zone control units (ZCLIs), and / or one or more domain control units (DCLIs) of the motor vehicle.
[0055] In the event that the at least one additional data processing device includes two or more further data processing devices, certain steps performed by the at least one further data processing device can be understood, for example, as different further data processing devices performing different steps or different parts of a step. In particular, it is not necessary for each further data processing device to perform the steps completely. In other words, the performance of the steps can be distributed among the two or more further data processing devices.
[0056] Each embodiment of the computer-implemented method results in a corresponding embodiment of a method for planning a path for the automatic parking of a motor vehicle, which is not purely computer-implemented, by including corresponding steps for generating the environmental data using the at least one environmental sensor system.
[0057] Receiving or receiving data or information can, for example, involve receiving or receiving the data, particularly by the relevant data processing system, from a sending entity; reading the data from a data storage device; receiving or receiving a data stream containing the data; or extracting the data from the data stream, and so on. This can be achieved, in particular, through wired or wireless data transmission. Specifically, data transmission 2024PF01881
[0058] 10
[0059] between a hardware and / or software interface of the sending entity and a hardware and / or software interface of the data processing system.
[0060] An environmental sensor system can be understood, for example, as a sensor system capable of generating sensor data or sensor signals that map, represent, or reproduce an environment. In particular, the ability to detect electromagnetic or other signals from the environment is not sufficient to classify a sensor system as an environmental sensor system. For example, cameras, especially those operating in the visible or infrared range, radar systems, lidar systems, or ultrasonic sensor systems can be considered environmental sensor systems.
[0061] Depending on the implementation of the at least one environmental sensor system, the environmental data may therefore include, for example, one or more camera images, thermal images, lidar point clouds, lidar depth images, radar data, ultrasound data and / or corresponding images generated on the basis of the radar or ultrasound data.
[0062] The further vehicle target position corresponds in particular to a position in a parking space for the motor vehicle and, for example, the further vehicle target orientation corresponds to a direction or orientation of the parking space.
[0063] The current vehicle position can be determined explicitly based on environmental data and / or using a self-localization algorithm. Similarly, the current vehicle orientation can be determined. Alternatively, the current vehicle position and orientation can be determined within a reference coordinate system, where the current vehicle position corresponds to a coordinate origin of the reference coordinate system. The current vehicle orientation can, for example, be at an angle of 0° with a predefined axis of the reference coordinate system, in particular a longitudinal axis of the reference coordinate system.The reference coordinate system can, for example, be a coordinate system fixed to the vehicle, the origin of which is located at the point mentioned above that defines the position of the vehicle, i.e., for example, at the center of the front axle or the rear axle of the vehicle. 2024PF01881.
[0064] 11
[0065] The reference coordinate system can, for example, be a Cartesian coordinate system, where one axis of the reference coordinate system corresponds to a longitudinal axis of the motor vehicle and another axis of the reference coordinate system corresponds to a transverse axis of the motor vehicle perpendicular to it.
[0066] For example, the path planning agent can be provided with multiple boundary conditions or sets of boundary conditions to determine at least one path. For each boundary condition or set of boundary conditions, the path planning agent can then, for example, determine a path for the vehicle from its current position with its current orientation to its next destination position with its next destination orientation. The boundary conditions might, for example, require that the vehicle's first maneuver be forward or reverse, or something similar.
[0067] Various criteria can be considered as a rule for selecting the path used in the automatic parking of a vehicle. The selection can, for example, be random. It is also possible to select the shortest path among the at least one available path, or the path with the fewest changes of direction of the vehicle along that path, or the path with the smallest deviation of the final vehicle orientation from the subsequent vehicle destination orientation. A more complex rule can also be used that considers two or more of these and / or other criteria. It should be noted that in the special case where the at least one available path consists of exactly one path, selecting the path is equivalent to selecting exactly that one path.
[0068] The selected path can be considered the result of the path planning process. The vehicle can then, in particular, be automatically guided along the selected path to park.
[0069] According to at least one embodiment, the at least one path includes at least two paths, each of which is characterized by a respective final vehicle position and a respective final vehicle orientation. For each of the at least two paths, a misorientation value is calculated, which corresponds to a deviation of the respective final vehicle orientation from the subsequent vehicle target orientation. The path to 2024PF01881
[0070] 12
[0071] The choice of use in the automatic guidance system of the motor vehicle depends on the calculated misorientation values.
[0072] In other words, the deviation of the vehicle's orientation from the corresponding target orientation at the end of the path is taken into account not only during the training of the path planning agent, but also during its productive use. This results in particularly suitable paths for parking or automatic parking of the vehicle.
[0073] For example, the path that has at least two paths can be selected as the path for use in automatic parking of the motor vehicle, the one that has the smallest misorientation value.
[0074] Alternatively, for each of the two possible paths, a weighted sum can be calculated from the respective misorientation value and at least one other characteristic value of that path. The path to be used for automatic parking of the vehicle can then be selected based on the calculated weighted sums, for example, the path with the lowest weighted sum.
[0075] The at least one further characteristic value can, for example, include the number of changes in direction of the motor vehicle during the corresponding path and / or the deviation of the final vehicle orientation from the further vehicle target orientation, and so on.
[0076] According to at least one embodiment, a further vehicle anchor position, which is shifted parallel to the further vehicle target orientation by a predefined further position offset relative to the further vehicle target orientation, and a further vehicle anchor orientation, which is equal to the further vehicle target orientation, are determined based on the environmental data. The at least one path for the vehicle is determined by the trained vehicle planning assistant from the current vehicle position with the current vehicle orientation, via the further vehicle anchor position with the further vehicle anchor orientation, to the further vehicle target position with the further vehicle target orientation. 2024PF01881
[0077] 13
[0078] It should be noted that such embodiments are possible regardless of whether the anchor position and anchor orientation were used during the training of the path planning agent.
[0079] In such embodiments, the probability is further increased that the further vehicle target orientation is achieved particularly accurately by the motor vehicle.
[0080] Further embodiments of the computer-implemented method according to the invention follow directly from the various configurations of the computer-implemented training method according to the invention and vice versa.
[0081] In particular, individual features and corresponding explanations as well as advantages regarding the various embodiments of the computer-implemented method according to the invention can be transferred analogously to corresponding embodiments of the computer-implemented training method according to the invention.
[0082] According to a further aspect of the invention, a method for automatically parking a motor vehicle is described. In this method, a computer-implemented procedure according to the invention is carried out for planning a path for automatically parking a motor vehicle, and the motor vehicle is parked automatically by being guided along the selected path.
[0083] For example, at least one control signal is generated for automatically guiding the vehicle along the selected path. To automatically guide the vehicle along the selected path, at least one actuator of the vehicle is controlled depending on this at least one control signal.
[0084] The at least one actuator can, for example, include one or more brake actuators and / or one or more steering actuators and / or one or more drive motors of the vehicle. Based on the at least one control signal, the one or more actuators can influence the longitudinal and / or lateral steering of the vehicle to automatically guide it along the selected path.
[0085] According to another aspect of the invention, a data processing system is provided which is configured to carry out a computer-implemented training method according to the invention. 2024PF01881
[0086] 14
[0087] The terms "data processing system" and "at least one data processing device" may be used interchangeably within the scope of this disclosure. In this disclosure, a data processing device may, for example, be understood as a device with processing circuits for processing data. A data processing device can thus perform arithmetic operations to process data. Indexed access to a data structure, such as a lookup table (LUT) or a database, may also be considered an arithmetic operation. Similarly, data processing that is partially or fully implemented in hardware may be considered an arithmetic operation.
[0088] A data processing device may, in particular, comprise one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more systems-on-a-chip (SoCs). A data processing device may also comprise one or more processors, for example, one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors (DSPs). The data processing device may also comprise a physical or virtual cluster of computers or other devices of the aforementioned type.
[0089] A data processing device may also include one or more hardware and / or software interfaces, for example for receiving and / or providing data.
[0090] A data processing device may also include one or more storage devices. A storage device may be implemented as volatile memory, such as dynamic random access memory (DRAM) or static random access memory (SRAM), or as non-volatile data storage, such as read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or flash EEPROM, ferroelectric random access memory (FRAM), magnetoresistive random access memory (MRAM), or phase-change random access memory (PCRAM).
[0091] According to another aspect of the invention, a further data processing system is specified which is configured to implement a computer-implemented system according to the invention.
[0092] 15
[0093] To carry out a procedure for planning a path for the automatic parking of a motor vehicle.
[0094] According to a further aspect of the invention, an arrangement comprising a data processing system configured to carry out a computer-implemented training method according to the invention and an electronic vehicle guidance system for a motor vehicle is specified.The electronic vehicle guidance system includes a further data processing system that is configured to receive environmental data representing the current environment of the vehicle, to determine a further vehicle target position and a further vehicle target orientation based on the environmental data, to determine at least one path for the vehicle from a current vehicle position with a current vehicle orientation to the further vehicle target position with the further vehicle target orientation using the trained path planning agent, to select a path for use in automatically parking the vehicle from the at least one path and to automatically guide the vehicle along the selected path.
[0095] In particular, the additional data processing system stores the trained path planning agent.
[0096] An electronic vehicle control system (EVS) can be understood as an electronic system designed to control a vehicle fully automatically or autonomously, in particular without requiring any intervention from a driver. The vehicle automatically performs all necessary functions, such as steering, braking, and / or acceleration maneuvers, monitoring and recording road traffic, and reacting accordingly. Specifically, the EVS can implement a fully automatic or fully autonomous driving mode of the vehicle according to Level 5 of the SAE J3016 classification. An EVS can also be understood as an advanced driver assistance system (ADAS), which supports the driver during partially automated or semi-autonomous driving.In particular, the electronic vehicle guidance system can implement a partially automated or semi-autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification. Here and in the following, "SAE J3016" refers to the corresponding standard in the April 2021 version .2024PF01881.
[0097] 16
[0098] At least partially automated vehicle control can therefore include driving the vehicle in accordance with a fully automated or fully autonomous driving mode of Level 5 according to SAE J3016. At least partially automated vehicle control can also include driving the vehicle in accordance with a partially automated or semi-autonomous driving mode according to Levels 1 to 4 of SAE J3016.
[0099] According to at least one embodiment, the electronic vehicle guidance system has at least one environmental sensor system configured to generate environmental data.
[0100] Further embodiments of the arrangement according to the invention follow directly from the various configurations of the computer-implemented training method according to the invention, and vice versa. In particular, individual features and corresponding explanations as well as advantages relating to the various embodiments of the training method according to the invention can be transferred analogously to corresponding embodiments of the arrangement according to the invention.
[0101] According to another aspect of the invention, a computer program with instructions is specified. When the instructions are executed by a data processing system, the instructions cause the data processing system to perform a computer-implemented training method according to the invention.
[0102] The instructions can be provided, for example, as program code. This program code can be provided, for example, as binary code or assembly language, and / or as source code in a programming language such as C, and / or as a program script, such as Python.
[0103] According to another aspect of the invention, a further computer program with further instructions is specified. When the further instructions are executed by a further data processing system, the further instructions cause the further data processing system to carry out a computer-implemented method according to the invention for planning a path for automatically parking a motor vehicle. 2024PF01881
[0104] 17
[0105] The additional instructions can be provided, for example, as program code. This program code can be provided, for example, as binary code or assembly language, and / or as source code in a programming language such as C, and / or as a program script, such as Python.
[0106] According to another aspect of the invention, a computer-readable storage medium is specified which stores a computer program according to the invention and / or a further computer program according to the invention.
[0107] The computer program, the further computer program, and the computer-readable storage medium are each computer program products containing the commands and / or the further commands.
[0108] Further features of the invention are evident from the claims, the figures, and the description of the figures. The features and combinations of features mentioned above in the description, as well as those mentioned below in the description of the figures and / or illustrated in the figures, may be encompassed by the invention not only in the combinations specified, but also in other combinations. In particular, embodiments and combinations of features that do not include all the features of an originally formulated claim may also be encompassed by the invention. Furthermore, embodiments and combinations of features that go beyond or deviate from the combinations of features mentioned in the claims may also include the invention.
[0109] The invention is explained in more detail below with reference to specific exemplary embodiments and corresponding schematic drawings. Identical or functionally equivalent elements in the drawings may be provided with the same reference numerals. The description of identical or functionally equivalent elements is not necessarily repeated with respect to the different figures.
[0110] The figures show 2024PF01881
[0111] 18
[0112] Fig. 1 shows a schematic representation of a motor vehicle during the execution of an exemplary embodiment of an inventive method for automatic parking;
[0113] Fig. 2 shows a schematic representation of a motor vehicle during the execution of another exemplary embodiment of an inventive method for automatic parking;
[0114] Fig. 3 shows a schematic representation of the states of a motor vehicle during the execution of another exemplary embodiment of an inventive method for automatic parking; and
[0115] Fig. 4 shows a schematic representation of an exemplary embodiment of a computer-implemented training method for training a path planning agent to automatically park a motor vehicle by reinforcement learning.
[0116] Fig. 1 shows a schematic representation of a motor vehicle 1 during the execution of an exemplary embodiment of a method according to the invention for automatic parking.
[0117] The motor vehicle 1 has an electronic vehicle guidance system 3 which includes a data processing system 2 which stores a path planning agent 16 (see Fig. 4) trained according to a computer-implemented training method according to the invention for the automatic parking of the motor vehicle 1.
[0118] The motor vehicle 1, for example the electronic vehicle guidance system 3, has at least one environmental sensor system 4, for example a camera and / or a radar system and / or a lidar system and / or an ultrasonic sensor system. The at least one environmental sensor system 4 is configured to generate environmental data representing the current environment of the motor vehicle 1 and to provide it to the data processing system 2.
[0119] The data processing system 2 can determine a vehicle target position 12, in particular within a parking space 5, and a vehicle target orientation 13, which for example corresponds to an orientation of the parking space 52024PF01881, based on the environmental data.
[0120] 19
[0121] The parking space 5 can, for example, be located between two obstacles 6a, 6b, such as between two other vehicles. Using the trained path planning agent 16, the data processing system 2 can determine at least one path for the vehicle 1 from its current position with its current orientation to its target position 12 with orientation 13, and select one path 7 from this at least one path for use in the automatic parking of the vehicle 1.That at least one path leads to the vehicle target position 12 with the vehicle target orientation 13 can be considered to be given, in particular, even if the respective path does not end exactly at the vehicle target position 12 but within a position tolerance range 14 around the vehicle target position 12 and / or with an orientation within an orientation tolerance range 15 around the vehicle target orientation 13.
[0122] The data processing system 2 can automatically guide the motor vehicle 1 along the selected path 7 in order to automatically park the motor vehicle 1 in the parking space 5. For this purpose, the data processing system 2 can, in particular, generate at least one control signal for automatically guiding the motor vehicle 1 along the selected path 7, and at least one actuator of the motor vehicle 1 can be controlled depending on the at least one control signal for automatically guiding the motor vehicle 1 along the selected path 7.
[0123] Fig. 2 shows a schematic representation of a motor vehicle 1 during the execution of a further exemplary embodiment of an inventive method for automatic parking, which is based on the embodiment according to Fig. 1.
[0124] In this process, a vehicle anchor position 8, which is shifted by a predefined position offset relative to the vehicle target position 12 parallel to the vehicle target orientation 13, and a further vehicle anchor orientation 9, which is equal to the further vehicle target orientation 13, are determined based on the environmental data.
[0125] The at least one path for the motor vehicle 1 is determined by the trained path planning agent 16 such that it leads from the current vehicle position with the current vehicle orientation, via the vehicle anchor position 8 with the further vehicle anchor orientation 9, to the vehicle target position 12 with the vehicle target orientation 13. The at least one path is therefore determined in two parts. In particular, the selected path 7 also has a first part.
[0126] 20
[0127] 7a from the current vehicle position with the current vehicle orientation to the vehicle anchor position 8 with the vehicle anchor orientation 9, and a second part 7b from the vehicle anchor position 8 with the vehicle anchor orientation 9 to the vehicle target position 12 with the vehicle target orientation 13.
[0128] Fig. 3 shows the states of the motor vehicle 1 during the execution of the operation described in Fig.
[0129] The procedure described in point 2 is visualized again.
[0130] Starting from a current state 300 of the vehicle 1 with its current vehicle position and orientation, the vehicle 1 is guided through one or more intermediate states 310, 320 with respective intermediate positions and orientations to an anchor state 330 with vehicle anchor position 8 and vehicle anchor orientation 9. The vehicle 1 is then guided, in particular by driving straight forward, from the anchor state 330 to a target vehicle state 340 with vehicle target position 12 and vehicle target orientation 13.
[0131] Fig. 4 schematically shows an exemplary embodiment of a computer-implemented training method for training a path planning agent 16 to automatically park a motor vehicle 1 by reinforcement learning. The trained path planning agent 16 can then be used, for example, in a method as described with respect to Figs. 1 to 3.
[0132] A variety of training scenarios 18, 19, 20 are provided for training the path planning agent 16. These can be created, in particular, through simulation. Each training scenario 18, 19, 20 is characterized by at least one vehicle starting position, one vehicle starting orientation, one vehicle destination position, and one vehicle destination orientation. Furthermore, the training scenarios 18, 19, 20 can be characterized, for example, by different positions and / or shapes of obstacles 6a, 6b, 6c, and, if applicable, by a different number of obstacles 6a, 6b, 6c, and accordingly by different traversable areas for the vehicle 1.
[0133] For each training scenario 18, 19, 20, the path planning agent 16 generates a simulated path from the respective vehicle starting position with the respective vehicle starting orientation to a simulated vehicle ending position with a simulated vehicle ending orientation, based on a reinforcement learning strategy. (At least 2024PF01881)
[0134] 21
[0135] A reward function value for the simulated path is calculated, and the reinforcement learning strategy is updated based on at least one of the reward function values. The reward function depends on an orientation term that corresponds to a deviation of the simulated vehicle end orientation from the actual vehicle destination orientation.
[0136] Furthermore, in some embodiments, the reward function may consist of a target position term, which corresponds to a deviation of the simulated vehicle end position from the respective vehicle target position, and / or a path length term, which corresponds to a path length of the simulated path, and / or a robustness term, which corresponds to a minimum distance of the simulated path from at least one obstacle 6a, 6b, 6c according to the respective training scenario 18, 19, 20, and / or an average distance of the simulated path from the at least one obstacle according to the respective training scenario 18, 19, 20, and / or further terms that specify a quality of the respective simulated path.
[0137] Optionally, each training scenario 18, 19, 20 can be characterized by a vehicle anchor position that is shifted relative to the respective vehicle target position by a predefined position offset parallel to the respective vehicle target orientation, and a vehicle anchor orientation that is equal to the respective vehicle target orientation. The respective simulated path has an intermediate vehicle state characterized by an intermediate vehicle position with a minimal deviation from the respective vehicle anchor position and a corresponding intermediate vehicle orientation. The reward function then depends on an anchor term that depends on the minimal deviation of the intermediate vehicle position from the respective vehicle anchor position and / or on a deviation of the intermediate vehicle orientation from the respective vehicle anchor orientation.
[0138] This effectively creates a "false" parking space located above the actual parking space 5. This increases the robustness of the path planning agent 16's behavior, as the simulated path must enter the real parking space 5 at a much steeper angle. This allows the path planning agent 16 to react much more flexibly to changes in the environment or environmental data.
[0139] Apart from the aspects described, 16 known reinforcement learning techniques can be used to train the path planning agent. 2024PF01881
[0140] 22
[0141] are, in particular, using a so-called environment 17. The environment 17 can be considered a simulator that generates the various training scenarios 18, 19, 20. The path planning agent 16 evaluates a current situation, which is referred to as state s t As described, an action leads to a tIt exits and receives feedback from the environment after each action, for example 17. The feedback is used as a reward r t described or, if applicable, as a punishment.
[0142] In some embodiments, the difficulty of the training scenarios 18, 19, 20 may increase during the course of the training, the difficulty being, for example, due to the width of the parking space 5 and / or the positions and / or sizes of the obstacles 6a, 6b, 6c and / or corresponding tolerances to be observed.
[0143] In training scenario 18, for example, parking lot 5 is particularly wide, and there are only two widely separated obstacles, 6a and 6b. Therefore, it is relatively easy for the path planning agent 16 to simulate a path to parking lot 5. However, training scenario 19 is already more demanding and thus presents a higher level of difficulty. The number of obstacles, 6a, 6b, and 6c, is increased by three or four, and parking lot 5 is narrower because the obstacles are closer together compared to training scenario 18. In training scenario 20, the difficulty is increased even further because the obstacles, 6a, 6b, and 6c, are wider and even closer together, making parking lot 5 even narrower.
[0144] An output from the trained path planning agent 16 can, for example, be converted into a steering angle profile 21 along the respective simulated path. The steering angle profile 21 is shown as a curve in Fig. 4. The horizontal axis shows a distance d from the current vehicle position, and the vertical axis shows a steering angle α of a steering system of the vehicle 1. The steering angle profile 21 here comprises, by way of example, three segments 22, 23, 24, each of which can be interpreted as the vehicle 1 traveling a certain distance with a constant or changing steering angle α.
[0145] The steering angle profile 21 can, for example, be provided to the data processing system 2 of the electronic vehicle guidance system 3, so that it can generate at least one control signal as described.
[0146] As described, the invention can increase the accuracy of path planning in the automatic parking of a motor vehicle.
Claims
2024PF01881 23 Patent claims 1. Computer-implemented training method for training a path planning agent (16) for automatically parking a motor vehicle (1) by reinforcement learning based on a variety of training scenarios (18, 19, 20), wherein each training scenario (18, 19, 20) is characterized by a vehicle starting position, a vehicle starting orientation, a vehicle destination position and a vehicle destination orientation, wherein for each training scenario (18, 19, 20) by the path planning agent (16) based on a reinforcement learning strategy, a simulated path is generated from the respective vehicle starting position with the respective vehicle starting orientation to a simulated vehicle end position with a simulated vehicle end orientation; at least one value of a reward function is calculated for the simulated path and the reinforcement learning strategy is updated depending on the at least one value of the reward function, where the reward function depends on an orientation term that corresponds to a deviation of the simulated vehicle end orientation from the respective vehicle destination orientation.
2. Computer-implemented training method according to claim 1, wherein each training scenario (18, 19, 20) is characterized by a vehicle anchor position which is shifted relative to the respective vehicle target position by a predefined position offset parallel to the respective vehicle target orientation, and a vehicle anchor orientation which is equal to the respective vehicle target orientation, wherein for each training scenario the respective simulated path has an intermediate vehicle state characterized by an intermediate vehicle position with minimal deviation from the respective vehicle anchor position and a corresponding intermediate vehicle orientation; and the reward function depends on an anchor term that is determined by the minimum deviation of the vehicle's intermediate position from the respective vehicle anchor position. 2024PF01881 24 and / or depends on a deviation of the vehicle intermediate orientation from the respective vehicle anchor orientation.
3. Computer-implemented training method according to one of the preceding claims, wherein the reward function depends on a target position term that corresponds to a deviation of the simulated vehicle end position from the respective vehicle target position; and / or a path length term that corresponds to the path length of the simulated path; and / or a robustness term which corresponds to a minimum distance of the simulated path from at least one obstacle (6a, 6b, 6c) according to the respective training scenario and / or an average distance of the simulated path from the at least one obstacle (6a, 6b, 6c) according to the respective training scenario (18, 19, 20).
4. Computer-implemented training method according to one of the preceding claims, wherein the path planning agent comprises an artificial neural network.
5. Computer-implemented method for planning a path for the automatic parking of a motor vehicle (1), wherein a path planning agent is trained using a computer-implemented training method according to one of the preceding claims, wherein Environmental data representing the current environment of the motor vehicle (1) are received; a further vehicle target position (12) and a further vehicle target orientation (13) are determined based on the environmental data; by the trained path planning agent (16) at least one path for the motor vehicle (1) is determined from a current vehicle position with a current vehicle orientation to the further vehicle target position (12) with the further vehicle target orientation (13); and a path (7) for use in the automatic parking of the motor vehicle (1) from which at least one path (7) is selected.
6. Computer-implemented method according to claim 5, wherein 2024PF01881 25 which includes at least one path and at least two paths, each of which is characterized by a respective final vehicle position and a respective final vehicle orientation; for each of the at least two paths a misorientation value is calculated which corresponds to a deviation of the respective final vehicle orientation from the further vehicle target orientation (13); the path (7) for use in the automatic parking of the motor vehicle (1) is selected depending on the calculated misorientation values.
7. Computer-implemented method according to claim 6, wherein the path of the at least two paths is selected as the path (7) for use in the automatic parking of the motor vehicle (1) which has the smallest misorientation value.
8. Computer-implemented method according to claim 6, wherein For each path of at least two paths, a weighted sum is calculated from the respective misorientation value and at least one other characteristic value of the respective path; and the path (7) for use in the automatic parking of the motor vehicle (1) is selected depending on the calculated weighted sums.
9. Computer-implemented method according to any one of claims 5 to 8, wherein a further vehicle anchor position (8), which is shifted by a predefined further position offset relative to the further vehicle target position (12) parallel to the further vehicle target orientation (13), and a further vehicle anchor orientation (9), which is equal to the further vehicle target orientation (13), are determined based on the environmental data; and the at least one path for the motor vehicle (1 ) is determined by the trained path planning agent (16) from the current vehicle position with the current vehicle orientation via the further vehicle anchor position (8) with the further vehicle anchor orientation (9) to the further vehicle target position (12) with the further vehicle target orientation (13).
10. Method for automatically parking a motor vehicle (1), wherein a computer-implemented method according to any one of claims 5 to 9 is carried out and 2024PF01881 26 the motor vehicle (1) is automatically guided along the selected path (7).
11. Method according to claim 10, wherein at least one control signal is generated for the automatic guidance of the motor vehicle (1) along the selected path (7); and for automatically guiding the motor vehicle (1) along the selected path (7) at least one actuator of the motor vehicle (1) is controlled depending on the at least one control signal.
12. Data processing system (2) configured to perform a computer-implemented training method according to any one of claims 1 to 4 and / or a computer-implemented method according to any one of claims 5 to 9.
13. Arrangement comprising a data processing system configured to perform a computer-implemented training method for training a path planning agent (16) according to any one of claims 1 to 4, and an electronic vehicle guidance system (3) for a motor vehicle (1), wherein the electronic vehicle guidance system (3) comprises a further data processing system (2) configured to to receive environmental data representing the current environment of the motor vehicle (1); to determine a further vehicle target position (12) and a further vehicle target orientation (13) based on the environmental data; to determine at least one path for the motor vehicle (1) from a current vehicle position of the motor vehicle (1) with a current vehicle orientation of the motor vehicle (1) to the further vehicle target position (12) with the further vehicle target orientation (13) using the trained path planning agent (16); to select a path (7) for use in the automatic parking of the motor vehicle (1) from the at least one path; and to automatically guide the motor vehicle (1) along the selected path (7).
14. Arrangement according to claim 13, wherein the electronic vehicle guidance system (3) comprises at least one environmental sensor system (4) configured to generate the environmental data. 2024PF01881 27 15. computer program product Commands which, when executed by a data processing system, cause the data processing system to perform a computer-implemented training method according to any one of claims 1 to 4; and / or further commands which, when executed by a data processing system (2), cause the data processing system (2) to carry out a computer-implemented method according to one of claims 5 to 9.