Training a path-planning algorithm for automatically parking a motor vehicle, planning a path when automatically parking a motor vehicle, and automatically parking a motor vehicle
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- VALEO SCHALTER & SENSOREN GMBH
- Filing Date
- 2026-01-21
- Publication Date
- 2026-07-30
Smart Images
Figure EP2026051385_30072026_PF_FP_ABST
Abstract
Description
[0001] 2024PF01880
[0002] 1
[0003] Training a path planning algorithm for automatic parking of a motor vehicle, path planning for automatic parking of a motor vehicle, and automatic parking of a motor vehicle.
[0004] The present invention relates to a computer-implemented training method for training a path planning algorithm for the automatic parking of a motor vehicle by reinforcement learning, a computer-implemented method for path planning in the automatic parking of a motor vehicle, wherein a path planning algorithm is trained and used using such a computer-implemented training method, and a method for the automatic parking of a motor vehicle, wherein such a computer-implemented method is carried out. The invention further relates to a data processing system for carrying out such a computer-implemented method or training method, an arrangement comprising a data processing system and an electronic vehicle guidance system, and corresponding computer program products.
[0005] In automated driving systems, numerous well-known algorithms exist for path planning, also known as trajectory planning. One class of such path planning algorithms is based on so-called agents, which have been trained using reinforcement learning.
[0006] The publication Y. Chen et al.: “Deep Reinforcement Learning in Autonomous Car Path Planning and Control: A Survey” (arXiv:2404.00340v1) provides an overview of literature on Deep Reinforcement Learning (DRL) in the field of path planning and control for vehicles.
[0007] Known path-planning algorithms for automatic parking may exhibit reduced accuracy or reliability, particularly in difficult or complex parking scenarios, due to inaccuracies or noise in the underlying environmental data provided by the vehicle's environmental sensor systems, which may have been preprocessed, and so on. 2024PF01880
[0008] 2
[0009] It is an object of the present invention to increase the accuracy and / or reliability of path planning in the automatic parking of a motor vehicle.
[0010] This problem is solved by the subject matter of the independent claim. Further embodiments and preferred developments are the subject matter of the dependent claims.
[0011] The invention is based on the idea of training two different path planning agents for the automatic parking of a motor vehicle by reinforcement learning, wherein the two path planning agents each determine only a part of the path for the motor vehicle.
[0012] According to one aspect of the invention, a computer-implemented training method for training a path planning algorithm for the automatic parking of a motor vehicle by reinforcement learning is provided, based on a plurality of training scenarios, in particular predefined training scenarios. Each training scenario of the plurality of training scenarios is characterized by a corresponding vehicle start state, comprising a vehicle start position and a vehicle start orientation; by a corresponding vehicle anchor state, comprising a vehicle anchor position and a vehicle anchor orientation; and by a corresponding vehicle target state, comprising a vehicle target position and a vehicle target orientation.
[0013] For each training scenario, a first path planning agent of the path planning algorithm generates a simulated first path segment from the respective vehicle start state to a simulated intermediate vehicle state, which has a simulated intermediate vehicle position and orientation, based on a first reinforcement learning strategy. A second path planning agent of the path planning algorithm generates a simulated second path segment from the simulated intermediate vehicle state to a simulated final vehicle state, which has a simulated final vehicle position and orientation, based on a second reinforcement learning strategy. The respective values of at least one reward or punishment function are determined, and the first and second reinforcement learning strategies are updated depending on these determined values.At least one reward or punishment function depends on a first deviation term according to a2024PF01880.
[0014] 3
[0015] Deviation of the simulated intermediate vehicle state from the respective vehicle anchor state and from a second deviation term corresponding to a deviation of the simulated final vehicle state from the respective vehicle target state.
[0016] Unless otherwise specified, all steps of the computer-implemented training procedure can be performed by a data processing system comprising at least one data processing device. In particular, the at least one data processing device is configured or adapted to perform the steps of the computer-implemented training procedure. For this purpose, the at least one data processing device may, for example, store a computer program containing instructions which, when executed by the at least one data processing device, cause it to execute the computer-implemented training procedure. The terms "data processing system" and "at least one data processing device" may be used interchangeably.
[0017] In the event that the at least one data processing device comprises two or more data processing devices, certain steps performed by the at least one data processing device can be understood, for example, as different data processing devices performing different steps or different parts of a step. In particular, it is not necessary for each data processing device to perform the steps completely. In other words, the execution of the steps can be distributed among the two or more data processing devices.
[0018] In reinforcement learning, the so-called environment comprises a set of states, particularly for the vehicle, and a set of possible actions, also particularly for the vehicle. The path-planning agents can interact with the environment at discrete points in time. At each point in time, one of the states of the environment exists, and the path-planning agents select one of the actions.
[0019] A reinforcement learning strategy, also known as a policy, is a collection of probability measures, each representing a preference of the respective path-planning agent to select a specific action at a given time and under a given state. Subsequently, the environment outputs a follow-up state depending on the selected action, and these steps are repeated until the corresponding path segment has been simulated. At least one 2024PF01880
[0020] 4
[0021] The reward or punishment function is evaluated particularly when the entire path has been simulated. However, depending on the specific design of the reinforcement learning method, an evaluation of at least one reward or punishment function, or a part thereof, can also be performed after each repetition.
[0022] The states each include, in particular, the position and orientation of the vehicle. Furthermore, the states can include the target position and orientation of the vehicle for the respective training scenario. In addition, the states can also include the positions and, where applicable, the shapes of obstacles and / or drivable areas for the vehicle. Furthermore, the states can include, for example, the steering position or wheel position of the vehicle and / or the direction of travel of the vehicle, in particular whether the vehicle is traveling forwards or backwards.
[0023] The actions may include, for example, maintaining or changing the direction of travel of the motor vehicle and / or maintaining or changing the steering position or wheel position of the motor vehicle.
[0024] A position for the motor vehicle can be understood here and in the following as the position of a predefined point, in particular on the motor vehicle.
[0025] For example, the center point of a front or rear axle of the motor vehicle can serve as a reference point for the position.
[0026] An orientation for the motor vehicle can be understood here and in the following as the direction of a vector originating from the point that defines the position for the motor vehicle, or originating from another predefined point.
[0027] By using a large number of training scenarios, at least one reward or punishment function can be optimized by varying the respective reinforcement learning strategy across the multitude of training scenarios. This allows the trained path planning agents, acting according to the respective trained reinforcement learning strategy, to then be used for productive path planning. In particular, each path planning agent generates a sequence of actions that correspond to a specific path segment for the vehicle. For the optimization or updating of the reinforcement learning strategies, see 2024PF01880
[0028] 5
[0029] Known methods of reinforcement learning can be used based on the value of at least one reward or punishment function.
[0030] Since the path planning agents, or rather the path planning algorithm (which may consist of, for example, a first and a second path planning agent), are to be trained for the automatic parking of the vehicle, the training scenarios are parking scenarios. These are characterized in particular by the fact that the vehicle's starting position, starting orientation, destination position, and destination orientation are predefined, as well as by the positions and, if applicable, shapes of obstacles and / or a drivable area for the vehicle. The various training scenarios differ primarily in one or more of these aspects. Therefore, a large number of training scenarios can be generated and used without significant computational effort, thus achieving high reliability and accuracy in path planning by the trained path planning algorithm.
[0031] Depending on the specific implementation of the computer-implemented training method, the first and second path planning agents can be trained separately. In this case, the at least one reward or punishment function comprises a first reward or punishment function for training the first path planning agent or the first reinforcement learning strategy, and a second reward or punishment function for training the second path planning agent or the second reinforcement learning strategy. In other embodiments, the first and second path planning agents can be trained together, for example, using end-to-end training. In this case, the at least one reward or punishment function can, for example, consist of a single reward or punishment function.Basically, determining the respective values of at least one reward or punishment function can be understood as meaning that a corresponding value is determined for each reward or punishment function of the at least one reward or punishment function.
[0032] Here and in the following, a state of the motor vehicle can be understood as including at least a corresponding position and orientation of the motor vehicle. The states specified by the training scenarios 2024PF01880
[0033] 6
[0034] They may also include, in particular, a corresponding tolerance for the respective position and / or a corresponding tolerance for the respective orientation.
[0035] A deviation of one state from another state, for example, the simulated intermediate vehicle state from the respective vehicle anchor state or the simulated final vehicle state from the respective vehicle target state, can include a deviation of the respective positions, i.e., the intermediate vehicle position from the respective vehicle anchor position or the simulated final vehicle position from the respective vehicle target position, and / or a deviation of the respective orientations, i.e., the intermediate vehicle orientation from the respective vehicle anchor orientation or the simulated final vehicle orientation from the respective vehicle target orientation.
[0036] A deviation between two positions is given, in particular, by a distance between the two positions. A deviation between two orientations can be given, for example, by an angle between the two corresponding vectors.
[0037] It should be noted that the at least one reward or punishment function may also include further terms that may depend, for example, on the number of changes of direction along the simulated path, the total length of the simulated path, the total number of magazines required for the simulated path, a minimum or average distance of the vehicle from obstacles along the simulated path, and so on. For example, the at least one reward or punishment function may include a weighted sum of the terms considered.
[0038] By splitting the path planning into the first path part and the second path part when training the path planning algorithm, two separate trained path planning agents are created.
[0039] The trained first path planning agent can be described as a scanner module that essentially scans the parking lot or generates a path segment for a preliminary parking position. In production use of the trained path planning algorithm, the scanner module can, for example, attempt to guide the vehicle to an entrance area of the parking lot. This allows the 2024PF01880
[0040] 7
[0041] For example, a vehicle may already be partially entering the parking space, maintaining a relatively large safety distance from any obstacles, such as other vehicles, pillars, walls, and so on. In this situation, the vehicle's environmental sensor system potentially has an improved, and in particular, direct and complete, view of the vehicle's target position. The vehicle's target position and / or orientation can then be verified or corrected, and the trained second path planning agent, which can also be called the parking module, can build upon this information to achieve the target vehicle state as accurately as possible. The parking module can, for example, plan directly to the vehicle's target position or, if necessary, plan a different, particularly more complex, parking maneuver.
[0042] The division according to the invention also increases the robustness of the trained path planning algorithm with regard to changes in environmental conditions or environmental data.
[0043] According to at least one embodiment, the at least one reward or punishment function has a first reward or punishment function that depends on a first deviation term. The first reinforcement learning strategy is updated depending on the value of the first reward or punishment function. Furthermore, the at least one reward or punishment function has a second reward or punishment function that depends on the second deviation term. The second reinforcement learning strategy is updated depending on the value of the second reward or punishment function.
[0044] This allows the two path planning agents, or reinforcement learning strategies, to be trained more flexibly and potentially more efficiently. It is also possible to train them in parallel. Alternatively, training the second path planning agent can be combined with training the first. Separating the reward or punishment functions for each path planning agent during training the path planning algorithm strengthens their independence, which can ultimately further increase the robustness of the trained path planning algorithm. 2024PF01880
[0045] 8
[0046] According to at least one embodiment, the first path planning agent includes a first artificial neural network and / or the second path planning agent includes a second artificial neural network.
[0047] This enables the path planning agents to process particularly complex training scenarios, or to handle particularly complex environmental scenarios after training.
[0048] According to at least one embodiment, the respective vehicle anchor state has an anchor position tolerance for the vehicle anchor position and an anchor orientation tolerance for the vehicle anchor orientation, and the first deviation term depends on the anchor position tolerance and / or the anchor orientation tolerance. Alternatively or additionally, the respective target vehicle state has an anchor position tolerance for the vehicle anchor position and an anchor position tolerance for the vehicle anchor orientation, and the second deviation term depends on the anchor position tolerance and / or the anchor orientation tolerance.
[0049] The dependence of the respective deviation term on the corresponding tolerance(s) can be understood, in particular, as meaning that the corresponding reward or punishment function only occurs if the corresponding position or orientation of the vehicle lies within the corresponding tolerance, or that the reward is lower the further the corresponding position or orientation of the vehicle lies outside the corresponding tolerance. The opposite applies if the function is formulated as a punishment function.
[0050] Such embodiments have the advantage that different tolerance standards can be applied to the two path planning agents or reinforcement learning strategies, further increasing the flexibility of the process. For example, it can be stipulated that the anchor position tolerance is greater than the target position tolerance and / or that the anchor orientation tolerance is greater than the target orientation tolerance. This ensures, on the one hand, that ultimately high accuracy is achieved with regard to the final result of the parking process. On the other hand, lower requirements can apply to the anchor state, since any deviations at this point can be compensated for by the planning of the second path segment.
[0051] 9
[0052] These can be corrected or compensated for. This allows for simpler and less complex initial path segments when parking the vehicle.
[0053] According to at least one embodiment, the anchor position tolerance corresponds to a maximum distance from the vehicle anchor position, in particular the vehicle's intermediate position from the vehicle anchor position. Specifically, the maximum distance is in the range [20 cm, 100 cm] or in the range [40 cm, 80 cm] or in the range [50 cm, 70 cm].
[0054] According to at least one embodiment, the target position tolerance corresponds to a maximum distance from the vehicle target position, in particular the vehicle end position from the vehicle target position. Specifically, the maximum distance is in the range [1 cm, 20 cm], or in the range [8 cm, 12 cm], or in the range [9 cm, 11 cm].
[0055] For example, the anchor position tolerance corresponds to a maximum distance from the vehicle anchor position which is in the range [20 cm, 100 cm], or in the range [40 cm, 80 cm], or in the range [50 cm, 70 cm], and the target position tolerance corresponds to a maximum distance from the vehicle target position which is in the range [1 cm, 20 cm], or in the range [8 cm, 12 cm], or in the range [9 cm, 11 cm].
[0056] According to at least one embodiment, the anchor orientation tolerance corresponds to a maximum orientation deviation from the vehicle anchor orientation, in particular the vehicle intermediate orientation from the vehicle anchor orientation. Specifically, the maximum orientation deviation is in the range [5°, 15°], or in the range [8°, 12°].
[0057] According to at least one embodiment, the target orientation tolerance corresponds to a maximum orientation deviation from the vehicle target orientation, in particular the vehicle end orientation from the vehicle target orientation. Specifically, the maximum orientation deviation is in the range [0.5°, 5°], or in the range [1°, 3°].
[0058] For example, the anchor orientation tolerance corresponds to a maximum orientation deviation from the vehicle anchor orientation, which lies in the range [5°, 15°] or in the range [8°, 12°], and the target orientation tolerance corresponds to the vehicle target orientation, which corresponds to a maximum orientation deviation from the vehicle target orientation, which lies in the range [0.5°, 5°] or in the range [1°, 3°]. 2024PF01880
[0059] 10
[0060] According to at least one embodiment, the respective vehicle anchor position is shifted by a predefined positional offset relative to the respective vehicle target position, parallel to the respective vehicle target orientation. For example, the vehicle anchor orientation is identical to the vehicle target orientation.
[0061] In such embodiments, the path planning agent is therefore rewarded if the simulated intermediate vehicle state exhibits a particularly small deviation of the intermediate vehicle position from the respective vehicle anchor position and / or a particularly small deviation of the intermediate vehicle orientation from the vehicle anchor orientation, and thus also from the vehicle target orientation. In particular, the reward can be greater the smaller the corresponding deviations are, or conversely, the penalty can be less severe the smaller the corresponding deviations are.
[0062] In the productive application of the path planning algorithm, paths are therefore preferentially generated where the vehicle already has an orientation as close as possible to the target orientation before reaching its destination position. This increases the flexibility of the path planning with regard to possible variations in environmental conditions or environmental data acquired by the vehicle's sensors, and any necessary planning corrections, etc. Overall, the reliability and / or accuracy of the path planning are further increased.
[0063] According to a further aspect of the invention, a computer-implemented method for path planning in the automatic parking of a motor vehicle is provided, wherein a computer-implemented training method according to one of the preceding claims is carried out. First environmental data, comprising an environment of the motor vehicle in a first vehicle state, representing a current vehicle position and a current vehicle orientation, are received, in particular from at least one environmental sensor system of the motor vehicle. A further vehicle target state, comprising a further vehicle target position and a further vehicle target orientation, is determined based on the first environmental data. A further vehicle anchor state, comprising a further vehicle anchor position and a further vehicle anchor orientation, is determined based on the first environmental data. 2024PF01880
[0064] 11
[0065] The trained first path planning agent determines at least one first path segment for the vehicle from the first vehicle state to the further vehicle anchor state. A first path segment for use in the automatic parking of the vehicle is selected from this at least one first path segment. Second environmental data, representing the vehicle's surroundings in the further vehicle anchor state, is received, in particular from the vehicle's at least one environmental sensor system. The further vehicle target state is verified or corrected based on this second environmental data. The trained second path planning agent determines at least one second path segment for the vehicle from the further vehicle anchor state to the verified or corrected further vehicle target state. A second path segment for use in the automatic parking of the vehicle is selected from this at least one second path segment.
[0066] Unless otherwise specified, all steps of the computer-implemented method can be performed by a further data processing system comprising at least one further data processing device, in particular by a further data processing system of the motor vehicle. Specifically, the at least one further data processing device is configured or adapted to perform the steps of the computer-implemented method. For this purpose, the at least one further data processing device can, for example, store a further computer program containing instructions which, when executed by the at least one further data processing device, cause the at least one further data processing device to execute the computer-implemented method.
[0067] All further data processing devices of the at least one additional data processing device can be part of the motor vehicle. However, it is also possible that all further data processing devices of the at least one additional data processing device are part of an external computing system outside the motor vehicle, for example, a mobile electronic device, a backend server, or a cloud computing system. It is also possible that the at least one additional data processing device comprises both at least one vehicle data processing device of the motor vehicle and at least one external data processing device of the external computing system. The at least one vehicle data processing device can, for example, consist of one or more control units, ECUs (electronic control units), and / or of one or more
[0068] 12
[0069] This may include several zone control units (ZCUs) and / or one or more domain control units (DCUs) of the motor vehicle.
[0070] In the event that the at least one additional data processing device includes two or more further data processing devices, certain steps performed by the at least one further data processing device can be understood, for example, as different further data processing devices performing different steps or different parts of a step. In particular, it is not necessary for each further data processing device to perform the steps completely. In other words, the performance of the steps can be distributed among the two or more further data processing devices.
[0071] Each embodiment of the computer-implemented method results in a corresponding embodiment of a method for path planning in the automatic parking of a motor vehicle, which is not purely computer-implemented, by including corresponding steps for generating the first environmental data and / or the second environmental data using the at least one environmental sensor system.
[0072] Receiving or receiving data or information can, for example, involve receiving or receiving the data, particularly by the relevant data processing system, from a sending entity; reading the data from a data storage device; receiving or receiving a data stream containing the data; or extracting the data from the data stream, and so on. This can be achieved, in particular, through wired or wireless data transmission. Specifically, data transmission can occur between a hardware and / or software interface of the sending entity and a hardware and / or software interface of the data processing system.
[0073] An environmental sensor system can, for example, be understood as a sensor system that is capable of generating sensor data or sensor signals that map, represent, or reproduce an environment of the environmental sensor system.
[0074] In particular, the ability to detect electromagnetic or other signals from the environment is not sufficient to classify a sensor system as an environmental sensor system. For example, cameras, especially those in the visible spectrum, cannot
[0075] 13
[0076] Working cameras or cameras operating in the infrared range, radar systems, lidar systems or ultrasonic sensor systems are considered environmental sensor systems.
[0077] Depending on the implementation of the at least one environmental sensor system, the first and second environmental data may therefore include, for example, one or more camera images, thermal images, lidar point clouds, lidar depth images, radar data, ultrasound data and / or corresponding images generated on the basis of the radar or ultrasound data.
[0078] The further vehicle target position corresponds in particular to a position in a parking space for the motor vehicle and, for example, the further vehicle target orientation corresponds to a direction or orientation of the parking space.
[0079] The current vehicle position can be determined explicitly based on environmental data and / or using a self-localization algorithm. Similarly, the current vehicle orientation can be determined. Alternatively, the current vehicle position and orientation can be determined within a reference coordinate system, where the current vehicle position corresponds to a coordinate origin of the reference coordinate system. The current vehicle orientation can, for example, be at an angle of 0° with a predefined axis of the reference coordinate system, in particular a longitudinal axis of the reference coordinate system.The reference coordinate system can, for example, be a coordinate system fixed to the motor vehicle, the origin of which is located at the point mentioned above that defines the position of the motor vehicle, i.e., for example, in the center of the front axle or the rear axle of the motor vehicle.
[0080] The reference coordinate system can, for example, be a Cartesian coordinate system, where one axis of the reference coordinate system corresponds to a longitudinal axis of the motor vehicle and another axis of the reference coordinate system corresponds to a transverse axis of the motor vehicle perpendicular to it.
[0081] For example, the path planning agent can be provided with multiple boundary conditions or sets of boundary conditions to determine the at least one path. For each boundary condition or set of boundary conditions, the 2024PF01880
[0082] 14
[0083] The path planning agent can then, for example, determine a path for the vehicle from its current position and orientation to its next destination position and orientation. The constraints might require, for instance, that the vehicle's first maneuver be forward or reverse, or something similar.
[0084] Various criteria can be considered as a rule for selecting a path segment for use in the automatic parking of a vehicle. The selection can, for example, be random. It is also possible to select the shortest path segment of the at least one path segment, or the path segment with the fewest changes of direction of the vehicle during that segment, or the path segment with the smallest deviation of the respective final vehicle orientation from the subsequent vehicle destination orientation. A more complex rule can also be used that considers two or more of these and / or other criteria. It should be noted that in the special case where the at least one path segment consists of exactly one path segment, selecting the path segment is equivalent to selecting exactly that one path segment.
[0085] The selected first path segment and the selected second path segment can be considered the result of path planning. The vehicle can then be automatically guided along the selected path segments, specifically to park the vehicle.
[0086] According to at least one embodiment, the first environmental data and / or the second environmental data depend on camera image data generated by at least one camera of the motor vehicle and / or on ultrasonic sensor data generated by at least one ultrasonic sensor of the motor vehicle and / or on radar sensor data generated by at least one radar sensor of the motor vehicle and / or on lidar sensor data generated by at least one lidar sensor of the motor vehicle.
[0087] The first environmental data and / or the second environmental data may also depend on appropriately fused and / or otherwise processed camera image data and / or ultrasonic sensor data and / or radar sensor data and / or lidar sensor data.
[0088] Further embodiments of the computer-implemented method according to the invention follow directly from the various configurations of the 2024PF01880
[0089] 15
[0090] The computer-implemented training method according to the invention and vice versa. In particular, individual features and corresponding explanations as well as advantages regarding the various embodiments of the computer-implemented method according to the invention can be transferred analogously to corresponding embodiments of the computer-implemented training method according to the invention.
[0091] According to a further aspect of the invention, a method for automatically parking a motor vehicle is described. In this method, a computer-implemented path planning method according to the invention is carried out for automatically parking a motor vehicle, and the motor vehicle is parked automatically by being guided automatically along the selected first path segment and along the selected second path segment.
[0092] For example, at least one control signal is generated for automatically guiding the vehicle along the selected first path segment and along the selected second path segment. To automatically guide the vehicle along the selected first path segment and along the selected second path segment, at least one actuator of the vehicle is controlled depending on this at least one control signal.
[0093] The at least one actuator can, for example, include one or more brake actuators and / or one or more steering actuators and / or one or more drive motors of the vehicle. Based on the at least one control signal, the one or more actuators can influence the longitudinal and / or lateral steering of the vehicle to automatically guide it along the selected path.
[0094] According to another aspect of the invention, a data processing system is provided which is configured to carry out a computer-implemented training method according to the invention.
[0095] The terms "data processing system" and "at least one data processing device" may be used interchangeably within the scope of this disclosure. In this disclosure, a data processing device may, for example, be understood as a device with processing circuits for processing data. A data processing device can thus perform arithmetic operations to process data. An indexed access to a data structure, for example, a Look-Up-2024PF01880
[0096] 16
[0097] A table, LUT, or database can also be considered a computational operation. Even data processing that is partially or fully implemented in hardware can be considered a computational operation.
[0098] A data processing device may, in particular, comprise one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more systems-on-a-chip (SoCs). A data processing device may also comprise one or more processors, for example, one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors (DSPs). The data processing device may also comprise a physical or virtual cluster of computers or other devices of the aforementioned type.
[0099] A data processing device may also include one or more hardware and / or software interfaces, for example for receiving and / or providing data.
[0100] A data processing device may also include one or more storage devices. A storage device may be implemented as volatile memory, such as dynamic random access memory (DRAM) or static random access memory (SRAM), or as non-volatile data storage, such as read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or flash EEPROM, ferroelectric random access memory (FRAM), magnetoresistive random access memory (MRAM), or phase-change random access memory (PCRAM).
[0101] According to a further aspect of the invention, a further data processing system is specified which is configured to carry out a computer-implemented method according to the invention for path planning of a path for the automatic parking of a motor vehicle.
[0102] According to another aspect of the invention, an arrangement comprising a data processing system configured to perform a computer-implemented training method according to the invention, and an electronic 2024PF01880
[0103] 17
[0104] A vehicle guidance system for a motor vehicle is specified. The electronic vehicle guidance system includes a further data processing system configured to receive initial environmental data representing the environment of the motor vehicle in a first vehicle state, including the current vehicle position and orientation. This further data processing system is configured to determine a further target vehicle state, including a further target vehicle position and orientation, based on the initial environmental data.The further data processing system is set up to determine, based on the first environment data, a further vehicle anchor state, a further vehicle anchor position and a further vehicle anchor orientation, to determine at least one first path part for the motor vehicle from the first vehicle state to the further vehicle anchor state using the trained first path planning agent, and to select a first path part for use in the automatic parking of the motor vehicle from the at least one first path part.
[0105] The additional data processing system is configured to receive secondary environmental data representing the vehicle's environment in the secondary vehicle anchor state, to verify or correct the secondary vehicle target state based on this secondary environmental data, and to determine at least one secondary path segment for the vehicle from the secondary vehicle anchor state to the verified or corrected secondary vehicle target state using the trained secondary path planning agent. The additional data processing system is configured to select a secondary path segment for use in automatically parking the vehicle from this secondary path segment and to automatically guide the vehicle along the selected secondary path segment.
[0106] In particular, the additional data processing system stores the trained path planning algorithm, especially the trained first path planning agent and the trained second path planning agent.
[0107] An electronic vehicle guidance system can be understood as an electronic system designed to control a vehicle fully automatically or autonomously, in particular without requiring any intervention by a driver. The vehicle performs all necessary functions, such as steering, 2024PF01880
[0108] 18
[0109] The electronic vehicle control system automatically performs braking and / or acceleration maneuvers, monitors and records road traffic, and reacts accordingly. In particular, the system can implement a fully automated or fully autonomous driving mode according to Level 5 of the SAE J3016 classification. An electronic vehicle control system can also be understood as an advanced driver assistance system (ADAS), which supports the driver during partially automated or semi-autonomous driving. Specifically, the electronic vehicle control system can implement a partially automated or semi-autonomous driving mode according to Levels 1 to 4 of the SAE J3016 classification. Here and in the following, "SAE J3016" refers to the corresponding standard in its April 2021 version.
[0110] At least partially automated vehicle control can therefore include driving the vehicle in accordance with a fully automated or fully autonomous driving mode of Level 5 according to SAE J3016. At least partially automated vehicle control can also include driving the vehicle in accordance with a partially automated or semi-autonomous driving mode according to Levels 1 to 4 of SAE J3016.
[0111] According to at least one embodiment, the electronic vehicle guidance system has at least one environmental sensor system configured to generate the first environmental data and / or the second environmental data.
[0112] Further embodiments of the arrangement according to the invention follow directly from the various configurations of the computer-implemented training method according to the invention, and vice versa. In particular, individual features and corresponding explanations as well as advantages relating to the various embodiments of the training method according to the invention can be transferred analogously to corresponding embodiments of the arrangement according to the invention.
[0113] According to another aspect of the invention, a computer program with instructions is specified. When the instructions are executed by a data processing system, the instructions cause the data processing system to perform a computer-implemented training method according to the invention.
[0114] The instructions can be in the form of program code, for example. The program code can be, for example, binary code or assembler and / or the source code of a 2024PF01880.
[0115] 19
[0116] It should be provided in a programming language, for example C, and / or as a program script, for example Python.
[0117] According to a further aspect of the invention, a further computer program with further instructions is provided. When the further instructions are executed by a further data processing system, the further instructions cause the further data processing system to carry out a computer-implemented method according to the invention for path planning for the automatic parking of a motor vehicle.
[0118] The additional instructions can be provided, for example, as program code. This program code can be provided, for example, as binary code or assembly language, and / or as source code of a programming language, such as C, and / or as a program script, such as Python.
[0119] According to another aspect of the invention, a computer-readable storage medium is specified which stores a computer program according to the invention and / or a further computer program according to the invention.
[0120] The computer program, the further computer program, and the computer-readable storage medium are each computer program products containing the commands and / or the further commands.
[0121] Further features of the invention are evident from the claims, the figures, and the description of the figures. The features and combinations of features mentioned above in the description, as well as those mentioned below in the description of the figures and / or illustrated in the figures, may be encompassed by the invention not only in the combinations specified, but also in other combinations. In particular, embodiments and combinations of features that do not include all the features of an originally formulated claim may also be encompassed by the invention. Furthermore, embodiments and combinations of features that go beyond or deviate from the combinations of features mentioned in the claims may also include the invention.
[0122] The invention is explained in more detail below with reference to specific exemplary embodiments and corresponding schematic drawings. In2024PF01880
[0123] 20
[0124] Identical or functionally equivalent elements in the drawings may be labelled with the same reference symbols. The description of identical or functionally equivalent elements is not necessarily repeated for each figure.
[0125] The figures show
[0126] Fig. 1 shows a schematic representation of a motor vehicle during the execution of an exemplary embodiment of an automatic parking method;
[0127] Fig. 2 shows a schematic representation of the states of a motor vehicle during the execution of a further exemplary embodiment of an inventive method for automatic parking; and
[0128] Fig. 3 shows a schematic representation of an exemplary embodiment of a computer-implemented training method for training a path planning algorithm for automatically parking a motor vehicle by reinforcement learning.
[0129] Fig. 1 shows a schematic representation of a motor vehicle 1 during the execution of an exemplary embodiment of a method according to the invention for automatic parking.
[0130] The motor vehicle 1 has an electronic vehicle guidance system 3, which includes a data processing system 2 that stores a path planning algorithm (see Fig. 4) trained according to a computer-implemented training method according to the invention for automatically parking the motor vehicle 1. The trained path planning algorithm has a trained first path planning agent and a trained second path planning agent.
[0131] The motor vehicle 1, for example the electronic vehicle guidance system 3, has at least one environmental sensor system 4, for example a camera and / or a radar system and / or a lidar system and / or an ultrasonic sensor system. The at least one environmental sensor system 4 is configured to generate initial environmental data representing the environment of the motor vehicle 1 and to provide this data to the data processing system 2. 2024PF01880
[0132] 21
[0133] Based on the initial environmental data, the data processing system 2 can determine a vehicle target position 12, in particular within a parking space 5, and a vehicle target orientation 13, which corresponds, for example, to the orientation of the parking space 5. Furthermore, based on the initial environmental data, the data processing system 2 can determine a vehicle anchor position 8 and a vehicle anchor orientation 9. The parking space 5 can, for example, be located between two obstacles 6a, 6b, such as between two other vehicles.
[0134] Using the trained first path planning agent, the data processing system 2 can determine at least one first path segment for the vehicle 1 from its current position and orientation to the vehicle anchor position 8 with orientation 9, and select a first path segment 7a from this first path segment for use in the automatic parking of the vehicle 1. The fact that the first path segment always leads to the vehicle anchor position 8 with orientation 9 can be considered to be true even if the respective first path segment does not end exactly at the vehicle anchor position 8, but within a position tolerance range 10 around the vehicle anchor position 8 and / or with an orientation within an orientation tolerance range 11 around the vehicle anchor orientation 9.
[0135] The data processing system 2 can automatically guide the motor vehicle 1 along the selected first path segment 7a. For this purpose, the data processing system 2 can, in particular, generate at least one control signal for automatically guiding the motor vehicle 1 along the selected first path segment 7a, and at least one actuator of the motor vehicle 1 can be controlled depending on the at least one control signal for automatically guiding the motor vehicle 1 along the selected first path segment 7a.
[0136] The at least one environmental sensor system 4 is configured to generate second environmental data representing the environment of the vehicle 1 and to provide this data to the data processing system 2 after the vehicle 1 has been automatically guided along the selected first path segment 7a. 2024PF01880
[0137] 22
[0138] Data processing system 2 can verify or correct the vehicle target position 12 and the vehicle target orientation 13 based on the second set of environmental data. Using the trained second path planning agent, data processing system 2 can determine at least one second path segment for the vehicle 1 from the vehicle anchor position 8 with vehicle anchor orientation 9 to the verified or corrected vehicle target position 12 with the verified or corrected vehicle target orientation 13, and select a second path segment 7b from the at least one second path segment for use in automatically parking the vehicle 1.That the at least one second path segment leads to the vehicle target position 12 with the vehicle target orientation 13 can be considered to be given, in particular, even if the respective second path segment does not end exactly at the vehicle target position 12, but within a position tolerance range 14 around the vehicle target position 12 and / or with an orientation within an orientation tolerance range 15 around the vehicle target orientation 13.
[0139] The data processing system 2 can automatically guide the motor vehicle 1 along the selected second path segment 7b. For this purpose, the data processing system 2 can, in particular, generate the at least one control signal for automatically guiding the motor vehicle 1 along the selected second path segment 7b, and the at least one actuator of the motor vehicle 1 can be controlled depending on the at least one control signal for automatically guiding the motor vehicle 1 along the selected second path segment 7b.
[0140] Fig. 2 shows the states of the motor vehicle 1 during the execution of the operation described in Fig.
[0141] The procedure described in point 2 is visualized again.
[0142] Starting from a current state 200 of the vehicle 1 with its current vehicle position and orientation, the vehicle 1 is guided via one or more intermediate states 210, 220 with respective intermediate positions and orientations to an anchor state 230 with vehicle anchor position 8 and vehicle anchor orientation 9. The vehicle 1 is then guided, in particular by driving straight forward, from the anchor state 230 to a target vehicle state 240 with vehicle target position 12 and vehicle target orientation 13. 2024PF01880
[0143] 23
[0144] Fig. 3 schematically shows an exemplary embodiment of a computer-implemented training method for training the second path planning agent 16 of the path planning algorithm by reinforcement learning.
[0145] The trained second path planning agent 16 can then be used, for example, in a procedure as described with regard to Fig. 1 and Fig. 2.
[0146] To train the second path planning agent 16, a variety of training scenarios 18, 19, 20 are provided. These can be created, in particular, through simulation. Each training scenario 18, 19, 20 is characterized by at least one vehicle anchor position, one vehicle anchor orientation, one vehicle target position, and one vehicle target orientation. Furthermore, the training scenarios 18, 19, 20 can be characterized, for example, by different positions and / or shapes of obstacles 6a, 6b, 6c, and, if applicable, by a different number of obstacles 6a, 6b, 6c, and accordingly by different traversable areas for the vehicle 1.
[0147] For each training scenario 18, 19, 20, the second path planning agent 16 generates a simulated second path segment from the respective vehicle anchor position with the respective vehicle anchor orientation to a simulated vehicle end position with a simulated vehicle end orientation, based on the second reinforcement learning strategy. At least one value of the second reward or punishment function for the simulated path is calculated, and the second reinforcement learning strategy is updated depending on this at least one value of the second reward or punishment function. The second reward or punishment function depends on a second deviation term, which includes a deviation of the simulated vehicle end position from the respective vehicle target position and / or a deviation of the simulated vehicle end orientation from the respective vehicle target orientation.
[0148] Furthermore, in some embodiments, the second reward or punishment function may consist of a path length term corresponding to the path length of the simulated second path segment, and / or a robustness term corresponding to a minimum distance of the simulated second path segment from at least one obstacle 6a, 6b, 6c according to the respective training scenario 18, 19, 20, and / or an average distance of the simulated second path segment from the at least one obstacle according to the respective training scenario 18, 19, 20, and / or further terms specifying a quality of the respective simulated second path segment. 2024PF01880
[0149] 24
[0150] Apart from the aspects described, 16 known techniques of reinforcement learning can be used to train the second path planning agent, in particular using a so-called environment 17. The environment 17 can be considered a simulator that generates the various training scenarios 18, 19, 20. The second path planning agent 16 evaluates a current situation, referred to as state St, performs an action at, and receives feedback from the environment 17 after each action, for example. The feedback is referred to as a reward R or, if applicable, as a punishment.
[0151] In some embodiments, the difficulty of the training scenarios 18, 19, 20 may increase during the training, the difficulty being due, for example, to the width of the parking space 5 and / or the positions and / or sizes of the obstacles 6a, 6b, 6c and / or to corresponding tolerances to be observed.
[0152] In training scenario 18, for example, parking lot 5 is particularly wide, and only two widely separated obstacles, 6a and 6b, are planned. Therefore, it is relatively easy for the second path planning agent, 16, to simulate a second path segment to parking lot 5. However, training scenario 19 is already more demanding and thus presents a higher level of difficulty. The number of obstacles, 6a, 6b, and 6c, is increased to three or four, and parking lot 5 is narrower because the obstacles, 6a, 6b, and 6c, are closer together compared to training scenario 18. In training scenario 20, the difficulty is increased even further because the obstacles, 6a, 6b, and 6c, are wider and even closer together, making parking lot 5 even narrower.
[0153] An output from the trained second path planning agent 16 can, for example, be converted into a steering angle profile 21 along the respective simulated second path segment. The steering angle profile 21 is shown as a curve in Fig. 3. The horizontal axis shows a distance d from the current vehicle position, and the vertical axis shows a steering angle α of a steering system of the vehicle 1. The steering angle profile 21 here comprises, by way of example, three segments 22, 23, 24, each of which can be interpreted as the vehicle 1 traveling a certain distance with a constant or changing steering angle α.
[0154] 25
[0155] The steering angle profile 21 can, for example, be provided to the data processing system 2 of the electronic vehicle guidance system 3, so that it can generate at least one control signal as described.
[0156] The first path planning agent can be trained in the same way as described for the second path planning agent (16), except that when training the first path planning agent, the vehicle destination position and vehicle destination orientation must be replaced by the vehicle anchor position and vehicle anchor orientation, respectively, and the vehicle anchor position and vehicle anchor orientation must be replaced by a vehicle start position and vehicle start orientation. Instead of the simulated vehicle end position and simulated vehicle end orientation, a simulated vehicle intermediate position and a simulated vehicle intermediate orientation are used when training the first path planning agent. Furthermore, the second reward or punishment function must be replaced by a first reward or punishment function.The first reward or punishment function depends on a first deviation term, which includes a deviation of the simulated intermediate vehicle position from the respective vehicle anchor position and / or a deviation of the simulated intermediate vehicle orientation from the respective vehicle anchor orientation.
[0157] As described, the invention can increase the accuracy of path planning in the automatic parking of a motor vehicle.
Claims
2024PF01880 Patent claims 1. Computer-implemented training method for training a path planning algorithm for automatically parking a motor vehicle (1) by reinforcement learning based on a plurality of training scenarios (18, 19, 20), wherein each training scenario (18, 19, 20) is characterized by a corresponding vehicle start state having a vehicle start position and a vehicle start orientation, by a corresponding vehicle anchor state having a vehicle anchor position and a vehicle anchor orientation, and by a corresponding vehicle target state having a vehicle target position and a vehicle target orientation, wherein for each training scenario (18, 19, 20) by a first path planning agent (16) of the path planning algorithm based on a first reinforcement learning strategy, a simulated first path segment from the respective vehicle start state to a simulated vehicle intermediate state, which has a simulated vehicle intermediate position and a simulated vehicle intermediate orientation, is generated; by a second path planning agent (16) of the path planning algorithm based on a second reinforcement learning strategy, a simulated second path part from the simulated vehicle intermediate state to a simulated vehicle final state, which has a simulated vehicle final position and a simulated vehicle final orientation, is generated; respective values of at least one reward or punishment function are determined and the first reinforcement learning strategy and the second reinforcement learning strategy are updated depending on the determined values, wherein the at least one reward or punishment function depends on a first deviation term corresponding to a deviation of the simulated intermediate vehicle state from the respective vehicle anchor state and on a second deviation term corresponding to a deviation of the simulated final vehicle state from the respective vehicle target state.
2. Computer-implemented training method according to claim 1, wherein 2024PF01880 27 which has at least one reward or punishment function, a first reward or punishment function that depends on the first deviation term, and the first reinforcement learning strategy is updated depending on the value of the first reward or punishment function; and which has at least one reward or punishment function, a second reward or punishment function that depends on the second deviation term, and the second reinforcement learning strategy is updated depending on the value of the second reward or punishment function.
3. Computer-implemented training method according to one of the preceding claims, wherein the first path planning agent (16) includes a first artificial neural network and / or the second path planning agent (16) includes a second artificial neural network.
4. Computer-implemented training method according to one of the preceding claims, wherein the respective vehicle anchor state has an anchor position tolerance for the vehicle anchor position and an anchor orientation tolerance for the vehicle anchor orientation, and the first deviation term depends on the anchor position tolerance (10) and / or the anchor orientation tolerance (11); and / or the respective vehicle target state has a target position tolerance (14) for the vehicle target position and a target orientation tolerance (15) for the vehicle target orientation and the second deviation term depends on the target position tolerance (14) and / or the target orientation tolerance (15).
5. Computer-implemented training method according to claim 4, wherein the anchor position tolerance is greater than the target position tolerance and / or the anchor orientation tolerance is greater than the target orientation tolerance.
6. Computer-implemented training method according to one of claims 4 or 5, wherein the anchor position tolerance corresponds to a maximum distance from the vehicle anchor position, which is in the range [20 cm, 100 cm] or in the range [40 cm, 80 cm]; and / or 2024PF01880 28 The anchor orientation tolerance corresponds to a maximum orientation deviation from the vehicle anchor orientation, which is in the range [5°, 15°] or in the range [8°, 12°].
7. Computer-implemented training method according to any one of claims 4 to 6, wherein the target position tolerance corresponds to a maximum distance from the vehicle target position which is in the range [1 cm, 20 cm] or in the range [8 cm, 12 cm]; and / or The target orientation tolerance corresponds to a maximum orientation deviation from the vehicle target orientation, which lies in the range [0.5°, 5°] or in the range [1°, 3°].
8. Computer-implemented training method according to one of the preceding claims, wherein the respective vehicle anchor position is shifted by a predefined position offset parallel to the respective vehicle target orientation with respect to the respective vehicle target position.
9. Computer-implemented method according to claim 8, wherein the vehicle anchor orientation is identical to the vehicle target orientation.
10. Computer-implemented method for path planning during automatic parking of a motor vehicle (1), wherein a computer-implemented training method according to one of the preceding claims is carried out, wherein first environmental data, which represent an environment of the motor vehicle (1) in a first vehicle state of the motor vehicle (1) and represent a current vehicle position and a current vehicle orientation, are received; a further vehicle target state, a further vehicle target position (12) and a further vehicle target orientation (13) is determined based on the first environment data; another vehicle anchor state, another vehicle anchor position (8) and another vehicle anchor orientation (9) is determined based on the first environment data; by the trained first path planning agent (16) at least one first path segment for the motor vehicle (1) from the first vehicle state to the further vehicle anchor state is determined; 2024PF01880 29 a first path part (7a) for use in the automatic parking of the motor vehicle (1) from which at least one first path part is selected; second environmental data, representing the environment of the motor vehicle (1) in the further vehicle anchor state, are received; the further vehicle target state is verified or corrected based on the second environmental data; by the trained second path planning agent (16) at least one second path part for the motor vehicle (1) from the further vehicle anchor state to the verified or corrected further vehicle target state is determined; and a second path part (7b) for use in the automatic parking of the motor vehicle (1) is selected from the at least one second path part.
11. Computer-implemented method according to claim 10, wherein the first environmental data and / or the second environmental data depend on camera image data generated by at least one camera of the motor vehicle (1) and / or depend on ultrasonic sensor data generated by at least one ultrasonic sensor of the motor vehicle (1) and / or depend on radar sensor data generated by at least one radar sensor of the motor vehicle (1) and / or depend on lidar sensor data generated by at least one lidar sensor of the motor vehicle (1).
12. Method for automatically parking a motor vehicle (1), wherein a computer-implemented method according to one of claims 10 or 11 is carried out and the motor vehicle (1) is automatically guided along the selected first path part (7a) and along the selected second path part (7b).
13. Method according to claim 12, wherein at least one control signal for the automatic guidance of the motor vehicle (1) along the selected first path segment (7a) and along the selected second path segment (7b) is generated; and For the automatic guidance of the motor vehicle (1) along the selected first path segment (7a) and along the selected second path segment (7b), at least one actuator of the motor vehicle (1) is controlled depending on the at least one control signal. 2024PF01880 30 14. Data processing system (2) configured to perform a computer-implemented training method according to any one of claims 1 to 9 and / or a computer-implemented method according to any one of claims 10 or 11.
15. Arrangement comprising a data processing system configured to perform a computer-implemented training method for training a path planning algorithm according to any one of claims 1 to 9, and an electronic vehicle guidance system (3) for a motor vehicle (1), wherein the electronic vehicle guidance system (3) comprises a further data processing system (2) configured to to receive initial environmental data, which represent an environment of the motor vehicle (1) in a first vehicle state of the motor vehicle (1) and represent a current vehicle position and a current vehicle orientation; to determine a further vehicle target state, a further vehicle target position (12) and a further vehicle target orientation (13) based on the first environmental data; to determine a further vehicle anchor state, a further vehicle anchor position (8) and a further vehicle anchor orientation (9) based on the first environmental data; to determine at least one first path segment for the motor vehicle (1) from the first vehicle state to the further vehicle anchor state using the trained first path planning agent (16); to select a first path part (7a) for use in the automatic parking of the motor vehicle (1) from the at least one first path part; to receive second environmental data representing the environment of the motor vehicle (1) in the further vehicle anchor state; to verify or correct the further vehicle target state based on the second environmental data; to determine at least one second path segment for the motor vehicle (1) from the further vehicle anchor state to the verified or corrected further vehicle target state using the trained second path planning agent (16); to select a second path segment (7b) for use in automatically parking the motor vehicle (1) from the at least one second path segment; and 2024PF01880 31 to guide the motor vehicle (1) automatically along the selected first path section (7a) and along the selected second path section (7b).
16. Arrangement according to claim 15, wherein the electronic vehicle guidance system (3) comprises at least one environment sensor system (4) configured to generate the first environment data and / or the second environment data.
17. computer program product Commands which, when executed by a data processing system, cause the data processing system to perform a computer-implemented training method according to any one of claims 1 to 9; and / or further commands which, when executed by a data processing system (2), cause the data processing system (2) to carry out a computer-implemented method according to one of claims 10 or 11.