Perception field based driving related operation

Abstract

To provide a method for perception field driving related operation for realizing complete automatic operation.SOLUTION: A method for perception field driving related operation may include: (i) obtaining object information regarding one or more objects located within an environment of a vehicle; (ii) determining, using one or more neural networks (NNs), one or more virtual forces that are applied on the vehicle, wherein the one or more virtual forces belong to a virtual physical model; and (iii) performing one or more driving related operation of the vehicle on the basis of the one or more virtual forces.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] Autonomous vehicles (AVs) have the potential to significantly reduce the number of traffic accidents and CO2 emissions, and contribute to a more efficient transportation system. However, current candidate AV technologies lack scalability in the following three respects: [Background technology]

[0002] Limited field of view, lighting and weather challenges, and obstructions all contribute to detection errors and noisy localization / kinematics. To address such poor real-world perception output, one approach to AV technology is to invest in expensive sensors or integrate specialized infrastructure into road networks. However, such attempts are extremely costly and, in the case of infrastructure, geographically limited, thus never leading to generally accessible AV technology.

[0003] AV technologies that are not based on expensive hardware and infrastructure rely entirely on machine learning, and therefore depend on data to handle real-world situations. While massive amounts of data and computational resources are needed to address detection errors and learn sufficient driving policies for complex driving tasks, there are still edge cases that go unhandled. The common thread in these edge cases is that machine learning models do not generalize well to invisible or confusing situations, and the black-box nature of deep neural networks makes it difficult to analyze malfunctioning behavior.

[0004] Current on-road autonomous driving is implemented in the form of individual ADAS functions such as ACC, AEB, and LCA. To achieve full autonomous driving, it is necessary not only to seamlessly combine existing ADAS functions but also to cover the currently unautomated gaps by adding such functions (such as lane changes and intersection handling). In short, current autonomous driving is not based on a comprehensive approach that can be easily extended to achieve full autonomous driving.

Summary of the Invention

[0005] Embodiments of the present disclosure will be more fully understood and recognized by reading the following detailed description in conjunction with the drawings.

Brief Description of the Drawings

[0006] [Figure 1] It is a diagram showing an example of a method. [Figure 2] It is a diagram showing an example of a method. [[ID= / / 19]] [Figure 3] It is a diagram showing an example of a method. [Figure 4] It is a diagram showing an example of a method. [Figure 5] It is a diagram showing an example of a vehicle. [Figure 6] It is a diagram showing an example of a method. [Figure 7] It is a diagram showing an example of a method. [Figure 8] It is a diagram showing an example of a method. [Figure 9] It is a diagram showing an example of a method.

Modes for Carrying Out the Invention

[0007] In the following detailed description, many specific details are set forth in order to provide a thorough understanding of the present invention. However, one skilled in the art will understand that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.

[0008] The subject matter considered to be part of the present invention is specifically pointed out and explicitly claimed in the concluding section of this specification. However, the present invention, with respect to both its configuration and method of operation, as well as its object, features, and advantages, can be best understood by referring to the following detailed description when read together with the accompanying drawings.

[0009] For the sake of simplicity and clarity, please understand that the elements shown in the diagrams are not necessarily drawn to a consistent scale. For example, the dimensions of some elements may be exaggerated relative to others for clarity. Furthermore, reference numbers may be repeated between drawings to indicate corresponding or similar elements where deemed appropriate.

[0010] Since the illustrated embodiments of the present invention can be implemented using electronic components and circuits largely known to those skilled in the art, further details will not be described beyond what is deemed necessary, as set forth above, in order to understand and appreciate the underlying concepts of the present invention and to avoid obscuring or distracting from the teachings of the present invention.

[0011] References to methods in this specification must be applied with necessary modifications to devices or systems capable of performing the methods, and / or to non-temporary computer-readable media containing instructions for performing the methods.

[0012] References to systems or devices herein should be applied mutatis mutandis to the methods that may be performed by the system, and / or to non-temporary computer-readable media that store instructions that can be executed by the system.

[0013] References to non-temporary computer-readable media in this specification must be applied to devices or systems capable of executing instructions stored in non-temporary computer-readable media, and / or may be applied to devices or systems capable of executing instructions, with necessary modifications to the method of executing those instructions.

[0014] Any combination of any figure, any part of the specification, and / or any module or unit listed in any claim may be provided.

[0015] Any of the units and / or modules shown in the application may be implemented in hardware and / or code, instructions and / or commands stored on non-temporary computer-readable media, and may be contained within or outside a vehicle, in a mobile device, on a server, etc.

[0016] The vehicles may be any type of vehicle, such as ground transport vehicles, air transport vehicles, and watercraft.

[0017] The specification and / or drawings may refer to images. Images are an example of a media unit. References to images can be applied to media units with necessary modifications. Media units may be examples of perceived information. References to media units are not limited to signals generated by nature, signals representing human behavior, signals representing operations related to stock markets, medical signals, financial series, geodetic signals, geophysical, chemical, molecular, text and numerical signals, time series, etc., but can be applied to any type of natural signal with the necessary modifications. Any reference to a media unit may be applied to the detected information after any necessary modifications have been made. The information that is detected can be of any kind and is detected by all types of sensors, including visible light cameras, sound sensors, infrared sensors, radar images, ultrasonic sensors, electro-optical sensors, radiographs, and LIDAR (light detection and ranging) sensors. Sensing can include generating samples (e.g., pixels, audio signals) that represent the transmitted signal, or receiving the signal at the sensor.

[0018] The specification and / or drawings may refer to a processor. A processor may also be a processing circuit. A processing circuit may be implemented as a central processing unit (CPU) and / or as one or more other integrated circuits, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a fully custom integrated circuit, or a combination of such integrated circuits.

[0019] Any combination of any steps of any method shown in the specification and / or drawings can be provided.

[0020] Any combination of the subject matter of any of the claims can be provided.

[0021] Any combination of systems, units, components, processors, and sensors shown in the specification and / or drawings can be provided.

[0022] All references to objects can be applied to patterns. Therefore, references to object detection can be applied to pattern detection with the necessary modifications.

[0023] Successful driving requires detouring based on the position and movement of surrounding road objects, but humans are known to be poor at kinematic estimation. It seems that humans use the internal representation of surrounding objects in the form of a virtual force field, which immediately implies action, thus avoiding the need for kinematic estimation. Consider a scenario where your vehicle is traveling in one lane, and a vehicle in the adjacent lane, diagonally ahead, begins to enter your lane. Human reactions to braking or changing direction are immediate and instinctive, and can be experienced as a virtual force pushing your vehicle away from the changing vehicle. This representation of a virtual force is learned and associated with specific road objects.

[0024] Inspired by the considerations above, we propose a new concept of perceptual fields. A perceptual field is a learned representation of road objects in the form of a virtual force field, "perceived" through the vehicle's control system in the form of ADAS and / or AV software. Here, the field is defined as a mathematical function that depends on spatial position (or a similar quantity).

[0025] An example of inference method 100 is shown in Figure 1, and includes the following:

[0026] Method 100 can be performed for one or more frames of the vehicle environment.

[0027] Step 110 of Method 100 may include detecting and / or tracking one or more objects (e.g., including one or more road users). Detection and / or tracking can be carried out in any manner. One or more objects could be any objects that could affect the behavior of a vehicle. For example, road users (pedestrians, other vehicles), the road and / or path the vehicle is traveling on (e.g., road or path conditions, road shape, e.g., curves, straight road segments), traffic signs, traffic lights, intersections, schools, kindergartens, etc. Step 110 may include obtaining additional information, such as kinematic and contextual variables, associated with one or more objects. Obtaining may include receiving or generating. Obtaining may include processing one or more frames to generate kinematic and contextual variables.

[0028] Note that step 110 may include obtaining kinematic variables (even without obtaining one or more frames).

[0029] Method 100 may also include step 120 of obtaining the respective perceptual fields associated with one or more objects. Step 120 may include determining which mappings between objects to retrieve and / or use.

[0030] Step 110 (and even step 120) may be followed by step 130, which determines one or more virtual forces associated with one or more objects by passing relevant input variables, such as kinematic variables and context variables, to the perceptual field (and one or more virtual physical model functions).

[0031] Step 130 may be followed by step 140, which determines the total virtual forces applied to the vehicle based on one or more virtual forces associated with one or more objects. For example, step 140 may include performing a vector-weighted sum (or other function) on one or more virtual forces associated with one or more objects.

[0032] Step 140 may be followed by step 150, which determines the desired (or target) virtual acceleration based on the sum of virtual forces, for example, on the basis of Newton's second law. The desired virtual acceleration may be a vector or it may have a direction.

[0033] Step 150 may be followed by step 160, which converts the desired virtual acceleration into one or more vehicle driving actions that propagate the vehicle according to the desired virtual acceleration.

[0034] For example, step 160 may include using the movement of the accelerator pedal, the movement of the brake pedal, and / or the angle of the steering wheel to convert the desired acceleration into acceleration or deceleration, or to change the direction of travel of the vehicle. The conversion may be based on a vehicle dynamics model with a specific control scheme.

[0035] The advantages of perceptual fields include explainability, generalizability, and robustness against noisy inputs.

[0036] Explainability Representing a vehicle's movement as a composition of individual perceptual fields means breaking down the action into more fundamental components, which in itself is a crucial step toward explainability. The possibility of visualizing these fields and applying intuition from physics to predict the vehicle's movement represents further explainability compared to typical end-to-end black-box deep learning approaches. This increased transparency allows passengers and drivers to trust AV and ADAS technologies more.

[0037] generalizability Representing a vehicle's reaction to unknown road objects as a virtual avoidance force field constitutes an induced bias in unseen situations. This representation has the potential advantage of handling edge cases safely with less training. Furthermore, the perception field model is holistic in the sense that the same approach can be used for all aspects of driving policy. It can also be broken down into narrower driving functions used in ADAS such as ACC, AEB, and LCA. Finally, the complex nature of the perception field allows the model to be trained in atomic scenarios and then appropriately handle more complex scenarios.

[0038] Robustness to noisy inputs Physical constraints on the temporal evolution of the perceptual field, combined with potential filtering of the input, may lead to better handling of noise in the input data compared to pure filtering of localization and kinematic data.

[0039] Physical or virtual forces enable mathematical formulations, such as quadratic ordinary differential equations, that constitute so-called dynamic systems. The advantage of directly expressing control policies is that they are easily influenced by intuition from dynamic systems theory and allow for the easy incorporation of external modules such as input / output prediction, navigation, and filtering.

[0040] Another advantage of the perceptual field approach is that it is not dependent on specific hardware and does not increase computational costs compared to existing methods.

[0041] Training process

[0042] The process of learning the perceptual field can be one of two types, or a combination thereof: behavioral cloning (BC) and reinforcement learning (RL). BC approximates control policies by fitting neural networks to observed human state-behavior pairs, while RL requires trial-and-error learning in a simulated environment without reference to expert demonstrations.

[0043] To combine these two classes of learning algorithms, one first learns a policy through BC and then uses it as the initial policy to fine-tune it using RL. Another way to combine the two approaches is to first learn the so-called reward function (used in RL) through behavior cloning to infer what constitutes desirable behavior for humans, and then train it through trial and error using regular RL. This latter approach is called inverse RL (IRL).

[0044] Figure 2 shows 200 examples of training methods employed for learning through BC.

[0045] Method 200 can be initiated by step 210, which involves collecting human data, which is considered an expert demonstration of how to handle the scenario.

[0046] In step 220, following step 210, a loss function can be constructed that penalizes the difference between the kinematic variables arising from the perceptual field model and the kinematic variables of the corresponding human behavior.

[0047] Step 220 may be followed by step 230, in which the parameters of the perceptual field and the auxiliary function (which may be a virtual physical model function different from the perceptual field) are updated by some optimization algorithm, such as gradient descent, to minimize the loss function.

[0048] Figure 3 shows an example of 250 training methods used in reinforcement learning.

[0049] Method 250 can begin with step 260, which involves constructing a realistic simulation environment.

[0050] Step 260 is followed by Step 270, which involves constructing the reward function, either by learning from expert explanations or by designing it manually.

[0051] Step 270 is followed by step 280, in which the episode is run in a simulation environment, and parameters of the perceptual field and assistive functions can be continuously updated by algorithms such as proximal policy optimization to maximize the expected cumulative reward.

[0052] Figure 4 shows an example of method 400.

[0053] Method 400 may be for operations related to perceptual field driving.

[0054] Method 400 can be initiated by initialization step 410.

[0055] The initialization step 410 may include receiving a group of NNs trained to perform step 440 of method 400.

[0056] Alternatively, step 410 may include training a group of NNs that perform step 440 of method 400.

[0057] The following are various examples of training groups of neural networks. A group of neural networks can be trained to map object information to one or more virtual forces using behavioral cloning. A group of neural networks (NNs) can be trained to map object information to one or more virtual forces using reinforcement learning. The NN group can be trained to map object information to one or more virtual forces using a combination of reinforcement learning and behavioral cloning. A group of neural networks (NNs) can be trained to map object information to one or more virtual forces using reinforcement learning with a reward function defined using behavioral cloning. Using reinforcement learning with an initial policy defined using behavioral cloning, a group of neural networks can be trained to map object information to one or more virtual forces. A group of neural networks (NNs) can be trained to map object information to one or more virtual forces and one or more virtual physical model functions that are different from the perceptual field. A group of neural networks (NNs) may include a first NN and a second NN, where the first NN is trained to map object information to one or more perceptual fields, and the second NN is trained to map object information to one or more virtual physical model functions.

[0058] The initialization step 410 may be followed by a step 420 in which object information is obtained about one or more objects located within the vehicle's environment. Step 410 can be repeated multiple times, and the following steps can also be repeated multiple times. The object information may include video, images, audio, or other sensed information.

[0059] Step 420 may be followed by step 440, which uses one or more neural networks (NNs) to determine one or more virtual forces to be applied to the vehicle.

[0060] One or more NNs may be the entire group of NNs (from initialization step 410), or they may be only a part of the group of NNs, leaving one or more unselected NNs in the group.

[0061] One or more virtual forces represent one or more influences of one or more objects on the vehicle's behavior. These influences can be future or present. An impact may alter the vehicle's trajectory.

[0062] One or more virtual forces belong to a virtual physical model. A virtual physical model is a virtual model to which physical laws (e.g., mechanical laws, electromagnetic laws, optical laws) can be virtually applied to vehicles and / or objects.

[0063] Step 440 may include at least one of the following steps: • Calculate the total virtual force applied to the vehicle based on one or more virtual forces applied to the vehicle. The desired virtual acceleration of the vehicle is determined based on the total virtual acceleration applied to the vehicle by all virtual forces. The desired virtual acceleration may or may not be equal to the sum of the virtual accelerations.

[0064] Method 400 may also include at least one of steps 431, 432, 433, 434, 435, and 436.

[0065] Step 431 may include determining the status of the vehicle based on object information.

[0066] Step 431 may be followed by step 432, which selects one or more neural networks (NNs) based on the situation.

[0067] Alternatively, step 431 may be followed by step 433, which provides situation metadata to one or more NNs.

[0068] Step 434 may include discovering the class of one or more objects based on their object information.

[0069] Step 434 may be followed by step 435, which selects one or more NNs based on the class of at least one of the objects.

[0070] Alternatively, step 434 may be followed by step 436, which provides one or more NNs with class metadata indicating the class of at least one of the objects.

[0071] Step 440 may be followed by step 450, in which one or more driving-related operations of the vehicle are performed based on one or more virtual forces.

[0072] Step 450 can be performed without human driver intervention and may include changing the vehicle's speed and / or acceleration and / or direction of travel. This may include performing autonomous driving or performing advanced driver-assistance system (ADAS) driving operations, which may include temporary control of the vehicle and / or one or more driving-related units of the vehicle. This may include setting the vehicle's acceleration to a desired virtual acceleration, with or without human driver involvement.

[0073] Step 440 may include suggesting to the driver that the vehicle's acceleration be set to a desired virtual acceleration.

[0074] FIG. 5 is an example of a vehicle. The vehicle may include one or more sensing units 501, one or more driving-related units 510 (such as an autonomous driving unit, an ADAS unit, etc.), a processor 560 configured to execute any of the above methods, a memory unit 508 for storing instructions and / or results of methods, functions, etc., and a communication unit 504 for in-vehicle and / or out-of-vehicle communication.

[0075] FIG. 6 shows an example of a method 600 for lane centering RL with lane sample points as inputs. The sample points of the lane are within the environment of the vehicle.

[0076] FIG. 6 shows inference steps 610-640 (some of which may include using one or more NNs), and a step 670 of adjusting the perception field calculated by one or more NNs using RL. RL may change the perception field calculated by one or more NNs.

[0077] RL assumes a simulation environment that can generate input data for implementing the policy (perception field) learned by an agent (ego vehicle).

[0078] Method 600 can start with step 610 of detecting the closest lane or side of road sample points (X L,i , Y L,i ) and (X R,i , Y R,i ), where L is left, R is right, and index i refers to the sample point. The speed of the ego vehicle (previously referred to as the vehicle) is V ego .

[0079] After step 610, concatenate the left lane input vector (X L,i , Y L,i ) and V ego into X L , and concatenate the right lane input vector (X R,i , Y R,i ) and V ego into XR Step 620, which connects to this, follows.

[0080] After step 620, the lane perception field f θ (X L ) and f θ (X R Step 630 may follow, which calculates ). This is done by one or more neural networks.

[0081] After step 630, the differential equation a=f describes the acceleration applied to the vehicle. θ (X L )+f θ (X R Step 640 may follow, which constructs the structure.

[0082] This could be the output of the inference process. Step 640 is followed by step 450 (not shown).

[0083] Method 600 may also include step 670, which evaluates the perceptual field by applying RL to perform acceleration in a simulated environment and evaluating the results by, for example, applying a reward function. Evaluation may include deciding to update the perceptual field by updating the weights of one or more NNs (for example, to maximize the average reward). Step 670 may include using any RL algorithm, e.g., PPO, SAC, TD3, etc.

[0084] Figure 7 shows a multi-object RL method 700 with visual input.

[0085] Step 710 of Method 700 involves a series of panoramic segments (images taken by the vehicle) from the ego's viewpoint over a short time window, with the relative distance X to each individual object. rel,i This may include receiving.

[0086] After step 710, the spatiotemporal CNN is applied to individual instances (objects) to obtain high-level spatiotemporal features X i Step 720 may follow, which involves capturing the object.

[0087] After step 720, individual perceptual fields f θ (X i, i) and total Σf θ (X rel,I ,X i, Step 730 may follow, calculating i).

[0088] Step 740 follows step 730, and a differential equation a = Σf is given that describes the acceleration applied to the vehicle. θ (X rel,I ,X i, i) construct the following.

[0089] This could be the output of the inference process. Step 740 is followed by step 450 (not shown).

[0090] Method 700 may also include step 770, which evaluates the perceptual field by applying RL, for example by applying a reward function, implementing acceleration in a simulated environment, and evaluating the results. Evaluation may include deciding to update the perceptual field by updating the weights of one or more NNs (for example, to maximize the average reward). Step 770 may include using any RL algorithm, such as PPO, SAC, TD3, etc.

[0091] Figure 8 shows the method 800 for multi-object BC using kinematic input.

[0092] Step 810 of Method 800 is to determine the relative kinematics of the detected object (X rel,i ,V rel,i This may include receiving a list of X rel,i This is the relative position of the detected object i- to the vehicle, and Vrel,i V is the relative velocity of the detected object i- to the vehicle. ego It also receives.

[0093] After step 810, perceptual field f for each object θ (X rel,i ,V rel,i ,V ego, Step 820 may follow, calculating i).

[0094] Step 820 may be followed by step 830, which sums the contributions from the individual perceptual fields. Step 830 may also include normalizing the resulting two-dimensional vector so that its magnitude is equal to the maximum magnitude of each individual term:N * Σf θ (X rel,i ,V rel,i ,V ego, i).

[0095] Step 840 follows step 830, and a differential equation a=N describes the acceleration applied to the vehicle. * Σf θ (X rel,i ,V rel,i ,V ego, i) construct the following.

[0096] This could be the output of the inference process. Step 840 is followed by step 450 (not shown).

[0097] This method may include updating one or more network parameters.

[0098] This method is based on initial conditions (Math 1) The program may include step 860 to calculate the vehicle's trajectory when TIFF2023039926000002.tif828 is given. This may include applying acceleration as an initial condition to determine the next condition.

[0099] After step 860, (Math 2) Step 870 may follow, which calculates TIFF2023039926000003.tif882. The loss calculated in step 870 can be propagated to modify at least some of the weights of one or more neural networks used in step 820.

[0100] Figure 9 shows the inference of Method 900, which adds a loss function to an adaptive cruise control model implemented with kinematic variables as inputs.

[0101] Step 910 of Method 900 is the position of the vehicle X ego , own vehicle speed ego The position of the nearest vehicle in front of your car X CIPV , and the speed V of the nearest vehicle in front of your vehicle CIPV This may include receiving.

[0102] After step 910, relative position X rel =X ego -X CIPV , and relative velocity V rel =V ego -V CIPV Step 920, which calculates the result, can follow.

[0103] Step 920 may be followed by the following step 930. • The first neural network allows the perceptual field function g θ (X rel ,V CIPV Calculate ). • Auxiliary function h ψ (V rel ) is calculated using the second neural network. ·g θ (X rel ,V CIPV ) to h ψ (V rel Multiply by () to find the target acceleration (equal to the target force).

[0104] This could be the output of the inference process. Step 930 is followed by step 450 (not shown).

[0105] This method may involve updating one or more NN parameters.

[0106] This method is based on initial conditions (Math 3) Given TIFF2023039926000004.tif828, the program can include step 960 to calculate the vehicle's trajectory.

[0107] After step 960, (Math 4) Step 970 may follow, which calculates TIFF2023039926000005.tif881. Then, at least some of the weights of one or more neural networks used in step 930, which propagates the loss, are modified.

[0108] In the aforementioned specification, the present invention has been described with reference to specific examples of embodiments of the invention. However, it will be apparent that various modifications and changes can be made without departing from the broader spirit and scope of the invention as described in the appended claims. Furthermore, terms such as “front,” “back,” “up,” “down,” “up,” and “down,” when used in the description and claims, are used for descriptive purposes only and not necessarily for descriptive purposes to describe permanent relative positions. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances so that the embodiments of the present invention described herein may operate in orientations other than those illustrated or described herein. Furthermore, the terms “assert” or “set” and “negate” (or “deassert” or “clear”) are used herein when referring to the process of setting a signal, status bit, or similar device to a logically true or logically false state, respectively. If a logically true state is at logical level 1, then a logically false state is at logical level 0. Conversely, if a logically true state is at logical level 0, then a logically false state is at logical level 1. Those skilled in the art will recognize that the boundaries between logic blocks are merely illustrative, and that alternative embodiments may merge logic blocks or circuit elements, or impose alternative functional decompositions on various logic blocks or circuit elements. Therefore, it should be understood that the architectures presented herein are merely examples, and in practice, many other architectures can be implemented to achieve the same functionality. Any arrangement of the components to achieve the same functionality will be effectively "associated" in such a way that the desired functionality is achieved. Therefore, any two components combined herein to achieve a particular function can be considered "related" to each other, regardless of whether they are architecture or intermediate components, so as to achieve the desired function. Similarly, any two components that are associated in this way can be considered to be "operably connected" or "operably coupled" to each other in order to achieve the desired function. Furthermore, those skilled in the art will recognize that the boundaries between the operations described above are merely illustrative. Multiple operations can be combined into a single operation, a single operation can be distributed into additional operations, and operations can be performed with at least partial overlap. Furthermore, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be modified in various other embodiments. Furthermore, for example, in one embodiment, the illustrated examples may be implemented as circuits arranged on a single integrated circuit or within the same device. Alternatively, these examples may be implemented as any number of separate integrated circuits or separate devices interconnected in an appropriate manner. However, other modifications, variations, and substitutions are possible. Therefore, the specification and drawings should be considered illustrative, not restrictive. In the claims, reference symbols placed in parentheses should not be interpreted as limiting the scope of the claims. The word "includes" does not preclude the existence of any elements or steps other than those described in the claim. Furthermore, the terms "a" or "an" are defined as one or more when used herein. Furthermore, the use of introductory phrases such as "at least one" or "one or more" in a claim applies to an invention containing only one such element, even if the introduction of another claim element with the indefinite article "a" or "an" in the same particular claim includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an". The same applies to the use of the definite article. Unless otherwise specified, terms such as "first" and "second" are used to arbitrarily distinguish between the elements they describe. Therefore, these terms are not necessarily intended to indicate any temporal or other prioritization of such elements. The mere fact that certain measures are described in different claims does not mean that combinations of these measures cannot be used to one's advantage. While certain features of the present invention are illustrated and described herein, many modifications, substitutions, alterations, and equivalents will be apparent to those skilled in the art. Therefore, it should be understood that the appended claims are intended to cover all modifications and alterations that fall within the true spirit of the invention. For clarity, it should be understood that various features of embodiments of this disclosure described in the context of separate embodiments may be provided in combination in a single embodiment. Conversely, for the sake of brevity, various features of the embodiments of the present disclosure described in the context of a single embodiment may be provided separately or in any suitable subcombination. Those skilled in the art will understand that the embodiments of the present disclosure are not limited to those specifically shown and described above. Rather, the scope of the embodiments of this disclosure is defined by the appended claims and their equivalents.

Claims

1. A method for operating related to a perception field, comprising: obtaining object information regarding one or more objects arranged in the environment of a vehicle; using one or more neural networks (NNs) to determine one or more virtual forces applied to the vehicle, wherein the one or more virtual forces represent one or more influences of the one or more objects on the behavior of the vehicle, and the one or more virtual forces belong to a virtual physical model; executing one or more driving-related operations of the vehicle based on the one or more virtual forces. A method.

2. The determination includes calculating a total virtual force applied to the vehicle based on the one or more virtual forces applied to the vehicle. The method according to claim 1.

3. Determining a desired virtual acceleration of the vehicle based on a total virtual acceleration applied to the vehicle by the total virtual force. The method according to claim 2.

4. The desired virtual acceleration is equal to the total virtual acceleration. The method according to claim 3.

5. Judging the situation of the vehicle based on the object information. The method according to claim 1.

6. Selecting one or more NNs based on the situation. The method according to claim 5.

7. Feeding situation metadata to the one or more NNs. The method according to claim 5.

8. Detecting the class of each of one or more objects. The method according to claim 1.

9. Selecting the one or more NNs based on the class of at least one of the one or more objects. The method according to claim 8.

10. The one or more NNs are trained to map the object information to the one or more virtual forces using reinforcement learning having a reward function defined using behavior cloning. The method according to claim 1.

Images