Redundant structure for automatic or autonomous driving

Abstract

A method for determining a trajectory of a vehicle includes accessing a plurality of sensors (106) of the vehicle to define a plurality of environmental models, the plurality of environmental models including a primary environmental model (112) and a plurality of secondary environmental models (114); wherein the primary environment model uses the sensor data of the plurality of sensors, and each secondary environment model uses the sensor data of a subset of the plurality of sensors. The method further includes calculating a primary trajectory of the vehicle using the primary environmental model (116) and verifying the primary trajectory using each of the plurality of secondary environmental models (120). In addition, the invention further defines a corresponding device and a vehicle.

Description

Technical Field

[0001] This application relates to a technique for determining trajectories for automated, autonomous, or semi-autonomous vehicles using a redundant architecture. It also relates to a method for determining vehicle trajectories, corresponding equipment, and a vehicle capable of automatically determining trajectories. Background Technology

[0002] Technological advancements have driven the rapid development of autonomous and automated vehicles, which have enormous potential to transform transportation in terms of safety and efficiency, while also reducing energy consumption. Autonomous or automated vehicles refer to vehicles that can navigate and drive with little or no human intervention. This is achieved through the use of built-in sensors, which are fed to computer vision and machine learning components that generate control signals for the autonomous or automated vehicles.

[0003] The automation level of autonomous or automated vehicles is defined by the Society of Automotive Engineers (SAE) J3016 standard. This standard describes the automation levels and classifies vehicles into levels 0 (no automation) to 5 (full automation) based on their capabilities. At Level 0 automation, the human driver is responsible for all aspects of driving, including vehicle control and environmental monitoring. Partial automation begins at Level 2; the driver still needs to focus on driving and monitoring the environment, but several automated functions work together to control the vehicle's operation. Therefore, autonomous or automated vehicles can also be called semi-autonomous vehicles. Level 5 full automation means the vehicle can achieve complete autonomy in all driving scenarios and operating conditions.

[0004] One of the key challenges facing autonomous vehicles at Level 2 and above automation is accurately and reliably determining their trajectories. A vehicle trajectory refers to its travel path, and the ability to plan and adjust these trajectories is crucial for ensuring the safety and efficiency of vehicle operation. The trajectory determination process not only needs sufficient accuracy to handle complex traffic scenarios but also sufficient adaptability to respond to dynamic changes in the environment, such as constantly changing road conditions, traffic signals, or moving objects.

[0005] Existing trajectory determination schemes rely on environmental information, which makes it difficult to achieve the expected reliability and robustness in complex and dynamic environments. They may also fail to provide results quickly enough to adapt to dynamic environmental changes. Errors in the trajectory can lead to serious consequences, including traffic accidents and congestion. Therefore, there is an urgent need for an efficient architecture with high reliability and robustness.

[0006] Safety-related standards for autonomous driving include ISO 26262 and Expected Functional Safety (SOTIF, ISO / PAS21448). Expected Functional Safety addresses the safety of autonomous driving functions under non-fault conditions, while ISO 26262 focuses on safety issues under fault conditions.

[0007] Redundancy is a widely used concept in various technical fields to improve the reliability and robustness of systems. Redundant systems typically involve using additional or duplicated components that can perform the same task in the event of a failure of the primary component.

[0008] Redundancy methods for determining the trajectory of autonomous vehicles have been discussed. For example, US Patent Application Publication No. US2019 / 0079513A1 proposes a fault-tolerant control method for autonomous vehicles. This method employs two control channels to receive data characterizing the autonomous vehicle's motion planning and dynamic state, and uses this data to execute the autonomous vehicle's motion planning. The multiple control channels include a first control channel and a second control channel. The first control channel executes the motion planning, and when a fault is detected during the execution of the motion planning, the motion state of the autonomous vehicle is adjusted based on the fault response parameters.

[0009] US Patent Application Publication No. 2021 / 0122384 A1 discloses a dual-path system that achieves hardware redundancy by calculating control information separately in independent units. The driver assistance device calculates the control inputs of downstream vehicle devices based on surrounding information, and the downstream devices calculate the control inputs of the driver assistance device based on surrounding information. A steering diagnostic unit in the downstream device determines whether the control inputs of the driver assistance device are consistent within a preset time, indicating normal operation; otherwise, they indicate an anomaly. However, this hardware redundancy-based safety scheme cannot meet the expected functional safety architecture goals of vehicles with Level 2 and higher levels of automation.

[0010] US Patent Application Publication No. 2022 / 0055651 A1 relates to the calculation of multiple trajectories. The method generates a set of candidate trajectories, calculates a score based on parameter values ​​associated with the corresponding final trajectory, uses this score to determine a trajectory and uses it as a warm-start trajectory for trajectory optimization, and uses this warm-start trajectory to develop the vehicle's final trajectory.

[0011] Existing methods have shortcomings in terms of efficiency, reliability, and robustness in autonomous vehicle trajectory planning, especially in meeting the expected functional safety requirements of vehicles with Level 2 and above automation. Therefore, there is an urgent need in the field for a technical solution that can achieve efficient trajectory determination and possess higher reliability and robustness. Summary of the Invention

[0012] The objective of this invention is achieved by the vehicle trajectory determination method, apparatus, and vehicle according to the independent claims. This application also defines one or more computer-readable media, and preferred embodiments are defined by the dependent claims.

[0013] A first aspect of this application provides a method for determining a vehicle trajectory, comprising: accessing multiple sensors of a vehicle to define multiple environmental models, the multiple environmental models including a primary environmental model and multiple secondary environmental models, wherein the primary environmental model uses sensor data from the multiple sensors, and each secondary environmental model uses sensor data from a subset of the multiple sensors; calculating a primary trajectory of the vehicle using the primary environmental model; and verifying the primary trajectory using at least one of the multiple secondary environmental models.

[0014] This method can be a computer-implemented method, executable on an electronic device including one or more mutually coupled processors or processing units (PUs) configured to perform the steps of this method. The one or more processors or processing units may implement the steps of this method through hardware and / or software modules. Examples may include an access processing unit or access interface, a computation processing unit or processor, and / or a verification processing unit or verifier capable of performing one or more steps according to the method or implementation of the first aspect.

[0015] The multiple environmental models are defined based on the vehicle's available sensors. A vehicle's environmental model may involve using various sensors to perceive the surrounding environment, analyzing the perceived information, and / or constructing a dynamic and detailed representation of the environment to provide a basis for safe and efficient vehicle navigation within that environment. This process may involve collecting (raw or pre-processed) sensor data using a set of available sensors (or a subset thereof). Since each type of sensor may have its advantages and disadvantages in terms of environmental conditions or constraints, sensor fusion can be employed to combine data from multiple sensors to construct a more complete, reliable, and accurate environmental representation for the vehicle. Defining the environmental model also includes identifying and classifying objects in the environment, such as distinguishing one or more of other vehicles, pedestrians, cyclists, road signs, obstacles, etc. This process may involve the application of trained machine learning models. The vehicle will also track identified objects to determine their position, speed, or trajectory over time. The environmental model can be constructed using one or more types of collected and processed data. The environmental model can represent static elements such as road surface structures, roads, lanes, or various static objects, as well as dynamic elements such as other vehicles, pedestrians, cyclists, or various moving objects. The environmental model may also include the current state of the vehicle. The environmental model is preferably a dynamic model. The environmental model can be continuously updated based on the latest sensor data and detected changes in the environment.

[0016] The multiple environmental models include a primary environmental model and at least one secondary environmental model. The primary environmental model can utilize all available sensors of the vehicle and their corresponding sensor data. Each secondary environmental model can utilize a subset of available sensors. The sensor subsets corresponding to each secondary environmental model can be different. However, at least some secondary environmental models can also have the same sensor subset. Considering the variability and uncertainty of sensor data, the multiple environmental models can each perform different analyses of the sensor data or make different predictions about the future state of the environment. This can be used to evaluate or validate the planned trajectory, ensuring the trajectory's robustness to uncertainty.

[0017] The master environment model is used to calculate the vehicle's master trajectory. For this purpose, the master trajectory can be determined, for example, by the vehicle's planning and control system. The trajectory generation process considers the vehicle's current state, including its position, speed, direction, and any combination of other vehicle parameters. The generation process also considers other factors, including the vehicle's target state, destination or intermediate waypoints, and / or constraints such as minimum speed, maximum speed, maximum acceleration, road conditions, and / or steering angle (to name just a few), in any combination.

[0018] Typically, a trajectory is validated using an environmental model used to calculate that trajectory. According to this application, the main trajectory is validated using at least one, at least part of, or each of the plurality of secondary environmental models. Validation may include one or more of, and in any combination of, collision detection, feasibility checks, efficiency considerations, and / or safety considerations. For example, potential collision detection may be performed on the main trajectory for objects in the corresponding secondary environmental models. This may involve simulating the movement of the vehicle along the main trajectory and detecting whether it intersects with the paths of other objects predicted in the corresponding secondary environmental models. The validation may also involve ensuring that the vehicle can actually follow the main trajectory under given dynamic characteristics and constraints. For example, if the trajectory requires the vehicle to make sharp turns beyond its capabilities, or to accelerate / decelerate beyond its capabilities, then the trajectory will be considered infeasible. Further validation may include efficiency and safety considerations. The main trajectory can be evaluated based on one or more of the following factors, in any combination: travel time, energy efficiency, ride comfort (e.g., avoiding sharp turns, abrupt acceleration / deceleration, and sudden starts / stops), traffic rule compliance, risk assessment, backup strategies, and safety buffer zones.

[0019] This method enables efficient trajectory determination based on a primary environment model utilizing data from all available vehicle sensors. Defining secondary environment models requires minimal additional hardware or processing resources. Since these multiple secondary environment models rely on different sensor data (or different environment model construction methods), they can highlight the strengths of each sensor or sensor combination, which may depend on conditions such as weather and lighting. Furthermore, different environment construction methods and algorithms can be employed during environment model construction to emphasize these strengths, for example, different fusion techniques or data processing algorithms used in the process. The primary trajectory generated using the primary environment model is validated using at least one secondary environment model, ensuring that if the primary trajectory passes validation, its reliability and robustness are improved.

[0020] Preferably, the method further includes adopting or rejecting the main trajectory based on the verification of the main trajectory using at least one of the plurality of secondary environment models. If at least one of the secondary environment models indicates that the main trajectory would cause a hazardous condition, the main trajectory can be rejected. If multiple or at least a majority of the secondary environment models vote against the main trajectory, the main trajectory can be rejected. Accordingly, if at least a majority of the secondary environment models confirm and / or vote in favor of the main trajectory, the main trajectory can be verified and used to control the vehicle.

[0021] In a further embodiment of the method described in the first aspect of this application, the method further includes: calculating at least one secondary trajectory of the vehicle using at least one of the plurality of secondary environment models. Preferably, the method further includes verifying the at least one secondary trajectory using the plurality of environment models. The at least one secondary trajectory may be verified using the primary environment model. Additionally or alternatively, the at least one secondary trajectory may be verified using the at least one secondary environment model.

[0022] At least one secondary trajectory can be computed in parallel with the primary trajectory. Therefore, while computing the primary trajectory (or substantially within the same timeframe), one or more secondary trajectories can be computed using one or more secondary environment models. The advantage of this approach is that if the primary trajectory is rejected, there are already alternative secondary trajectories available, and these can be directly verified using the multiple environment models. This shortens the decision-making time.

[0023] In a further embodiment of the method described in the first aspect of this application, the at least one secondary trajectory is calculated only if the primary trajectory is rejected. Therefore, the at least one secondary trajectory can be calculated after the primary trajectory. At least one or more secondary trajectories can be calculated using the plurality of secondary environment models, wherein each secondary trajectory is calculated by a different secondary environment model. The at least one or more secondary trajectories can also be calculated using those secondary environment models that voted against the primary trajectory. Additionally or alternatively, the secondary environment models may have a hierarchical order or be ranked. For example, a secondary environment model that relies on sensor data with lower resolution or lower reliability may have a lower ranking. Conversely, a secondary environment model that relies on high-resolution sensors or sensor data with higher reliability may have a higher ranking. One or more secondary trajectories can be calculated using one or more secondary environment models with the highest ranking or higher hierarchical order. These methods can be used in combination. Therefore, multiple secondary trajectories can be calculated based on secondary environment models that voted against the primary trajectory, and further based on one or more higher-ranked secondary environment models. The method of calculating the primary trajectory first and then the secondary trajectory can be implemented on devices with limited processing resources. Furthermore, it can be dynamically determined based on the vehicle's available processing resources. This enables efficient computation even on devices with limited processing resources.

[0024] In a further embodiment of the method described in the first aspect of this application, the method further includes: selecting a trajectory from the main trajectory and the at least one secondary trajectory based on the verification of the main trajectory and the at least one secondary trajectory relative to the plurality of environmental models. The verification may include ranking the trajectories, including the main trajectory and / or at least one secondary trajectory, the ranking being determined based on various verification parameters, including collision detection, feasibility verification, efficiency and safety considerations, or one or more of the corresponding tests (to name a few), and in any combination thereof. The trajectories are ranked according to the corresponding test or verification results. The trajectory with the highest ranking may be selected as the target trajectory for the vehicle.

[0025] In a further embodiment of the method described in the first aspect of this application, verifying the main trajectory and / or the at least one secondary trajectory includes voting on the main trajectory and the at least one secondary trajectory through one or more of the plurality of environment models. In one embodiment, verifying the main trajectory may include voting on the main trajectory through at least one secondary environment model among the plurality of environment models. In one embodiment, verifying the at least one secondary trajectory may include voting on the at least one secondary trajectory through the (main environment model or secondary) environment model involved in the plurality of environment models. Preferably, during the verification process, the main environment model may be used based on one or more technical factors such as processing resources. Voting may be conducted using a majority vote. If the voting mechanism is not based on an odd number of decisions (or environment models), a tie can be resolved by establishing a hierarchical ranking for the environment models, wherein votes can be weighted according to the hierarchy or ranking of the corresponding environment model.

[0026] In a preferred embodiment of the method described in the first aspect of this application, if a majority of the environmental models vote against the primary trajectory, the selection includes choosing a trajectory from the at least one secondary trajectory that receives a majority of votes. The majority vote may also be determined based on weighted voting. If the primary trajectory is rejected, the secondary trajectory that receives the most (weighted) votes may be selected as the target trajectory. The verification can be performed in parallel if sufficient processing resources are available. This ensures a rapid decision-making process.

[0027] In a further embodiment of the method described in the first aspect of this application, the voting produces a "safe" or "unsafe" result for each trajectory. This can be applied to the primary trajectory or one or more secondary trajectories. The environmental model can evaluate one or more trajectories as "safe." The voting mechanism can indicate the trajectory that the majority of the environmental model considers safe, and can select that trajectory as the target trajectory for controlling the vehicle.

[0028] Preferably, the multiple environment models used for voting (such as the multiple sub-environment models) may include an odd number of environment models. This simplifies the voting process, avoids ties, and ensures a majority vote.

[0029] In a further embodiment of the method described in the first aspect of this application, the selection of the trajectory includes a matrix lookup. This application also envisions similar solutions. Matrix lookup is an efficient and simple selection strategy where votes for each trajectory can be compiled into a matrix. The matrix can be organized such that each row represents a trajectory (including primary and secondary trajectories), and each column represents votes from an environment model. In one embodiment, the correspondence between rows and columns can also be interchanged. Thus, each cell of the matrix represents a vote for a specific trajectory from a specific environment model. This matrix can be used to select a trajectory. This can be as simple as selecting the trajectory with the highest total number of votes. Alternatively, more complex decision rules can be employed in the embodiments. For example, in the embodiments, priority can be given to environment models that may be concerned with specific factors, such as prioritizing safety over comfort, or requiring a trajectory to obtain a minimum number of safety votes before it can be considered.

[0030] In a further embodiment of the method described in the first aspect of this application, the trajectory is validated using the plurality of environmental models, including validation based on safety objectives. Validation based on safety objectives may include any combination of considerations of one or more of the following: collision, safe distance, traffic rules, vehicle physical state (including stability and limits), environmental (or road) conditions, emergency situations, etc. For example, the trajectory can be validated by examining whether it would result in a collision with any other road user or object (including other vehicles, pedestrians, obstacles) in the corresponding environmental model. Furthermore, the trajectory should ensure that the vehicle maintains a safe distance from other road users and objects. This safe distance may vary based on the object's speed, size, and / or behavior, as well as current road, surface, or weather conditions. The trajectory should also preferably comply with all applicable traffic rules, including speed limits, stop signs, traffic lights, etc. The trajectory may also be validated for vehicle physical factors. In particular, the trajectory should preferably be within the vehicle's stability and handling limits, thereby avoiding requiring the vehicle to perform excessive acceleration, deceleration, or sharp turns beyond its safe capabilities. Furthermore, adjustments to vehicle speed and following distance may need to be made based on road conditions such as slippery or icy conditions, which can be considered during trajectory verification. It should be understood that these are merely examples of safety objectives, and embodiments of this application may arbitrarily combine any one or more of these objectives, factors, parameters, or considerations to meet specific safety requirements and / or available processing resources and efficiency constraints.

[0031] In a further embodiment of the method described in the first aspect of this application, the method further includes: verifying the main trajectory using the main environment model. Verification based on the environment model used to calculate the trajectory itself constitutes double verification to identify possible calculation errors, thereby improving the security level of trajectory calculation. It should be understood that secondary trajectories can also be verified using the corresponding secondary environment model from which the trajectory was calculated.

[0032] In a further embodiment of the method described in the first aspect of this application, the plurality of secondary environmental models are independent of each other. The independence of the models can be statistical independence, which can be achieved based on processed sensor data and processing methods. For example, this can be achieved through differences in the sensor data used and / or different algorithms (e.g., fusion and parsing of data within the environmental models).

[0033] Preferably, the corresponding sensor subsets used to construct each secondary environment model each contain at least one different sensor from the plurality of sensors to achieve mutual independence of the secondary environment models. Alternatively, at least two secondary environment models may depend on subsets with the same sensors. However, the at least two secondary environment models may employ conceptually independent environment model construction methods. If the construction concepts of the environment models are different and independent, model independence can be achieved even if the same set of sensors is used. For example, an environment model constructed by performing early fusion and processing the results is a different (and independent) environment model from another environment model obtained by training and applying sensor images later. This application is not limited to a specific concept of independence, as long as the environment model reveals different (and independent) behaviors regarding trajectory calculation and verification.

[0034] In a further embodiment of the method described in the first aspect of this application, the method further includes: calculating an actuation sequence for the vehicle based on a trajectory (e.g., a selected trajectory, also referred to as a final trajectory or target trajectory). The actuation sequence of the vehicle determines how the vehicle will actually move to follow the selected trajectory. This process may involve translating the selected trajectory into a series of low-level instructions or actions executable by vehicle actuators, including one or more of a steering wheel, accelerator, and brake. If the selected trajectory is a continuous curve, it can be discretized or decomposed into a series of control segments. Each control segment can represent a short distance that the vehicle intends to travel in a short period of time. The current state of the vehicle can be estimated using available sensor data and / or the vehicle's internal data, including any combination of one or more operating states such as current position, orientation, speed, and acceleration. For each segment of the trajectory, a control point that the vehicle needs to reach can be defined. The control point may include a target position, target speed, or target orientation, and any combination of one or more other parameters. The control algorithm calculates the actuation sequence that the actuators intend to follow to move the vehicle from its current state to a target state. This process may include steering control and / or speed control of the vehicle. The calculated actuation sequence can then be transmitted to the corresponding actuators. For example, a steering command may be sent to the vehicle's steering system, and / or an accelerator or brake command may be sent to the vehicle's engine control unit (ECU) or brakes.

[0035] In a further embodiment of the method described in the first aspect of this application, the method further includes: verifying the actuation sequence with respect to the result of convergence to a selected trajectory within a safety envelope. This verification ensures that the vehicle indeed follows the selected or final trajectory within a preset safety envelope. This may involve continuously comparing the vehicle's actual position, orientation, speed, and / or other states and parameters (to name only) in any combination with expected values ​​defined by the selected trajectory, for example, at discrete control segments. Therefore, deviations relative to the trajectory can be detected to avoid problems with the vehicle control system or the trajectory itself.

[0036] Preferably, the safety envelope is defined by vehicle dynamics, environmental constraints, and one or more thresholds for uncertainties and errors. The safety envelope can be defined around (or within a certain distance range) the selected or final trajectory. This safety envelope represents a safe area where the vehicle can deviate from the precise trajectory without posing a safety risk. The size and shape of the safety envelope can depend on various factors, including any combination of one or more factors such as vehicle size and characteristics, current traffic and road conditions, and / or the expected behavior of other road users and moving objects. Deviations can be reflected by the one or more thresholds. If the vehicle is within the safety envelope, it is considered to be safely following the trajectory. If the vehicle is outside the safety envelope or beyond a threshold distance, it indicates a potential safety risk and may trigger corrective measures. Furthermore, progress toward the target state of the selected or final trajectory can be examined. If the vehicle converges toward the target state, it is considered to be approaching the trajectory along the path. If the vehicle deviates from the target state (and the deviation is within the threshold confidence interval defined by the one or more thresholds), it indicates a problem with the vehicle control system or the trajectory itself and may trigger corrective measures.

[0037] In a further embodiment of the method described in the first aspect of this application, the plurality of sensors includes one or more (optical) cameras, and / or one or more radars, and / or one or more lidars, and / or one or more thermal imaging sensors or cameras. The (optical) cameras may include any combination of low, medium, and high angular resolution cameras, stereo camera pairs, fisheye cameras, narrow-angle cameras, wide-angle cameras, and other types. The lidar sensors may include low, medium, and high angular resolution lidars with distance measurement capabilities. The radar sensors may include low, medium, and high angular resolution radars with direct distance and velocity measurement capabilities. It should be understood that this application does not limit the sensor group or sensor type. Rather, sensors capable of assisting semi-autonomous, autonomous, or automatic driving can be used in the sensor group.

[0038] In a further embodiment of the method described in the first aspect of this application, the method further includes: combining the sensor data with map data and / or the vehicle's self-motion data or the self-motion data of one of the plurality of sensor data. This enables the construction of a comprehensive and accurate environmental model. This combination improves the understanding of the vehicle's environment and its own state within that environment, enhancing its robustness. The map data can be obtained from pre-loaded maps or real-time map services and can provide detailed information on road networks. The map data is used to provide an understanding of the overall structure of the environment. The self-motion data reflects the motion of the sensors and / or the vehicle's own motion and may include motion-based parameters and states of the sensors or the vehicle, which can be obtained from sensors and systems within the vehicle. Combining these different types of data allows the vehicle to perceive its environment more comprehensively and accurately. For example, multiple sensor data may indicate the presence of other vehicles ahead, map data may show that the vehicle is about to enter a narrowing section of road, and the vehicle's self-motion data may show that the vehicle is maintaining its current speed. By combining this information, the vehicle can infer that it is approaching a potentially dangerous or hazardous area of ​​existing traffic ahead and may need to brake or slow down immediately. The vehicle can then plan its movements accordingly.

[0039] A second aspect of this application provides a computer-readable storage medium storing program code, the program code including instructions that, when executed by a processor, cause the processor to perform the method described in the first aspect of this application or any embodiment thereof. Preferably, the program code stored in the computer-readable storage medium contains instructions that, when executed, cause the processor to perform vehicle trajectory determination, including: accessing multiple sensors of the vehicle to define multiple environmental models, the environmental models including a primary environmental model and multiple secondary environmental models, wherein the primary environmental model uses sensor data from the multiple sensors, and each of the secondary environmental models uses sensor data from a subset of the multiple sensors; calculating a primary trajectory for the vehicle using the primary environmental model; and verifying the primary trajectory using at least one of the multiple secondary environmental models.

[0040] A third aspect of this application provides an apparatus including at least one processing component configured to perform the method described in the first aspect of this application or any embodiment thereof. Preferably, the apparatus is configured to perform vehicle trajectory determination, including: accessing a plurality of sensors of a vehicle to define a plurality of environmental models, the environmental models including a primary environmental model and a plurality of secondary environmental models, wherein the primary environmental model uses sensor data from the plurality of sensors, and each of the secondary environmental models uses sensor data from a subset of the plurality of sensors; calculating a primary trajectory for the vehicle using the primary environmental model; and verifying the primary trajectory using at least one of the plurality of secondary environmental models.

[0041] In one embodiment of the device described in the third aspect of this application, the device includes one or more of the following: a trajectory planner configured to calculate the main trajectory and / or the at least one secondary trajectory of the vehicle; a trajectory verifier configured to verify the main trajectory and / or the at least one secondary trajectory using the plurality of environment models; and a trajectory selector configured to select a trajectory from the main trajectory and the at least one secondary trajectory based on the verification of the main trajectory and the at least one secondary trajectory relative to the plurality of environment models.

[0042] A fourth aspect of this application provides a vehicle including multiple sensors and the device described in the third aspect of this application or any embodiment thereof. Preferably, the device is configured to perform vehicle trajectory determination, including: accessing multiple sensors of the vehicle to define multiple environmental models, the environmental models including a primary environmental model and multiple secondary environmental models, wherein the primary environmental model uses sensor data from the multiple sensors, and each of the secondary environmental models uses sensor data from a subset of the multiple sensors; calculating a primary trajectory for the vehicle using the primary environmental model; and verifying the primary trajectory using at least one of the multiple secondary environmental models.

[0043] In one embodiment of the vehicle described in the fourth aspect of this application, the vehicle is an automatic vehicle, an autonomous vehicle, or a semi-autonomous vehicle.

[0044] It should be understood that embodiments of the third or fourth aspect of this application may configure logic, processors, or functional units according to the features of the first aspect of this application or one of its implementations. Preferably, the device and vehicle may be configured according to any combination of embodiments of the first aspect of this application. Similarly, the method described in the embodiments of the first aspect of this application may include processing steps that reflect the structural features of the device or vehicle in any combination.

[0045] According to other aspects and embodiments of this application, a vehicle trajectory determination system and software architecture design are also defined, which can be configured to perform the methods described in the first, second, third, and fourth aspects and their respective embodiments of this application in any combination, or to implement the device. Attached Figure Description

[0046] To illustrate the technical features of the embodiments of this application, the accompanying drawings used to explain the embodiments are briefly described below. It should be noted that the drawings described below are only some embodiments of this application, and modifications can be made to these embodiments without departing from the scope of protection of the present invention as defined by the claims.

[0047] Figure 1 This is a flowchart of a method according to an embodiment of this application.

[0048] Figure 2 This is a flowchart of a method according to another embodiment of this application.

[0049] Figure 3 This is a flowchart of a method according to an embodiment of this application.

[0050] Figure 4 This is a schematic diagram of a vehicle having a processing component according to an embodiment of this application. Detailed Implementation

[0051] In the following description, various embodiments are illustrated with reference to the accompanying drawings. Furthermore, several examples will be used to describe various embodiments. It will be understood that the embodiments of this application may be modified in design and structure without departing from the scope of protection defined by the claims.

[0052] The technology described in this application can be implemented in various computing systems, examples of which will be described in detail below. Such systems typically involve configuring suitable computing devices to implement multiple modules, each providing one or more operations required to perform this technology; the implementation of each module may differ and need not be consistent. As used herein, a module can be a structural component in the system that performs an operational role; this component can be a portion of a software element or an entire software element (e.g., a function of a process, a standalone process, or other suitable implementation). Modules may contain computer-executable instructions encoded on a computer storage medium. Modules may execute in parallel or serial mode as needed and may exchange information with each other using shared memory on the computer on which they run, using message passing protocols, or other suitable methods. Exemplary modules performing one or more tasks will be described below. It should be noted that the described modules and task divisions are only illustrative of the types of modules implementing the exemplary technology of this application, and the scope of protection of this invention is not limited to a specific number, division method, or type of modules. In some implementations, all functions may be implemented by a single module. Furthermore, for ease of explanation, the modules are described below as executing on a single computing device. It should be noted that in some implementations, the modules may be implemented on independent computing devices that communicate with each other. Specifically, according to one or more embodiments of this application, the modules may be implemented on a device that processes analog sensor data, on a host system, or in a simulated test environment, and these implementations may be combined arbitrarily.

[0053] This application uses various abbreviations. For example, ODD can be used to refer to the Operational Design Domain. MRM can be used to refer to Minimum Risk Maneuver. ASIL can be used to refer to the Vehicle Safety Integrity Level. SOTIF can be used to refer to the ISO 21448 standard, i.e., Expected Functional Safety. ECU can be used to refer to the Electronic Control Unit. SW can be used to refer to software. LiDAR can be used to refer to lidar, and HD can be used to refer to high definition, such as when used in conjunction with maps. Other abbreviations well known to those skilled in the art may also be used in this application.

[0054] Figure 1 The flowchart below illustrates a method according to an embodiment of this application. Method 100 may be a vehicle trajectory determination method. Method 100 may begin with step 102. Sensor data 104 may be acquired from multiple sensors 106 of the vehicle 108.

[0055] Using sensor data 104, more than 110 environmental models are defined for vehicle 108, including a main environmental model 112 and multiple secondary environmental models 114; the main environmental model 112 uses (and is defined by) sensor data from the multiple sensors 106, and the secondary environmental models 114 use sensor data from a subset of the multiple sensors 106.

[0056] The multiple environmental models, including the primary environmental model 112 and multiple secondary environmental models 114, can be independent of each other. Therefore, while the primary environmental model can utilize sensor data from all available sensors 106, each subset of sensors used to define the data basis of the secondary environmental models 114 can include at least one different sensor from the multiple sensors. However, the same sensors can be used in at least two secondary environmental models 114. In this case, their independence can be achieved by employing a conceptually independent environmental model construction method for the at least two secondary environmental models 114.

[0057] In step 116, the main trajectory 118 is calculated using the main environment model 112, which is an all-in model combining all sensor data. In step 120, the main trajectory 118 generated using the main environment model 112 is verified using at least one of multiple secondary environment models 114. As shown by the dashed line, as an optional approach, the main trajectory 118 can also be verified using the main environment model 112 to validate the calculation in step 116. Method 100 may end at step 122.

[0058] Although the secondary environment model 114 may not use all available sensor data from the vehicle 108, it has advantages for specific operating conditions or is specifically designed for those conditions. Therefore, by using at least one secondary environment model 114 to validate the primary trajectory 118 determined using the full (primary) environment model 112, the safety, robustness, and effectiveness of the primary trajectory 118 can be improved before it is used to control the vehicle 108.

[0059] For example, different types of sensors may have different capabilities and different focuses under different environmental conditions. Therefore, the secondary environment model 114 can emphasize the capabilities and focuses of the types of sensors used to better reflect and highlight the corresponding environmental conditions.

[0060] For example, an optical camera may have the highest angular resolution or information density, but lacks direct distance measurement capabilities. Camera sensor data may be affected by low light and low visibility conditions. A lidar sensor may have moderate angular resolution or information density. LiDAR typically has direct distance measurement capabilities and is unaffected by lighting conditions, but is affected by rain, snow, fog, etc. Radar may have the lowest angular resolution or information density, but has direct distance and speed measurement capabilities; however, radar sensors typically cannot provide lane information, but are unaffected by lighting, rain, snow, fog, etc. Other sensors, such as thermal imaging cameras, acoustic sensors, or similar sensors, may also be used in any combination, and all such combinations are covered within the scope of this application.

[0061] Regarding the possible number of sensors in one or more embodiments of this application, preferably, multiple cameras may be used. The multiple cameras may include any combination of different types of cameras such as stereo camera pairs, fisheye cameras, narrow-angle cameras, and wide-angle cameras. Although only one LiDAR is typically used in various applications, multiple LiDAR sensors can also be configured. Furthermore, one or more radars may be used. For example, long-range and short-range radars (or other types of radars) may be configured in vehicle 108.

[0062] This application proposes a trajectory determination and planning scheme suitable for automatic / autonomous driving, including method, equipment, vehicle, system, and software architecture design. The embodiments of this application retain the driving capability based on the richest and best-performing combination of multiple sensors 106 of the vehicle 108 in the main environment model 112, while not reducing the overall safety, robustness, and availability requirements of the system under adverse environmental conditions or system failures.

[0063] This is achieved by employing an optimal or full primary environment model 112, which fully utilizes the different sensor modes and information sources of multiple sensors 106 to achieve the best possible sensing performance. Verification 120 is performed based on independent secondary environment models 114, which neither limits the performance of the optimal (primary) environment model 112 nor requires setting stringent security integrity levels and statistical verification requirements for the primary environment model 112. Verification 120 verifies the trajectory based on environment models 112 and 114 to ensure safety and avoid decision ambiguity. This can rely on majority decisions made during verification 120.

[0064] Figure 2 This is a flowchart of a method according to another embodiment of this application, where at least a portion of the processing of method 200 is compatible with... Figure 1 The processing corresponds to method 100 in China.

[0065] Method 200 may be a vehicle trajectory determination method and may begin at step 202. Sensor data 204 may be acquired from multiple sensors 206 of the vehicle 208.

[0066] Sensor data 204 can be used to define 210 multiple environmental models for vehicle 208, including a primary environmental model 212 and multiple secondary environmental models 214. The primary environmental model 212 uses (and is defined by) sensor data from the multiple sensors 206. The secondary environmental models 214 use sensor data from a subset of the multiple sensors 206.

[0067] In step 216, the main trajectory 218 is calculated using the main environment model 212. In step 220, the main trajectory 218 is verified based on at least one of the multiple secondary environment models 214. Alternatively, the main trajectory 218 can also be verified based on the main environment model 212 in step 220. In step 220, the available environment models can vote on, rank, or employ other suitable techniques, combining any combination of multiple parameters and conditions such as collision detection, feasibility verification, efficiency assessment, or safety assessment, to verify the main trajectory 218.

[0068] The result of verification 220 is used in step 222 to approve or reject the main trajectory 218. If the main trajectory 218 is approved, method 200 may proceed to step 224 to select the main trajectory 218. This selected trajectory may be used to control vehicle 208. Based on new sensor data, method 200 may iteratively execute the process of determining the next trajectory.

[0069] If the primary trajectory 218 is rejected, method 200 may proceed to step 226, using at least one of the multiple secondary environment models 214 to calculate at least one secondary trajectory 228 for the vehicle 208. Each secondary environment model may be used separately to calculate the secondary environment trajectory.

[0070] In step 230, at least one secondary trajectory 228 can be verified using one or more of multiple environment models (including the primary environment model 212 and multiple secondary environment models 214). The verification process may exclude or include the secondary environment model used to calculate the corresponding secondary trajectory to be verified in step 230. The verification method in step 230 may be similar to the verification method for the primary trajectory in step 220.

[0071] Based on the verification results of step 230 (e.g., voting or ranking of the at least one secondary trajectory 228), in step 232, a trajectory is selected from the at least one secondary trajectory 228, which can be used to control the vehicle 208. If new sensor data is available, method 200 can iteratively execute the process of determining the next trajectory. Method 200 can end at step 234.

[0072] Method 200 first calculates the primary trajectory 218 using the primary environment model 212. If the primary trajectory 218 is rejected in step 222, a secondary trajectory 228 is calculated in step 226. This may include secondary trajectories 228 from all secondary environment models 214, or may include secondary trajectories generated by one or more selected secondary environment models 214 (such as secondary trajectories from secondary environment models with higher hierarchical order among the secondary environment models 214). Alternatively, the secondary trajectory 228 may be calculated in step 226 using only the secondary environment models that rejected the primary trajectory 218 in steps 220 and 222. Calculating and verifying the primary trajectory 218 first, and then calculating the secondary trajectory 228, reduces the simultaneous computational resources required, thus enabling implementation on resource-constrained or limited devices.

[0073] Regarding the correspondence between method 200 and method 100, sensor data 204 can at least partially correspond to sensor data 104; sensor 206 can at least partially correspond to sensor 106; vehicle 208 can at least partially correspond to vehicle 108; main environment model 212 can at least partially correspond to main environment model 112; the plurality of secondary environment models 214 can at least partially correspond to the plurality of secondary environment models 114; and / or main trajectory 218 can at least partially correspond to main trajectory 118, and the above correspondences can be any combination. Furthermore, the processing of step 210 can at least partially correspond to step 110; the processing of step 216 can at least partially correspond to step 116; and / or the processing of step 220 can at least partially correspond to step 120, and the above correspondences can be any combination. Therefore, the features disclosed in method 100 can be used and implemented, at least partially, as corresponding features in method 200, and vice versa.

[0074] Figure 3 This is a flowchart of a method according to an embodiment of this application. At least some steps of method 300 may be combined with... Figure 1 Method 100 and / or Figure 2 The processing steps of method 200 correspond to this.

[0075] Method 300 can be a vehicle trajectory determination method. Method 300 may begin at step 302. Sensor data 304 may be acquired from multiple sensors 306 of the vehicle 308.

[0076] Sensor data 304 can be used to define 310 multiple environmental models for vehicle 308, including a primary environmental model 312 and multiple secondary environmental models 314. The primary environmental model 312 uses (and is defined by) sensor data from the multiple sensors 306. The secondary environmental models 314 use sensor data from a subset of the multiple sensors 306.

[0077] In step 316, the main trajectory 318 is calculated using the main environment model 312. In step 320, multiple secondary trajectories 322 are calculated using the multiple secondary environment models 314. The main trajectory 318 and the multiple secondary trajectories 322 can be calculated in at least partially parallel, but they can also be calculated sequentially or in a sequential manner.

[0078] In step 324, the trajectory, including the main trajectory 318 and the multiple secondary trajectories 322, is verified based on the main environment model 312 and / or one or more of the multiple secondary environment models 314. This verification can be performed in parallel, or alternatively, it can be performed serially or sequentially.

[0079] Therefore, if the primary trajectory 318 is calculated first and verified as the first trajectory in step 324, and if the secondary trajectory 322 is subsequently calculated and verified, the processing steps of method 300 can be at least partially related to... Figure 2 The sequential processing steps correspond to those in method 200.

[0080] Using the verification result from step 324, a trajectory is selected in step 326. The verification result may include the voting results of the environmental models that participated in the verification in step 324. For example, if a majority of environmental models vote against the main trajectory 318, then the secondary trajectory that received the majority of votes among multiple secondary trajectories 322 can be selected.

[0081] Votes can be weighted or prioritized based on the primary or secondary environmental model used. The primary environmental model can have a higher ranking. Secondary environmental models can be prioritized based on the sensor data or sensor type used to construct them. The priorities may also include the environmental conditions or parameters of vehicle 308.

[0082] For example, sensor type priorities can be set as follows: During the day and / or in clear conditions, the camera's priority may be higher than that of the lidar, which in turn may be higher than that of the radar; at night, the lidar's priority may be higher than that of the camera, which in turn may be higher than that of the radar; in adverse weather conditions, the radar's priority may be higher than that of the camera, which in turn may be higher than that of the lidar. These conditions reflect exemplary conditions of the selected sensor list. In embodiments, different combinations of conditions and other types of sensors, in any form, can be considered to set priorities.

[0083] The selected trajectory can be used to control vehicle 308. This control may include calculating an actuation sequence for vehicle 308 and transmitting that sequence to the actuators of vehicle 308. For example, a steering command in the actuation sequence may be sent to the steering system, and an accelerator or brake command may be sent to the engine control unit (ECU). The operating status of vehicle 308 can be continuously monitored and evaluated based on a safety envelope. This process may involve accessing more sensor data. Therefore, method 308 may continue with sensor data access, updating and adjusting environmental models 312 and 314, and the subsequent calculation steps of trajectories 318 and 322. Method 300 may terminate at step 328.

[0084] Regarding the correspondence between method 300 and methods 100 and 200, sensor data 304 can at least partially correspond to sensor data 104 or 204; sensor 306 can at least partially correspond to sensor 106 or 206; vehicle 308 can at least partially correspond to vehicle 108 or 208; main environment model 312 can at least partially correspond to main environment model 112 or 212; the plurality of secondary environment models 314 can at least partially correspond to the plurality of secondary environment models 114 or 214; main trajectory 318 can at least partially correspond to main trajectory 118 or 218; secondary trajectory 322 can at least partially correspond to secondary trajectory 228, and the above correspondences can be any combination. Furthermore, the processing in step 310 may correspond at least partially to step 110 or 210; the processing in step 316 may correspond at least partially to step 116 or 216; the processing in step 320 may correspond at least partially to step 226; step 324 may correspond at least partially to steps 220 and 230, or 120; or step 326 may correspond at least partially to steps 232 and / or 224, and the above correspondences may also be in any combination. Therefore, the features disclosed in method 100 or 200 may be used and implemented at least partially as corresponding features in method 300, and vice versa.

[0085] Figure 4 This is a schematic diagram of a vehicle with processing components according to an embodiment of this application. The vehicle 400 can be an autonomous vehicle, a semi-autonomous vehicle, or an automatic vehicle. Although Figure 4 The vehicle in question is in the form of a car, but this application is not limited to passenger cars or land vehicles. It also covers all types of vehicles, including passenger cars, trucks, two-wheelers, bicycles, public transport vehicles, rail vehicles and other types of land vehicles. It may also include water vehicles, aircraft or space vehicles, as well as any other type of vehicle that can drive in an automatic, autonomous or semi-autonomous manner and requires a determined trajectory for vehicle control.

[0086] For example, Figure 4The diagram shows a vehicle outline view from a side perspective. Vehicle 400 may include multiple sensors 402 of various types. Exemplarily, sensors 402 may include one or more lidar sensors 402a, which may be positioned on or on multiple sides and locations of vehicle 400, and can map the environment by emitting and receiving laser beams; one or more cameras 402b, 402c, preferably positioned around the vehicle (e.g., at the front, rear, corners, and sides), for acquiring visual data of the environment; and / or one or more radars 402d, preferably positioned at the front, rear, and corners of vehicle 400, and may also be positioned on the sides, for detecting the distance and relative speed of objects by emitting radio waves. Sensors 402 may also include ultrasonic sensors (not shown), which may be positioned on the bumper or lower part of vehicle 400, and can be used for close-range detection; and GPS or other satellite-based positioning sensors 402e, which can be used to enable vehicle 400 to determine its global position. The vehicle 400 may also include an inertial measurement unit (IMU) 402f, which can be used to measure the acceleration and angular velocity of the vehicle 400 to determine the relative changes in its position and orientation.

[0087] It should be understood that Figure 4 Only exemplary selections of different types of sensors 402 are shown, which are arranged on vehicle 400 in exemplary positions and layouts. This application may cover fewer or more sensors, different types of sensors, and the positions of the sensors on or within vehicle 400 may vary, and their number and combination may be arbitrarily set.

[0088] Data from sensor 402 can be input to processing unit 404, and this transmission process can be completed using one or more wired or wireless data bus systems of vehicle 400. Processing unit 404 can process the data and control actuators 406 of vehicle 400. Actuators 406 may include at least one of a steering actuator 406a (for controlling the direction of vehicle 400), a throttle actuator (for controlling the acceleration and speed of the vehicle), and a brake actuator 406b (for controlling the braking mechanism of the vehicle), and may also include other types of actuators corresponding to various components of vehicle 400. These actuators can be used in any number and combination. Similar to sensor 402, Figure 4 The number, type, and location of the actuators 406 are merely examples. This application may cover fewer or more actuators, actuators of different types, and the locations of the actuators on or within the vehicle 400 may vary. Their number and combination can be arbitrarily set.

[0089] Processing unit 404 may include one or more dedicated units, components, or modules configured to perform specific tasks and may be implemented in hardware, software, or a combination of hardware and software. For example, processing unit 404 may include trajectory determination component 408, which may be configured to determine the trajectory of vehicle 400 based on current sensor data from sensors 402. Trajectory determination component 408 may be configured to receive sensor data from multiple sensors 402 of vehicle 400 to define multiple environmental models, including a primary environmental model and multiple secondary environmental models. The primary environmental model may depend on sensor data from all of the multiple sensors 402; conversely, the secondary environmental models may depend on sensor data from a subset of the multiple sensors 402. Trajectory determination component 408 may be configured to calculate the primary trajectory of vehicle 400 using the primary environmental model and to verify the primary trajectory using at least one of the multiple secondary environmental models.

[0090] For each task, the trajectory determination component 408 may include a trajectory planner 410, a trajectory verifier 412, and a trajectory selector 414. The trajectory planner 410 may be configured to calculate a primary trajectory and / or at least one secondary trajectory for the vehicle 400. The trajectory verifier 412 may be configured to verify the primary trajectory and / or at least one secondary trajectory using multiple environment models. The trajectory selector 414 may be configured to select a trajectory from the primary trajectory and at least one secondary trajectory based on the verification of the primary trajectory and at least one secondary trajectory relative to the multiple environment models. It should be understood that other units or modules of the trajectory determination component 408 may be used to perform the tasks disclosed in the embodiments of this application.

[0091] If the verification passes, the trajectory determination component 408 can calculate the actuation sequence for the vehicle 400 based on the selected trajectory to achieve vehicle control. If the primary trajectory is rejected, the trajectory determination component 408 can be configured to further determine the trajectory, such as using a secondary trajectory calculated using a secondary environment model. The trajectory determination component 408 can be configured to, for example, perform... Figure 1 Method 100 Figure 2 Method 200 or Figure 3 Method 300 includes one or more steps, and the above steps can be combined arbitrarily.

[0092] The embodiments of this application employ a master environment model utilizing all available sensor information and data, reflecting optimal fusion capabilities to construct a master environment model around vehicle 400. The master environment model can also be referred to as a full-scale environment model. Because the master environment model utilizes all available sensor data and information, there is no need to prove its independence or require it to have a high level of security integrity.

[0093] One or more alternative (secondary) environment models can be defined, each employing a subset of the available sensors 402. By using different sensors in different secondary environment models, proof of independence between models is ensured; such models can also be referred to as "independent" environment models. Exemplarily, in an architecture using cameras 402b, 402c, LiDAR 402a, and radar 402d, at least three secondary environment models can be constructed: a camera-only secondary environment model, a LiDAR-only secondary environment model, and a radar-only secondary environment model. It should be understood that three secondary environment models are merely a feasible example; any other number of secondary environment models can be used, such as at least four, five, six, or more. Selecting an odd number of secondary environment models is preferred to avoid a tie, but embodiments of this application can also use an even number of secondary environment models and resolve the tie problem through ranking or other methods.

[0094] The trajectory planner 410 can be used to calculate the target trajectory of the vehicle for all environment models. The trajectory validator 412 can validate the output of the trajectory planner 410 using one or more (or all) environment models. This validation may include validation for critical safety objectives, such as any combination of collisions with obstacles, departure from drivable surfaces, etc. This process may require a moderate safety integrity level (ASIL B or C).

[0095] The trajectory selector 414 can select the optimal trajectory from a performance and safety perspective based on the verification results from the trajectory verifier 412. This process may require a high security integrity level (ASIL D).

[0096] Furthermore, the controller of processing unit 404 can be used to calculate the desired actuation sequence based on the selected trajectory, a process that may require a moderate safety integrity level (ASIL B). The control checker can verify whether the actuation sequence generated by the controller enables the vehicle to converge to the trajectory selected by trajectory selector 414 within a safety envelope.

[0097] The embodiments of this application can achieve autonomous driving based on the best possible environment model (i.e., the main environment model) under all conditions because the main environment model uses all possible sensor inputs to reconstruct the environment. In one example, this may include any combination of various technologies such as cameras, radar, lidar, high-definition maps, or self-motion information. Since sensor modalities as a single source of information may have systematic deficiencies under certain conditions, it is expected that under normal vehicle system conditions, the main environment model can generate the optimal trajectory with the highest availability. However, the system architecture provided by this application can also detect errors in the main environment model. In addition, the system architecture also has the ability to maintain minimum risk condition MRM operation when some sensors or systems fail.

[0098] To achieve the above effects, embodiments of this application use "independent" secondary environment models to monitor the "full" primary environment model, and / or provide seamless failover capabilities when errors are detected in the primary environment model. By performing trajectory verification based on multiple environment models (e.g., regarding predefined safety objectives), the ambiguity of the monitoring task can be reduced. For example, verifying whether a trajectory will collide with obstacles or deviate from the desired drivable area based on environment models can be objectively evaluated, and the trajectory verifier will provide a clear "safe" or "unsafe" vote.

[0099] The embodiments of this application minimize the complexity and computational requirements of modules such as high-security integrity software modules. The selection process for verified trajectories needs to be simplified as much as possible. Based on this, the trajectory verifier 412 provides easily parsable output to the trajectory selector 414. The trajectory selector can be implemented as a matrix lookup, for example, using a two-out-of-three voting mechanism when employing three environment models, and / or introducing dynamic priority rules for secondary environment models. This can be based on operational design domain conditions and the understanding of systematic defects in specific secondary environment models.

[0100] If the trajectory validator 412 and / or the trajectory selector 414 votes against the main trajectory generated by the full main environment model, in the case of three secondary environment models, the two secondary environment models that voted against the main trajectory can be prioritized for executing the trajectory planning of the minimum risk maneuver. The trajectory planning of the minimum risk maneuver can be implemented through a high integrity feedback loop, by reconfiguring the trajectory planner 410 using the corresponding secondary environment models to complete the minimum risk maneuver planning. In this way, there is no need to set up a specific module for the minimum risk maneuver planning; instead, modules considered error-free can be reused to direct the minimum risk maneuver.

[0101] The selected trajectory (including the primary trajectory, a secondary trajectory, or the minimum-risk maneuver trajectory) may be deemed valid and used to manipulate vehicle 400. This process may involve a closed-loop controller. Embodiments of this application employ a design principle that decomposes the trajectory determination task into components with high complexity but low safety integrity levels, and components with low complexity but high safety integrity levels. To ensure the controller output achieves the expected integrity level, a traditional executor-checker mechanism can be used. This mechanism involves one entity (executor) performing the task, and another entity (checker) independently verifying the correctness of the task completion, thereby improving reliability. Therefore, the controller output needs to be monitored to confirm whether it converges to the selected trajectory. It also needs to be verified whether the execution of the actuation sequence keeps the state of vehicle 400 within the safety envelope around the selected trajectory.

[0102] In a preferred embodiment, this application also covers a processing device comprising a plurality of means configured to perform the relevant functions of embodiments of the present invention. Specifically, the device may be configured to determine a trajectory for a vehicle and may include means for accessing multiple sensors of the vehicle to define multiple environmental models, the environmental models comprising a primary environmental model and multiple secondary environmental models, wherein the primary environmental model uses sensor data from the multiple sensors, and each secondary environmental model uses sensor data from a subset of the multiple sensors; means configured to calculate a primary trajectory for the vehicle using the primary environmental model; and means configured to verify the primary trajectory using each of the multiple secondary environmental models. Preferred embodiments of this device may also include other means to implement the details of the first aspect of this application and its embodiments, and may also be combined with… Figures 1 to 4 And the embodiments disclosed in the corresponding description can be combined in any way.

[0103] It should be understood that Figures 1 to 4 The specific implementation details provided are merely preferred examples. This application may also employ other implementation methods that include different components, modules, block diagrams, units, circuits, connections, and links. The scope of protection of this application is not limited to any particular hardware implementation.

[0104] Although this application has described some embodiments in detail, it should be understood that the technical solutions of this application can be implemented in various forms. Specifically, the technical solutions claimed in this application can be implemented in ways different from those exemplified in this specification, and the various features and technical characteristics recorded in this specification can be arbitrarily combined. The embodiments shown herein are only used to illustrate the technical solutions of this application and are not intended to limit the scope of protection of this invention as defined by the claims.

Claims

1. A method for determining a vehicle trajectory, comprising: Access multiple sensors of the vehicle to define multiple environmental models, the multiple environmental models including a main environmental model and multiple secondary environmental models, wherein the main environmental model uses sensor data from the multiple sensors, and each of the secondary environmental models uses sensor data from a subset of the multiple sensors; The main trajectory of the vehicle is calculated using the main environment model. The main trajectory is verified using at least one of the plurality of secondary environment models.

2. The method according to claim 1, further comprising: Based on the verification of the main trajectory using at least one of the plurality of secondary environment models, the main trajectory is adopted or rejected.

3. The method according to claim 1 or 2, further comprising: At least one secondary trajectory of the vehicle is calculated using at least one of the plurality of secondary environment models.

4. The method according to claim 3, wherein, The at least one secondary trajectory is calculated in parallel with the primary trajectory.

5. The method according to claim 3, wherein, The at least one secondary trajectory is calculated only if the primary trajectory is rejected.

6. The method according to any one of claims 3 to 5, further comprising: Based on the verification of the main trajectory and the at least one secondary trajectory relative to the plurality of environmental models, a trajectory is selected from the main trajectory and the at least one secondary trajectory.

7. The method according to any one of the preceding claims, preferably claim 4, wherein, Validating the main trajectory and / or the at least one secondary trajectory includes voting on the main trajectory or the at least one secondary trajectory through the multiple environmental models.

8. The method according to claim 7, wherein, If a majority of the environmental models vote against the primary trajectory, the selection includes choosing a trajectory from the at least one secondary trajectory that receives the majority of votes.

9. The method according to claim 7 or 8, wherein, The voting process generates a "safe" or "unsafe" result for each trajectory.

10. The method according to any one of claims 6 to 9, wherein, Selecting the trajectory includes matrix lookup.

11. The method according to any one of the preceding claims, wherein, The trajectory is validated using the multiple environmental models, including validation based on safety objectives.

12. The method according to any one of the preceding claims, further comprising: The main trajectory is verified using the main environment model.

13. The method according to any one of the preceding claims, wherein, The multiple secondary environment models are independent of each other.

14. The method according to any one of the preceding claims, wherein, Each subset includes at least one different sensor among the plurality of sensors.

15. The method according to any one of the preceding claims, preferably claim 6, further comprising: The actuation sequence for the vehicle is calculated based on the selected trajectory.

16. The method of claim 15, further comprising: Verify the actuation sequence with respect to the results of converging to the selected trajectory within the safety envelope.

17. The method according to claim 16, wherein, The safety envelope is defined by vehicle dynamics, environmental constraints, and one or more thresholds for uncertainty and error.

18. The method according to any one of the preceding claims, wherein, The plurality of sensors include one or more cameras, and / or one or more radars, and / or one or more lidars, and / or one or more thermal imaging cameras.

19. The method according to any one of the preceding claims, further comprising: The sensor data is combined with map data and / or the vehicle's self-motion data or the self-motion data of one of the plurality of sensor data.

20. At least one computer-readable medium having instructions stored thereon, which, when executed by a computing device, cause the computing device to perform the method of any of the preceding claims.

21. An apparatus comprising: At least one processing component is configured to perform the method according to any one of claims 1 to 19.

22. The device of claim 21, further comprising one or more of the following: A trajectory planner configured to calculate the primary trajectory and / or at least one secondary trajectory of the vehicle; A trajectory validator is configured to validate the primary trajectory and / or at least one secondary trajectory using the plurality of environment models; and A trajectory selector is configured to select a trajectory from the main trajectory and the at least one secondary trajectory based on the verification of the main trajectory and the at least one secondary trajectory relative to the plurality of environment models.

23. A vehicle comprising a plurality of sensors and a device according to any one of claims 21 to 22.

24. The vehicle according to claim 23, wherein, The vehicle is an automated vehicle, an autonomous vehicle, or a semi-autonomous vehicle.

Images