Method for cognitive situation awareness using attention-based event structure

Abstract

A method for contextual awareness of a vehicle using perception-inspired event generation includes: receiving perception input data from vehicle sensors; and processing the perception input data to classify and generate parameters related to external entities near the vehicle. The method includes generating a hierarchical event structure that classifies and prioritizes the perception input data by categorizing external entities into regions of interest and prioritizing external entities within those regions based on a risk level value. Higher risk level values ​​indicate higher priority within the region of interest. The method also includes developing vehicle behavior planning based on the hierarchical event structure.

Description

Technical Field

[0001] This disclosure generally relates to vehicles controlled by an automated driving system, and in particular to vehicles configured to automatically control the steering, acceleration and braking of the vehicle during a driving cycle without human intervention. Background Technology

[0002] The operation of modern vehicles is becoming increasingly automated, meaning that driving control is being provided with less and less driver intervention. Vehicle automation has been categorized into numerical levels ranging from zero (corresponding to no automation with complete human control) to five (corresponding to full automation without human control). Various automated driver assistance systems (such as cruise control, adaptive cruise control, and parking assistance systems) correspond to lower levels of automation, while truly “driverless” vehicles correspond to higher levels of automation. Summary of the Invention

[0003] Embodiments according to this disclosure offer numerous advantages. For example, embodiments according to this disclosure include an event structure based on priority-based attention to allow for effective contextual awareness.

[0004] In one aspect of this disclosure, a method for controlling a vehicle includes: receiving perceived input data from vehicle sensors by a controller; and processing the perceived input data using human perception-inspired cognitive analysis by the controller to classify and generate parameters related to external entities near the vehicle. The method includes generating a hierarchical event structure by the controller to classify and prioritize the perceived input data by categorizing external entities into one of high-attention areas, low-attention areas, and no-attention areas. The method also includes developing behavioral planning for the vehicle by the controller and generating control signals by the controller to control the vehicle's actuators.

[0005] In some aspects, the perception input data includes entity data related to external entities near the vehicle, including the lane position of the external entity near the vehicle, the predicted path of the external entity relative to the vehicle, and one or more of the position and orientation of one or more traffic lanes relative to the vehicle. The perception input data also includes vehicle characteristic data, including one or more of the vehicle speed, braking, and the vehicle's planned (projected) driving path.

[0006] In some respects, using human perception-inspired cognitive analytics to process perceptual input data includes generating regional attention level values ​​for external entities, estimating behavioral attention level values ​​for external entities, calculating risk level values ​​for external entities, determining whether anomalies are detected, and changing regional attention level values ​​for external entities when anomalies are detected.

[0007] In some aspects, generating regional attention level values ​​for external entities includes assessing the predicted path of the external entity relative to the vehicle, the position and orientation of one or more lanes relative to the vehicle, and the planned driving path of the vehicle.

[0008] In some aspects, generating a hierarchical event structure involves prioritizing external entities within a region of interest based on their risk level values, where a higher risk level value indicates a higher priority within the region of interest.

[0009] In some respects, it is expressed as x i The regional attention level value of external entities is calculated as follows: L ZA (x i ) = S xi ( Z +α C ( x i ), where Z is the baseline regional attention level value of the external entity. C ( x i ) is the calculation of the complexity of the external entity, and S xi It is the sigmoid function.

[0010] In some respects, the baseline regional attention level value Z is 0 for regions with no attention, 0.4 for regions with low attention, and 0.8 for regions with high attention.

[0011] In some respects, it is expressed as x i The external entity's behavior attention level value is calculated as follows: ,in, Represents external entities x i Position, speed, and heading angle It is the desired location of the external entity. It is the expected speed of the external entity relative to the speed limit. It is the expected heading angle of the external entity.

[0012] In some respects, performing risk level analysis on external entities involves calculating the risk value of the external entity, expressed as... x i The risk value of external entities is calculated as follows: .

[0013] In some respects, external entities x iThe behavioral attention level value is calculated as follows: ,in, α and β The weights are such that (0 ≤ α + β ≤ 1), Represents external entities x i Position, speed, and heading angle It is the desired location of the external entity. It is the expected speed of the external entity relative to the speed limit. It is the expected heading angle of the external entity. S m ( n () is the sigmoid function that converges to the "m" component outside of the minimum and maximum "n" values.

[0014] In another aspect of this disclosure, a motor vehicle includes: a plurality of environmental sensors configured to detect external features near the motor vehicle; a plurality of vehicle sensors configured to detect vehicle characteristics; actuators configured to control steering, acceleration, braking, or gear shifting of the vehicle; and at least one controller electronically communicating with a corresponding sensor among the plurality of environmental sensors, the plurality of vehicle sensors, and the actuator. The at least one controller is programmed with an autonomous driving system control algorithm and configured to automatically control the actuators based on the autonomous driving system control algorithm. The autonomous driving control system algorithm includes: a perception system configured to receive perception input data from the plurality of environmental sensors and vehicle characteristic data from the plurality of vehicle sensors; and a cognitive context awareness module configured to process the perception input data and vehicle characteristic data using human perception-inspired cognitive analysis to analyze external entities near the motor vehicle. This analysis includes generating a hierarchical event structure that classifies and prioritizes the perception input data by categorizing external entities into one of high-attention areas, low-attention areas, and no-attention areas. The autonomous driving control system algorithm also includes a vehicle control module configured to develop behavior planning for the motor vehicle, including planning a driving path. The at least one controller is further programmed to control the actuator according to behavior planning.

[0015] In some respects, the cognitive analysis performed by the cognitive context awareness module includes generating regional attention level values ​​for external entities, estimating behavioral attention level values ​​for external entities, calculating risk level values ​​for external entities, determining whether anomalies are detected, and changing the regional attention level values ​​for external entities when anomalies are detected.

[0016] In another aspect of this disclosure, a method for contextual awareness of a vehicle using perception-inspired event generation includes: receiving perception input data from vehicle sensors by a controller; and processing the perception input data by the controller to classify and generate parameters related to external entities near the vehicle. The method also includes generating a hierarchical event structure by the controller, which classifies and prioritizes the perception input data by categorizing external entities into regions of interest and prioritizing external entities within those regions based on their risk level values. Higher risk level values ​​indicate higher priority within the region of interest. The method further includes developing vehicle behavior planning by the controller based on the hierarchical event structure.

[0017] In some respects, the attention zone is one of the high attention zone, low attention zone, and no attention zone.

[0018] In some aspects, the perception input data includes entity data related to external entities near the vehicle, including the lane position of the external entity near the vehicle, the predicted path of the external entity relative to the vehicle, and one or more of the position and orientation of one or more driving lanes relative to the vehicle. The perception input data also includes vehicle characteristic data, including one or more of the vehicle speed, braking, and the vehicle's planned driving path.

[0019] In some aspects, processing perceived input data includes generating regional attention level values ​​for external entities, estimating behavioral attention level values ​​for external entities, calculating risk level values ​​for external entities, determining whether anomalies are detected, and changing the regional attention level values ​​for external entities when anomalies are detected.

[0020] In some respects, external entities are categorized into areas of urgent concern when anomalies are detected.

[0021] In some aspects, generating regional attention level values ​​for external entities includes assessing the predicted path of the external entity relative to the vehicle, the position and orientation of one or more lanes relative to the vehicle, and the planned driving path of the vehicle.

[0022] In some respects, external entities x i The behavioral attention level value is calculated as follows: ,in, α and β The weights are such that (0 ≤ α + β ≤ 1), Represents external entities x i Position, speed, and heading angle It is the desired location of the external entity. It is the expected speed of the external entity relative to the speed limit. It is the expected heading angle of the external entity. S m ( n () is the sigmoid function that converges to the "m" component outside of the minimum and maximum "n" values.

[0023] In some respects, the risk level value is compared with a predetermined risk threshold, and when the risk level value is lower than the predetermined risk threshold, the external entity is classified in a no-concern area, and the perceived data about the external entity is not stored by the controller.

[0024] Option 1. A method for controlling a vehicle, comprising:

[0025] The controller receives perception input data from the vehicle's sensors;

[0026] The controller uses human-perception-inspired cognitive analytics to process perceptual input data to classify and generate parameters related to external entities near the vehicle.

[0027] The controller generates a hierarchical event structure that classifies and prioritizes perceived input data by categorizing external entities into one of three regions: high attention, low attention, and no attention.

[0028] Vehicle behavior planning is developed using the controller; and

[0029] The controller generates control signals to control the vehicle's actuators.

[0030] Option 2. According to the method of Option 1, wherein the perception input data includes entity data related to external entities near the vehicle, including the lane position of the external entity near the vehicle, the predicted path of the external entity relative to the vehicle, and one or more of the position and orientation of one or more driving lanes relative to the vehicle, and the perception input data also includes vehicle characteristic data of the vehicle, including one or more of the vehicle speed, braking and the planned driving path of the vehicle.

[0031] Option 3. According to the method described in Option 2, wherein using human perception-inspired cognitive analysis to process perceptual input data includes generating regional attention level values ​​for external entities, estimating behavioral attention level values ​​for external entities, calculating risk level values ​​for external entities, determining whether an anomaly is detected, and changing the regional attention level value of external entities when an anomaly is detected.

[0032] Option 4. According to the method in Option 3, generating the regional attention level value of the external entity includes evaluating the predicted path of the external entity relative to the vehicle, the position and orientation of one or more lanes relative to the vehicle, and the planned driving path of the vehicle.

[0033] Option 5. According to the method described in Option 4, generating a hierarchical event structure includes prioritizing external entities within the attention area based on their risk level values, wherein a higher risk level value indicates a higher priority within the attention area.

[0034] Option 6. The method according to Option 4, wherein, is represented as x i The regional attention level value of external entities is calculated as follows: L ZA (x i ) = S xi ( Z +αC ( x i ), where Z is the baseline regional attention level value of the external entity. C ( x i ) is the calculation of the complexity of the external entity, and S xi It is the sigmoid function.

[0035] Option 7. According to the method described in Option 5, the baseline regional attention level value Z is 0 for regions with no attention, 0.4 for regions with low attention, and 0.8 for regions with high attention.

[0036] Option 8. The method according to Option 6, wherein, is represented as x i The external entity's behavior attention level value is calculated as follows: ,in, Represents external entities x i Position, speed, and heading angle It is the desired location of the external entity. It is the expected speed of the external entity relative to the speed limit. It is the expected heading angle of the external entity.

[0037] Option 9. According to the method described in Option 7, wherein performing risk level analysis on external entities includes calculating the risk value of the external entity, expressed as... x i The risk value of external entities is calculated as follows: .

[0038] Option 10. The method according to Option 3, wherein, is represented as x i The external entity's behavior attention level value is calculated as follows: ,in, α and β The weights are such that (0 ≤ α + β ≤ 1), Represents external entities x i Position, speed, and heading angle It is the desired location of the external entity. It is the expected speed of the external entity relative to the speed limit. It is the expected heading angle of the external entity. S m ( n () is the sigmoid function that converges to the "m" component outside of the minimum and maximum "n" values.

[0039] Option 11. A motor vehicle, comprising:

[0040] Multiple environmental sensors are configured to detect external features near motor vehicles;

[0041] Multiple vehicle sensors configured to detect vehicle characteristics;

[0042] An actuator configured to control vehicle steering, acceleration, braking, or gear shifting; and

[0043] At least one controller, which electronically communicates with corresponding sensors and actuators among the plurality of environmental sensors, the plurality of vehicle sensors, and the actuator itself. The at least one controller is programmed with and configured to automatically control the actuators based on an autonomous driving system control algorithm. The autonomous driving control system algorithm includes:

[0044] A perception system configured to receive perception input data from the plurality of environmental sensors and vehicle characteristic data from the plurality of vehicle sensors;

[0045] A cognitive context awareness module, configured to use human perception-inspired cognitive analysis to process perceptual input data and vehicle characteristic data to analyze external entities near a motor vehicle. This analysis includes generating a hierarchical event structure that classifies and prioritizes perceptual input data by categorizing external entities into one of three regions: high-attention areas, low-attention areas, and no-attention areas.

[0046] A vehicle control module configured to develop behavior planning for motor vehicles, including planning driving routes;

[0047] The at least one controller is further programmed to control the actuator according to behavior planning.

[0048] Option 12. The motor vehicle according to Option 11, wherein the cognitive analysis performed by the cognitive context awareness module includes generating a regional attention level value of an external entity, estimating a behavioral attention level value of an external entity, calculating a risk level value of an external entity, determining whether an anomaly is detected, and changing the regional attention level value of the external entity when an anomaly is detected.

[0049] Option 13. A method for generating contextual awareness for vehicles using perception-inspired event generation, comprising:

[0050] The controller receives perception input data from the vehicle's sensors;

[0051] The controller processes the perceived input data to classify and generate parameters related to external entities near the vehicle.

[0052] The controller generates a hierarchical event structure, which classifies and prioritizes perceived input data by categorizing external entities into regions of interest and prioritizing them based on their risk level values. A higher risk level value indicates a higher priority within the region of interest.

[0053] The controller develops vehicle behavior planning based on a hierarchical event structure.

[0054] Option 14. The method according to Option 13, wherein the attention region is one of a high attention region, a low attention region, and a no attention region.

[0055] Option 15. The method according to Option 13, wherein the perception input data includes entity data related to external entities near the vehicle, including lane position of the external entity near the vehicle, predicted path of the external entity relative to the vehicle, and one or more of the position and orientation of one or more driving lanes relative to the vehicle, and the perception input data also includes vehicle characteristic data of the vehicle, including one or more of vehicle speed, braking, and planned driving path of the vehicle.

[0056] Option 16. According to the method in Option 15, the processing of perceived input data includes generating a regional attention level value of an external entity, estimating a behavioral attention level value of an external entity, calculating a risk level value of an external entity, determining whether an anomaly is detected, and changing the regional attention level value of the external entity when an anomaly is detected.

[0057] Option 17. The method according to Option 16, wherein, upon detection of an anomaly, the external entity is classified into an area of ​​urgent concern.

[0058] Option 18. The method according to Option 16, wherein generating the regional attention level value of the external entity includes evaluating the predicted path of the external entity relative to the vehicle, the position and orientation of one or more lanes relative to the vehicle, and the planned driving path of the vehicle.

[0059] Option 19. The method according to Option 16, wherein, is represented as x i The external entity's behavior attention level value is calculated as follows: ,in, α and β The weights are such that (0 ≤ α + β ≤ 1), Represents external entities x i Position, speed, and heading angle It is the desired location of the external entity. It is the expected speed of the external entity relative to the speed limit. It is the expected heading angle of the external entity. S m ( n () is the sigmoid function that converges to the "m" component outside of the minimum and maximum "n" values.

[0060] Option 20. The method according to Option 13, wherein a risk level value is compared with a predetermined risk threshold, and when the risk level value is lower than the predetermined risk threshold, the external entity is classified in a no-concern area, and the perception data about the external entity is not stored by the controller. Attached Figure Description

[0061] This disclosure will be described in conjunction with the following drawings, wherein the same reference numerals denote the same elements.

[0062] Figure 1 This is a schematic diagram of a communication system including an autonomously controlled vehicle according to an embodiment of the present disclosure.

[0063] Figure 2 This is a schematic block diagram of an automated driving system (ADS) for a vehicle according to embodiments of the present disclosure.

[0064] Figure 3 This is a high-level flowchart of a method for cognitive contextual awareness using attention-based event structures according to embodiments of this disclosure.

[0065] Figure 4 This is a high-level flowchart of a hierarchical event structure according to embodiments of the present disclosure.

[0066] Figure 5 This is a schematic diagram of a cognitive situational awareness event according to an embodiment of the present disclosure.

[0067] Figure 6 This is a schematic diagram of another cognitive situational awareness event according to an embodiment of the present disclosure.

[0068] Figure 7 This is a schematic diagram of a cognitive context awareness event according to an embodiment of the present disclosure, illustrating the merging of similar attention areas.

[0069] Figure 8 This is a graphical representation of probability-based behavioral attention according to embodiments of the present disclosure.

[0070] The foregoing and other features of this disclosure will become more fully apparent from the following description and the appended claims, taken in conjunction with the accompanying drawings. It should be understood that these drawings illustrate only a few embodiments according to this disclosure and should not be considered as limiting the scope of this disclosure, which will be described with additional specificity and detail using the drawings. Any dimensions disclosed in the drawings or elsewhere herein are for illustrative purposes only. Detailed Implementation

[0071] This document describes embodiments of the present disclosure. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take different and alternative forms. The drawings are not necessarily drawn to scale; some features may be exaggerated or minimized to show detail of particular components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but merely as a representative basis for teaching those skilled in the art to use the present disclosure in various ways. As will be understood by those skilled in the art, various features illustrated and described with reference to any of the drawings may be combined with features illustrated in one or more other drawings to produce embodiments that are not explicitly illustrated or described. The combinations of illustrated features provide representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of this disclosure may be desired for particular applications or implementations.

[0072] Certain terms used in the following description may be for illustrative purposes only and are not intended to be limiting. For example, terms such as “above” and “below” refer to orientations in the referenced figures. Terms such as “front,” “rear,” “left,” “right,” “rear,” and “side” describe the orientation and / or position of portions of a part or element within a consistent but arbitrary frame of reference, as will become clear from the context of the part or element under discussion and the associated figures. Furthermore, terms such as “first,” “second,” “third,” etc., may be used to describe individual parts. Such terms may include words specifically mentioned above, their derivatives, and words with similar meanings.

[0073] Figure 1 The diagram schematically illustrates the operating environment of a mobile vehicle communication and control system 10 for a motor vehicle 12. The communication and control system 10 for the vehicle 12 generally includes one or more wireless carrier systems 60, a terrestrial communication network 62, a computer 64, a mobile device 57 such as a smartphone, and a remote access center 78.

[0074] Figure 1 The vehicle 12 shown schematically is depicted as a passenger car in the illustrated embodiment, but it should be understood that any other vehicle may be used, including motorcycles, trucks, sports utility vehicles (SUVs), station wagons (RVs), boats, aircraft, etc. Vehicle 12 includes a propulsion system 13, which in various embodiments may include an internal combustion engine, an electric motor such as a traction motor, and / or a fuel cell propulsion system.

[0075] The vehicle 12 also includes a transmission 14 configured to transmit power from the propulsion system 13 to a plurality of vehicle wheels 15 according to a selectable speed ratio. According to various embodiments, the transmission 14 may include a stepped automatic transmission, a continuously variable (CVT) transmission, or other suitable transmission.

[0076] The vehicle 12 also includes wheel brakes 17 configured to provide braking torque to the vehicle wheels 15. In various embodiments, wheel brakes 17 may include friction brakes, regenerative braking systems such as electric motors, and / or other suitable braking systems.

[0077] The vehicle 12 also includes a steering system 16. Although depicted as including a steering wheel for illustrative purposes, in some embodiments contemplated within the scope of this disclosure, the steering system 16 may not include a steering wheel.

[0078] Vehicle 12 includes a wireless communication system 28 configured to communicate wirelessly with other vehicles (“V2V”) and / or infrastructure (“V2I”). In an exemplary embodiment, wireless communication system 28 is configured to communicate via a dedicated short-range communication (DSRC) channel. A DSRC channel refers to a one-way or two-way short-to-medium-range wireless communication channel designed specifically for automotive use and with a corresponding set of protocols and standards. However, wireless communication systems configured to communicate via additional or optional wireless communication standards, such as IEEE 802.11 and cellular data communication, are also considered to be within the scope of this disclosure.

[0079] The propulsion system 13, transmission 14, steering system 16, and wheel brakes 17 communicate with or are under the control of at least one controller 22. Although depicted as a single unit for illustrative purposes, controller 22 may additionally include one or more other controllers, collectively referred to as the "controller". Controller 22 may include a microprocessor or central processing unit (CPU) that communicates with various types of computer-readable storage devices or media. For example, computer-readable storage devices or media may include volatile and non-volatile storage devices in read-only memory (ROM), random access memory (RAM), and persistent active memory (KAM). KAM is persistent or non-volatile memory that can be used to store various operational variables when the CPU is powered off. The computer-readable storage device or media may be implemented using any of a variety of known memory devices, such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or any other electrical, magnetic, optical, or combined memory device capable of storing data, some of which represents executable instructions used by controller 22 in controlling the vehicle.

[0080] Controller 22 includes an Automated Driving System (ADS) 24 for automatically controlling various actuators in the vehicle. In an exemplary embodiment, ADS 24 is a so-called Level 4 or Level 5 automation system. A Level 4 system means “high automation,” referring to all aspects of a dynamic driving task performed by the automated driving system in a specific driving mode, even if the driver does not respond appropriately to an intervention request. A Level 5 system means “full automation,” referring to all aspects of a dynamic driving task performed by the automated driving system at all times under all road and environmental conditions that can be managed by a human driver. In an exemplary embodiment, ADS 24 is configured to control the propulsion system 13, transmission 14, steering system 16, and wheel brakes 17 via multiple actuators 30 in response to inputs from multiple sensors 26, without human intervention, to control vehicle acceleration, steering, and braking, respectively. The multiple sensors 30 may include GPS, RADAR, LIDAR, optical cameras, thermal imaging cameras, ultrasonic sensors, and / or appropriate additional sensors.

[0081] Figure 1 Several networked devices capable of communicating with the wireless communication system 28 of vehicle 12 are illustrated. One of the networked devices capable of communicating with vehicle 12 via the wireless communication system 28 is a mobile device 57. Mobile device 57 may include computer processing power, a transceiver capable of communicating using short-range wireless protocols, and a visual smartphone display 59. The computer processing power includes a microprocessor in the form of a programmable device, which includes one or more instructions stored in an internal memory structure and applied to receiving binary input to create binary output. In some embodiments, mobile device 57 includes a GPS module capable of receiving GPS satellite signals and generating GPS coordinates based on those signals. In other embodiments, mobile device 57 includes cellular communication capabilities, enabling mobile device 57 to perform voice and / or data communications on a wireless carrier system 60 using one or more cellular communication protocols, as discussed herein. The visual smartphone display 59 may also include a touchscreen graphical user interface.

[0082] The radio carrier system 60 is preferably a cellular telephone system, comprising multiple cell towers 70 (only one shown), one or more mobile switching centers (MSCs) 72, and any other networking components required to connect the radio carrier system 60 to the terrestrial communication network 62. Each cell tower 70 includes transmit and receive antennas and a base station, with base stations from different cell towers connected directly to the MSC 72 or via intermediate equipment (such as a base station controller). The radio carrier system 60 can implement any suitable communication technology, including, for example, analog technologies (such as AMP), or digital technologies such as CDMA (e.g., CDMA2000) or GSM / GPRS. Other cell tower / base station / MSC arrangements are also possible and can be used with the radio carrier system 60. For example, base stations and cell towers can coexist in the same location, or they can be located far apart from each other; each base station can serve a single cell tower, or a single base station can serve various cell towers, or various base stations can be connected to a single MSC, to name just a few possible arrangements.

[0083] In addition to using wireless carrier system 60, a second wireless carrier system in the form of satellite communication can be used to provide one-way or two-way communication with vehicle 12. This can be accomplished using one or more communication satellites 66 and uplink transmission station 67. One-way communication may include, for example, satellite radio service, in which program content (news, music, etc.) is received by transmission station 67, packaged for uploading, and then sent to satellite 66, which broadcasts the program to the user. Two-way communication may include, for example, satellite telephone service using satellite 66 to relay telephone communication between vehicle 12 and station 67. Satellite telephone can be utilized in addition to or in place of wireless carrier system 60.

[0084] Terrestrial network 62 may be a conventional terrestrial-based telecommunications network connected to one or more terrestrial telephone lines and connecting wireless carrier system 60 to remote access center 78. For example, terrestrial network 62 may include a Public Switched Telephone Network (PSTN), such as a network used to provide hard-wired telephone, packet-switched data communications, and Internet infrastructure. One or more segments of terrestrial network 62 may be implemented using standard wired networks, fiber optic or other optical networks, cable networks, power lines, other wireless networks (such as wireless local area networks (WLANs) or networks providing broadband wireless access (BWAs) or any combination thereof). Furthermore, remote access center 78 does not need to be connected via terrestrial network 62, but may include wireless telephone equipment that can communicate directly with wireless networks (such as wireless carrier system 60).

[0085] Despite Figure 1While shown as a single device, computer 64 may include multiple computers accessible via private or public networks such as the Internet. Each computer 64 may be used for one or more purposes. In an exemplary embodiment, computer 64 may be configured as a web server accessible by vehicle 12 via wireless communication system 28 and wireless carrier 60. Other computers 64 may include, for example, a service center computer (where diagnostic information and other vehicle data can be uploaded from the vehicle via wireless communication system 28) or a third-party repository (where vehicle data or other information can be provided to or obtained from a third-party repository), whether through communication with vehicle 12, remote access center 78, mobile device 57, or a combination thereof. Computer 64 may maintain a searchable database and a database management system that allows data entry, removal, and modification, as well as the receipt of requests to locate data within the database. Computer 64 may also be used to provide Internet connectivity, such as DNS services, or as a network address server that uses DHCP or other suitable protocols to assign IP addresses to vehicle 12. In addition to vehicle 12, computer 64 may also communicate with at least one supplementary vehicle. Vehicle 12 and any supplementary vehicles may be collectively referred to as a fleet.

[0086] like Figure 2 As shown, ADS 24 includes several different control systems, including at least a perception system 32 for determining the presence, location, classification, and path of features or targets detected near the vehicle. The perception system 32 is configured to receive input from various sensors, such as… Figure 1 The sensor 26 shown is used to synthesize and process sensor inputs to generate parameters that serve as inputs for other control algorithms of ADS24.

[0087] The sensing system 32 includes a sensor fusion and preprocessing module 34, which processes and synthesizes sensor data 27 from various sensors 26. The sensor fusion and preprocessing module 34 performs calibrations on the sensor data 27, including but not limited to LIDAR-to-LIDAR calibration, camera-to-LIDAR calibration, LIDAR-to-chassis calibration, and LIDAR beam intensity calibration. The sensor fusion and preprocessing module 34 outputs preprocessed sensor outputs 35.

[0088] The classification and segmentation module 36 receives preprocessed sensor output 35 and performs target classification, image classification, traffic light classification, target segmentation, ground segmentation, and target tracking processes. Target classification includes, but is not limited to, identifying and classifying targets in the surrounding environment, including identifying and classifying traffic signals and signs, radar fusion and tracking to consider sensor placement and field of view (FOV) via LiDAR fusion, and false positive rejection to eliminate many false positives present in urban environments, such as manhole covers, bridges, elevated trees or lampposts, and other obstacles with high RADAR cross-sections that do not impede a vehicle's ability to travel along its path. Additional target classification and tracking processes performed by the classification and segmentation module 36 include, but are not limited to, free space detection and high-level tracking, which fuse data from RADAR trajectories, LiDAR segmentation, LiDAR classification, image classification, target shape fitting models, semantic information, motion prediction, raster maps, static obstacle maps, and other sources to generate high-quality target trajectories. The classification and segmentation module 36 further performs traffic control device classification and the fusion of traffic control device and lane association and traffic control device behavior models. The classification and segmentation module 36 generates target classification and segmentation outputs 37, including target recognition information.

[0089] The localization and mapping module 40 uses the target classification and segmentation output 37 to calculate parameters, including but not limited to estimates of the vehicle 12's position and orientation in both typical and challenging driving scenarios. These challenging driving scenarios include, but are not limited to, dynamic environments with many vehicles (e.g., dense traffic), environments with large-scale obstacles (e.g., road construction or building sites), hilly terrain, multi-lane roads, single-lane roads, various road markings and buildings or a lack of said markings and buildings (e.g., residential areas versus commercial areas), and bridges and overpasses (both above and below the vehicle's current road segment).

[0090] The positioning and mapping module 40 also incorporates new data collected as a result of expanded map areas obtained by the onboard mapping functions performed by the vehicle 12 during operation, as well as map data “push” to the vehicle 12 via the wireless communication system 28. The positioning and mapping module 40 updates previous map data with new information (e.g., new lane markings, new building structures, addition or removal of building areas, etc.) without modifying unaffected map areas. Examples of map data that can be generated or updated include, but are not limited to, yield line classification, lane boundary generation, lane connection, classification of secondary and primary roads, classification of left and right turns, and intersection lane creation. The positioning and mapping module 40 produces a positioning and mapping output 41 that includes the position and orientation of the vehicle 12 relative to detected obstacles and road features.

[0091] The vehicle odometer module 46 receives data 27 from the vehicle sensor 26 and generates a vehicle odometer output 47, which includes, for example, vehicle heading and speed information. The absolute positioning module 42 receives the positioning and map output 41 and the vehicle odometer information 47 and generates a vehicle position output 43, which is used for the separate calculations discussed below.

[0092] The target prediction module 38 uses the target classification and segmentation output 37 to generate parameters, including but not limited to the position of the detected obstacles relative to the vehicle, the predicted path of the detected obstacles relative to the vehicle, and the position and orientation of the driving lane relative to the vehicle. Data on the predicted paths of targets (including pedestrians, surrounding vehicles, and other moving targets) are output as target prediction output 39 and used for separate calculations discussed below.

[0093] ADS 24 also includes an observation module 44 and an interpretation module 48. The observation module 44 generates observation output 45 received by the interpretation module 48. The observation module 44 and the interpretation module 48 allow access to the remote access center 78. The interpretation module 48 generates interpretation output 49, which includes additional input (if any) provided by the remote access center 78.

[0094] The path planning module 50 processes and synthesizes the target prediction output 39, the interpretation output 49, and the additional route information 79 received from an online database or remote access center 78 to determine the vehicle path to follow to keep the vehicle on the desired route while complying with traffic regulations and avoiding any detected obstacles. The path planning module 50 employs algorithms configured to avoid any detected obstacles near the vehicle, keep the vehicle in the current lane, and keep the vehicle on the desired route. The path planning module 50 outputs vehicle path information as path planning output 51. Path planning output 51 includes a commanded vehicle path based on the vehicle route, the vehicle's position relative to the route, the position and orientation of the lane, and the presence of any detected obstacles.

[0095] The first control module 52 processes and synthesizes the route planning output 51 and the vehicle position output 43 to generate the first control output 53. In the case of remote takeover mode of vehicle operation, the first control module 52 also incorporates the route information 79 provided by the remote access center 78.

[0096] The vehicle control module 54 receives the first control output 53 and the speed and heading information 47 received from the vehicle odometer 46, and generates a vehicle control output 55. The vehicle control output 55 includes a set of actuator commands to implement the command path from the vehicle control module 54, including but not limited to steering commands, gear shift commands, throttle commands, and braking commands.

[0097] Vehicle control output 55 is transmitted to actuator 30. In an exemplary embodiment, actuator 30 includes steering control, shift control, throttle control, and brake control. The steering control can, for example, control... Figure 1 The steering system 16 shown. The shift control unit can, for example, control the steering system 16. Figure 1 The transmission 14 shown. The throttle control unit can, for example, control the following: Figure 1 The propulsion system 13 is shown. The braking control unit can, for example, control such as... Figure 1 The wheel brake 17 shown.

[0098] This disclosure describes methods and systems for generating event structures to represent and understand situations occurring in the environment surrounding an autonomous vehicle while it is driving on a road. By analyzing and prioritizing environmental and behavioral concerns, multiple events can be abstracted such that a single representation produces the same response to the corresponding autonomous entities. Traditional event description structures use geometric region-based or entity-based classifications without adequate abstraction and generalization, thus requiring significant storage to represent a large number of events. Therefore, this disclosure addresses the need to represent a large number of events as experiences under normal operating (i.e., driving) conditions. The methods and systems disclosed herein utilize novel priority-based concern event structures to allow for effective situational awareness.

[0099] Several advantages of the methods and systems disclosed herein include, for example, but not limited to, human-perception-inspired event generation for effective situational awareness, a hierarchical structure including attention zones and behavioral attention, and risk level analysis to determine the priority of events / obstacles / vehicles / pedestrians / etc. within each attention zone. Furthermore, the methods and systems disclosed herein include emergency attention zones for handling anomalous events / obstacles / vehicles / pedestrians / etc., which cause immediate intervention to the autonomous operation of vehicle 12. Finally, the methods and systems disclosed herein effectively compress traffic situational information for efficient data processing. Using the methods and systems disclosed herein, information obtained from sensors of autonomous vehicles such as vehicle 12 can be applied to perceive, reason, and understand the surrounding situation with more human-like capabilities and less computational complexity, without losing key details of events and the environment, leading to improved navigation decisions for ADS 24.

[0100] Appropriate situational awareness is particularly useful for autonomous driving, not only allowing for the safe operation of vehicle 12 but also enabling the understanding of the surrounding environment and the making of appropriate navigation and vehicle control decisions. While a wide variety of information may be expected to be used and stored during the autonomous driving decision-making process performed by ADS 24, for practical reasons, the input data to ADS 24 should be represented, stored, and used efficiently. Therefore, ADS 24 should utilize methods and systems designed for both efficiency and adequacy in decision-making. The methods and systems disclosed herein assess the urgency and threat of the vehicle's current and planned driving paths by considering the adjacent situation around vehicle 12. By combining regional and behavioral concerns and assigning weights to entities within each concern region to focus on the immediate environment, ADS 24 is allowed to handle multiple adjacent entities and complex scenarios.

[0101] Figure 3 A high-level diagram of a method 100 for generating cognitive contextual awareness using attention-based event structures, according to an embodiment, is illustrated. Method 100 can be used in conjunction with various modules of vehicle 12 and ADS 24 as discussed herein. According to an exemplary embodiment, method 100 can be used in conjunction with controller 22 as discussed herein, or by other systems associated with or independent of the vehicle. The order of operation of method 100 is not limited to... Figure 3 The steps may be performed in one or more different orders, or the steps may be performed simultaneously, as applicable to this disclosure.

[0102] At point 102, ADS 24 receives perception input from sensor 26 of vehicle 12. In various embodiments, the perception input includes sensor data from various sensors, including GPS, RADAR, LIDAR, optical cameras, thermal imaging cameras, ultrasonic sensors, and / or suitable additional sensors. The perception input includes data about the surrounding environment and data about vehicle characteristics, including, but not limited to, speed, braking, planned driving path, etc. In various embodiments, the perception input is sensor data related to external features, such as other vehicles, targets, pedestrians, etc., near vehicle 12. In various embodiments, the perception input is received from sensor 26 via perception system 32 of ADS 24.

[0103] Various modules of ADS 24 process sensor data and transmit the data in the form of tokens to the cognitive context awareness module, as shown at 104. In some embodiments, the cognitive context awareness module is a module of ADS 24 and works in conjunction with the localization and mapping module 40 to estimate the position of vehicle 12 in both typical and challenging driving scenarios. Furthermore, the cognitive context awareness module works in conjunction with the target prediction module 38 of ADS 24 to further classify and generate parameters related to the position of detected obstacles relative to vehicle 12, predicted paths of detected obstacles relative to vehicle 12, and the position and orientation of the driving lane relative to vehicle 12. As discussed in more detail herein, the cognitive context awareness function includes the allocation of regional attention to any detected obstacles or entities in the environment surrounding vehicle 12, estimation of behavioral attention to detected obstacles or entities, risk level analysis of detected obstacles or entities, identification of any anomalous entities or behaviors, and reallocation of regional attention to any anomalous entities or behaviors.

[0104] The cognitive situational awareness function generates a corresponding hierarchical event structure, as shown at point 114. The hierarchical event structure is... Figure 4The event structure information is then used to develop behavior planning, as shown at 116. Behavior planning can be performed by the target prediction module 38 of ADS 24 or by another module of ADS 24. Decision behavior, typically in the form of a trajectory, is generated from behavior planning and synthesized with other information used by path planning module 50 to generate a vehicle path to be followed to keep the vehicle on the desired route while complying with traffic regulations and avoiding any detected obstacles. The path planning output 51, including the decision behavior, is sent to a vehicle controller, such as vehicle control module 54, as shown at 118. As described herein, vehicle control module 54 generates one or more control signals or vehicle control outputs 55, which are sent to the hardware of vehicle 12, such as one or more actuators 30, to implement the commanded vehicle path, including but not limited to steering commands, braking commands, and throttle commands. In various embodiments, depending on the computational load, etc., Figure 3 The method 100 outlined herein may be executed by a single controller (e.g., controller 22) or may be distributed across multiple controllers of vehicle 12.

[0105] Continue to refer to Figure 3 More specifically, in the cognitive context awareness step shown at point 104, once the controller 22 receives perceived data from sensor 26, the vehicle electronic control unit (ECU) system, and external environmental feeds, it assigns a regional attention level to each external entity near the vehicle based on the environmental data and the planned path or desired trajectory of vehicle 12, as shown at point 106. At point 108, it performs a behavioral attention estimation for each external entity within each region, considering the entity's actions relative to environmental conditions. At point 110, it performs a risk level analysis for each entity based on its assigned regional attention and behavioral attention. The risk level analysis can reveal anomalies such as unexpected targets, unexpected entity behavior, and / or areas of urgent attention. If necessary, at point 112, it reassigns the attention and behavioral regions of each entity based on any detected anomalies. The environmental attention and behavioral attention data generated for each entity in cognitive context awareness step 104 are stored as a hierarchical event structure at point 114.

[0106] According to the embodiment, the hierarchical event structure 124 is in Figure 4As shown in the diagram. The highest level of event structure 124 includes header information 130, an emergency concern zone 132, a high concern zone 134, a low concern zone 136, and a no concern zone 138. If an unusual target or unusual action occurs in the environment surrounding vehicle 12, the target or action entity is listed in the emergency concern zone 132. Each zone is listed in descending order of priority; that is, entities classified in the emergency concern zone 132 receive the highest priority, entities classified in the high concern zone 134 receive the second highest priority, and so on. Entities within each zone are assigned a risk level. Figure 4 As shown, entities 140 and 142 within the emergency concern zone 132 are assigned risk levels. Entities within each zone are ranked by risk level, with entities at the highest risk level ranked higher than those at lower risk levels. Similarly, entities 144 and 146 are classified in the high concern zone 134, assigned risk levels, and appropriately ranked. Furthermore, entities 148 and 150 are classified in the low concern zone 136 and ranked according to their assigned risk levels. Environmental considerations, such as known traffic patterns in the area, allow for the selective storage of entities at or above a predetermined risk threshold. Entities classified in the no-concern zone 138 are not stored to reduce computational storage requirements.

[0107] Based on road structure (e.g., the area directly in front of vehicle 12, merging lanes, etc.), various factors are used to determine the area of ​​concern in the environment surrounding vehicle 12, including, but not limited to, the planned path of vehicle 12 (e.g., left turn, right turn, etc.), traffic environment (straight roads, intersections, number of lanes, position of vehicle 12 on the road, etc.), and possible paths that could lead to a collision with a target or other vehicles.

[0108] Two examples of regional attention allocation for common autonomous driving scenarios Figure 5 and 6 Shown in. Figure 5 The image shows vehicle 12 approaching an intersection intending to make a right turn. The area of ​​the intersection itself is classified as a high-concern area 134. Once vehicle 12 has made a right turn at the intersection, other high-concern areas 134 include the lanes in front of and behind vehicle 12. Furthermore, lanes traveling in the opposite direction to vehicle 12's intended path are also classified as high-concern areas 134. The lanes classified as high-concern areas 134... Figure 5 Each area is the area that vehicle 12 intends to enter during the planning of its driving path and / or the area where other vehicles or pedestrians may interfere with the planned path of vehicle 12. In addition, areas where other vehicles with the right of way have priority are also classified as high concern areas 134.

[0109] Continue to refer to Figure 5 The area at the intersection directly opposite vehicle 12 is designated as low-concern area 136. Areas classified as low-concern areas 136 are areas where other vehicles or pedestrians may be present, but the probability that these vehicles or pedestrians will interfere with the planned driving path of vehicle 12 is lower than that of areas classified as high-concern areas 134.

[0110] like Figure 5 As shown, two areas are classified as non-concern areas 138. Non-concern area 138 is an area in which other vehicles, targets, and / or pedestrians may be present but are classified as unlikely to interfere with the planned driving path of vehicle 12 (unless these other vehicles, targets, and / or pedestrians exhibit anomalous behavior). Anomalous behavior includes, for example, but not limited to, vehicles leaving their intended driving lanes or pedestrians crossing the street outside of designated crossing areas.

[0111] Another example of regional attention allocation is in Figure 6 As shown in the diagram. In this example, vehicle 12 travels along a road with multiple lanes in each direction. The area directly in front of vehicle 12 and the adjacent lane to the left of vehicle 12 in the opposite direction are classified as high-concern areas 134. The area behind vehicle 12 and the adjacent lane to the right of vehicle 12 are classified as low-concern areas 136. Finally, the lanes extending in the opposite direction behind vehicle 12 (i.e., lanes in which vehicles have passed vehicle 12 and vehicle 12 is moving away from these vehicles) and the opposite lanes separated from vehicle 12 by at least one lane are classified as no-concern areas 138.

[0112] As discussed herein, if the sensors of vehicle 12 detect unexpected behavior that could interfere with the planned path of vehicle 12, vehicles, targets, and / or pedestrians in any area (particularly in non-concern area 138) can be classified as areas or targets of urgent concern. Areas of urgent concern are assigned when estimating risk values. In various embodiments, Figure 5 and 6 The attention zones shown are based on a merging of attention zone types assigned to cell or region elements in the environment surrounding vehicle 12. These cell or region elements are displayed... Figure 7 In the left panel, the merged attention area is displayed Figure 7 In the right panel.

[0113] In various embodiments, the information used for assigning area allocations is obtained from two sources: prior map data from a navigation system (e.g., GPS of vehicle 12) and perception data from sensor 26 of vehicle 12. As mentioned herein, controller 22 performs accurate and robust environment-to-map mapping and perception output via various modules of ADS 24. Once area attention is assigned to adjacent area elements (e.g., other vehicles, targets, obstacles, pedestrians, etc.), elements with the same area attention level are merged, such as... Figure 7 As shown in the right panel.

[0114] Each adjacent vehicle, target, or pedestrian in the area x i Each region has its own regional attention level value assigned by the corresponding attention area. L ZA (x i ) In one example, the high-attention area will have a regional attention level value. L ZA (x i ) = 0.8 is assigned to external entities (including vehicles, targets, or pedestrians) within the high-attention area 134, and the low-attention area will have its regional attention level value... L ZA (x i ) = 0.4 is assigned to vehicles, targets, or pedestrians in low attention area 136, and zero attention level value is assigned to vehicles, targets, or pedestrians in low attention area 138 for no attention area.

[0115] In another example, the regional attention level value is calculated as follows:

[0116] L ZA (x i ) = S xi ( Z +αC ( x i ))

[0117] Where Z is the baseline regional attention level value {0, 0.4, 0.8} used for the no, low, and high attention regions 138, 136, and 134, respectively; C ( x i ) is the complexity calculation of external entities (i.e., vehicles, targets, or pedestrians);S xi It is the sigmoid function.

[0118] Each attention zone can contain multiple entities or agents, each with its own behavior attention level value assigned based on various factors. These factors include, but are not limited to, the entity's position relative to the corresponding lane of the road or the travel path of vehicle 12, the entity's speed relative to the expected speed of vehicle 12, and the heading angle of the entity relative to the corresponding lane of the road or the travel path of vehicle 12.

[0119] To obtain a behavior attention level value, these factors are combined using one of the following exemplary methods. In one method, the kinematic information of the entity is used. The behavior attention level value assigned to the entity depends on the entity's relative position, velocity, and heading. Relativity is determined by the difference between the entity's actual behavior and its expected behavior (i.e., the difference between the entity's actual position, velocity, and heading and its expected position, velocity, and heading). For an autonomous vehicle, such as vehicle 12, the entity's actual path should be aligned with the expected entity trajectory. In various embodiments, the behavior attention level value is obtained from the following equation:

[0120]

[0121] in, The corresponding entity x i Position, speed, and heading angle It refers to the desired position of an entity within a road lane when the entity is not intended to change lanes. It is the expected speed of the entity relative to the speed limit. It is the expected heading angle of an entity within a road lane when the entity is not intended to change lanes.

[0122] In various embodiments, the behavioral attention level value is obtained as a weighted sum of the sigmoid functions of each component, expressed as:

[0123]

[0124] in, α and β The weights are such that (0 ≤ α + β ≤ 1), S m ( n The sigmoid function is the sigmoid function for the m components that converges outside the minimum and maximum values ​​of "n". As the deviations of the components increase, This also means that the entity's attention level value has increased.

[0125] In various embodiments, pre-trained information is used to obtain the behavioral attention level value for each entity. Assume that an entity x in the environment has m possible training paths. (in, i =1… m () represents the possible paths of an entity. From time... tk arrive t Observations The probability of each feasible training path can be constructed. In various embodiments, probabilities are obtained through probability estimation. In normal situations (i.e., situations where the entity does not exhibit any abnormal behavior or problems), at least one expected action should have a higher probability than the unexpected behavior by a threshold probability (denoted as ). p th A higher probability. Mathematically, the following relationship occurs:

[0126] .

[0127] Optionally, in abnormal situations (i.e., situations where entities exhibit abnormal behavior or problems), all possible expected actions cannot have a higher degree of abnormality than... p th A higher probability, and the following relationship occurs:

[0128] .

[0129] Therefore, the behavioral attention level value is obtained as a function of the expected action probability and the anomaly threshold probability for each entity, as shown below:

[0130] .

[0131] function f BA2 ( It can be defined as the reciprocal of the exponent of the expected action probability, such as... Figure 8 As shown. Figure 8 As shown, if the probability becomes less than the threshold, the behavior attention level value becomes significantly larger (shifting from right to left in the curve shown).

[0132] In mathematical terms, the behavioral attention level value is calculated as follows:

[0133] if

[0134] if

[0135] if

[0136] in, It is the maximum level of attention given to a behavior. yes p th The level of attention given to behavior at that time p min It is to achieve The probability, α and β It is the coefficient of the exponent, where, α > β Using this mathematical expression, The rate of change is less than the probability p th It becomes higher at times.

[0137] Obtain the regional attention level value for each entity. L ZA and behavioral attention level value L BA Subsequently, controller 22 estimates the risk value of the corresponding entity relative to the planned path or current position of vehicle 12. In various embodiments, the risk value is estimated as follows:

[0138] .

[0139] In some embodiments, It is the product of two attention level values:

[0140] .

[0141] Once the risk value of each entity is known, multiple entities within the same area of ​​interest (e.g., high-interest area 134 and low-interest area 136) can be sorted, such as... Figure 4 As shown. Only those risk values ​​exceeding a certain risk threshold are considered. r th Only entities that meet the criteria will be sorted in the hierarchical event structure 124.

[0142] In addition, if ,in r UA If the risk threshold for an entity in an emergency concern area is met, then the entity is added to an existing emergency concern area 132 or an emergency concern area 132 is created (if it does not already exist in the hierarchical event structure 124). Entities within emergency concern area 132 are given the highest priority; that is, ADS 24 considers these entities most important when determining whether any changes should be made to the planned path of vehicle 12. (Greater than...) rUA The risk value of an entity is usually (if not always) caused by abnormal traffic situations.

[0143] When generating new event queues for similar events, two events are compared in order of regional concern level (i.e., comparing the regional concern level of urgency, then the regional concern level of high concern, and then the regional concern level of low concern). Within each region, the corresponding entities are considered in order of risk value.

[0144] One benefit of the method 100 for generating cognitive contextual awareness using the hierarchical event structure 124 is more efficient use of storage space. For most traffic scenarios, such as the two examples shown in Figures 5 and 6, the number of meaningful entities (i.e., entities with risk values ​​greater than the risk threshold) is approximately 10. The information for each entity, including the type of attention area, its position or posture relative to vehicle 12, and its risk value, requires significantly less storage space than other methods used to assess environmental conditions and potential interactions with vehicle 12. As discussed herein, using cognitive contextual awareness effectively reduces the amount of traffic contextual information available for efficient data processing by the controller.

[0145] It should be emphasized that many variations and modifications can be made to the embodiments described herein, and the elements of these embodiments should be understood as existing in other acceptable examples. All such modifications and variations are intended to be included within the scope of this disclosure and protected by the appended claims. Furthermore, any steps described herein may be performed simultaneously or in a different order than those listed herein. Moreover, it should be understood that the features and properties of the specific embodiments disclosed herein can be combined in different ways to form additional embodiments, all of which fall within the scope of this disclosure.

[0146] Unless otherwise specifically stated, or otherwise understood in the context in which they are used, conditional language such as “can,” “may,” “possibly,” “can,” “e.g.,” as used herein (among others) is generally intended to convey that certain embodiments include certain features, elements, and / or states that are not included in other embodiments. Therefore, such conditional language is not generally intended to imply that one or more embodiments require features, elements, and / or states in any way, or to imply that one or more embodiments necessarily include logic for determining, with or without author input or prompting, whether such features, elements, and / or states are included in any particular embodiment or will be executed in any particular embodiment.

[0147] In addition, the following terms may have been used herein. The singular forms “a,” “an,” and “the” include plural references unless the context clearly specifies otherwise. Thus, for example, a reference to one item includes a reference to one or more items. The term “a” refers to one, two, or more, and is generally applicable to selecting some or all of a quantity. The term “multiple” refers to two or more items. The terms “about” or “approximately” mean that quantities, dimensions, sizes, formulations, parameters, shapes, and other characteristics do not need to be precise, but can be approximate and / or larger or smaller as needed, reflecting acceptable tolerances, conversion factors, rounding, measurement errors, and other factors known to those skilled in the art. The term “substantially” means that the described characteristic, parameter, or value does not need to be precisely achieved, but deviations or variations (including, for example, tolerances, measurement errors, measurement accuracy limitations, and other factors known to those skilled in the art) may occur in amounts that do not preclude the effects that the characteristic is intended to provide.

[0148] For convenience, multiple items may be presented in a common list. However, these lists should be interpreted as if each member of the list were individually identified as a separate and unique member. Therefore, without the contrary indication, no single member of such a list should be construed as a de facto equivalent of any other member of the same list simply based on their presentation in the common group. Furthermore, when the terms “and” and “or” are used in conjunction with a list of items, they will be interpreted broadly, as any one or more of the listed items can be used individually or in combination with other listed items. The term “optionally” refers to selecting one of two or more alternatives and is not intended to limit the selection to only those listed alternatives or only one listed alternative at a time, unless the context explicitly indicates otherwise.

[0149] The processes, methods, or algorithms disclosed herein can be submitted to, implemented by, or fed to a processing device, controller, or computer, which may include any existing programmable electronic control unit or dedicated electronic control unit. Similarly, the processes, methods, or algorithms can be stored in many forms as data and instructions executable by a controller or computer, including but not limited to: information permanently stored on non-writable storage media (e.g., ROM devices) and information variablely stored on writable storage media (e.g., floppy disks, magnetic tapes, CDs, RAM devices, and other magnetic and optical media). The processes, methods, or algorithms can also be implemented as software-executable objects. Optionally, the processes, methods, or algorithms can be implemented wholly or partially using suitable hardware components, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers, or other hardware components or devices, or a combination of hardware, software, and firmware components. Such example devices can be in-vehicle as part of a vehicle computing system or located outside the vehicle and communicating remotely with one or more devices in the vehicle.

[0150] While exemplary embodiments have been described above, these embodiments are not intended to describe all possible forms covered by the claims. The language used in this specification is descriptive rather than restrictive, and it should be understood that various changes may be made without departing from the spirit and scope of this disclosure. As previously stated, features of various embodiments may be combined to form other exemplary aspects of this disclosure that may not be explicitly described or illustrated. While various embodiments may have been described as providing advantages or superiority over other embodiments or prior art with respect to one or more desired characteristics, those skilled in the art will recognize that one or more features or characteristics may be compromised to achieve desired overall system properties depending on the specific application and implementation. These properties may include, but are not limited to, cost, strength, durability, lifecycle cost, merchantability, appearance, packaging, size, maintainability, weight, manufacturability, ease of assembly, etc. Thus, embodiments described with respect to one or more characteristics as less desirable compared to other embodiments or prior art implementations are not outside the scope of this disclosure and may be desirable for a particular application.

Claims

1. A method for controlling a vehicle, comprising: The controller receives perception input data from the vehicle's sensors; The controller uses human-perception-inspired cognitive analytics to process perceived input data to calculate risk level values ​​for external entities near the vehicle. The controller generates a hierarchical event structure, which classifies and prioritizes perceived input data by classifying external entities into attention regions and prioritizing external entities within the classified attention regions according to their risk level values. The attention regions are one of high attention regions, low attention regions, and no attention regions. The controller develops vehicle behavior planning based on a hierarchical event structure; as well as The controller generates control signals based on behavior planning to control the vehicle's actuators.

2. The method according to claim 1, wherein, The perception input data includes entity data related to external entities near the vehicle, including the lane position of the external entity near the vehicle, the predicted path of the external entity relative to the vehicle, and one or more of the position and orientation of one or more driving lanes relative to the vehicle. The perception input data also includes vehicle characteristic data, including one or more of the vehicle speed, braking, and the vehicle's planned driving path.

3. The method according to claim 2, wherein, Using human perception-inspired cognitive analytics to process perceptual input data includes generating regional attention level values ​​for external entities, estimating behavioral attention level values ​​for external entities, calculating risk level values ​​for external entities based on regional and behavioral attention level values, determining whether anomalies are detected, and changing the attention region of external entities when anomalies are detected.

4. The method according to claim 3, wherein, The generation of regional attention level values ​​for external entities includes evaluating the predicted path of the external entity relative to the vehicle, the position and orientation of one or more lanes relative to the vehicle, and the planned driving path of the vehicle.

5. The method according to claim 4, wherein, A higher risk level value indicates a higher priority within the area of ​​concern.

6. The method according to claim 4, wherein, Represented as x i The regional attention level value of external entities is calculated as L. ZA (x i ) = S xi (Z+αC(x i ), where Z is the baseline regional attention level value of the external entity, C(x) i ) is the computation of the complexity of the external entity, and S xi It is a sigmoid function, and α is the weight.

7. The method according to claim 6, wherein, The baseline regional attention level value Z is 0 for areas with no attention, 0.4 for areas with low attention, and 0.8 for areas with high attention.

8. The method according to claim 6, wherein, Represented as x i The level of attention to the behavior of external entities is calculated as L. BA (x i )=f BA1 (p xi -p D ,v xi -v D ,h xi -h D ), where (p xi v xi h xi ) represents the external entity x i Position, velocity, and heading angle, p D It is the desired location of the external entity, v D h is the expected speed of the external entity relative to the speed limit. D It is the expected heading angle of the external entity.

9. The method according to claim 8, wherein, Represented as x i The risk level value of external entities is calculated as R(x) i ) = L ZA (x i )·L BA (x i ).

10. The method according to claim 3, wherein, Represented as x i The level of attention to the behavior of external entities is calculated as L. BA (x i )=α·S p (p xi -p D )+β·S v (v xi -v D )+(1-α-β)·S h (h xi -h D ), where α and β are weights such that (0≤α+β≤1), (p xi v xi h xi ) represents the external entity x i Position, velocity, and heading angle, p D It is the desired location of the external entity, v D h is the expected speed of the external entity relative to the speed limit. D It is the expected heading angle of the external entity, S m (n) is the sigmoid function for the "m" components that converge outside the minimum and maximum "n" values.

11. A motor vehicle, comprising: Multiple environmental sensors are configured to detect external features near motor vehicles; Multiple vehicle sensors configured to detect vehicle characteristics; An actuator configured to control vehicle steering, acceleration, braking, or gear shifting; as well as At least one controller, which electronically communicates with corresponding sensors and actuators among the plurality of environmental sensors, the plurality of vehicle sensors, and the actuator itself. The at least one controller is programmed with and configured to automatically control the actuators based on an autonomous driving system control algorithm. The autonomous driving control system algorithm includes: A perception system configured to receive perception input data from the plurality of environmental sensors and vehicle characteristic data from the plurality of vehicle sensors; A cognitive context awareness module is configured to process perceptual input data and vehicle characteristic data using human perception-inspired cognitive analysis to analyze external entities near a motor vehicle. This analysis includes calculating risk level values ​​for external entities and generating a hierarchical event structure. This hierarchical event structure classifies and prioritizes perceptual input data by categorizing external entities into attention regions and prioritizing external entities within each attention region based on their risk level values. The attention regions are one of high attention regions, low attention regions, and no attention regions. The vehicle control module is configured to develop motor vehicle behavior planning based on a hierarchical event structure, including planning driving paths. The at least one controller is further programmed to control the actuator according to behavior planning.

12. The motor vehicle according to claim 11, wherein, The cognitive analysis performed by the cognitive context awareness module includes generating regional attention level values ​​for external entities, estimating behavioral attention level values ​​for external entities, calculating risk level values ​​for external entities based on regional and behavioral attention level values, determining whether an anomaly is detected, and changing the attention region of external entities when an anomaly is detected.

13. A method for generating contextual awareness for vehicles using perception-inspired event generation, comprising: The controller receives perception input data from the vehicle's sensors; The controller processes the perceived input data to calculate the risk level value of external entities near the vehicle; The controller generates a hierarchical event structure, which classifies and prioritizes perceived input data by classifying external entities into areas of concern and prioritizing external entities within those areas based on their risk level values. A higher risk level value indicates a higher priority within the area of ​​concern. as well as The controller develops vehicle behavior planning based on a hierarchical event structure.

14. The method according to claim 13, wherein, The attention zone is one of the high attention zone, low attention zone, and no attention zone.

15. The method according to claim 13, wherein, The perception input data includes entity data related to external entities near the vehicle, including the lane position of the external entity near the vehicle, the predicted path of the external entity relative to the vehicle, and one or more of the position and orientation of one or more driving lanes relative to the vehicle. The perception input data also includes vehicle characteristic data, including one or more of the vehicle speed, braking, and the vehicle's planned driving path.

16. The method according to claim 15, wherein, Processing perceived input data includes generating regional attention level values ​​for external entities, estimating behavioral attention level values ​​for external entities, calculating risk level values ​​for external entities based on regional and behavioral attention level values, determining whether an anomaly has been detected, and changing the attention region of external entities when an anomaly is detected.

17. The method according to claim 16, wherein, When an anomaly is detected, the external entity is classified in the area of ​​urgent concern.

18. The method according to claim 16, wherein, The generation of regional attention level values ​​for external entities includes evaluating the predicted path of the external entity relative to the vehicle, the position and orientation of one or more lanes relative to the vehicle, and the planned driving path of the vehicle.

19. The method of claim 16, wherein, Represented as x i The level of attention to the behavior of external entities is calculated as L. BA (x i )=α·S p (p xi -p D )+β·S v (v xi -v D )+(1-α-β)·S h (h xi -h D ), where α and β are weights such that (0≤α+β≤1), (p xi v xi , h xi ) represents the external entity x i Position, velocity, and heading angle, p D It is the desired location of the external entity, v D h is the expected speed of the external entity relative to the speed limit. D It is the expected heading angle of the external entity, S m (n) is the sigmoid function for the "m" components that converge outside the minimum and maximum "n" values.

20. The method according to claim 13, wherein, The risk level value is compared with a predetermined risk threshold, and when the risk level value is lower than the predetermined risk threshold, the external entity is classified in the no-concern area, and the perceived data about the external entity is not stored by the controller.

Images