Method and system for controlling an autonomous vehicle

Abstract

A method includes obtaining, using at least one processor (304), sensor data associated with an environment in which an autonomous vehicle (702) operates according to a first trajectory of a plurality of trajectories, the environment including a road user at a location; obtaining, using the at least one processor, a rule indicative of a targeted expressive operation of the autonomous vehicle (702) in the vicinity of the road user, the rule comprising a first criterion for initiating an application of the rule, and a second criterion; determining, using the at least one processor, whether the sensor data complies with a first criterion, and applying a rule in response to the sensor data complying with the first criterion; wherein applying the rule comprises: determining, using the at least one processor (304), a violation metric based on the sensor data and a second criterion; evaluating, using at least one processor, a first trajectory of the autonomous vehicle based on the violation metric; and selecting, using the at least one processor, a second trajectory from the plurality of trajectories based on the violation metric.

Description

Background Art

[0001] Road users such as pedestrians rely on certain cues from vehicles (driver gaze, vehicle speed / position, etc.) in certain situations (such as before crossing a road, etc.). However, introducing autonomous vehicles into communities and communities of road users such as pedestrians may impact these cues dynamics. Communicating autonomous vehicle intentions to road users may be a challenge. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Figure 1 is an example environment in which a vehicle including one or more components of an autonomous system may be implemented;

[0003] Figure 2 is a diagram of one or more example systems of a vehicle including an autonomous system;

[0004] Figure 3 yes Figure 1 and Figure 2 diagrams of components of one or more example devices and / or one or more example systems;

[0005] Figure 4 is a diagram of certain components of an example autonomous system;

[0006] Figure 5 is a diagram of an example implementation of a process for autonomous vehicle yielding;

[0007] Figure 6 is a diagram of an example implementation of a process for autonomous vehicle yielding;

[0008] FIG. 7A to FIG. 7B is a diagram of an example implementation of a process for autonomous vehicle yielding;

[0009] FIG. 8A to FIG. 8C is a diagram of an example implementation of a process for autonomous vehicle yielding; and

[0010] Fig. 9 is a flow chart of an example process for autonomous vehicle yielding. DETAILED DESCRIPTION

[0011] In the following description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent that the embodiments described in the present disclosure can be implemented without these specific details. In some instances, well-known configurations and devices are illustrated in block diagram form to avoid unnecessarily obscuring aspects of the present disclosure.

[0012] In the accompanying drawings, for ease of description, the specific arrangement or order of schematic elements (such as those representing systems, devices, modules, instruction blocks and / or data elements, etc.) is illustrated. However, those skilled in the art will understand that, unless explicitly described, the specific order or arrangement of the schematic elements in the accompanying drawings is not intended to mean that a specific processing order or sequence, or separation of processing is required. In addition, unless explicitly described, the inclusion of a schematic element in the accompanying drawings is not intended to mean that such an element is required in all embodiments, nor is it intended to mean that the features represented by such an element may not be included in some embodiments or cannot be combined with other elements in some embodiments.

[0013] In addition, in the accompanying drawings, connecting elements (such as solid or dotted lines or arrows, etc.) are used to illustrate the connection, relationship or association between or among two or more other schematic elements, and the absence of any such connecting elements is not intended to mean that there can be no connection, relationship or association. In other words, some connections, relationships or associations between elements are not illustrated in the accompanying drawings so as not to obscure the present disclosure. In addition, for ease of illustration, a single connecting element can be used to represent multiple connections, relationships or associations between elements. For example, if the connecting element represents the communication of a signal, data or instruction (e.g., "software instruction"), it will be understood by those skilled in the art that such an element can represent one or more than one signal path (e.g., bus) that may be needed to affect the communication.

[0014] Although the terms "first", "second" and / or "third", etc. are used to describe various elements, these elements should not be limited by these terms. The terms "first", "second" and / or "third" are only used to distinguish one element from another. For example, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact without departing from the scope of the described embodiments. Both the first contact and the second contact are contacts, but they are not the same contacts.

[0015] The terms used in the specification of the various embodiments described herein are included only for the purpose of describing specific embodiments and are not intended to be limiting. As used in the specification of the various embodiments described and the appended claims, the singular forms "a", "an" and "the" are also intended to include plural forms and can be used interchangeably with "one or more than one" or "at least one", unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more than one of the associated listed items. It will also be understood that when the terms "include", "comprise", "have" and / or "have" are used in this specification, the stated features, integers, steps, operations, elements and / or components are specifically stated, but the presence or addition of one or more than one other features, integers, steps, operations, elements, components and / or groups thereof are not excluded.

[0016] As used herein, the terms "communication" and "communicating" refer to at least one of receiving, receiving, transmitting, transmitting and / or providing information (or information represented by, for example, data, signals, messages, instructions and / or commands, etc.). For a unit (e.g., a device, a system, a component of a device or system, and / or a combination thereof) to communicate with another unit, this means that the unit is able to directly or indirectly receive information from the other unit and / or send (e.g., transmit) information to the other unit. This can refer to a direct or indirect connection that is wired and / or wireless in nature. In addition, even if the transmitted information can be modified, processed, relayed and / or routed between the first unit and the second unit, the two units can communicate with each other. For example, even if the first unit passively receives information and does not actively transmit information to the second unit, the first unit can communicate with the second unit. As another example, if at least one intermediary unit (e.g., a third unit located between the first unit and the second unit) processes the information received from the first unit and transmits the processed information to the second unit, the first unit can communicate with the second unit. In some embodiments, a message may refer to a network packet (eg, a data packet, etc.) that includes data.

[0017] As used herein, the term "if" is optionally interpreted to mean "when," "at," "in response to being determined to be," and / or "in response to being detected," etc., depending on the context. Similarly, the phrases "if it is determined" or "if [the stated condition or event] is detected" are optionally interpreted to mean "when determining," "in response to being determined to be" or "when [the stated condition or event] is detected," and / or "in response to being detected," etc., depending on the context. In addition, as used herein, the terms "have," "have," or "possess," etc. are intended to be open-ended terms. Furthermore, unless expressly stated otherwise, the phrase "based on" is intended to mean "based at least in part on."

[0018] “At least one” and “one or more than one” include a function being performed by one element, a function being performed by more than one element, such as in a distributed manner, several functions being performed by one element, several functions being performed by several elements, or any combination of the above.

[0019] Some embodiments of the present disclosure are described herein in conjunction with thresholds. As described herein, satisfying (such as meeting, etc.) a threshold may refer to a value being greater than a threshold, more than a threshold, above a threshold, greater than or equal to a threshold, less than a threshold, less than a threshold, below a threshold, less than or equal to a threshold, and / or equal to a threshold, etc.

[0020] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments described. However, it will be apparent to one of ordinary skill in the art that the various embodiments described may be implemented without these specific details. In other cases, well-known methods, processes, components, circuits, and networks have not yet been described in detail in order not to unnecessarily obscure aspects of the embodiments.

[0021] General Overview

[0022] In some aspects and / or embodiments, the systems, methods, and computer program products described herein include and / or implement techniques that provide a set of rules for how an autonomous vehicle (AV) should yield when approaching a road user, particularly for expressing the AV's intent to the road user. For example, the disclosed techniques can be used to provide the expressive intent of an autonomous vehicle to a road user, such as through the use of rules and expressive motion of the autonomous vehicle. If a first criterion is met, the rules are applied and then evaluated to determine if there are any violations of the rules.

[0023] With the implementation of the systems, methods, and computer program products described herein, techniques for autonomous vehicles to yield can advantageously include improving the safety of road users when encountering autonomous vehicles. Improving safety can include, for example, improving the sense of safety of road users (e.g., pedestrians, cyclists, traffic controllers) with respect to autonomous vehicles. Some of the advantages of the systems, methods, and computer program products described herein can include the ability to communicate the intended operation of the autonomous vehicle to road users. This communication can improve the quality of communication between the autonomous vehicle and the road user because the likelihood that the intended operation of the autonomous vehicle is correctly perceived by the road user can be increased.

[0024] Reference now Figure 1 , illustrates an example environment 100 in which vehicles including autonomous systems and vehicles not including autonomous systems operate. As illustrated, the environment 100 includes vehicles 102a-102n, objects 104a-104n, routes 106a-106n, areas 108, vehicle-to-infrastructure (V2I) devices 110, a network 112, a remote autonomous vehicle (AV) system 114, a fleet management system 116, and a V2I system 118. The vehicles 102a-102n, the vehicle-to-infrastructure (V2I) devices 110, the network 112, the autonomous vehicle (AV) system 114, the fleet management system 116, and the V2I system 118 are interconnected (e.g., establish connections for communication, etc.) via wired connections, wireless connections, or a combination of wired or wireless connections. In some embodiments, objects 104a-104n are interconnected with at least one of vehicles 102a-102n, vehicle-to-infrastructure (V2I) devices 110, networks 112, autonomous vehicle (AV) systems 114, fleet management systems 116, and V2I systems 118 via wired connections, wireless connections, or a combination of wired or wireless connections.

[0025] Vehicles 102a-102n (individually referred to as vehicles 102 and collectively referred to as vehicles 102) include at least one device configured to transport goods and / or people. In some embodiments, vehicles 102 are configured to communicate with V2I devices 110, remote AV systems 114, fleet management systems 116, and / or V2I systems 118 via network 112. In some embodiments, vehicles 102 include cars, buses, trucks, and / or trains, etc. In some embodiments, vehicles 102 are similar to vehicles 200 described herein (see Figure 2). In some embodiments, vehicles 200 in the set of vehicles 200 are associated with an autonomous queue manager. In some embodiments, vehicles 102 travel along respective routes 106a-106n (individually referred to as routes 106 and collectively referred to as routes 106) as described herein. In some embodiments, one or more vehicles 102 include an autonomous system (e.g., an autonomous system that is the same as or similar to autonomous system 202).

[0026] Objects 104a-104n (individually referred to as objects 104 and collectively referred to as objects 104) include, for example, at least one vehicle, at least one pedestrian, at least one cyclist, and / or at least one structure (e.g., a building, a sign, a fire hydrant, etc.), etc. Each object 104 is stationary (e.g., located at a fixed location and over a period of time) or moves (e.g., has a speed and is associated with at least one trajectory). In some embodiments, objects 104 are associated with corresponding locations in area 108.

[0027] Routes 106a-106n (individually referred to as routes 106 and collectively referred to as routes 106) are each associated with (e.g., specify a series of actions (also referred to as trajectories) connecting states along which the AV can navigate. Each route 106 begins at an initial state (e.g., a state corresponding to a first spatiotemporal location and / or speed, etc.) and ends at a final target state (e.g., a state corresponding to a second spatiotemporal location different from the first spatiotemporal location) or a target zone (e.g., a subspace of acceptable states (e.g., terminal states)). In some embodiments, the first state includes a location where one or more individuals will board the AV, and the second state or zone includes one or more locations where one or more individuals boarding the AV will disembark. In some embodiments, routes 106 include multiple acceptable state sequences (e.g., multiple spatiotemporal location sequences) that are associated with (e.g., define multiple trajectories). In an example, routes 106 include only high-level actions or imprecise state locations, such as a series of connecting roads indicating a change of direction at a roadway intersection, etc. Additionally or alternatively, the route 106 may include more precise actions or states, such as, for example, specific target lanes or precise locations within lane regions and target speeds at those locations, etc. In an example, the route 106 includes a plurality of precise state sequences along at least one high-level action with a limited look-ahead horizon to an intermediate target, wherein a combination of consecutive iterations of the limited horizon state sequences cumulatively correspond to a plurality of trajectories that collectively form a high-level route terminating at a final target state or region.

[0028] The area 108 includes a physical area (e.g., a geographic region) in which the vehicle 102 can navigate. In an example, the area 108 includes at least one state (e.g., a country, a province, a separate state of a plurality of states included in a country, etc.), at least a portion of a state, at least one city, at least a portion of a city, etc. In some embodiments, the area 108 includes at least one named thoroughfare (referred to herein as a "road"), such as a highway, an interstate highway, a parkway, a city street, etc. Additionally or alternatively, in some examples, the area 108 includes at least one unnamed road, such as a driveway, a section of a parking lot, a section of an open space and / or undeveloped area, a dirt road, etc. In some embodiments, the road includes at least one lane (e.g., a portion of the road that the vehicle 102 can traverse). In an example, the road includes at least one lane associated with (e.g., identified based on) at least one lane marking line.

[0029] The vehicle-to-infrastructure (V2I) device 110 (sometimes referred to as a vehicle-to-infrastructure or vehicle-to-everything (V2X) device) includes at least one device configured to communicate with the vehicle 102 and / or the V2I system 118. In some embodiments, the V2I device 110 is configured to communicate with the vehicle 102, the remote AV system 114, the fleet management system 116, and / or the V2I system 118 via the network 112. In some embodiments, the V2I device 110 includes a radio frequency identification (RFID) device, a sign, a camera (e.g., a two-dimensional (2D) and / or three-dimensional (3D) camera), lane markings, street lights, parking meters, etc. In some embodiments, the V2I device 110 is configured to communicate directly with the vehicle 102. Additionally or alternatively, in some embodiments, the V2I device 110 is configured to communicate with the vehicle 102, the remote AV system 114, and / or the fleet management system 116 via the V2I system 118. In some embodiments, the V2I device 110 is configured to communicate with the V2I system 118 via the network 112.

[0030] The network 112 includes one or more wired and / or wireless networks. In an example, the network 112 includes a cellular network (e.g., a long-term evolution (LTE) network, a third generation (3G) network, a fourth generation (4G) network, a fifth generation (5G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber-based network, a cloud computing network, etc., and / or a combination of some or all of these networks, etc.

[0031] The remote AV system 114 includes at least one device configured to communicate with the vehicle 102, the V2I device 110, the network 112, the fleet management system 116, and / or the V2I system 118 via the network 112. In an example, the remote AV system 114 includes a server, a server group, and / or other similar devices. In some embodiments, the remote AV system 114 is co-located with the fleet management system 116. In some embodiments, the remote AV system 114 participates in the installation of some or all of the components of the vehicle (including autonomous systems, autonomous vehicle computing, and / or software implemented by autonomous vehicle computing, etc.). In some embodiments, the remote AV system 114 maintains (e.g., updates and / or replaces) these components and / or software during the life of the vehicle.

[0032] The queue management system 116 includes at least one device configured to communicate with the vehicles 102, the V2I devices 110, the remote AV system 114, and / or the V2I system 118. In an example, the queue management system 116 includes a server, a server group, and / or other similar devices. In some embodiments, the queue management system 116 is associated with a ridesharing company (e.g., an organization for controlling the operation of multiple vehicles (e.g., vehicles including autonomous systems and / or vehicles not including autonomous systems)).

[0033] In some embodiments, the V2I system 118 includes at least one device configured to communicate with the vehicle 102, the V2I device 110, the remote AV system 114, and / or the fleet management system 116 via the network 112. In some examples, the V2I system 118 is configured to communicate with the V2I device 110 via a connection other than the network 112. In some embodiments, the V2I system 118 includes a server, a server group, and / or other similar devices. In some embodiments, the V2I system 118 is associated with a municipality or a private agency (e.g., a private agency for maintaining the V2I device 110, etc.).

[0034] In some embodiments, Fig. 9 As illustrated, the apparatus 300 is configured as software instructions to perform one or more steps of the disclosed method.

[0035] supply Figure 1 The number and arrangement of elements illustrated are examples. Figure 1 There may be additional elements, fewer elements, different elements, and / or differently arranged elements than those illustrated. Additionally or alternatively, at least one element of environment 100 may be described as being Figure 1Additionally or alternatively, at least one set of elements of environment 100 may perform one or more functions described as being performed by at least one different set of elements of environment 100.

[0036] Reference now Figure 2 , vehicle 200 includes autonomous system 202, powertrain control system 204, steering control system 206, and braking system 208. In some embodiments, vehicle 200 is compatible with vehicle 102 (see Figure 1 ) is the same or similar. In some embodiments, the vehicle 200 has autonomous capabilities (e.g., implements at least one function, feature, and / or device, etc., which enables the vehicle 200 to operate partially or completely without human intervention, including but not limited to fully autonomous vehicles (e.g., vehicles that abandon reliance on human intervention) and / or highly autonomous vehicles (e.g., vehicles that abandon reliance on human intervention in certain circumstances), etc.). For a detailed description of fully autonomous vehicles and highly autonomous vehicles, reference can be made to SAE International's standard J3016: Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle AutomatedDriving Systems, the entire contents of which are incorporated by reference. In some embodiments, the vehicle 200 is associated with an autonomous queue manager and / or a ride-sharing company.

[0037] Autonomous system 202 includes a sensor suite that includes one or more devices such as camera 202a, LiDAR sensor 202b, Radar sensor 202c, and microphone 202d. In some embodiments, autonomous system 202 may include more or fewer devices and / or different devices (e.g., ultrasonic sensors, inertial sensors, GPS receivers (discussed below), and / or odometer sensors for generating data associated with an indication of the distance that vehicle 200 has traveled, etc.). In some embodiments, autonomous system 202 uses one or more devices included in autonomous system 202 to generate data associated with environment 100 described herein. Data generated by one or more devices of autonomous system 202 can be used by one or more systems described herein to observe the environment (e.g., environment 100) in which vehicle 200 is located. In some embodiments, autonomous system 202 includes communication device 202e, autonomous vehicle computing 202f, and safety controller 202g.

[0038] The camera 202a includes a communication device 202e, an autonomous vehicle computer 202f, and / or a safety controller 202g configured to communicate with the communication device 202e via a bus (e.g., Figure 3 The camera 202a includes at least one device for communicating with the autonomous vehicle computing 202f (e.g., an image processing unit 202a, a bus ... Figure 1In some embodiments, the autonomous vehicle computing 202f determines a depth to one or more objects in a field of view of at least two of the plurality of cameras based on image data from the at least two cameras. In some embodiments, the camera 202a is configured to capture images of objects within a distance relative to the camera 202a (e.g., up to 100 meters and / or up to 1 kilometer, etc.). Thus, the camera 202a includes features such as sensors and lenses that are optimized for sensing objects at one or more distances relative to the camera 202a.

[0039] In an embodiment, the camera 202a includes at least one camera configured to capture one or more images associated with one or more traffic lights, street signs, and / or other physical objects that provide visual navigation information. In some embodiments, the camera 202a generates traffic light data associated with the one or more images. In some examples, the camera 202a generates TLD data associated with one or more images including a format (e.g., RAW, JPEG, and / or PNG, etc.). In some embodiments, the camera 202a that generates TLD data differs from other systems incorporating cameras described herein in that the camera 202a may include one or more cameras with a wide field of view (e.g., a wide-angle lens, a fisheye lens, and / or a lens with a viewing angle of approximately 120 degrees or greater, etc.) to generate images related to as many physical objects as possible.

[0040] The laser detection and ranging (LiDAR) sensor 202b includes a communication device 202e, an autonomous vehicle computing device 202f, and / or a safety controller 202g via a bus (e.g., Figure 3The LiDAR sensor 202b includes at least one device that communicates with a bus (the same or similar bus as the bus 302 of the embodiment of the present invention). The LiDAR sensor 202b includes a system configured to emit light from a light emitter (e.g., a laser emitter). The light emitted by the LiDAR sensor 202b includes light outside the visible spectrum (e.g., infrared light, etc.). In some embodiments, during operation, the light emitted by the LiDAR sensor 202b encounters a physical object (e.g., a vehicle) and is reflected back to the LiDAR sensor 202b. In some embodiments, the light emitted by the LiDAR sensor 202b does not penetrate the physical object encountered by the light. The LiDAR sensor 202b also includes at least one light detector that detects the light emitted from the light emitter after encountering the physical object. In some embodiments, at least one data processing system associated with the LiDAR sensor 202b generates an image (e.g., a point cloud and / or a combined point cloud, etc.) representing objects included in the field of view of the LiDAR sensor 202b. In some examples, at least one data processing system associated with the LiDAR sensor 202b generates an image representing the boundaries of the physical object and / or the surface of the physical object (e.g., the topology of the surface), etc. In such examples, the image is used to determine the boundaries of the physical object in the field of view of the LiDAR sensor 202b.

[0041] The radio detection and ranging (Radar) sensor 202c includes a sensor configured to communicate with the communication device 202e, the autonomous vehicle computing 202f, and / or the safety controller 202g via a bus (e.g., Figure 3 At least one device for communicating with a bus (same or similar bus as bus 302 of the embodiment of the present invention). Radar sensor 202c includes a system configured to transmit (pulsed or continuous) radio waves. The radio waves transmitted by Radar sensor 202c include radio waves within a predetermined spectrum. In some embodiments, during operation, the radio waves transmitted by Radar sensor 202c encounter physical objects and are reflected back to Radar sensor 202c. In some embodiments, the radio waves transmitted by Radar sensor 202c are not reflected by some objects. In some embodiments, at least one data processing system associated with Radar sensor 202c generates a signal representing an object included in the field of view of Radar sensor 202c. For example, at least one data processing system associated with Radar sensor 202c generates an image representing the boundary of a physical object and / or the surface of a physical object (e.g., the topology of the surface), etc. In some examples, the image is used to determine the boundary of a physical object in the field of view of Radar sensor 202c.

[0042] The microphone 202d includes a microphone configured to communicate with the communication device 202e, the autonomous vehicle computing device 202f, and / or the safety controller 202g via a bus (e.g., Figure 3 At least one device for communicating with the vehicle 200 (the same or similar bus as bus 302 of FIG. 1 ). Microphone 202d includes one or more microphones (e.g., an array microphone and / or an external microphone, etc.) that capture an audio signal and generate data associated with (e.g., representing) the audio signal. In some examples, microphone 202d includes a transducer device and / or the like. In some embodiments, one or more systems described herein can receive the data generated by microphone 202d and determine the position (e.g., distance, etc.) of an object relative to vehicle 200 based on an audio signal associated with the data.

[0043] The communication device 202e includes at least one device configured to communicate with the camera 202a, the LiDAR sensor 202b, the Radar sensor 202c, the microphone 202d, the autonomous vehicle computing 202f, the safety controller 202g, and / or the DBW system 202h. For example, the communication device 202e may include at least one device configured to communicate with the camera 202a, the LiDAR sensor 202b, the Radar sensor 202c, the microphone 202d, the autonomous vehicle computing 202f, the safety controller 202g, and / or the DBW system 202h. Figure 3 The communication device 202e may be a device that is the same as or similar to the communication interface 314 of the vehicle. In some embodiments, the communication device 202e includes a vehicle-to-vehicle (V2V) communication device (eg, a device for enabling wireless communication of data between vehicles).

[0044] Autonomous vehicle computing 202f includes at least one device configured to communicate with camera 202a, LiDAR sensor 202b, Radar sensor 202c, microphone 202d, communication device 202e, safety controller 202g, and / or DBW system 202h. In some examples, autonomous vehicle computing 202f includes devices such as client devices, mobile devices (e.g., cellular phones and / or tablet computers, etc.), and / or servers (e.g., computing devices including one or more central processing units and / or graphics processing units, etc.). In some embodiments, autonomous vehicle computing 202f is the same or similar to autonomous vehicle computing 400 described herein. Additionally or alternatively, in some embodiments, autonomous vehicle computing 202f is configured to communicate with an autonomous vehicle system (e.g., with Figure 1 remote AV system 114 of the same or similar autonomous vehicle system), a fleet management system (e.g., Figure 1 of the same or similar queue management system as the queue management system 116), V2I devices (e.g., Figure 1 V2I device 110 that is the same as or similar to V2I device 110) and / or V2I system (e.g., Figure 1The V2I system 118 may communicate with the same or similar V2I system.

[0045] Safety controller 202g includes at least one device configured to communicate with camera 202a, LiDAR sensor 202b, Radar sensor 202c, microphone 202d, communication device 202e, autonomous vehicle computing 202f, and / or DBW system 202h. In some examples, safety controller 202g includes one or more controllers (electrical controllers and / or electromechanical controllers, etc.) configured to generate and / or transmit control signals to operate one or more devices of vehicle 200 (e.g., powertrain control system 204, steering control system 206, and / or braking system 208, etc.). In some embodiments, safety controller 202g is configured to generate control signals that take precedence over (e.g., override) control signals generated and / or transmitted by autonomous vehicle computing 202f.

[0046] The DBW system 202h includes at least one device configured to communicate with the communication device 202e and / or the autonomous vehicle computing 202f. In some examples, the DBW system 202h includes one or more controllers (e.g., electrical controllers and / or electromechanical controllers, etc.) configured to generate and / or transmit control signals to operate one or more devices of the vehicle 200 (e.g., powertrain control system 204, steering control system 206, and / or braking system 208, etc.). Additionally or alternatively, one or more controllers of the DBW system 202h are configured to generate and / or transmit control signals to operate at least one different device of the vehicle 200 (e.g., turn signal lights, headlights, door locks, and / or windshield wipers, etc.).

[0047] The powertrain control system 204 includes at least one device configured to communicate with the DBW system 202h. In some examples, the powertrain control system 204 includes at least one controller and / or actuator, etc. In some embodiments, the powertrain control system 204 receives a control signal from the DBW system 202h, and the powertrain control system 204 causes the vehicle 200 to start moving forward, stop moving forward, start moving backward, stop moving backward, accelerate in a certain direction, decelerate in a certain direction, turn left and / or turn right, etc. In an example, the powertrain control system 204 increases, maintains the same, or decreases the energy (e.g., fuel and / or electricity, etc.) provided to the motor of the vehicle, thereby rotating or not rotating at least one wheel of the vehicle 200.

[0048] The steering control system 206 includes at least one device configured to rotate one or more wheels of the vehicle 200. In some examples, the steering control system 206 includes at least one controller and / or actuator, etc. In some embodiments, the steering control system 206 rotates the two front wheels and / or the two rear wheels of the vehicle 200 to the left or right to turn the vehicle 200 left or right.

[0049] Braking system 208 includes at least one device configured to actuate one or more brakes to decelerate and / or hold vehicle 200 stationary. In some examples, braking system 208 includes at least one controller and / or actuator configured to cause one or more calipers associated with one or more wheels of vehicle 200 to close on respective rotors of vehicle 200. Additionally or alternatively, in some examples, braking system 208 includes an automatic emergency braking (AEB) system and / or a regenerative braking system, etc.

[0050] In some embodiments, vehicle 200 includes at least one platform sensor (not explicitly illustrated) for measuring or inferring a property of a state or condition of vehicle 200. In some examples, vehicle 200 includes platform sensors such as a global positioning system (GPS) receiver, an inertial measurement unit (IMU), wheel rate sensors, wheel brake pressure sensors, wheel torque sensors, engine torque sensors, and / or steering angle sensors.

[0051] Reference now Figure 3 , a schematic diagram of an exemplary device 300. As illustrated, the device 300 includes a processor 304, a memory 306, a storage component 308, an input interface 310, an output interface 312, a communication interface 314, and a bus 302. In some embodiments, the device 300 corresponds to: at least one device of the vehicle 102 (e.g., at least one device of a system of the vehicle 102); at least one device of the remote AV system 114, the fleet management system 116, the V2I system 118; and / or one or more devices of the network 112 (e.g., one or more devices of a system of the network 112). In some embodiments, one or more devices of vehicle 102 (e.g., one or more devices of a system of vehicle 102 (such as remote AV system 114, fleet management system 116, and at least one device of V2I system 118), and / or one or more devices of network 112 (e.g., one or more devices of a system of network 112) include at least one device 300 and / or at least one component of device 300. Figure 3As shown, apparatus 300 includes a bus 302 , a processor 304 , a memory 306 , a storage component 308 , an input interface 310 , an output interface 312 , and a communication interface 314 .

[0052] The bus 302 includes components for communication between components of the licensing device 300. In some embodiments, the processor 304 is implemented in hardware, software, or a combination of hardware and software. In some examples, the processor 304 includes a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), and / or an accelerated processing unit (APU), etc.), a microphone, a digital signal processor (DSP), and / or any processing component that can be programmed to perform at least one function (e.g., a field programmable gate array (FPGA) and / or an application-specific integrated circuit (ASIC), etc.). The memory 306 includes a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic and / or static storage device (e.g., flash memory, magnetic memory, and / or optical memory, etc.) that stores data and / or instructions for use by the processor 304.

[0053] Storage component 308 stores data and / or software related to the operation and use of device 300. In some examples, storage component 308 includes a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk, etc.), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a tape, a CD-ROM, a RAM, a PROM, an EPROM, a FLASH-EPROM, an NV-RAM, and / or another type of computer-readable medium, and a corresponding drive.

[0054] The input interface 310 includes components that permit the device 300 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, and / or a camera, etc.). Additionally or alternatively, in some embodiments, the input interface 310 includes a sensor for sensing information (e.g., a global positioning system (GPS) receiver, an accelerometer, a gyroscope, and / or an actuator, etc.). The output interface 312 includes components for providing output information from the device 300 (e.g., a display, a speaker, and / or one or more light emitting diodes (LEDs), etc.).

[0055] In some embodiments, communication interface 314 includes a transceiver-like component (e.g., a transceiver and / or a separate receiver and transmitter, etc.) that permits device 300 to communicate with other devices via a wired connection, a wireless connection, or a combination of a wired connection and a wireless connection. In some examples, communication interface 314 permits device 300 to receive information from another device and / or provide information to another device. In some examples, communication interface 314 includes an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, interface and / or cellular network interface, etc.

[0056] In some embodiments, the device 300 performs one or more processes described herein. The device 300 performs these processes based on the processor 304 executing software instructions stored by a computer-readable medium such as a memory 306 and / or a storage component 308. Computer-readable media (e.g., non-transitory computer-readable media) are defined herein as non-transitory memory devices. Non-transitory memory devices include storage space located within a single physical storage device or storage space distributed across multiple physical storage devices.

[0057] In some embodiments, the software instructions are read into the memory 306 and / or storage component 308 from another computer-readable medium or from another device via the communication interface 314. The software instructions stored in the memory 306 and / or storage component 308, when executed, cause the processor 304 to perform one or more processes described herein. Additionally or alternatively, hardwired circuitry is used in place of or in combination with the software instructions to perform one or more processes described herein. Therefore, unless expressly stated otherwise, the embodiments described herein are not limited to any specific combination of hardware circuitry and software.

[0058] The memory 306 and / or the storage component 308 include a data storage unit or at least one data structure (e.g., a database, etc.). The device 300 can receive information from the data storage unit or at least one data structure in the memory 306 or the storage component 308, store information in the data storage unit or at least one data structure, communicate information to the data storage unit or at least one data structure, or search for information stored in the data storage unit or at least one data structure. In some examples, the information includes network data, input data, output data, or any combination thereof.

[0059] In some embodiments, the device 300 is configured to execute software instructions stored in the memory 306 and / or the memory of another device (e.g., another device that is the same as or similar to the device 300). As used herein, the term "module" refers to at least one instruction stored in the memory 306 and / or the memory of another device, which, when executed by the processor 304 and / or the processor of another device (e.g., another device that is the same as or similar to the device 300), causes the device 300 (e.g., at least one component of the device 300) to perform one or more processes described herein. In some embodiments, the module is implemented in software, firmware, and / or hardware, etc.

[0060] supply Figure 3 The number and arrangement of components illustrated are examples. Figure 3 The device 300 may include additional components, fewer components, different components, or differently arranged components than those illustrated. Additionally or alternatively, a set of components (e.g., one or more components) of the device 300 may perform one or more functions described as being performed by another component or set of components of the device 300.

[0061] Reference now Figure 4, illustrates an example block diagram of an autonomous vehicle computing 400 (sometimes referred to as an "AV stack"). As illustrated, the autonomous vehicle computing 400 includes a perception system 402 (sometimes referred to as a perception module), a planning system 404 (sometimes referred to as a planning module), a positioning system 406 (sometimes referred to as a positioning module), a control system 408 (sometimes referred to as a control module), and a database 410. In some embodiments, the perception system 402, the planning system 404, the positioning system 406, the control system 408, and the database 410 are included in and / or implemented in an automatic navigation system of a vehicle (e.g., the autonomous vehicle computing 202f of the vehicle 200). Additionally or alternatively, in some embodiments, the perception system 402, the planning system 404, the positioning system 406, the control system 408, and the database 410 are included in one or more independent systems (e.g., one or more systems that are the same or similar to the autonomous vehicle computing 400, etc.). In some examples, the perception system 402, planning system 404, positioning system 406, control system 408, and database 410 are included in one or more independent systems located in the vehicle and / or at least one remote system as described herein. In some embodiments, any and / or all of the systems included in the autonomous vehicle computing 400 are implemented in software (e.g., software instructions stored in a memory), computer hardware (e.g., by a microprocessor, microcontroller, application specific integrated circuit [ASIC] and / or field programmable gate array (FPGA)), or a combination of computer software and computer hardware. It will also be understood that in some embodiments, the autonomous vehicle computing 400 is configured to communicate with a remote system (e.g., an autonomous vehicle system that is the same or similar to the remote AV system 114, a fleet management system 116 that is the same or similar to the fleet management system 116, and / or a V2I system that is the same or similar to the V2I system 118, etc.).

[0062] In some embodiments, the perception system 402 receives data associated with at least one physical object in the environment (e.g., data used by the perception system 402 to detect at least one physical object) and classifies the at least one physical object. In some examples, the perception system 402 receives image data captured by at least one camera (e.g., camera 202a), the image being associated with (e.g., representing) one or more physical objects within the field of view of the at least one camera. In such examples, the perception system 402 classifies the at least one physical object based on one or more groups of physical objects (e.g., bicycles, vehicles, traffic signs, and / or pedestrians, etc.). In some embodiments, based on the classification of the physical object by the perception system 402, the perception system 402 transmits data associated with the classification of the physical object to the planning system 404.

[0063] In some embodiments, planning system 404 receives data associated with a destination and generates data associated with at least one route (e.g., route 106) along which a vehicle (e.g., vehicle 102) can travel toward the destination. In some embodiments, planning system 404 periodically or continuously receives data (e.g., data associated with the classification of physical objects described above) from perception system 402, and planning system 404 updates at least one trajectory or generates at least one different trajectory based on the data generated by perception system 402. In some embodiments, planning system 404 receives data associated with an updated position of a vehicle (e.g., vehicle 102) from positioning system 406, and planning system 404 updates at least one trajectory or generates at least one different trajectory based on the data generated by positioning system 406.

[0064] In some embodiments, the positioning system 406 receives data associated with (e.g., representing) a location of a vehicle (e.g., vehicle 102) in an area. In some examples, the positioning system 406 receives LiDAR data associated with at least one point cloud generated by at least one LiDAR sensor (e.g., LiDAR sensor 202b). In some examples, the positioning system 406 receives data associated with at least one point cloud from multiple LiDAR sensors, and the positioning system 406 generates a combined point cloud based on each point cloud. In these examples, the positioning system 406 compares the at least one point cloud or the combined point cloud with a two-dimensional (2D) and / or three-dimensional (3D) map of the area stored in the database 410. Then, based on the positioning system 406 comparing the at least one point cloud or the combined point cloud with the map, the positioning system 406 determines the position of the vehicle in the area. In some embodiments, the map includes a combined point cloud of the area generated before the navigation of the vehicle. In some embodiments, the map includes, but is not limited to, a high-precision map of roadway geometry, a map describing the road network connectivity, a map describing the physical properties of the roadway (such as traffic speed, traffic volume, the number of vehicle and bicycle traffic lanes, lane width, lane traffic direction or the type and location of lane markings, or a combination thereof, etc.), and a map describing the spatial location of road features (such as crosswalks, traffic signs, or various types of other driving lights, etc.). In some embodiments, the map is generated in real time based on data received by the perception system.

[0065] In another example, positioning system 406 receives global navigation satellite system (GNSS) data generated by a global positioning system (GPS) receiver. In some examples, positioning system 406 receives GNSS data associated with a location of a vehicle in an area, and positioning system 406 determines the latitude and longitude of the vehicle in the area. In such an example, positioning system 406 determines the position of the vehicle in the area based on the latitude and longitude of the vehicle. In some embodiments, positioning system 406 generates data associated with the position of the vehicle. In some examples, based on the location of the vehicle determined by positioning system 406, positioning system 406 generates data associated with the position of the vehicle. In such an example, the data associated with the position of the vehicle include data associated with one or more semantic properties corresponding to the position of the vehicle.

[0066] In some embodiments, the control system 408 receives data associated with at least one trajectory from the planning system 404, and the control system 408 controls the operation of the vehicle. In some examples, the control system 408 receives data associated with at least one trajectory from the planning system 404, and the control system 408 controls the operation of the vehicle by generating and transmitting control signals to operate a powertrain control system (e.g., DBW system 202h and / or powertrain control system 204, etc.), a steering control system (e.g., steering control system 206), and / or a braking system (e.g., braking system 208). In the example, in the case where the trajectory includes a left turn, the control system 408 transmits a control signal to cause the steering control system 206 to adjust the steering angle of the vehicle 200, thereby causing the vehicle 200 to turn left. Additionally or alternatively, the control system 408 generates and transmits control signals to cause other devices of the vehicle 200 (e.g., headlights, turn signals, door locks, and / or windshield wipers, etc.) to change state.

[0067] In some embodiments, perception system 402, planning system 404, positioning system 406, and / or control system 408 implement at least one machine learning model (e.g., at least one multi-layer perceptron (MLP), at least one convolutional neural network (CNN), at least one recurrent neural network (RNN), at least one autoencoder, and / or at least one transformer, etc.). In some examples, perception system 402, planning system 404, positioning system 406, and / or control system 408 implement at least one machine learning model alone or in combination with one or more of the above systems. In some examples, perception system 402, planning system 404, positioning system 406, and / or control system 408 implement at least one machine learning model as part of a pipeline (e.g., a pipeline for identifying one or more objects located in an environment, etc.).

[0068] Database 410 stores data transmitted to, received from, and / or updated by perception system 402, planning system 404, positioning system 406, and / or control system 408. In some examples, database 410 includes a storage component (e.g., a storage component) for storing data and / or software related to operations and using at least one system of autonomous vehicle computing 400. Figure 3In some embodiments, database 410 stores data associated with a 2D and / or 3D map of at least one area. In some examples, database 410 stores data associated with a 2D and / or 3D map of a portion of a city, portions of multiple cities, multiple cities, counties, states, and / or countries (State) (e.g., a country), etc. In such an example, a vehicle (e.g., a vehicle that is the same or similar to vehicle 102 and / or vehicle 200) can be driven along one or more drivable areas (e.g., single-lane roads, multi-lane roads, highways, remote roads, and / or off-road roads, etc.) and cause at least one LiDAR sensor (e.g., a LiDAR sensor that is the same or similar to LiDAR sensor 202b) to generate data associated with an image representing an object included in the field of view of the at least one LiDAR sensor.

[0069] In some embodiments, database 410 can be implemented across multiple devices. In some examples, database 410 includes a vehicle (e.g., a vehicle that is the same or similar to vehicle 102 and / or vehicle 200), an autonomous vehicle system (e.g., an autonomous vehicle system that is the same or similar to remote AV system 114), a fleet management system (e.g., a vehicle that is the same or similar to remote AV system 114), and a fleet management system (e.g., a vehicle that is the same or similar to remote AV system 114). Figure 1 The same or similar queue management system as the queue management system 116 of FIG. 1 and / or the V2I system (e.g., Figure 1 The V2I system 118 is the same as or similar to the V2I system 118).

[0070] Reference now Figure 5 , a diagram illustrating an implementation of a process for autonomous vehicle yielding, such as system 500. In some embodiments, the implementation includes AV computing 540 and a vehicle (with Figure 2 In some embodiments, the system 500 is similar to the vehicle 200 of the present invention, such as an autonomous vehicle, etc. Figure 2 , Figure 3 and Figure 4 The illustrated system (such as (for example, Figure 2 , Figure 3 and Figure 4 The system 500 may be the same or similar to the AV, AV system, remote AV system, fleet management system, and / or V2I system (exemplified in FIG. 5 ). The system 500 may be used to operate an autonomous vehicle. The system 500 may not be used to operate an autonomous vehicle.

[0071] The present disclosure relates to systems, methods and computer program products that can provide a set of rules for how an autonomous vehicle should yield when approaching a road user, such as at a crosswalk, and in particular for expressing the autonomous vehicle's intentions to the road user. The rule set can be applicable to operating an autonomous vehicle when encountering jaywalkers and / or traffic controllers. The present disclosure relates to systems, methods and computer program products that provide violation metrics based on the rule set and the sensed behavior of the autonomous vehicle in the mentioned situations. The rule set can be viewed as defining the successful behavior of the autonomous vehicle, for example in providing an expressive intent that can be understood by the road user.

[0072] For example, one or more mathematical rules are disclosed herein, which can be used to govern the operation of an autonomous vehicle, such as when approaching a road user, for example, at a crosswalk. (One or more) rules may include one or more expressions, which are designed to characterize the target behavior of the movement of the AV (such as (one or more) ideal autonomous vehicle movement behaviors, etc.), violation metrics that can punish behaviors that deviate from the target behavior (e.g., non-ideal behaviors), and behavior entry and / or exit criteria (e.g., rule ranges). The rule can be considered to define a rule range, which characterizes entry criteria for entering the rule and / or exit criteria for exiting the rule. The rule can be used to provide a definition of successful behavior, a metric for behavior violations, and behavior of entry / exit criteria (e.g., rule ranges). In some embodiments, the planning system can access data including rules used for planning. For example, a formal language (e.g., using Boolean logic) is used to specify rules. In the situation encountered by the AV, at least some of the rules may be applicable to the situation. For example, the rules apply to a given situation as follows: based on information available to the AV (e.g., information about the surrounding environment), whether entry criteria are met.

[0073] Autonomous vehicles can advantageously communicate the intentions of the autonomous vehicle. Ideal behavior of an autonomous vehicle may include stopping far enough at a crosswalk to yield the right of way. The present disclosure may formalize target or desired behaviors of an AV (such as target expressive operations, etc.), as well as behavioral entry and / or exit criteria and / or entry and / or exit triggers. In addition, the present disclosure formalizes metrics (e.g., violation metrics) to penalize deviating behaviors. Autonomous vehicle motion behaviors can significantly improve pedestrian safety, confidence, and / or intent understanding.

[0074] Advantageously, expressive autonomous vehicle motion can produce statistically significant improvements in pedestrian sense of safety, confidence, and understanding of the autonomous vehicle's intent. For example, the expressive autonomous vehicle motion exemplified in the present disclosure can be a better indicator to pedestrians than other current technologies (such as light and / or sound, etc.). This may be in part because expressive autonomous vehicle motion more closely simulates the real-world experience of pedestrians. Happy pedestrians can benefit the community's acceptance of autonomous vehicles within their communities. Advantageously, the present disclosure can benefit post-log processing and evaluation of autonomous vehicles' on-road behavior. For example, the present disclosure uses violation metrics for measurement and evaluation of autonomous vehicles, such as for whether rules are violated while the autonomous vehicle is on the road.

[0075] System 500 is disclosed herein. In some embodiments or examples, the system includes at least one processor. In some embodiments or examples, system 500 includes at least one non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause at least one processor to perform operations, including: using at least one processor, obtaining sensor data associated with an environment in which an autonomous vehicle operates according to a first trajectory in a plurality of trajectories. The environment may include a road user at a certain location. In some embodiments or examples, the operation includes: obtaining a rule indicating a target expressive operation of an autonomous vehicle in the vicinity of a road user, the rule including a first criterion for initiating the application of the rule, and a second criterion. In one or more examples, the second criterion is used to detect satisfaction of the rule. In some embodiments or examples, the operation includes: determining whether the sensor data meets the first criterion. In some embodiments or examples, the operation includes: applying the rule in response to the sensor data meeting the first criterion. In some embodiments or examples, applying the rule includes: determining a violation based on the sensor data and the second criterion. In some embodiments or examples, the operation includes: using at least one processor, evaluating the first trajectory of the autonomous vehicle based on a violation metric. In some embodiments or examples, the operations include: selecting, using at least one processor, a second trace from a plurality of traces based on a violation metric.

[0076] The term trajectory can be considered as a path for navigating an AV from a first location to a second location. A trajectory can be considered as a series of actions connecting states along which an autonomous vehicle can navigate. A trajectory can be considered as a maneuver, such as lane changes and / or navigation via a crosswalk, etc. A location can be considered as a spatiotemporal location. In one or more examples, a first location is referred to as an initial or starting location, and a second location is referred to as a destination, a final location, a target, a target location, or a target location. In one or more examples, a trajectory includes one or more segments (e.g., segments of a road), and each segment includes one or more blocks (e.g., portions of a lane or intersection) (e.g., associated with one or more blocks). In one or more examples, a location corresponds to a real-world location. For example, a location can include a pickup location and / or a drop-off location for picking up people or goods and / or unloading people or goods.

[0077] System 500 (e.g., AV computing 540) can be viewed as a system for expressive operations (such as for expressive yielding, etc.). For example, system 500 can be viewed as an expressive system. For example, system 500 is "expressive" because the system operates the AV in a manner that communicates intent to road users in the environment (e.g., other vehicles, pedestrians, cyclists, traffic controllers, etc.). Intent can be viewed as a purpose, goal, and / or target, which can be communicated as an operation, such as to follow a trajectory. For example, system 500 is "expressive" because the system communicates the operating state of the AV to road users in the environment (e.g., objects, other vehicles, pedestrians, cyclists, traffic controllers, etc.).

[0078] In one or more examples, communications of system 500 may be considered "expressive" communications, that is, in the form of expressive maneuvers (e.g., involving one or more trajectories of the AV and / or one or more operations of the AV (e.g., via control features of the AV), such as deceleration, acceleration, or steering behavior of the AV (e.g., steering angle change)). In one or more examples, "expressive" communications may include expressive indications (e.g., involving one or more of the speakers, lights, and displays of the AV). For example, expressive communications may include trajectories that include one or more of deceleration, acceleration, and steering behaviors, for example, without involving one or more of the speakers, lights, and displays of the AV, or the expressive communications may include deceleration, acceleration, or steering behaviors in addition to involving one or more of the speakers, lights, and displays of the AV.

[0079] For example, system 500 can be configured to apply a rule indicating a target expressive operation in response to sensor data (e.g., sensor data 503) meeting a first criterion. This can provide a road user with an expressive indication of motion related to actions that the autonomous vehicle can take. System 500 can be configured to evaluate an action or operation (such as an expressive indication of motion, etc.) taken by the autonomous vehicle to determine whether the rule is violated (such as due to a conflict in the rules of system 500, etc.).

[0080] Sensor data 503 may be one or more of the following: Radar sensor data associated with a Radar image, image sensor data associated with an image, audio sensor data associated with a sound, and LiDAR sensor data associated with a point cloud. The specific type of sensor data is not restrictive. Sensor data 503 may indicate the environment around the autonomous vehicle. For example, sensor data 503 may indicate an object and / or multiple objects in the environment around the autonomous vehicle. In particular, sensor data 503 may indicate an environment in which the autonomous vehicle operates according to a first track in a plurality of tracks. For example, sensor data 503 indicates a road user at a certain location. For example, sensor data 503 indicates a road user at a certain location when the autonomous vehicle is traveling or planning to continue traveling according to the first track.

[0081] Sensors, such as sensor 502, may be one or more sensors associated with an autonomous vehicle, such as onboard sensors. The autonomous vehicle may include one or more sensors that may be configured to generate sensor data for monitoring the environment in which the autonomous vehicle is operating (such as via sensor 502). For example, one or more sensors may generate sensor data 503 that indicates what is happening in the environment around the autonomous vehicle (such as for determining one or more trajectories of the autonomous vehicle). Sensor 502 may be Figure 2 One or more of the sensors illustrated in . A sensor such as sensor 502 may be one or more of the following: a Radar sensor, a non-Radar sensor, a camera sensor, a microphone, an infrared sensor, an image sensor, and a LiDAR sensor. In some embodiments or examples, sensor 502 may be selected from the group consisting of a Radar sensor, a camera sensor, and a LiDAR sensor. The type of sensor is not limiting.

[0082] The environment may be the environment (such as an area) in which the autonomous vehicle is operating. The environment may include Figure 1Any and / or all features of. Sensor data (such as sensor data 503, etc.) may indicate certain aspects of the environment (such as zones, locations, segments, parts, etc.). For example, sensor data indicates one or more of a road user, a road segment, a road boundary, an object within the environment, a lane boundary, a crosswalk, and a non-crosswalk. The specific aspects of the environment indicated by the sensor data are not restrictive. The environment may include a road user. Sensor data 503 may indicate a road user. A road user may be an object in the environment as indicated by the sensor data. A road user may be one or more of a vehicle, a bicycle, and a pedestrian. For example, a road user may be a pedestrian at a crosswalk. A road user may be a dynamically moving object in the environment as indicated by the sensor data. A road user may be considered an agent. An agent such as a road user may be a physical object located in the environment. An agent may include one or more physical objects located in the environment. The type of agent may be set by the system or user. The type of agent may be automatically determined, such as by sensing and machine learning. In some embodiments or examples, an agent may be an object that can move dynamically over time. An agent may be any object captured by a sensor (such as sensor 502) or included in sensor data (such as sensor data 503). In some embodiments or examples, a road user is one or more of a user-operated vehicle and a pedestrian. Road users are not limited thereto and may be one or more of a pedestrian, a jaywalker, a traffic controller, a cyclist, a user-operated vehicle, a user-operated scooter, a user-operated car, a user-operated truck, and a user-operated motorcycle. A road user may not be another autonomous vehicle. Sensor data 503 may indicate road objects such as traffic lights, crosswalks, and the like.

[0083] Sensor data (such as sensor data 503, etc.) can indicate a road user at a certain location. The location can be a location in an environment. The location can be a part of an environment (such as a section, etc.). The location can be one or more than one of a crosswalk, a sidewalk near a crosswalk, and a road portion. The location can be considered as a location of a road user, such as a location surrounding a road user, such as an area associated with a road user, etc. In one or more examples, the sensor data indicates a vehicle and / or a pedestrian (e.g., a road user) operated by a user at a crosswalk (e.g., a certain location). In some embodiments or examples, the location of a road user includes an area surrounding the location of a road user.

[0084] System 500 can be configured to operate an autonomous vehicle according to one or more than one of a plurality of trajectories. For example, system 500 can use control system 510 (which can be used with Figure 4 The system 500 may be configured such as in the planning system 508 (which may be similar to the control system 408 of FIG. 1 ). Figure 4 The plurality of trajectories may be generated and / or planned similarly to planning system 404 of the system 500. For example, the plurality of trajectories may include potential trajectories that the system 500 may operate the autonomous vehicle to follow. For example, the plurality of potential trajectories may be considered as candidate trajectories to be selected by the AV, for example.

[0085] The plurality of trajectories may include one or more of a rate, acceleration, and direction at which the autonomous vehicle may operate. The system 500 may be configured to determine the plurality of trajectories. The plurality of trajectories may include a first trajectory. The plurality of trajectories may include a second trajectory and optionally a third trajectory, and optionally a fourth trajectory. The number of trajectories in the plurality of trajectories is not limiting. The plurality of trajectories may include one or more trajectories that coincide in time, such as potential trajectories that may occur simultaneously. The plurality of trajectories may include one or more trajectories in chronological order.

[0086] In some embodiments or examples, the system 500 is configured to determine the area around the road user's location based on sensor data. For example, the area may include a boundary around the road user. The area may include a horizontal plane around the road user. The area may be a vertical plane around the road user. The area may include horizontal and vertical planes around the user, such as defining a rectangular area and / or a polygonal area, etc. The area may be centered on the road user. The area may not be centered on the road user. The system 500 may be configured to determine the volume of the location around the road user based on sensor data, such as including horizontal and vertical planes around the user. For example, the system 500 determines a location including a horizontal area of ​​three square meters centered on the road user. In some embodiments or examples, the area around the road user's location includes a zone for communicating the intention of the maneuvering action of the autonomous vehicle. The zone may include areas where road users can perceive the autonomous vehicle and / or areas where road users can visually see the autonomous vehicle and / or areas where road users can auditorily hear the autonomous vehicle and / or areas where the autonomous vehicle can express its intentions to road users and / or areas where the autonomous vehicle can communicate expressively to road users and / or areas where road users can understand the autonomous vehicle's intended expressive operations (such as expressive intentions, etc.). For example, the zone is a zone used by an expressive autonomous vehicle that can be considered an expressive robot.

[0087] The targeted expressive operation may be viewed as indicating the operation of an autonomous vehicle as operated by the system 500, wherein the operation is configured to communicate the intent of the AV. The targeted expressive operation may include the operation of the AV that indicates the expressive intent of the autonomous vehicle. The targeted expressive operation may be viewed as a maneuver of the autonomous vehicle, such as a maneuver according to a trajectory, etc. For example, the targeted expressive operation may be viewed as indicating the operation of one or more of a control operation, such as one or more of an acceleration, deceleration, continuing at a certain rate, and changing direction of the autonomous vehicle, etc.

[0088] In some embodiments or examples, system 500 is configured to obtain rules indicating objective expressive operation of an autonomous vehicle in the vicinity of road users and / or places (e.g., crosswalks). The rules can be part of rule system 504. System 500 can optionally include perception system 506, which can include rule system 504. System 500 can be configured to forgo obtaining rules indicating objective expressive operation of an autonomous vehicle when sensor data does not indicate that the AV is in the vicinity of road users and / or places.

[0089] For example, system 500 may obtain rules only when sensor data indicates that the autonomous vehicle is in the vicinity of a road user. The vicinity of a road user may be an area around the road user, such as a horizontal area, etc. System 500 may determine the boundaries of proximity. System 500 may obtain different proximity sizes according to rules. Proximity may be considered to be within the sensing range of a sensor (such as sensor 502, etc.) providing sensor data 503. Proximity may be defined based on a distance relative to a road user and / or relative to an area associated with a road user. A proximity range may be defined based on an autonomous vehicle range relative to a road user and / or relative to an area associated with a road user. The proximity range of a road user may be the same as the range from an AV to a zone for communicating the intention of a maneuvering action of an autonomous vehicle. The proximity range of a road user may be different from the zone for communicating the intention of a maneuvering action of an autonomous vehicle.

[0090] The rule indicating the target expressive operation of the autonomous vehicle can be regarded as a rule that evaluates and optionally implements the expressive operation of the AV, for example, by selecting a trajectory based on a violation metric indicating non-violation of the rule. For example, the rule indicating the target expressive operation of the autonomous vehicle can be regarded as a rule configured to facilitate the operation of the AV (such as the specific behavior of the AV near road users, etc.). The rule indicating the target expressive operation of the autonomous vehicle can be regarded as an expressive rule, such as a rule for expressing yielding, etc. The rule indicating the target expressive operation may include a rule scope and / or a rule statement. The rule scope may include a first criterion for initiating the application of the rule. The first criterion may be regarded as an entry criterion for entering the rule. The rule statement may include a second criterion for detecting the satisfaction of the rule. The rule may include one or more criteria related to a set of operations (such as actions, boundaries and / or constraints, etc.) within which the system 500 can be configured to operate the vehicle. The rule can be based on evaluating the operation of the autonomous vehicle by the system 500 for one or more criteria. The rule may detect whether the operation has been an expressive operation of the system 500 on the autonomous vehicle (such as an expressive maneuver, etc.). The rule may be configured to detect whether the operation of the system 500 on the autonomous vehicle is a violation. The rule may be obtained from a plurality of rules (such as a list of rules and / or a database of rules, etc.). The rule may be obtained from a single rule.

[0091] In some embodiments or examples, the system 500 is configured to determine whether the sensor data 503 meets the first criterion. The system 500 can be configured to apply a rule in response to the sensor data meeting the first criterion. For example, in response to the sensor data meeting the first criterion, the system 500 can be configured to apply a rule for evaluating whether the operation of the autonomous vehicle captured by the sensor data meets the rule. For example, the application of the rule can involve evaluating how the autonomous vehicle slows down when reaching the crosswalk based on the sensor data. For example, the expressive operation of the AV can provide a clear signal to the road user that they can cross the crosswalk. The target expressive operation can include expressive stopping and / or expressive yielding. The target expressive operation can be regarded as an operation targeted by the system so that the AV can express its intention to the road user (such as the intention to yield to the road user, etc.). In some embodiments or examples, the target expressive operation of the autonomous vehicle near the road user includes yielding at the crosswalk. For example, the system 500 may be configured to operate the autonomous vehicle to yield (such as slow down, etc.) when approaching a road user, such as at a crosswalk and / or before a crosswalk. For example, the system 500 may be configured to operate the autonomous vehicle to yield (such as slow down, etc.) when approaching a jaywalker, a traffic controller, and / or a cyclist. The rules may be viewed as expressive rules, such as expressive yield rules, etc.

[0092] The rule may include a first criterion for application of the rule. The first criterion may be considered an entry criterion. For example, the first criterion is used to determine when to apply the rule. For example, the first criterion is used to initiate the rule (e.g., to initiate operation of the autonomous vehicle according to the rule) and / or enter the rule. For example, when the system 500 determines that the sensor data meets the first criterion, the system 500 applies the rule. When the sensor data does not meet the first criterion, the system 500 may not initiate the application of the rule. For example, when the sensor data does not meet the first criterion, the system 500 may abandon the application of the rule.

[0093] For example, when the sensor data indicates that the autonomous vehicle is in the vicinity of a road user and / or a location (e.g., a crosswalk), the system 500 may determine that the sensor data meets the first criterion (such as the first criterion is satisfied, etc.). For example, when the sensor data indicates the presence of a road user, the system 500 may determine that the sensor data meets the first criterion (such as the first criterion is satisfied, etc.). For example, when the sensor data indicates the presence of a location such as a crosswalk, the system 500 may determine that the sensor data meets the first criterion (such as the first criterion is satisfied, etc.).

[0094] The rule may also include a second criterion. The second criterion may be used to detect whether the system 500 (such as the operation of an autonomous vehicle, etc.) has satisfied (such as complied with, etc.) the rule. When the system 500 determines that the second criterion is met, the rule may be considered satisfied. When the system 500 determines that the second criterion is not met, the rule may be considered not satisfied. The system 500 may be configured to determine whether the sensor data complies with the second criterion. For example, when the system 500 determines that the sensor data complies with the second criterion, the rule may be considered satisfied. When the system 500 determines that the sensor data does not comply with the second criterion, the rule may be considered not satisfied.

[0095] In some embodiments or examples, the system 500 is configured to determine whether the sensor data meets the first criterion. In response to the system 500 determining that the sensor data does not meet the first criterion, the system 500 can be configured to abandon the application of the rule. In response to the system 500 determining that the sensor data does meet the first criterion, the system 500 can be configured to apply the rule.

[0096] Applying the rules may include evaluating the operation of the autonomous vehicle by the system 500. Applying the rules may include determining a violation metric based on the sensor data and the second criterion. The system 500 may be configured to determine a violation metric based on the sensor data and the second criterion. The violation metric may be considered as a metric indicating a violation of the rule or a zero violation (such as no violation, etc.). The violation metric may include a parameter that quantifies the violation. For example, the violation metric may include a weight parameter that quantifies the violation.

[0097] In response to the system 500 determining that the sensor data meets the second criterion, the system 500 can determine a violation metric indicating zero violations. In response to the system 500 determining that the sensor data does not meet the second criterion, the system 500 can determine a violation metric indicating a violation. In response to the system 500 determining that the sensor data does not meet the second criterion, the system 500 can determine the violation metric as indicating a violation of the distance parameter. The violation can be a violation of a rule. For example, if the system 500 operates the autonomous vehicle in response to a rule with a higher priority than the current rule, a rule can be violated.

[0098] In some embodiments or examples, the system 500 is configured to evaluate the first trajectory of the autonomous vehicle based on the violation metric. The system 500 can be configured to evaluate the first trajectory of the autonomous vehicle during operation of the autonomous vehicle along the first trajectory. The system 500 can be configured to evaluate the first trajectory of the autonomous vehicle after operation of the autonomous vehicle along the first trajectory. Evaluating the assessment of the system 500 for the autonomous vehicle can be advantageous to enhance the expressive operation of the AV.

[0099] In some embodiments or examples, evaluating the first trajectory of the autonomous vehicle based on the violation metric using at least one processor is performed after completion of the first trajectory. For example, the system 500 can be backward looking and configured to evaluate the operation of the autonomous vehicle after the operation of the autonomous vehicle has occurred. For example, the system 500 can be backward looking and configured to evaluate the first trajectory of the autonomous vehicle after the first trajectory of the autonomous vehicle has been driven.

[0100] In some embodiments or examples, the system 500 is configured to select a second trajectory from the plurality of trajectories based on the violation metric. The second trajectory may be different from the first trajectory. The second trajectory may be selected in the planning system 508 (which may be associated with the first trajectory). Figure 4 The second trajectory may be the same as the first trajectory.

[0101] The first criterion may be a plurality of different types of criteria. The first criterion may include one or more criteria such as discussed herein. The first criterion may include a plurality of criteria to be met for the system 500 to apply a rule (e.g., an entry rule). The first criterion may include a single criterion to be met for the system 500 to apply a rule. The first criterion may be met when one or more conditions are met.

[0102] In some embodiments or examples, the sensor data meets the first criterion in response to the sensor data indicating that the autonomous vehicle is in the vicinity of a road user and / or a location (e.g., a crosswalk). For example, when the sensor data indicates that the autonomous vehicle is in the vicinity of a road user and / or a location (e.g., a crosswalk), the system 500 determines that the sensor data meets the first criterion.

[0103] For example, system 500 may obtain sensor data indicating a road user. The first criterion may be whether the sensor data indicates that a road user is in the vicinity of an autonomous vehicle and / or a location (e.g., a crosswalk). Based on system 500 obtaining sensor data 503 indicating that the autonomous vehicle is in the vicinity of a road user and / or a location (e.g., a crosswalk), system 500 determines that sensor data 503 meets the first criterion. Based on system 500 obtaining sensor data 503 indicating that the autonomous vehicle is not in the vicinity of a road user and / or a location (e.g., a crosswalk) (e.g., within a distance relative to the road user and / or location (e.g., a crosswalk)), system 500 determines that sensor data 503 meets the first criterion.

[0104] For example, the system 500 may operate the autonomous vehicle to approach a crosswalk. The system 500 may obtain sensor data indicating that a road user (such as a jaywalker, a cyclist, and / or a traffic controller) is at the location of the crosswalk and determine that the sensor data 503 meets the first criterion.

[0105] In one or more examples, the first criterion includes one or more conditions (e.g., including the first condition). In other words, the first criterion can include a first condition that is met when the sensor data indicates that the autonomous vehicle is in the vicinity of a road user and / or a location (e.g., a crosswalk). Within proximity is, for example, within 30 meters, within 20 meters, within 10 meters of a road user and / or a location. In one or more examples, the sensor data meets the first criterion when the sensor data meets the first condition.

[0106] In some embodiments or examples, in response to the sensor data indicating that the autonomous vehicle is in the lane closest to the road boundary, the sensor data meets the first criterion. For example, when the sensor data indicates that the autonomous vehicle is in the lane closest to the road boundary, the system 500 determines that the sensor data meets the first criterion.

[0107] The road boundary can be a boundary of a road. For example, the road boundary is a shoulder and / or a median. The road boundary can be the location where the drivable road ends. System 500 can be configured to obtain sensor data indicating a lane in which the autonomous vehicle is operating. In response to the sensor data indicating that the autonomous vehicle is in the lane closest to the lane boundary, system 500 can determine that the sensor data meets the first criterion. In response to the sensor data not indicating that the autonomous vehicle is in the lane closest to the lane boundary, system 500 can determine that the sensor data does not meet the first criterion.

[0108] For example, when the sensor data may indicate that the autonomous vehicle is in the rightmost lane closest to the lane boundary, the system 500 may determine that the sensor data meets the first criterion. For example, when the sensor data may indicate that the autonomous vehicle is in the leftmost lane closest to the lane boundary, the system 500 may determine that the sensor data meets the first criterion.

[0109] In other words, the first criterion may include a second condition that is met when the system 500 obtains sensor data indicating that the autonomous vehicle is in a lane closest to a road boundary (e.g., the rightmost or leftmost lane closest to a road boundary and / or a lane boundary). In one or more examples, the sensor data meets the first criterion when the sensor data meets the second condition.

[0110] In some embodiments or examples, the sensor data meets the first criterion in response to the sensor data indicating that the road user has left the road surface. The road surface may be a drivable road surface. The road surface may be a surface on which the system 500 is configured to operate the autonomous vehicle.

[0111] In some embodiments or examples, the system 500 is configured to obtain sensor data indicating that the road user left the road surface. In response to the sensor data indicating that the road user left the road surface, the system 500 may determine that the sensor data meets the first criterion. In response to the sensor data not indicating that the road user left the road surface, the system 500 may determine that the sensor data does not meet the first criterion.

[0112] For example, the system 500 may obtain sensor data indicating that a pedestrian has left the road. In other words, the first criterion may include a third condition that is met when the system 500 obtains sensor data indicating that a road user has left the road. In one or more examples, the sensor data meets the first criterion when the sensor data meets the third condition.

[0113] In some embodiments or examples, the sensor data meets the first criterion in response to the sensor data indicating that the road user is in a lane closest to a road boundary.

[0114] In some embodiments or examples, the system 500 is configured to obtain sensor data indicating that the road user is in the lane closest to the road boundary. In response to the sensor data indicating that the road user is in the lane closest to the road boundary, the system 500 may determine that the sensor data meets the first criterion. In response to the sensor data not indicating that the road user is in the lane closest to the road boundary, the system 500 may determine that the sensor data does not meet the first criterion.

[0115] For example, system 500 may obtain sensor data indicating that a pedestrian is in the rightmost lane closest to the road boundary. In other words, the first criterion may include a fourth condition that is met when system 500 obtains sensor data indicating that the road user is in the lane closest to the road boundary. In one or more examples, the sensor data meets the first criterion when the sensor data meets the fourth condition.

[0116] In some embodiments or examples, the sensor data meets the first criterion in response to the sensor data indicating that the distance between the location of the road user and the location of the crosswalk does not meet the distance threshold.

[0117] In some embodiments or examples, the system 500 is configured to obtain sensor data indicating a distance between a location of a road user and a location of a crosswalk. In response to the sensor data indicating that the distance is below a distance threshold, the system 500 may determine that the sensor data meets the first criterion. In response to the sensor data indicating that the distance is at and / or above the distance threshold, the system 500 may determine that the sensor data does not meet the first criterion.

[0118] In other words, the first criterion may include a fifth condition that is met when the system 500 obtains sensor data indicating that the distance between the location of the road user and the location of the crosswalk is below a distance threshold. For example, the pedestrian may be within a specific distance threshold relative to the location of the crosswalk. The distance may be 0.5m, 1m, 5m, and / or 10m. In one or more examples, the sensor data meets the first criterion when the sensor data meets the fifth condition.

[0119] In one or more examples, when the sensor data meets the first condition and / or the second condition and / or the third condition and / or the fourth condition and / or the fifth condition, the sensor data meets the first criterion. In one or more examples, when the sensor data meets the first condition and the second condition and the fifth condition, the sensor data meets the first criterion. In one or more examples, when the sensor data meets the third condition or the fourth condition, the sensor data meets the fifth condition. In some embodiments or examples, determining whether the sensor data meets the first criterion includes: determining that the sensor data meets the first criterion based on the location of the road user indicated by the sensor data being near a crosswalk.

[0120] In some embodiments or examples, the system 500 is configured to obtain sensor data indicating the location of the road user and the location of the crosswalk. The sensor data may indicate the distance between the road user and the crosswalk. In response to the sensor data indicating that the distance is below a distance threshold, the system 500 may determine that the first criterion is met. In response to the sensor data indicating that the distance is at and / or above a distance threshold, the system 500 may determine that the first criterion is not met.

[0121] In some embodiments or examples, determining, using at least one processor, the violation metric based on the sensor data and the second criterion includes determining, using at least one processor, whether the sensor data complies with the second criterion. System 500 may be configured to determine whether the sensor data complies with the second criterion.

[0122] In some embodiments or examples, determining whether the sensor data meets the second criterion using at least one processor includes: determining a stopping distance parameter based on the sensor data using at least one processor, the stopping distance parameter indicating the distance between the location of the autonomous vehicle and the location of the road user when the autonomous vehicle reaches the stopping point. In some embodiments or examples, determining whether the sensor data meets the second criterion using at least one processor includes: determining whether the stopping distance parameter is below a threshold (e.g., different from a distance threshold) using at least one processor. In some embodiments or examples, determining whether the sensor data meets the second criterion using at least one processor includes: determining that the sensor data does not meet the second criterion in response to the stopping distance parameter being below the threshold.

[0123] In some embodiments or examples, the system 500 is configured to obtain sensor data indicating the distance between the location of the autonomous vehicle and the location of the road user when the autonomous vehicle reaches the stopping point. The system 500 can be configured to determine a stopping distance parameter indicating the distance. For example, the stopping distance parameter includes the distance between the location of the autonomous vehicle and the location of the road user when the autonomous vehicle reaches the stopping point.

[0124] In response to the sensor data indicating that the stopping distance parameter is below a threshold, the system 500 may determine that the sensor data does not meet the second criterion. In response to the stopping distance parameter indicating that the distance is at and / or above a distance threshold, the system 500 may determine that the sensor data meets the second criterion. The distance threshold may be considered an expressive threshold (such as an expressive yield threshold, etc.).

[0125] In response to the sensor data not meeting the second criterion (such as the stopping distance parameter indicating a distance below a threshold, etc.), the system 500 can be configured to determine a violation metric indicating a violation. In response to the sensor data meeting the second criterion (such as the stopping distance parameter indicating a distance at and / or above a threshold, etc.), the system 500 can be configured to determine a violation metric indicating zero violation.

[0126] In some embodiments or examples, determining, using at least one processor, the violation metric based on the sensor data and the second criterion includes determining the violation metric as indicating zero violation in response to the sensor data satisfying the second criterion.

[0127] In some embodiments or examples, determining the violation metric based on the sensor data and the second criterion using at least one processor includes: in response to the sensor data not meeting the second criterion, determining the violation metric as indicating a violation distance parameter. The violation distance parameter can be a violation metric indicating a violation (such as a quantified violation, etc.).

[0128] In some embodiments or examples, the violation distance parameter includes a distance at which the stopping distance parameter does not meet a threshold value. The system 500 can be configured to determine a quantifiable amount of distance at which the rule has been violated based on the sensor data. For example, the system 500 can determine the distance at which the stopping distance parameter does not meet the threshold value based on the sensor data indicating that the distance at which the autonomous vehicle has stopped is below a threshold value (such as the difference between the threshold value and the stopping distance parameter). For example, the violation distance parameter includes a distance at which the stopping distance parameter is less than (e.g., lower than and / or does not reach) a threshold value.

[0129] In one or more example systems, the system 500 is configured to control the autonomous vehicle based on the selected second trajectory using at least one processor. The system 500 can be configured to express specific operations (e.g., target expressive operations) of the autonomous vehicle to the road user based on rules (such as by controlling the autonomous vehicle based on the selected second trajectory, etc.).

[0130] In some embodiments or examples, the system 500 is configured to perform operations including controlling the autonomous vehicle based on the first trajectory. The system 500 can be configured to generate control data associated with the control of the autonomous vehicle based on the second trajectory and / or the violation metric. For example, the system can be configured to transmit the control data to a controller of the autonomous vehicle and / or actually operate the autonomous vehicle.

[0131] In one or more embodiments or examples, the system 500 is configured to control the autonomous vehicle by generating control data for the control system of the autonomous vehicle. In one or more embodiments or examples, the system 500 is configured to control the autonomous vehicle by providing control data to the control system of the autonomous vehicle. In one or more embodiments or examples, the system 500 is configured to control the autonomous vehicle by transmitting control data to, for example, the control system of the autonomous vehicle and / or an external system. In one or more embodiments or examples, the system 500 is configured to control the autonomous vehicle by controlling the control system and / or the external system of the autonomous vehicle based on the control data. The operation of controlling the autonomous vehicle can be performed by the system 500 in the control system 510.

[0132] In some embodiments or examples, at least one non-transitory computer-readable medium stores instructions that, when executed by at least one processor, cause the at least one processor to perform operations that include using the at least one processor to determine a total violation metric for one or more than one (e.g., all) locations to which the rule is applied. For example, the system 500 can be configured to combine some and / or all violation metrics indicating violations. As an example, the system 500 applies the square root of the sum of violations for all crosswalks to which the rule is applied. Other methods for determining a total violation metric may also be used.

[0133] In some embodiments or examples, at least one non-transitory computer-readable medium stores instructions that, when executed by at least one processor, cause the at least one processor to perform operations including using the at least one processor to provide data indicating a total violation metric for control of the autonomous vehicle. The system 500 may be configured to transmit data indicating a total violation metric. The system 500 may be configured to generate data indicating a total violation metric. The system 500 may be configured to transmit data indicating a total violation metric external to the system 500.

[0134] In some embodiments or examples, at least one non-transitory computer-readable medium stores instructions that, when executed by at least one processor, cause the at least one processor to perform operations including, using the at least one processor, selecting a second trace from a plurality of traces based on a total violation metric.

[0135] In some embodiments or examples, the rule includes a third criterion for exiting the rule. In response to the system 500 obtaining sensor data that meets the third criterion, the system 500 can be configured to exit the rule, such as stopping (e.g., terminating and / or abandoning, etc.) applying the rule, etc. In response to the system 500 obtaining sensor data that does not meet the third criterion, the system 500 can be configured not to exit the rule, such as continuing to apply the rule, etc. For example, when the sensor data stops indicating the presence of a road user in the vicinity of the autonomous vehicle, the system can determine that the sensor data meets the third criterion.

[0136] Reference now Figure 6 , Figure 6 Shows a rule system such as Figure 5 FIG. 6 is a diagram of an example implementation 600 of a process for autonomous vehicles to yield to a pedestrian, such as a rule system 504 or the like. As shown, a system such as system 500 may obtain a signal indicating a crosswalk (at a Figure 6 In addition, the system may obtain sensor data indicating road users at a certain location.

[0137] The system may obtain a rule indicative of a targeted expressive operation of an autonomous vehicle in the vicinity of a road user, the rule including a first criterion for initiating application of the rule. The rule may have Figure 6 , which includes example first criteria such as those discussed herein. The rule scope may include a condition 606 that is met when the sensor data indicates that the autonomous vehicle (e.g., the ego) is the closest vehicle approaching the crosswalk. Condition 606 may be considered a first condition (e.g., as for Figure 5 and 9 The rule scope may include a condition 608 that is met when the sensor data indicates that the autonomous vehicle is in the rightmost lane closest to the lane boundary. Condition 608 may be considered a second condition (e.g., as for Figure 5 and 9 The rule scope may include a condition 610 that is met when the sensor data indicates that a road user (e.g., a pedestrian) is off the road surface (e.g., on a pavement or sidewalk). Condition 610 may be considered a third condition (e.g., as for Figure 5 and 9 The rule range may include a condition 612 that is met when the sensor data indicates that a road user (e.g., a pedestrian) is in the rightmost lane. Condition 612 may be considered a fourth condition (e.g., as for Figure 5 and 9The rule range may include a condition 614 that is met when the sensor data indicates that a road user (e.g., a pedestrian) is within a certain distance (e.g., a distance threshold) relative to the crosswalk. Condition 614 may be considered a fifth condition (e.g., as in the case of a pedestrian crossing). Figure 5 and 9 described above).

[0138] In one or more examples, the first criterion may include one or more of the following: condition 608, condition 606, condition 610, condition 612, and condition 614. For example, the first criterion may be met according to the system 500 obtaining sensor data indicating that the autonomous vehicle (e.g., ego) is the vehicle closest to the approaching crosswalk as illustrated by condition 606. The first criterion may be met according to the system obtaining sensor data indicating that the autonomous vehicle is in the rightmost lane closest to the lane boundary as illustrated by condition 608.

[0139] In addition, the first criterion may be met based on sensor data obtained by the system, which sensor data indicates that a road user (e.g., a pedestrian) has left the road surface as illustrated by condition 610, or that a road user (e.g., a pedestrian) is in the rightmost lane as illustrated by condition 612, and that a road user (e.g., a pedestrian) is within a certain distance (e.g., a distance threshold) relative to a crosswalk as illustrated by condition 614.

[0140] exist Figure 6 In the example, when conditions 606, 608, and 614 are met, the first criterion is met. For example, when conditions 610 or 612 are met, condition 614 is checked. Figure 6 In , when conditions 606, 608, 610 and 614 are met, the first criterion is met. Figure 6 In the example, when conditions 606, 608, 612, and 614 are met, the first criterion is met. When conditions 606, 608, and 614 of the first criterion are met, the system may apply a method such as Figure 6 The rule shown in rule statement 616.

[0141] The system can operate the autonomous vehicle, such as via a trajectory in a plurality of trajectories, such as a first trajectory or a second trajectory, etc. For example, the system can be configured to operate the autonomous vehicle to maintain a certain spacing relative to the crosswalk as illustrated in operation 618, and / or to stop the autonomous vehicle as illustrated in operation 620.

[0142] Such as Figure 6 As shown, the system can determine a violation metric indicating zero violation or a violation based on the second criterion and the sensor data, and evaluate the first trajectory of the autonomous vehicle based on the violation metric.

[0143] For example, when the sensor data indicates that the stopping distance parameter between the autonomous vehicle and the road user is at and / or above a threshold value (illustrated by 618), the system determines the violation metric 622 as indicating a zero violation ( Figure 6 ). The determination of zero violation can be associated with, for example, the AV stopping or the autonomous vehicle maintaining a certain distance (e.g., spacing) relative to the crosswalk. For example, when the sensor data indicates that a stopping distance parameter between the autonomous vehicle and the road user is at and / or above a threshold, the sensor data meets the second criterion and the system can determine a violation metric indicating zero violation.

[0144] For example, when the sensor data indicates that the stopping distance parameter between the autonomous vehicle and the road user is below a threshold, the system determines the violation metric 624 as indicating a violation ( Figure 6 ). For example, the AV did not stop, or the autonomous vehicle did not maintain a certain clearance relative to the crosswalk. For example, when the sensor data indicates that the stopping distance parameter between the autonomous vehicle and the road user is below a threshold, the sensor data meets the second criterion and the system can determine a violation metric indicating a violation. This can be shown as indicating that the stopping distance parameter of the autonomous vehicle relative to the crosswalk is less than a threshold distance (m).

[0145] Reference now FIG. 7A to FIG. 7B , FIG. 7A to FIG. 7B A diagram of an example implementation 700 of a process for autonomous vehicle yielding utilizing an autonomous vehicle 702 is shown. Fig. 7A A violation metric indicating zero violations is illustrated. Figure 7B An example of a violation metric (bottom) indicating a violation. For example, autonomous vehicle 702 Figure 7B The autonomous vehicle 702 is too close to the pedestrian crossing and thus violates the rules. Fig. 7A is in a suitable stopping place.

[0146] The violation metric may be determined as discussed herein.Example mathematical models that may be used to determine the violation metric are further discussed.

[0147] For example, a rule applied by system 500 may have a rule scope, which may be represented, for example, as:

[0148]

[0149] in,

[0150]

[0151] as well as

[0152]

[0153] Scope of the rule Indicates that if the autonomous vehicle is the closest vehicle approaching a pedestrian crossing in the rightmost lane and there is one or more road users outside the road surface or in the rightmost lane within a certain distance relative to the protection zone of the pedestrian crossing, the rules apply for the pedestrian crossing.

[0154] A rule may include a rule statement related to the satisfaction of the rule, which may be expressed, for example, as:

[0155]

[0156] in,

[0157] A rule statement Π may indicate that the autonomous vehicle satisfies the rule if it satisfies the rule for all crosswalks in the scenario. i ) When the rules apply and the autonomous vehicle stops or maintains a certain distance from the crosswalk, the autonomous vehicle may meet the rules relative to the crosswalk.

[0158] An example instantaneous violation metric may be determined by the system, where the instantaneous violation is related to time t and may be expressed, for example, as:

[0159]

[0160] otherwise,

[0161] For pedestrian crossings i=1,...,n crw , for whichσ i (t).

[0162] An example violation metric for a given crosswalk may be determined by the system in the following manner:

[0163] exist For pedestrian crossings ρ T，i =max*ρ i (t),.

[0164] The violation metric for a crosswalk that is in range at any time during operation>T,I (e.g., all entry criteria are met, so the rules apply) is the maximum instantaneous violation of the autonomous vehicle relative to that crosswalk during those time intervals when the crosswalk was nearby.

[0165] The system can determine the total number of crosswalks that are in range at any time during a scenario, for example:

[0166]

[0167] The system can determine the total violation metric> total (unit m), which can be expressed, for example, as:

[0168]

[0169] Certain equations and / or mathematical formulas have been used herein and may be used by a system, such as system 500, etc. For example, a list of regular constants that may be used for any of the described equations:

[0170]

[0171]

[0172] For example, a list of rule parameters that can be used with any of the equations described:

[0173]

[0174] In one or more examples or embodiments, the autonomous vehicle 702 can be located between other agents in the environment, such as other vehicles, etc. In other cases, the autonomous vehicle 702 is located in front of or behind another agent. Similar to the discussion of rules for scenarios with pedestrians and / or crosswalks, rules can be applied for appropriate spacing between the autonomous vehicle 702 and other vehicles in the environment. The spacing can vary depending on whether the other vehicles are in front of the autonomous vehicle 702 or behind the autonomous vehicle 702. In addition, additional vehicles in the environment may affect the rules applied for pedestrians and / or crosswalks.

[0175] Reference now FIG. 8A to FIG. 8C , showing example implementations of scenarios that an autonomous vehicle may experience during operation of system 500. Fig. 8A Implementation 800 is illustrated in which autonomous vehicle 802 encounters a bicyclist at a crosswalk. For example, system 500 can obtain sensor data indicating that the environment includes a road user (e.g., bicyclist 804).

[0176] The system can be configured to obtain rules that indicate the intended expressive operation of autonomous vehicle 802 in the vicinity of road users. For example, system 500 can be configured to slow autonomous vehicle 802 and stop well before a crosswalk. This can provide an expressive intent to cyclist 804, indicating that it is safe to cross the crosswalk.

[0177] The rule may include a first criterion for initiation of the rule. In response to sensor data indicating that the autonomous vehicle is in a lane closest to a road boundary and the autonomous vehicle 802 is in the vicinity of a cyclist 804 (e.g., a road user), the system 500 may determine that the first criterion is met. Other first criteria may also be used in conjunction with or in place of the listed first criteria.

[0178] Because in Fig. 8A In the illustrated implementation 800, the first criterion is determined to be met, so the system 500 applies the rule. The application of the rule can provide an expressive intent to the cyclist 804. In addition, the system 500 can be configured to determine a violation metric based on the sensor data and the second criterion of the rule.

[0179] The second criterion may not be met based on the sensor data indicating that the stopping distance parameter indicating distance D2 is below threshold T. In other words, autonomous vehicle 802 is stopped too close to cyclist 804. Then, system 500 can determine a violation metric indicating a violation of distance parameter D1=T-D2. Violation of distance parameter D1 includes the distance at which stopping distance parameter D2 does not meet threshold T.

[0180] Figure 8B An implementation 870 is illustrated in which an autonomous vehicle 802 does not encounter jaywalkers at a crosswalk. Figure 8B As shown, for the application of the rule, a crosswalk is not required. Therefore, except when the road user is a jaywalker 806 rather than a cyclist and there is no crosswalk, Figure 8B The implementation of 870 relative to Fig. 8A same.

[0181] In addition, the system 500 can be configured to determine a violation metric based on the sensor data and a second criterion of the rule. The second criterion can be met based on the sensor data indicating that the stopping distance parameter indicating the distance D3 is equal to the threshold T. In other words, the autonomous vehicle 802 stops appropriately, thereby providing an expressive intent to the jaywalker 806. In this example, the system determines a violation metric indicating zero violation.

[0182] Figure 8C An implementation 850 of an autonomous vehicle 802 encountering a traffic controller is illustrated. Figure 8C Examples and Figure 8B A similar situation, except that the road user is a traffic controller on the road 808 rather than a jaywalker. Figure 8B Similarly, the vehicle stops appropriately at a stopping distance parameter D4 above the threshold, thereby providing an expressive intent to the traffic controller, and a violation metric indicating zero violation.

[0183] Reference now Fig. 9, a flowchart illustrating a method or process 900 for autonomous vehicle yielding, such as for operating and / or controlling an autonomous vehicle, etc. The method can be implemented by a system disclosed herein (such as AV computing 400 and Figure 1 , Figure 2 , Figure 3 , Figure 4 The vehicles 102, 200 and Figure 5 AV computing 540, system 500 and Figures 6 to 8C The disclosed system may include at least one processor that may be configured to perform one or more of the operations of method 900. Method 900 may be performed by another device or device group (e.g., completely and / or partially, etc.) separate from the system disclosed herein or including the system disclosed herein.

[0184] In one or more example methods, method 900 includes: using at least one processor, obtaining 902 sensor data associated with an environment in which an autonomous vehicle operates according to a first trajectory in a plurality of trajectories, the environment including a road user at a certain location. In one or more example methods, method 900 includes: using at least one processor, obtaining 904 a rule indicating a target expressive operation of an autonomous vehicle in the vicinity of a road user, the rule including a first criterion for initiating the application of the rule, and a second criterion. In one or more example methods, method 900 includes: using at least one processor, determining 906 whether the sensor data meets the first criterion. In one or more example methods, method 900 includes: in response to the sensor data meeting the first criterion, applying 908 the rule. In one or more example methods, applying 908 the rule includes: using at least one processor, determining a violation metric based on the sensor data and the second criterion. In one or more example methods, method 900 includes: using at least one processor, evaluating 910 the first trajectory of the autonomous vehicle based on the violation metric. In one or more example methods, method 900 includes selecting 912 , using at least one processor, a second trace from a plurality of traces based on a violation metric.

[0185] In one or more example methods, method 900 includes: in response to the sensor data not meeting the first criterion, abandoning 907 applying the rule. When the first criterion (such as an entry criterion, etc.) is met, the first criterion can be used to apply the rule. When the sensor data indicates the presence of the road user, the autonomous vehicle can be in the vicinity of the road user. The proximity can be considered to be within the sensing range of the sensor providing the sensor data. The rule can be considered an expressive rule, such as an expressive yield rule, etc. The target expressive operation can include an expressive stop and / or an expressive yield.

[0186] In one or more examples, the second criterion is used to detect whether the rule is satisfied. The violation metric can be considered as a metric indicating a violation or non-violation of the rule. The violation metric can include a parameter that quantifies the violation. The second trajectory can be the same as the first trajectory. In some examples, the second trajectory can be different from the first trajectory.

[0187] In one or more example methods, method 900 includes: using at least one processor, controlling the autonomous vehicle based on the selected second trajectory. Controlling may include: generating control data associated with controlling the autonomous vehicle based on the second trajectory and / or the violation metric, and / or transmitting the control data to a controller of the autonomous vehicle and / or actually operating the autonomous vehicle.

[0188] In one or more example methods, determining, using at least one processor, a violation metric based on the sensor data and the second criterion includes determining, using at least one processor, whether the sensor data complies with the second criterion.

[0189] In one or more example methods, determining, using at least one processor, whether the sensor data meets the second criterion includes: determining, using at least one processor, a stopping distance parameter based on the sensor data, the stopping distance parameter indicating a distance between a location of the autonomous vehicle and a location of a road user when the autonomous vehicle reaches a stopping point; and determining, using at least one processor, whether the stopping distance parameter is below a threshold, wherein in response to the stopping distance parameter being below the threshold, the sensor data does not meet the second criterion. The threshold may be considered an expressive threshold, such as an expressive yield threshold, etc.

[0190] In one or more example methods, determining, using at least one processor, a violation metric based on the sensor data and the second criterion includes determining the violation metric as indicating zero violation in response to the sensor data satisfying the second criterion.

[0191] In one or more example methods, determining, using at least one processor, a violation metric based on the sensor data and the second criterion includes determining the violation metric as indicating a violation of the distance parameter in response to the sensor data not meeting the second criterion.

[0192] In one or more example methods, the violation distance parameter includes a distance at which the stopping distance parameter does not meet a threshold. The distance may be a distance at which the autonomous vehicle has violated a rule, such as a difference between a threshold and the stopping distance parameter.

[0193] In one or more example methods, the rule includes a third criterion for exiting the rule. The method 900 may include determining whether the sensor data meets the third criterion. The method 900 may include exiting the rule (such as terminating application of the rule, etc.) in response to the sensor data meeting the third criterion. For example, the sensor data meets the third criterion when the sensor data stops indicating the presence of a road user in the vicinity of the autonomous vehicle.

[0194] In one or more example methods, the sensor data satisfies a first criterion in response to the sensor data indicating that the autonomous vehicle is in the vicinity of a road user. For example, the autonomous vehicle is close to a crosswalk, a jaywalker, a cyclist, and / or a traffic controller. In other words, the first criterion may include a first condition that is met when the sensor data indicates that the autonomous vehicle is in the vicinity of a road user.

[0195] In one or more example methods, in response to the sensor data indicating that the autonomous vehicle is in the lane closest to the road boundary, the sensor data meets the first criterion. For example, the rightmost lane closest to the road boundary or the leftmost lane closest to the road boundary. In other words, the first criterion may include a second condition that is met when the sensor data indicates that the autonomous vehicle is in the lane closest to the lane boundary (e.g., the rightmost or leftmost). In one or more example methods, in response to the sensor data indicating that the road user leaves the road surface, the sensor data meets the first criterion. For example, a pedestrian leaves the road surface. In other words, the first criterion may include a third condition that is met when the sensor data indicates that the road user leaves the road surface. In one or more example methods, in response to the sensor data indicating that the road user is in the lane closest to the road boundary, the sensor data meets the first criterion. For example, a pedestrian is in the rightmost lane. In other words, the first criterion may include a fourth condition that is met when the sensor data indicates that the road user is in the lane closest to the road boundary. In one or more example methods, in response to the sensor data indicating that the distance between the location of the road user and the location of the crosswalk does not meet the distance threshold, the sensor data meets the first criterion. For example, the pedestrian is in the rightmost lane. In other words, the first criterion may include a fifth condition that is met when the distance between the location indicated by the sensor data as the road user and the location of the crosswalk is below a distance threshold. For example, the pedestrian may be within a certain distance threshold from the location of the crosswalk.

[0196] In one or more example methods, the location of the road user includes an area surrounding the location of the road user. In one or more example methods, the area surrounding the location of the road user includes a zone for communicating the intention of the autonomous vehicle's maneuver. For example, an expressive autonomous vehicle that can be viewed as an expressive robot, such as a zone of an expressive robot.

[0197] In one or more example methods, method 900 includes: using at least one processor, determining a total violation metric for all locations where the rule is applied. For example, the total violation metric can be based on the square root of the sum of violations for all crosswalks where the rule is applied.

[0198] In one or more example methods, method 900 includes: using at least one processor, providing data indicative of a total violation metric for use in controlling an autonomous vehicle. Providing may include transmitting and / or generating data indicative of a total violation metric.

[0199] In one or more example methods, method 900 includes, using at least one processor, selecting a second trace from the plurality of traces based on the total violation metric.

[0200] In one or more example methods, the road user may be one or more of a user-operated vehicle and a pedestrian. The road user may be a user-operated vehicle including a cyclist and / or a user-operated scooter. For example, the road user may be one or more of a pedestrian and a user-operated vehicle near a crosswalk.

[0201] In one or more example methods, determining 906 whether the sensor data meets the first criterion includes determining that the sensor data meets the first criterion based on the sensor data indicating that the location of the road user is near a crosswalk.

[0202] In one or more example methods, a goal-expressive operation of an autonomous vehicle in the vicinity of a road user includes yielding at a crosswalk.

[0203] In one or more example methods, evaluating 910 , using at least one processor, a first trajectory of an autonomous vehicle based on a violation metric is performed after completion of the first trajectory.

[0204] In the previous description, aspects and embodiments of the present disclosure have been described with reference to many specific details, which may vary depending on the implementation. Therefore, the description and the accompanying drawings should be regarded as illustrative, not restrictive. The only and exclusive indication of the scope of the invention, and the applicant's expectation that the content of the scope of the invention is the literal and equivalent scope of the claims issued from this application in the specific form of the claims, including any subsequent amendments. Any definition of the terms used to be included in such claims that are clearly set forth herein should be based on the meaning of such terms as used in the claims. In addition, when the term "also includes" is used in the previous description or the attached claims, the following of the phrase may be an additional step or entity, or a sub-step / sub-entity of the previously described step or entity.

[0205] A non-transitory computer-readable medium is disclosed, comprising instructions stored thereon, which, when executed by at least one processor, cause the at least one processor to perform one or more operations according to the method disclosed herein.

[0206] Also disclosed are methods, non-transitory computer-readable media, and systems according to any of the following clauses.

[0207] Item 1. A method comprising:

[0208] obtaining, using at least one processor, sensor data associated with an environment in which the autonomous vehicle is operating according to a first trajectory of a plurality of trajectories, the environment including road users at a location;

[0209] obtaining, using the at least one processor, a rule indicative of a goal-expressive operation of the autonomous vehicle in the vicinity of the road user, the rule comprising a first criterion for initiating application of the rule, and a second criterion;

[0210] determining, using the at least one processor, whether the sensor data meets the first criterion,

[0211] In response to the sensor data meeting the first criterion, applying the rule;

[0212] Wherein, applying the rules comprises:

[0213] determining, using the at least one processor, a violation metric based on the sensor data and the second criterion;

[0214] evaluating, using the at least one processor, the first trajectory of the autonomous vehicle based on the violation metric; and

[0215] Using the at least one processor, a second trace is selected from the plurality of traces based on the violation metric.

[0216] Item 2. The method of Item 1, comprising: using the at least one processor to control the autonomous vehicle based on the selected second trajectory.

[0217] Clause 3. The method of clause 1, wherein determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises:

[0218] Using the at least one processor, a determination is made as to whether the sensor data meets the second criterion.

[0219] Item 4. The method of item 3, wherein determining, using the at least one processor, whether the sensor data meets the second criterion comprises:

[0220] determining, using the at least one processor, a stopping distance parameter based on the sensor data, the stopping distance parameter indicating a distance between a location of the autonomous vehicle and a location of the road user when the autonomous vehicle reaches a stopping point; and

[0221] determining, using the at least one processor, whether the stopping distance parameter is below a threshold;

[0222] In response to the stopping distance parameter being lower than a threshold, the sensor data does not meet the second criterion.

[0223] Clause 5. The method of any one of clauses 3 to 4, wherein determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises:

[0224] Responsive to the sensor data satisfying the second criterion, the violation metric is determined to indicate zero violation.

[0225] Clause 6. The method of any one of clauses 3 to 5, wherein determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises:

[0226] In response to the sensor data failing to comply with the second criterion, the violation metric is determined to indicate a violation of a distance parameter.

[0227] Item 7. The method of Item 6, wherein the violation distance parameter comprises a distance at which the stopping distance parameter does not satisfy the threshold.

[0228] Item 8. A method according to any of the preceding items, wherein the rule includes a third criterion for exiting the rule.

[0229] Item 9. A method according to any of the preceding items, wherein the sensor data meets the first criterion in response to the sensor data indicating that the autonomous vehicle is in the vicinity of the road user and / or the location.

[0230] Item 10. A method according to any of the preceding items, wherein the sensor data meets the first criterion in response to the sensor data indicating that the autonomous vehicle is in a lane closest to a road boundary.

[0231] Item 11. A method according to any of the preceding items, wherein the sensor data meets the first criterion in response to the sensor data indicating that the road user has left the road surface.

[0232] Item 12. A method according to any of the preceding items, wherein the sensor data meets the first criterion in response to the sensor data indicating that the road user is in a lane closest to a road boundary.

[0233] Item 13. A method according to any of the preceding items, wherein the sensor data meets the first criterion in response to the sensor data indicating that the distance between the location of the road user and the location of the crosswalk does not meet a distance threshold.

[0234] Item 14. A method according to any one of the preceding items, wherein the location of the road user includes an area surrounding the location of the road user.

[0235] Item 15. A method according to Item 14, wherein the area surrounding the location of the road user includes a zone for communicating the intention of the autonomous vehicle's maneuver.

[0236] Item 16. The method according to any one of the preceding items, comprising:

[0237] Using the at least one processor, a total violation metric is determined for all locations to which the rule is applied.

[0238] Item 17. The method of Item 16, comprising: using the at least one processor to provide data indicative of the total violation metric for use in controlling the autonomous vehicle.

[0239] Item 18. The method of any one of Items 16 to 17, the method comprising: using the at least one processor, selecting the second trajectory from the plurality of trajectories based on the total violation metric.

[0240] Item 19. A method according to any of the preceding items, wherein the road user can be one or more of the following: a user-operated vehicle and a pedestrian.

[0241] Item 20. A method according to any one of the preceding items, wherein determining whether the sensor data meets the first criterion includes: determining that the sensor data meets the first criterion based on the sensor data indicating that the location of the road user is near a crosswalk.

[0242] Item 21. A method according to any of the preceding items, wherein the objective expressive operation of the autonomous vehicle in the vicinity of the road user includes yielding at a crosswalk.

[0243] Item 22. A method according to any of the preceding items, wherein evaluating the first trajectory of the autonomous vehicle based on the violation metric using the at least one processor is performed after the first trajectory is completed.

[0244] Item 23. A system comprising:

[0245] at least one processor; and

[0246] at least one non-transitory computer-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

[0247] obtaining, using at least one processor, sensor data associated with an environment in which the autonomous vehicle is operating according to a first trajectory of a plurality of trajectories, the environment including road users at a location;

[0248] obtaining, using the at least one processor, a rule indicative of a goal-expressive operation of the autonomous vehicle in the vicinity of the road user, the rule comprising a first criterion for initiating application of the rule, and a second criterion;

[0249] determining, using the at least one processor, whether the sensor data meets the first criterion,

[0250] In response to the sensor data meeting the first criterion, applying the rule; wherein applying the rule comprises:

[0251] determining, using the at least one processor, a violation metric based on the sensor data and the second criterion;

[0252] evaluating, using the at least one processor, the first trajectory of the autonomous vehicle based on the violation metric; and

[0253] Using the at least one processor, a second trace is selected from the plurality of traces based on the violation metric.

[0254] Item 24. The system of Item 23, wherein the at least one non-transitory computer-readable medium stores instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

[0255] The autonomous vehicle is controlled, using the at least one processor, based on the selected second trajectory.

[0256] Clause 25. The system of clause 23, wherein determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises:

[0257] Using the at least one processor, a determination is made as to whether the sensor data meets the second criterion.

[0258] Clause 26. The system of clause 25, wherein determining, using the at least one processor, whether the sensor data meets the second criterion comprises:

[0259] determining, using the at least one processor, a stopping distance parameter based on the sensor data, the stopping distance parameter indicating a distance between a location of the autonomous vehicle and a location of the road user when the autonomous vehicle reaches a stopping point; and

[0260] determining, using the at least one processor, whether the stopping distance parameter is below a threshold;

[0261] In response to the stopping distance parameter being lower than a threshold, the sensor data does not meet the second criterion.

[0262] Clause 27. The system of any one of clauses 25 to 26, wherein determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises:

[0263] Responsive to the sensor data satisfying the second criterion, the violation metric is determined to indicate zero violation.

[0264] Clause 28. The system of any one of clauses 25 to 27, wherein determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises:

[0265] In response to the sensor data failing to comply with the second criterion, the violation metric is determined to indicate a violation of a distance parameter.

[0266] Item 29. A system according to Item 28, wherein the violation distance parameter includes a distance at which the stopping distance parameter does not meet the threshold.

[0267] Item 30. A system according to any one of Items 23 to 29, wherein the rule includes a third criterion for exiting the rule.

[0268] Item 31. A system according to any one of Items 23 to 30, wherein the sensor data meets the first criterion in response to the sensor data indicating that the autonomous vehicle is in the vicinity of the road user and / or the location.

[0269] Item 32. A system according to any one of Items 23 to 31, wherein the sensor data meets the first criterion in response to the sensor data indicating that the autonomous vehicle is in a lane closest to a road boundary.

[0270] Item 33. A system according to any one of Items 23 to 32, wherein the sensor data complies with the first criterion in response to the sensor data indicating that the road user has left the road surface.

[0271] Item 34. A system according to any one of Items 23 to 33, wherein the sensor data meets the first criterion in response to the sensor data indicating that the road user is in a lane closest to a road boundary.

[0272] Item 35. A system according to any one of Items 23 to 34, wherein the sensor data meets the first criterion in response to the sensor data indicating that the distance between the location of the road user and the location of the crosswalk does not meet a distance threshold.

[0273] Item 36. A system according to any one of Items 23 to 35, wherein the location of the road user includes an area surrounding the location of the road user.

[0274] Item 37. A system according to Item 36, wherein the area surrounding the location of the road user includes a zone for communicating the intention of the autonomous vehicle's maneuver.

[0275] Item 38. The system of any one of Items 23 to 37, wherein the at least one non-transitory computer-readable medium stores instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

[0276] Using the at least one processor, a total violation metric is determined for all locations to which the rule is applied.

[0277] Item 39. The system of Item 38, wherein the at least one non-transitory computer-readable medium stores instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

[0278] Using the at least one processor, data indicative of the total violation metric is provided for use in control of the autonomous vehicle.

[0279] Item 40. The system of any one of Items 38 to 39, wherein the at least one non-transitory computer-readable medium stores instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

[0280] Using the at least one processor, the second trace is selected from the plurality of traces based on the total violation metric.

[0281] Item 41. A system according to any one of Items 23 to 40, wherein the road user can be one or more of the following: a user-operated vehicle and a pedestrian.

[0282] Item 42. A system according to any one of Items 23 to 41, wherein determining whether the sensor data meets the first criterion includes: determining that the sensor data meets the first criterion based on the sensor data indicating that the location of the road user is near a crosswalk.

[0283] Item 43. A system according to any one of Items 23 to 42, wherein the objective expressive operation of the autonomous vehicle in the vicinity of the road user includes yielding at a crosswalk.

[0284] Item 44. A system according to any one of Items 23 to 43, wherein evaluating the first trajectory of the autonomous vehicle based on the violation metric using the at least one processor is performed after the first trajectory is completed.

[0285] Item 45. A non-transitory computer-readable medium comprising instructions stored thereon, the instructions, when executed by at least one processor, causing the at least one processor to perform operations comprising:

[0286] obtaining, using at least one processor, sensor data associated with an environment in which the autonomous vehicle is operating according to a first trajectory of a plurality of trajectories, the environment including road users at a location;

[0287] obtaining, using the at least one processor, a rule indicative of a goal-expressive operation of the autonomous vehicle in the vicinity of the road user, the rule comprising a first criterion for initiating application of the rule, and a second criterion;

[0288] determining, using the at least one processor, whether the sensor data meets the first criterion,

[0289] In response to the sensor data meeting the first criterion, applying the rule;

[0290] Wherein, applying the rules comprises:

[0291] determining, using the at least one processor, a violation metric based on the sensor data and the second criterion;

[0292] evaluating, using the at least one processor, the first trajectory of the autonomous vehicle based on the violation metric; and

[0293] Using the at least one processor, a second trace is selected from the plurality of traces based on the violation metric.

[0294] Item 46. The non-transitory computer-readable medium of Item 45, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform operations comprising:

[0295] The autonomous vehicle is controlled, using the at least one processor, based on the selected second trajectory.

[0296] Clause 47. The non-transitory computer-readable medium of clause 45, wherein determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises:

[0297] Using the at least one processor, a determination is made as to whether the sensor data meets the second criterion.

[0298] Clause 48. The non-transitory computer-readable medium of clause 47, wherein determining, using the at least one processor, whether the sensor data meets the second criterion comprises:

[0299] determining, using the at least one processor, a stopping distance parameter based on the sensor data, the stopping distance parameter indicating a distance between a location of the autonomous vehicle and a location of the road user when the autonomous vehicle reaches a stopping point; and

[0300] determining, using the at least one processor, whether the stopping distance parameter is below a threshold;

[0301] In response to the stopping distance parameter being lower than a threshold, the sensor data does not meet the second criterion.

[0302] Item 49. The non-transitory computer-readable medium of any one of Items 47 to 48, wherein determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises:

[0303] Responsive to the sensor data satisfying the second criterion, the violation metric is determined to indicate zero violation.

[0304] Item 50. The non-transitory computer-readable medium of any one of Items 47 to 49, wherein determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises:

[0305] In response to the sensor data failing to comply with the second criterion, the violation metric is determined to indicate a violation of a distance parameter.

[0306] Item 51. A non-transitory computer-readable medium according to Item 50, wherein the violation distance parameter comprises a distance at which the stopping distance parameter does not satisfy the threshold.

[0307] Item 52. A non-transitory computer-readable medium as described in any one of Items 45 to 51, wherein the rule includes a third criterion for exiting the rule.

[0308] Item 53. A non-transitory computer-readable medium as described in any one of Items 45 to 52, wherein the sensor data meets the first criterion in response to the sensor data indicating that the autonomous vehicle is in the vicinity of the road user.

[0309] Item 54. A non-transitory computer-readable medium as described in any one of Items 45 to 53, wherein the sensor data complies with the first criterion in response to the sensor data indicating that the autonomous vehicle is in a lane closest to a road boundary.

[0310] Item 55. A non-transitory computer-readable medium according to any one of Items 45 to 54, wherein the sensor data meets the first criterion in response to the sensor data indicating that the road user has left the road surface.

[0311] Item 56. A non-transitory computer-readable medium according to any one of Items 45 to 55, wherein the sensor data meets the first criterion in response to the sensor data indicating that the road user is in a lane closest to a road boundary.

[0312] Item 57. A non-transitory computer-readable medium according to any one of Items 45 to 56, wherein the sensor data meets the first criterion in response to the sensor data indicating that the distance between the location of the road user and the location of the crosswalk does not meet a distance threshold.

[0313] Item 58. A non-transitory computer-readable medium as described in any one of Items 45 to 57, wherein the location of the road user includes an area surrounding the location of the road user.

[0314] Item 59. A non-transitory computer-readable medium as described in Item 58, wherein the area surrounding the location of the road user includes a zone for communicating the intention of the autonomous vehicle's maneuver.

[0315] Item 60. The non-transitory computer-readable medium of any one of Items 45 to 59, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform operations comprising:

[0316] Using the at least one processor, a total violation metric is determined for all locations to which the rule is applied.

[0317] Item 61. The non-transitory computer-readable medium of Item 60, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform operations comprising:

[0318] Using the at least one processor, data indicative of the total violation metric is provided for use in control of the autonomous vehicle.

[0319] Item 62. The non-transitory computer-readable medium of any one of Items 60 to 61, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform operations comprising:

[0320] Using the at least one processor, the second trace is selected from the plurality of traces based on the total violation metric.

[0321] Item 63. A non-transitory computer-readable medium as described in any one of Items 45 to 62, wherein the road user can be one or more of the following: a user-operated vehicle and a pedestrian.

[0322] Item 64. A non-transitory computer-readable medium according to any one of Items 45 to 63, wherein determining whether the sensor data meets the first criterion includes: determining that the sensor data meets the first criterion based on an indication by the sensor data that the location of the road user is near a crosswalk.

[0323] Item 65. A non-transitory computer-readable medium as described in any of Items 45 to 64, wherein the objective expressive operation of the autonomous vehicle in the vicinity of the road user includes yielding at a crosswalk.

[0324] Item 66. A non-transitory computer-readable medium as described in any one of Items 45 to 65, wherein evaluating the first trajectory of the autonomous vehicle based on the violation metric using the at least one processor is performed after the first trajectory is completed.

Claims

1. A method comprising: obtaining, using at least one processor, sensor data associated with an environment in which the autonomous vehicle is operating according to a first trajectory of a plurality of trajectories, the environment including road users at a location; obtaining, using the at least one processor, a rule indicative of a goal-expressive operation of the autonomous vehicle in the vicinity of the road user, the rule comprising a first criterion for initiating application of the rule, and a second criterion; determining, using the at least one processor, whether the sensor data meets the first criterion, In response to the sensor data meeting the first criterion, applying the rule; Wherein, applying the rules comprises: determining, using the at least one processor, a violation metric based on the sensor data and the second criterion; evaluating, using the at least one processor, the first trajectory of the autonomous vehicle based on the violation metric; and Using the at least one processor, a second trace is selected from the plurality of traces based on the violation metric.

2. The method according to claim 1, comprising: The autonomous vehicle is controlled, using the at least one processor, based on the selected second trajectory.

3. The method according to claim 1, wherein: Determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises: Using the at least one processor, a determination is made as to whether the sensor data meets the second criterion.

4. The method according to claim 3, wherein: Determining, using the at least one processor, whether the sensor data meets the second criterion comprises: determining, using the at least one processor, a stopping distance parameter based on the sensor data, the stopping distance parameter indicating a distance between a location of the autonomous vehicle and a location of the road user when the autonomous vehicle reaches a stopping point; and determining, using the at least one processor, whether the stopping distance parameter is below a threshold; In response to the stopping distance parameter being lower than a threshold, the sensor data does not meet the second criterion.

5. The method according to any one of claims 3 to 4, wherein: Determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises: Responsive to the sensor data satisfying the second criterion, the violation metric is determined to indicate zero violation.

6. The method according to any one of claims 3 to 5, wherein: Determining, using the at least one processor, the violation metric based on the sensor data and the second criterion comprises: In response to the sensor data failing to comply with the second criterion, the violation metric is determined to indicate a violation of a distance parameter.

7. The method according to claim 6, wherein: The violation distance parameter includes a distance at which the stopping distance parameter does not satisfy the threshold.

8. A method according to any one of the preceding claims, wherein: The rule includes a third criterion for exiting the rule.

9. A method according to any one of the preceding claims, wherein: In response to the sensor data indicating that the autonomous vehicle is in the vicinity of the road user and / or the location, the sensor data meets the first criterion.

10. A method according to any one of the preceding claims, wherein: In response to the sensor data indicating that the autonomous vehicle is in a lane closest to a road boundary, the sensor data meets the first criterion.

11. A method according to any one of the preceding claims, wherein: In response to the sensor data indicating that the road user has left the road surface, the sensor data meets the first criterion.

12. A method according to any one of the preceding claims, wherein: In response to the sensor data indicating that the road user is in a lane closest to a road boundary, the sensor data meets the first criterion.

13. A method according to any one of the preceding claims, wherein: In response to the sensor data indicating that the distance between the location of the road user and the location of the crosswalk does not meet a distance threshold, the sensor data meets the first criterion.

14. A method according to any one of the preceding claims, wherein: The location of the road user includes an area surrounding the location of the road user.

15. The method according to claim 14, wherein: The area surrounding the location of the road user includes a zone for communicating the intent of the autonomous vehicle's maneuver.

16. A method according to any one of the preceding claims, comprising: Using the at least one processor, a total violation metric is determined for all locations to which the rule is applied.

17. The method according to claim 16, comprising: Using the at least one processor, data indicative of the total violation metric is provided for use in control of the autonomous vehicle.

18. The method according to any one of claims 16 to 17, comprising: Using the at least one processor, the second trace is selected from the plurality of traces based on the total violation metric.

19. A system comprising: at least one processor; as well as at least one non-transitory computer-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising: obtaining, using the at least one processor, sensor data associated with an environment in which the autonomous vehicle is operating according to a first trajectory of a plurality of trajectories, the environment including a road user at a location; obtaining, using the at least one processor, a rule indicative of a goal-expressive operation of the autonomous vehicle in the vicinity of the road user, the rule comprising a first criterion for initiating application of the rule, and a second criterion; determining, using the at least one processor, whether the sensor data meets the first criterion, In response to the sensor data meeting the first criterion, applying the rule; Wherein applying the rule comprises: using the at least one processor, determining a violation metric based on the sensor data and the second criterion; evaluating, using the at least one processor, the first trajectory of the autonomous vehicle based on the violation metric; and Using the at least one processor, a second trace is selected from the plurality of traces based on the violation metric.

20. A non-transitory computer readable medium comprising instructions stored thereon, which, when executed by at least one processor, cause the at least one processor to perform operations comprising: obtaining, using the at least one processor, sensor data associated with an environment in which the autonomous vehicle is operating according to a first trajectory of a plurality of trajectories, the environment including a road user at a location; obtaining, using the at least one processor, a rule indicative of a goal-expressive operation of the autonomous vehicle in the vicinity of the road user, the rule comprising a first criterion for initiating application of the rule, and a second criterion; determining, using the at least one processor, whether the sensor data meets the first criterion, In response to the sensor data meeting the first criterion, applying the rule; Wherein applying the rule comprises: using the at least one processor, determining a violation metric based on the sensor data and the second criterion; evaluating, using the at least one processor, the first trajectory of the autonomous vehicle based on the violation metric; and Using the at least one processor, a second trace is selected from the plurality of traces based on the violation metric.

Images