System, method, and storage medium for greeting a vehicle

Abstract

The present invention relates to systems, methods, and storage media for hailing vehicles. Generally, an indication of an intent of a potential rider to use an autonomous vehicle is received through a user interface. In response to receiving the indication, a hail request is sent to at least one autonomous vehicle through a signaling pattern, which the at least one autonomous vehicle can receive directly according to the signaling pattern.

Description

[0001] (This application is a divisional application of the application filed on August 17, 2017, with application number 201780064624.1 and title "Calling for Vehicles".)

[0002] Cross-references to related applications

[0003] This application claims priority to U.S. Application Serial No. 15 / 240,072, filed August 18, 2016, and U.S. Application Serial No. 15 / 240,150, filed August 18, 2016. Technical Field

[0004] This manual relates to the greeting of vehicles such as autonomous vehicles. Background Technology

[0005] like Figure 1 As shown, in a common method where person 10 hails a vehicle such as a taxi or ride-sharing vehicle, the person enters a request by interacting with a mobile application. The request is processed by the mobile application and then forwarded via a cellular phone network or the Internet to a central server 12, where the request is processed and then forwarded via the cellular phone network to the driver of vehicle 14. Summary of the Invention

[0006] Typically, in one aspect, an indication of a potential passenger's intention to use an autonomous vehicle is received via a user interface. In response to receiving the indication, a greeting request is sent via a signaling mode to one or more autonomous vehicles, which can directly receive the greeting request according to the signaling mode.

[0007] The implementation may include one or a combination of two or more of the following features: Presenting the user interface via a mobile device's display; Receiving the instruction via input features of a mobile device; Receiving the instruction via input features of a device at a fixed location; Sending the greeting request from the mobile device; Sending the greeting request from a device at a fixed location; Sending the greeting request via a combination of two or more signaling modes. The signaling modes include a visual mode; The visual mode includes the display of graphical elements; The visual mode includes the emission of light; The signaling mode includes an audible mode; The audible mode includes sound; The audible mode includes speech; The greeting request is configured to include greeting information related to the intended use of the autonomous vehicle; The greeting information includes the destination location; The greeting information includes the category or type of service; The type of service may be for use by potential passengers; The type of service may be for parcel delivery; The greeting information includes the number of passengers; The greeting information includes information about the potential passengers or the parcel to be delivered; The greeting information includes the priority of the service; Receiving confirmation from the autonomous vehicle. In response to receiving the confirmation of receipt, a greeting confirmation is sent to the autonomous vehicle.

[0008] Typically, in one aspect, a greeting request carried in signaling mode is received directly at the receiving equipment of the autonomous vehicle. This greeting request corresponds to an indication of a potential passenger's intention to use the autonomous vehicle. The greeting request is processed at the autonomous vehicle to determine whether to respond to it.

[0009] The implementation may include one or a combination of two or more of the following features: Receiving the greeting request via direct wireless communication; Receiving the greeting request from a mobile device; Receiving the greeting request from a device at a fixed location; Receiving the greeting request from the potential passenger or a representative of the potential passenger; Receiving the greeting request via a combination of two or more signaling modes. The signaling modes include a visual mode; The visual mode includes the display of graphic elements; The visual mode includes images or light; The signaling mode includes an audible mode; The audible mode includes sound; The audible mode includes speech; The signaling mode includes human gestures or facial expressions or both; The greeting request includes greeting information related to the use of the autonomous vehicle; The greeting information includes the destination location; The greeting information includes the category, type, or priority of the service; The greeting information includes the number of passengers; The greeting information includes information about the potential passenger or the package to be delivered; Sending a receipt confirmation from the autonomous vehicle; Receiving a greeting confirmation after the receipt confirmation has been sent.

[0010] Typically, in one aspect, there is a device for hailing an autonomous vehicle. This device includes input features to receive an indication of a potential passenger's intention to use the autonomous vehicle. The device also includes output features to send a hailing request to at least one autonomous vehicle via a signaling pattern, the at least one autonomous vehicle being able to directly receive the hailing request according to the signaling pattern.

[0011] The implementation may include one or a combination of two or more of the following features. The device includes a mobile device. The device is fixed in a location. The input feature includes a touch-sensitive display. The input feature includes a microphone. The input feature includes a button. The greeting request carries greeting information related to the use of the autonomous vehicle, and the device includes a processor to embed at least some of the greeting information into the greeting request. The greeting information includes the destination location. The greeting information includes the category, type, or priority of the service. The greeting information includes the number of passengers. The greeting information includes information about the potential passenger or the package to be delivered. The output feature includes a transmitter. The output feature includes a display. The output feature includes a light transmitter. The output feature includes a sound transmitter. The signaling mode includes a visual mode. The visual mode includes the display of graphic elements. The visual mode includes images or light. The signaling mode includes an audible mode. The audible mode includes sound. The audible mode includes speech. The signaling mode includes human gestures or facial expressions or both. The greeting request is sent via a combination of two or more signaling modes. The device includes a receiver feature to receive an acknowledgment from the autonomous vehicle after the greeting request has been sent.

[0012] Typically, in one aspect, the equipment for use at an autonomous vehicle includes a receiving feature for receiving a greeting request. The greeting request corresponds to an indication from a potential passenger that they wish to use the autonomous vehicle. The greeting request is received directly from equipment near the potential passenger, according to a signaling pattern. The equipment includes a processor for processing the greeting request.

[0013] The implementation may include one or a combination of two or more of the following features. The greeting request carries greeting information related to the use of the autonomous vehicle, and the device includes a processor to embed at least some of the greeting information into the greeting request. The greeting information includes a destination location. The greeting information includes the category or type of service. The greeting information includes the number of passengers. The greeting information includes information about the potential passenger or the package to be delivered. The receiving feature includes a camera. The receiving feature includes a light detector. The receiving feature includes a microphone. The signaling mode includes a visual mode. The visual mode includes the display of graphic elements. The visual mode includes images or light. The signaling mode includes an audible mode. The audible mode includes sound. The audible mode includes speech. The signaling mode includes human posture or facial expression or both. The greeting request is received via a combination of two or more signaling modes. The processor includes a process for determining an action based on the greeting request. The action includes accepting the greeting request. The action includes rejecting the greeting request. The action includes providing a pick-up location for the autonomous driving features of the autonomous vehicle. The device at the autonomous vehicle receives a greeting confirmation.

[0014] Typically, in one aspect, the autonomous vehicle includes a processor and software executed by the processor to cause the autonomous vehicle to travel to the identified location. The autonomous vehicle also includes devices for: (a) receiving a call request corresponding to a potential passenger's intention to use the autonomous vehicle, receiving the call request directly from a device near the potential passenger according to a signaling mode; and (b) providing a pick-up location to the processor based on the call request.

[0015] These and other aspects, features, implementation methods, and advantages can be expressed as methods, apparatus, systems, components, program products, business methods, devices or steps for performing functions, and in other ways.

[0016] These and other aspects, features, implementations and advantages will become apparent from the following description and claims. Attached Figure Description

[0017] Figure 1 , Figure 2 , Figure 9 and Figure 10 It's a block diagram.

[0018] Figure 3 , Figure 4 and Figure 8 This is a diagram illustrating how vehicles can be hailed.

[0019] Figure 5 , Figure 6 and Figure 7 It displays a screenshot.

[0020] Figure 11 This is a schematic diagram of the vehicle.

[0021] Figure 12 and Figure 13 This is a schematic diagram. Detailed Implementation

[0022] In this paper, we describe a system and technology used by people to hail vehicles such as autonomous vehicles. Figure 2 As shown, in some implementations of the systems and techniques described herein, user 16 calls vehicle 20 by communicating directly with the vehicle using one or more wireless signaling modes 18. In some cases, server 12 may also, or alternatively, communicate directly with the vehicle.

[0023] We sometimes refer to this system and technology broadly as a “calling system.” We use the term “calling” or “calling to” to include, for example, any action by any person in any situation (e.g., a pedestrian) to signal, call, flag down, sound an alarm, or otherwise attract the attention of a vehicle in order to indicate that person’s intention to ride (alone or with others) in the vehicle (such as an autonomous vehicle), to hire the vehicle for a service, to have the vehicle deliver or otherwise utilize the vehicle.

[0024] A greeting can be made by a greeting device, through a medium of a greeting device, or with the assistance of a greeting device. We use the phrase “greeting device” broadly to include, for example, any device, equipment, instrument, or other equipment associated with greeting, such as a telephone or other handheld device, a smartwatch or bracelet or other wearable device, or a fixed device mounted on or near the edge of a sidewalk or road, or otherwise positioned on or near the edge of a sidewalk or road, to name just a few. We use the phrase “potential passenger” broadly to include, for example, anyone who, under any circumstances, wishes to ride (alone or with others) in a vehicle (such as an autonomous vehicle), hire a vehicle for service, make a delivery in the vehicle, or otherwise utilize the vehicle. We use the term “hire a vehicle for service” broadly to include, for example, obtaining access to a vehicle in any way and under any circumstances, in order to, for example, ride in the vehicle, make a delivery in the vehicle, or otherwise utilize the vehicle. While we sometimes refer to autonomous vehicles in our discussion, greeting systems can also be used for semi-autonomous vehicles, and in some cases for other types of vehicles driven in other ways. We sometimes use the term “autonomous vehicle” broadly to refer to any such vehicle. The autonomous vehicle can be called upon when it passes by a potential passenger on a road, when it is parked, or in other circumstances.

[0025] Autonomous vehicles can drive safely without human intervention for part or all of the journey. For example... Figure 10 and Figure 11 As shown, autonomous vehicles include sensors, actuators, computers, and communication equipment to enable the automatic generation and following of routes through the environment. Autonomous vehicles typically include one or more video sensors, with at least one typically oriented in a forward direction to capture data related to the scene ahead of the vehicle. Additionally, autonomous vehicles may include one or more lidar sensors. Autonomous vehicles are typically equipped with two-way wireless communication capabilities to access data and information stored on servers in cloud services. Also as... Figure 10 and Figure 11 As shown, the autonomous vehicle can communicate wirelessly with a server located at a central position.

[0026] These and other capabilities enable autonomous vehicles to respond to detected visual or other physical features in the environment, including specific shapes, colors, textures, human postures, and other visual stimuli. Autonomous vehicles may also be equipped with auditory sensors for capturing and responding to sounds in the environment, as well as horns, loudspeakers, bells, or other sound-emitting devices. We will refer to these visual and auditory capabilities and devices, as well as a broad range of other detection capabilities and devices, by the collective term "stimulus detectors."

[0027] Therefore, as Figure 12 As shown, the stimulus detector 202 of the autonomous vehicle 200 may include, for example, video 204, audio 206, image 208, light 210, and RF 212.

[0028] Autonomous vehicles used with the greeting system described herein may also be equipped with lights, displays, light-emitting diodes, or other light-emitting devices 216 mounted on, for example, the exterior of the vehicle so that they can be seen from the outside of the vehicle. Autonomous vehicles may have displays 218, lights, sirens, car horns, or other sound transmitters 220, as well as other signaling devices mounted on, for example, the exterior of the vehicle. We refer to these signaling devices and technologies, as well as a broad range of other signaling capabilities and equipment, as “signaling equipment.” Typically, this signaling equipment operates in a mode that is perceptible and understandable to humans.

[0029] Traditionally, potential passengers can use a ride-hailing software process 230 to hire a vehicle (such as a human-driven taxi or vehicle for ride-hailing services) for service. This ride-hailing software process 230 can run on a processor unit 232 embedded in a smartphone, laptop, tablet, or other computing device 234. Potential passengers 236 typically input 238 their desired pick-up location into the ride-hailing software process, and may also input their desired drop-off (i.e., destination) location (e.g., a destination address or street intersection).

[0030] In known systems, the desired pick-up location is then wirelessly transmitted to a central server, whose task is to assign directions of movement to one or more, and potentially large, fleets of vehicles (e.g., hundreds or thousands). The central server then runs route planning software that wirelessly provides the pick-up locations of potential passengers to specific vehicles. The specific vehicle is then driven to the desired pick-up location to meet the potential passenger. Once the potential passenger enters the vehicle and completes the identification process (not necessarily in that order), the vehicle is considered hired for service and begins its journey to the desired drop-off location.

[0031] In some typical use cases of autonomous vehicles, the desired pick-up or destination location can be specified by an algorithm (which may reside on a central server in the cloud and whose task is to optimize the location of the autonomous vehicle fleet with the goal of minimizing passenger wait time when a passenger calls for a vehicle) or provided by another process (e.g., an emergency process that identifies the nearest hospital as the destination location due to a medical emergency detected in a vehicle).

[0032] In the implementation of the hail-a-ride system described herein, potential passengers of a vehicle (e.g., an autonomous vehicle) may wish to hire a vehicle for service without having to input their desired pick-up location into a smartphone, laptop, tablet, or other computing device, but simply hail a vehicle (e.g., an unoccupied autonomous vehicle passing by on the road). The hail-a-ride system described herein can also be applied to other scenarios, such as hailing an autonomous vehicle parked in a specific location.

[0033] like Figure 3 As shown, in our implementation of the greeting system, greeting the autonomous vehicle 30 can be accomplished by sending information from the potential passenger 32, or a representative of the potential passenger 32, or an associate of the potential passenger 32, to the autonomous vehicle using one signaling mode or a combination of two or more signaling modes, including those discussed below. We broadly use the term "signaling agency" to include, for example, any device, apparatus, or system through which information about the greeting can be sent to the autonomous vehicle. We broadly use the term "signaling mode" to include, for example, any form in which information can be transmitted via wireless signals or other methods, including non-electronic methods. Signaling modes can include: displaying images or image sequences, emitting sequences of light signals, emitting sequences of sound signals, transmitting wireless communication signals, or making gestures or facial expressions, to name just a few.

[0034] In most cases, information signaling mechanisms and patterns are designed to increase the rate of true detection (i.e., the autonomous vehicle correctly identifies a hello request as being directed to another vehicle) and to reduce the rate of false detection (i.e., the autonomous vehicle incorrectly identifies a hello request as being directed to another vehicle when no hello request has been sent or when a hello request has been sent to another vehicle).

[0035] like Figure 3 As shown, a potential passenger 32 can use a greeting device 34, such as a mobile device (or a variety of other devices), to signal to the autonomous vehicle.

[0036] In some cases, the greeting can be made by a potential passenger (or someone else, such as a companion) carrying a greeting device (e.g., a smartphone). In some cases (see...) Figure 4A greeting can be performed using a greeting device installed or mounted in a fixed location (such as on a structure where vehicle greetings typically occur, for example, outside a busy hotel). In some cases, the greeting device can be a typical human ability exercised by a potential passenger (or someone on her behalf), who does not carry any external greeting device but uses other signaling mechanisms to transmit information, such as hands, or face, other gestures, or whistling, shouting, or making another sound. In some implementations, greetings can be performed by interacting with a device attached to the autonomous vehicle itself (e.g., a smart card reader that can read the user's smart card or a smart device that can communicate with the user's smartphone).

[0037] The use of a greeting system can be accomplished in accordance with a predefined and generally accepted greeting protocol, which can cover how the signaling mechanism operates, the information to be transmitted, the format of that information, and various other aspects. Various entities operating the autonomous vehicles and their fleets to be employed for service, as well as entities producing the software, hardware, and other equipment for the autonomous vehicles and greeting devices, can subscribe to such greeting protocols.

[0038] The desired aspect of this call-out protocol is that it allows potential passengers to call a vehicle more quickly (because more than one fleet operator's vehicle is eligible for the protocol), compared to calling an autonomous vehicle operated by a designated fleet operator using a proprietary call-out method. Interoperability between fleet operators may also require a common central cloud or system capable of broadcasting the call-out request to all fleet operators.

[0039] Another desired aspect of the ride-hailing system described in this paper is that potential passengers can hail a ride much faster than in implementations where the passenger needs to input their desired pick-up location (which is not required in the current implementation), wait for the system to assign a vehicle, and then wait for the vehicle to arrive at the pick-up location. Another desired aspect of some implementations of the ride-hailing system is that potential passengers can hail a ride anonymously, for example, without using ride-hailing software processes that might be linked to user accounts containing personally identifiable information. In other words, potential passengers do not need to reveal their destination or expose any of their personal information while using the ride-hailing system.

[0040] On the other hand, depending on the signaling authority, certain types of information associated with a call request can be transmitted to the autonomous vehicle via the call system. This information may include, but is not limited to: information about the number of passengers, the type or cost of the desired service (which can be used to exclude vehicles of a specific size or category from responding to a call request), information about priority or emergency call requests, or personally identifiable information about passengers, or any combination of two or more of these, and other information. The transmitted information can be encrypted using any of a number of possible encryption techniques.

[0041] Greeting devices may include one or more of the following (and various others):

[0042] 1. For example Figure 12 As shown in the example, a smartphone, tablet, handheld PC, smartwatch, bracelet, or other wearable device or other configurable mobile device is equipped with at least a processor 232, a memory unit 244, an input device or process 246 associated with a user interface 248, and (in some cases) a display screen, multiple light-emitting diodes, luminescent materials, electronic ink screens, or other light-emitting or light-modulating media 250. Such display and luminescent media may be capable of displaying or modulating sequences of images, color sequences, or light sequences, or any combination of two or more of them, as well as other visual cues. The greeting device may be equipped with one or more video sensors 262 capable of detecting visual features in the environment. The device may be equipped with one or more microphones or speakers (or both) capable of detecting and emitting sound 264. The device may be equipped with a wired or wireless receiver and transmitter that enables the device to communicate, in particular, with a central cloud or server 242.

[0043] 2. Printed paper or cards.

[0044] 3. A whistle or other sound-producing device, including a programmable sound-producing device equipped with at least a processor and a speaker, and possibly a microphone.

[0045] Modes for hailing autonomous vehicles

[0046] Exemplary methods for hailing autonomous vehicles include, but are not limited to, the following methods, which may be employed independently or in combination.

[0047] Image display

[0048] like Figure 5 As shown, waving at an autonomous vehicle can be achieved by presenting the waving device 48 in a manner (e.g., in an orientation such that one or more images are displayed facing the road at shoulder height or higher) on the waving device 48. Figure 12A specific image or image sequence 44, 46 (e.g., alternating at a fixed repetition rate) is displayed on the video sensor 204 mounted on a passing autonomous vehicle. This specific image or image sequence 44, 46 may be within the line of sight of the video sensor 204 mounted on the vehicle. For example, the beckoning device may be located at a sidewalk or road edge, or at an intersection where a potential passenger is standing.

[0049] One or more images to be displayed can exhibit one of the following properties or any two or more of the following properties:

[0050] 1. A pattern of geometric features 50 (e.g., image features composed of geometric elements) or features 52, such as features not less than those features that can be reliably resolved by an autonomous vehicle at a nominal greeting distance (e.g., 1-30 meters) using, for example, a detection system based on a vision sensor (multiple) 208 mounted on the vehicle and having a typical field of view (e.g., 90 degrees horizontally x 60 degrees vertically) and resolution (e.g., 640 pixels horizontally x 480 pixels vertically). Figure 13 As shown, the software process 300 running on the greeting device may include a geometric element process 302 for forming and presenting geometric features.

[0051] 2. In black and white (e.g.) Figure 5 Grayscale or color image formation. Displaying a grayscale or color image instead of a black and white image may be advantageous because grayscale and color images may contain more information than black and white images, and therefore can enable the encoding of a larger amount of information related to the greeting request, or can reduce the occurrence of false detections, or both. Detection of the color information contained in the image implicitly requires the autonomous vehicle to be equipped with a color video sensor. The software process 300 running on the greeting device may include black and white, grayscale, and color processes 304 to produce an appropriate image.

[0052] 3. Texture (e.g., the arrangement of colors or intensities in a region or throughout an image) can be reliably resolved by the autonomous vehicle at a nominal greeting distance using a detection system based on multiple visual sensors mounted on the vehicle and having typical field of view and resolution. Software process 300 running on the greeting device may include texture generation process 306.

[0053] 4. Textures can include a format (such as, e.g., ...) Figure 6 The QR code 60 shown is an example of information encoded in binary format. This allows more information to be transmitted to the vehicle. Additional information can be encoded in the greeting request by increasing the resolution of this format. The software process 300 running on the greeting device may include an encoding process 308 for this purpose.

[0054] 5. For example Figure 7 As shown, the sequence of colors 62, 64, and 66 can be displayed by a calling device at a fixed repetition rate. Figure 7 The display shows 68 ( Figure 12 The emitted light is colored light (250 in the signaling device). Other light sources can be used, such as LEDs, flashlights, and strobe lights, to name a few. The emitted light can be in the visible or invisible spectrum. The light can be a single wavelength or contain multiple wavelengths. The pattern can be changed temporarily, as can the intensity of the light. As discussed later, sound can be used as a signaling mode, and the frequency of the sound can be controlled similarly. The temporal and spectral control of the light can be handled by the light modulation process 310 running on the signaling device 300.

[0055] 6. The appearance of the image should be uncommon to reduce the possibility of false detections caused by sensors on the autonomous vehicle detecting similar or identical images not generated by the greeting device. For example, the image should not resemble those commonly found on traffic signs or other common pedestrian features such as newsboxes, bus shelters, etc.

[0056] Information related to a greeting request (e.g., the type or cost of the desired service, or the occurrence of a priority or urgent greeting request) can be encoded in the image using encoding and optical modulation processes 308 and 310 using various methods, including but not limited to the following methods or combinations thereof:

[0057] 1. Modify image features, for example, display unique images or image sequences based on information related to the greeting request. For instance, a specific image may be displayed for a greeting request from a potential passenger as a single passenger to an autonomous vehicle offering standard-category services, while another specific image may be displayed for a greeting request from a potential passenger representing two passengers to an autonomous vehicle offering premium-category services.

[0058] 2. Modify the displayed image series, including changing the temporal attributes of the displayed image sequence. For example, a specific image sequence may be displayed for a potential passenger requesting a ride to an autonomous vehicle offering standard-category service, while another specific image sequence may be displayed for a potential passenger requesting a ride to an autonomous vehicle offering premium-category service. Temporal attributes such as the display duration of each image may be modulated to encode additional information or reduce the occurrence of false detections.

[0059] 3. Change the black-and-white, grayscale, or color attributes of the displayed image or image sequence, including changing the temporal attributes of the image's color. For example, a specific red image could be displayed for a potential passenger's request to hail an autonomous vehicle offering standard category services, while the same specific image could be displayed in green for a group of two potential passengers' requests to hail an autonomous vehicle offering premium category services.

[0060] 4. Change the texture properties of the displayed image or image sequence. For example, a specific black and white checkerboard image can be displayed for a potential passenger calling an autonomous vehicle that provides standard category services, while a specific green and yellow checkerboard image can be displayed for a potential passenger calling an autonomous vehicle that provides premium category services, representing a group of two passengers.

[0061] 5. The image to be displayed can be a company logo or other symbol conveying an advertising or promotional message. The advertisement can be targeted at an autonomous vehicle company (e.g., a company operating autonomous vehicles that can respond to a call) or another business entity.

[0062] Emit light

[0063] A gesture to an autonomous vehicle can be made by emitting light from a gesture device in a manner that (e.g., in an orientational presentation device such that the emitted light is directed toward the road at shoulder height or higher) that the light may be within the line of sight of a video sensor mounted on the autonomous vehicle (e.g., from a sidewalk or road edge, or through a potential passenger standing at an intersection).

[0064] Based on the encoding and light modulation processes 308 and 310 performed on the greeting device 300, the emitted light to be displayed can exhibit one of the following properties or a combination of two or more of the following properties:

[0065] 1. The wavelength of the light may be in the visible spectrum, or, assuming the greeting device has sufficient emission capability, the wavelength may be in the near-infrared spectrum or other spectra, provided that the wavelength can be detected by a video sensor mounted on the autonomous vehicle.

[0066] 2. The wavelengths of light in the visible spectrum can lie in bands associated with a particular color or many colors.

[0067] 3. The wavelength of light can be uniform or variable.

[0068] 4. The wavelength of light can be controlled or uncontrollable.

[0069] 5. The light intensity can be uniform or variable. This light intensity should be reliably detectable by an autonomous vehicle at a nominal greeting distance using a detection system based on multiple vision sensors or multiple light detection sensors with typical detection sensitivity.

[0070] 6. The intensity of light can be controllable or uncontrollable.

[0071] 7. The characteristics of the emitted light (e.g., color, wavelength, intensity-time frequency (i.e., flash rate), or combinations thereof) should be unusual in order to minimize the possibility that sensors on the autonomous vehicle might mistakenly detect emitted light with similar or identical characteristics as a greeting request, resulting in false detection. For example, the characteristics of the emitted light should not be similar to those emitted by a typical vehicle turn indicator.

[0072] Information related to a call-out request (e.g., the type or cost of the desired service, or the occurrence of a priority or urgent call-out request) can be encoded in the emitted light using various methods through the encoding process 308. These various methods include, but are not limited to, the following methods or combinations thereof:

[0073] 1. Uniformly change the wavelength of light on the display of the calling device. For example, the display device can show a color changing from red to green according to a specified time pattern.

[0074] 2. The wavelength of light can be varied according to a specific spatial or temporal pattern on the display (e.g., different spatial areas or the entire display can display different wavelengths of light uniformly or in a time-varying manner). For example, a specific chessboard image can be displayed, in which the individual cells of the chessboard change color at different frequencies.

[0075] 3. To uniformly change the light intensity across the entire display, or to change the light intensity according to a specific spatial or temporal pattern on the display.

[0076] Making a gesture

[0077] like Figure 8 As shown, greeting a passing autonomous vehicle can be achieved by performing a (e.g., uncommon) gesture or gesture sequence 70 at a location and in an orientation, which may be within the visual line of sight of a video or lidar sensor mounted on the autonomous vehicle (e.g., from a sidewalk or road edge, or when a potential passenger is standing at an intersection). Software process 282 runs on a processor 280 mounted on the autonomous vehicle. Figure 12 The data captured from the video or lidar sensors will then be analyzed to detect the appearance of gestures (multiple) intended to indicate a greeting request.

[0078] The poses (multiple) to be displayed can exhibit the following attributes:

[0079] 1. One or more gestures may include: a static or dynamic position or movement of one or more of the arms or legs of a potential passenger or other person, using a position and movement substantially different from normal limb positioning and movement, which can be correctly resolved by the autonomous vehicle at a nominal calling distance using a detection system based on multiple on-vehicle vision sensors and having a typical field of view and resolution. One or more gestures may also include facial expressions, which can be correctly resolved by the autonomous vehicle at a nominal calling distance using a detection system based on multiple on-vehicle vision sensors and having a typical field of view and resolution. An example of a facial gesture is a sustained gaze at the vehicle for a minimum duration. This gesture may be combined with the use of the calling device 72.

[0080] 2. The appearance of one or more gestures should be uncommon in order to minimize the possibility that sensors on the autonomous vehicle might detect similar or identical gestures not intended to signal the autonomous vehicle, leading to false detections. For example, the appearance of one or more gestures should not resemble those used by cyclists to indicate turns or by police officers or flag bearers to regulate traffic flow.

[0081] 3. When combined with a secondary visually distinct stimulus to reduce false detections, a common gesture (such as waving a hand or arm) can be used to greet the autonomous vehicle. One such secondary visual stimulus could be a potential passenger looking at or tracking the autonomous vehicle through her head posture while performing the common gesture. A software process 282 running on a processor 280 mounted on the autonomous vehicle then analyzes the captured data from video or lidar sensors, or both, to detect the presence of both the gesture and the secondary visual stimulus, the combination of which is intended to indicate a greeting request.

[0082] Information related to a greeting request (e.g., the type or cost of the desired service, or the occurrence of a priority or urgent greeting request) can be encoded in a gesture(s) using various methods, including but not limited to the following methods or combinations thereof:

[0083] 1. Change a specific gesture, for example, display a unique gesture or sequence of gestures based on information related to the greeting request. For example, a specific gesture may be displayed for a single passenger's greeting request to an autonomous vehicle that provides standard category services, while another specific gesture may be displayed for a greeting request from potential passengers representing a group of two passengers to an autonomous vehicle that provides premium category services.

[0084] 2. Modify the displayed gesture sequence, including changing the temporal attributes of the displayed gesture sequence. For example, a specific gesture sequence can be displayed for a single passenger's request to hail an autonomous vehicle offering standard-category service, while another specific gesture sequence can be displayed for a group of two potential passengers' requests to hail an autonomous vehicle offering premium-category service. Temporal attributes such as the display duration of each gesture can be modulated to encode additional information or reduce the occurrence of false detections.

[0085] Make a sound

[0086] Greeting a passing autonomous vehicle can be achieved by emitting an unusually loud sound or sound sequence from a greeting device (including a smartphone or similar device, a horn or similar device, or a potential passenger using their mouth), such a sound or sound sequence being detectable by sensors installed on the passing autonomous vehicle without causing false detection. The emission of the sound, as well as the characteristics and properties of the emitted sound, can be controlled and modulated by one of the encoding or sound modulation processes 308, 312 running on the greeting device.

[0087] The one or more sounds to be emitted can exhibit the following properties:

[0088] 1. The sound frequency can be within the typical human audible frequency range, or, assuming the greeting device has sufficient transmission capability, it can be outside the typical human audible frequency range. Transmission of a specific sound frequency depends on auditory sensors installed on passing autonomous vehicles capable of detecting such frequencies.

[0089] 2. Sound frequencies in the typical human audible frequency range can be located in the frequency bands associated with a specific note or many notes.

[0090] 3. The sound frequency can be uniform or variable.

[0091] 4. The sound frequency can be controllable or uncontrollable.

[0092] 5. The sound volume can be uniform or variable. The volume level should be selected so that it can be reliably detected by one or more sound measurement sensors mounted on the autonomous vehicle with typical detection sensitivity and under given typical expected ambient noise levels.

[0093] 6. The sound volume can be controlled or uncontrollable.

[0094] 7. One or more sounds may be uncommon in order to reduce the likelihood that similar or identical ambient sounds will be detected by sensors mounted on the autonomous vehicle and interpreted as a greeting request, leading to false detections. For example, one or more sounds should not be inherently similar to common sounds, such as car horns, car alarms, or emergency vehicle sirens.

[0095] Information related to the greeting request (e.g., the type or cost of the service requested, or the occurrence of a priority or urgent greeting request) can be encoded in the emitted sound using encoding and sound modulation processes 308, 312 using various methods, including but not limited to the following methods or combinations thereof:

[0096] 1. Variable the frequency of a sound uniformly or according to a specific time pattern. The pattern of the sound frequency to be emitted can be a short song, tune, or advertising slogan conveying an advertising or promotional message. The advertisement can be targeted at an autonomous vehicle company (e.g., a company that manages autonomous vehicles that can respond to a call) or another business entity.

[0097] 2. Change the volume of the sound evenly or according to a specific time pattern.

[0098] direct wireless communication

[0099] A calling device can be used to directly communicate with the autonomous vehicle wirelessly, eliminating the need for the calling request to travel from a potential passenger through a central server to the autonomous vehicle. The calling device can be a mobile device or can be fixed in a specific location. Both the calling device and the communication device on the autonomous vehicle will need to support appropriate communication technologies; for example, both need to include a transmitter, a receiver, and communication software. Many existing communication technologies, such as radio frequency (RF), infrared (IR), WiFi, WiMax, and Bluetooth, can be used or are suitable for this purpose. The calling device can broadcast the request to nearby vehicles, and once a specific autonomous vehicle responds, the calling device can then uniquely communicate with that vehicle. In the case of the calling device, communication process 314 can handle the communication and adhere to appropriate protocols.

[0100] In some examples, direct wireless communication can be achieved by using a vehicle-to-infrastructure (V2I) compatible protocol. In such an example, the greeting device can send a greeting request (using a V2I compatible protocol) to the infrastructure. The infrastructure then relays the greeting request to nearby vehicles (e.g., within a 500-meter range). The autonomous vehicle retrieves the relayed greeting request using its V2I-compatible receiver. This greeting method is direct because it does not involve a central server. The infrastructure can be a variety of devices, such as a WiFi access point installed, for example, on a streetlight.

[0101] Combination of greeting patterns

[0102] Each of the various signaling modes we have discussed can be used in combination with one or more other modes, and can also be combined with additional modes not explicitly mentioned. Using combined modes can reduce the incidence of false detections. For example, a greeting method can rely on performing a specific gesture (e.g., waving one's arm above a person's head) while displaying a greeting device emitting a specific time-sequence of colors. A wide range of other combinations of signaling modes are also possible.

[0103] Fixed greeting equipment

[0104] As mentioned above, autonomous vehicles can also be called by calling devices such as calling devices that form part of a kiosk fixed in a location. This is particularly relevant to, but not limited to, areas where autonomous vehicles are typically called, or where there is a high probability of autonomous vehicles being present nearby, or areas with high pedestrian traffic, or areas at autonomous vehicle stations. Calling devices in fixed locations can provide various types of user interfaces, from simple buttons to more complex combinations including screens, speakers, microphones, touchscreens, or similar input / output components. The user interface allows potential passengers (or their representatives) to indicate the destination location, the number of passengers, the type of service, the arrival time, and various other information related to the call. Calling devices in fixed locations can broadcast a calling request to one or more nearby vehicles by attempting to communicate directly with one or more nearby vehicles or through a central system, using appropriate wireless communication equipment including a receiver and transmitter for this destination.

[0105] Process the received greeting requests

[0106] Once a call request is received on an autonomous vehicle (e.g., in...), Figure 12One of the communication devices 240 shown detects a greeting request transmitted using one of the signaling modes and devices described above or other similar signaling modes or devices. The processor on the autonomous vehicle processes the information contained in the greeting request and, in particular, decides whether to accept the request. This decision can be made by a process running on the autonomous vehicle, by a central system to which the vehicle is connected, or a combination of both. The decision can be based on a combination of factors. These factors include, but are not limited to, the vehicle's current location, the vehicle's current speed, the current traffic conditions in the surrounding area, and the vehicle's ability to determine whether to safely stop and pick up passengers. Nearby autonomous vehicles can also communicate with each other using vehicle-to-vehicle (V2V) communication to ensure that multiple vehicles do not respond to the same greeting request. A centralized system can also be used to achieve vehicle-to-vehicle communication for this purpose.

[0107] Communication between autonomous vehicles and greeting devices

[0108] Although we have described various methods of waving an autonomous vehicle by sending a waving request from a potential passenger (or a representative of a potential passenger) or from a waving device (thus creating one-way communication between the potential passenger and the autonomous vehicle), in some cases, for the purpose of confirming the waving request, it may be desirable to achieve two-way communication between the potential passenger (or associated waving device) and the autonomous vehicle.

[0109] To confirm a greeting request, upon detecting an image-based, light-based, auditory, gestural, or other greeting request from a potential passenger, the autonomous vehicle may transmit an image-based, light-based, or auditory confirmation signal. For example, the intent is that this confirmation will be uniquely identified as a response signal by one or more sensors mounted on the greeting device. Upon receiving this confirmation, the greeting device may transmit a different greeting confirmation signal to the autonomous vehicle than the original greeting request. Upon receiving this confirmation, the autonomous vehicle will initiate a pick-up sequence, including stopping near the potential passenger for the purpose of picking them up.

[0110] Exemplary signaling patterns used to send confirmation of receipt include those used to send a greeting request. More specifically, such methods include, but are not limited to:

[0111] 1. Responding to a greeting request by displaying a black-and-white image, grayscale image, or color image or sequence of images on a display screen mounted on the autonomous vehicle (e.g., on the exterior of the autonomous vehicle). Image attributes (e.g., geometric features, texture, and appearance) should be selected such that they can be reliably resolved by one or more visual sensors mounted on the greeting device and having typical field of view and resolution.

[0112] 2. Responding to a greeting request by emitting light or sound from one or more displays, light emitting devices, light-emitting diodes, sirens, car horns, or other signaling devices mounted on the autonomous vehicle (e.g., on the exterior of the autonomous vehicle). The light intensity should be selected such that it can be reliably detected by one or more visual, light, or auditory sensors mounted on the greeting device and having typical detection sensitivity, or reliably perceived by a potential passenger.

[0113] 3. A greeting request is responded to by emitting a sound or a sequence of sounds from one or more speakers mounted on the vehicle (e.g., on the exterior of the vehicle). The volume level should be selected such that, given a typical expected ambient noise level, it can be reliably detected by sound measurements mounted on the greeting device and one or more sensors with typical detection sensitivity, or reliably perceived by a potential passenger.

[0114] One or more images displayed, one or more lights displayed, and one or more sounds emitted by an autonomous vehicle can exhibit exemplary properties, including, for example, those described above for one or more images displayed, one or more lights displayed, and one or more sounds emitted by a hail device. Information can also be encoded in one or more images displayed, one or more lights displayed, and one or more sounds emitted by an autonomous vehicle using methods such as those described above for one or more images displayed, one or more lights displayed, and one or more sounds emitted by a hail device.

[0115] Once the paging device has received an acknowledgment, it can transmit a paging confirmation to the autonomous vehicle. The signaling patterns used to send the paging confirmation can include those that can be used to transmit the paging request. However, it may not be necessary to encode the paging confirmation signal information associated with the paging request (e.g., the type or cost of the desired service, or the occurrence of a priority or emergency request). As a result, the paging confirmation signal can be simpler than the paging request.

[0116] Oral communication can also be conducted via voice output (speech) from one or more speakers mounted on the exterior of the vehicle and verbal responses (speech) received from potential passengers via one or more microphones mounted on the autonomous vehicle. To do this, a processor on the autonomous vehicle can execute a speech synthesis program or dialogue system included in the vehicle, play back recorded speech, or broadcast speech received from a human remote operator connected to the vehicle via a wireless link. The volume can be selected to suit the distance between the potential passenger and the autonomous vehicle. Once the autonomous vehicle receives a verbal response, the on-vehicle voice recognizer or decoding program can determine whether to interpret the received audio signal as a greeting confirmation of a greeting request.

[0117] like Figure 9 As shown in some examples of the greeting system, the process of greeting an autonomous vehicle begins, for example, with a potential passenger sending a greeting request using a signaling pattern provided via a greeting device or gesture. 90 The autonomous vehicle detects the greeting request and decodes the information embedded within it. 92 A process running on the autonomous vehicle decides whether to accept or reject the request. 94 In doing so, it may communicate with a central server or other vehicles. The autonomous vehicle then sends a response to the greeting device for that user using the signaling pattern. 96 The greeting device may forward the response to the user by notifying the user using a sound or visual indicator on a mobile device or kiosk. 98 If the autonomous vehicle accepts the request, it changes its route accordingly and safely stops near the passenger. 100 In some implementations, this is not required. Figure 9 All the steps shown.

[0118] Other implementations are within the scope of the appended claims.

Claims

1. A system for hailing a vehicle, comprising: One or more computers and one or more storage devices storing instructions thereon that, when executed by the one or more computers, are capable of causing the one or more computers to perform operations comprising: receiving, at one or more vision sensors of a vehicle, data associated with a first body gesture sequence (i) performed by a first person at a first location and (ii) within a perception range of the one or more vision sensors; decoding one or more body gestures from the first body gesture sequence, wherein each body gesture includes a respective gesture attribute; identifying, based on the respective gesture attributes of the one or more body gestures, a request for transportation services at the first location, wherein the request for transportation services includes information comprising at least one of a category or cost of a desired hail service and a priority or occurrence of an urgent hail request; selecting, based on the identified request for transportation services, a parking location at or near the first location; and causing the vehicle to park at the parking location.

2. The system of claim 1, wherein, The one or more vision sensors include one or more video sensors configured to capture video of a scene proximate to the vehicle.

3. The system of claim 1, wherein, The one or more vision sensors include one or more LiDAR sensors.

4. The system of claim 1, wherein, The first body gesture sequence includes one or more facial expressions.

5. The system of claim 4, wherein, A first facial expression of the one or more facial expressions includes the first person continuously gazing at the vehicle for a minimum duration of time.

6. The system of claim 1, wherein, The gesture attribute of a body gesture includes a display duration attribute that measures how long the first person performs the body gesture, and wherein identifying, based on the respective gesture attributes of the one or more body gestures, a request for transportation services at the first location includes: identifying a request for transportation services based on the display duration attribute of the one or more body gestures exceeding a minimum duration of time.

7. The system of claim 1, wherein, Identifying, based on the respective gesture attributes of the one or more body gestures, a request for transportation services at the first location includes: identifying a first gesture and a second gesture of the one or more body gestures, wherein the first gesture and the second gesture, performed consecutively by the first person, are associated with a request for transportation services.

8. The system of claim 1, wherein, The transportation services include a plurality of service categories, wherein each service category is associated with (i) a particular level of service provided by the vehicle to the first person and (ii) a particular body gesture sequence, and wherein identifying, based on the respective gesture attributes of the one or more body gestures, a request for transportation services at the first location includes: determining that the first body gesture sequence matches a particular body gesture sequence associated with a particular service category of the plurality of service categories; and in response to determining that the first body gesture sequence matches the particular body gesture sequence associated with the particular service category of the plurality of service categories: identifying the request for transportation services as a request for a particular service category, and providing, by the vehicle to the first person, a particular level of service associated with the particular service category.

9. One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising: receiving, at one or more visual sensors of a vehicle, data associated with (i) a first person at a first location and (ii) a first sequence of body poses performed by the first person within a perception range of the one or more visual sensors; decoding, from the first sequence of body poses, one or more body poses, wherein each body pose includes a respective pose attribute; identifying, based on the respective pose attributes of the one or more body poses, a request for transportation services at the first location, wherein the request for transportation services includes information comprising a category or cost of a desired hail service and at least one of a priority or an occurrence of an urgent hail request; selecting, based on the identified request for transportation services, a parking location at or near the first location; and causing the vehicle to park at the parking location.

10. The computer-readable storage medium of claim 9, wherein, the one or more visual sensors include one or more video sensors configured to capture video of a scene proximate to the vehicle.

11. The computer-readable storage medium of claim 9, wherein, the one or more visual sensors include one or more LiDAR sensors.

12. The computer-readable storage medium of claim 9, wherein, the first sequence of body poses includes one or more facial expressions.

13. The computer-readable storage medium of claim 12, wherein, a first facial expression of the one or more facial expressions includes the first person continuously gazing at the vehicle for a minimum duration of time.

14. The computer-readable storage medium of claim 9, wherein, the pose attribute of a body pose includes a display duration attribute that measures how long the first person performs the body pose, and wherein identifying, based on the respective pose attributes of the one or more body poses, a request for transportation services at the first location comprises: identifying a request for transportation services based on the display duration attribute of the one or more body poses exceeding a minimum duration of time.

15. The computer-readable storage medium of claim 9, wherein, the transportation services include a plurality of service categories, wherein each service category is associated with (i) a particular level of service provided by the vehicle to the first person and (ii) a particular sequence of body poses, and wherein identifying, based on the respective pose attributes of the one or more body poses, a request for transportation services at the first location comprises: determining that the first sequence of body poses matches a particular sequence of body poses associated with a particular service category of the plurality of service categories; and in response to determining that the first sequence of body poses matches a particular sequence of body poses associated with a particular service category of the plurality of service categories: identifying the request for transportation services as a request for the particular service category, and providing, by the vehicle to the first person, the particular level of service associated with the particular service category.

16. A method for hailing a vehicle, comprising: receiving, at one or more vision sensors of a vehicle, data associated with a first body gesture sequence (i) by a first person at a first location and (ii) within a perception range of the one or more vision sensors; decoding one or more body gestures from the first body gesture sequence, wherein each body gesture includes a respective gesture attribute; identifying, based on the respective gesture attributes of the one or more body gestures, a request for transportation service at the first location, wherein the request for transportation service includes information comprising at least one of a category or cost of a desired hail service and a priority or occurrence of an urgent hail request; selecting, based on the identified request for transportation service, a parking location at or near the first location; and causing the vehicle to park at the parking location.

17. The method of claim 16, wherein, the first body gesture sequence includes one or more facial expressions.

18. The method of claim 16, wherein, the gesture attribute of a body gesture includes a display duration attribute that measures how long the first person performs the body gesture, and wherein identifying, based on the respective gesture attributes of the one or more body gestures, a request for transportation service at the first location includes: identifying a request for transportation service based on the display duration attribute of the one or more body gestures exceeding a minimum duration.

19. The method of claim 16, wherein, identifying, based on the respective gesture attributes of the one or more body gestures, a request for transportation service at the first location includes: identifying a first gesture and a second gesture of the one or more body gestures, wherein the first gesture and the second gesture, which the first person performs consecutively, are associated with a request for transportation service.

20. The method of claim 16, wherein, the transportation service includes a plurality of service categories, wherein each service category is associated with (i) a particular level of service provided by the vehicle to the first person and (ii) a particular body gesture sequence, and wherein identifying, based on the respective gesture attributes of the one or more body gestures, a request for transportation service at the first location includes: determining that the first body gesture sequence matches a particular body gesture sequence associated with a particular service category of the plurality of service categories; and in response to determining that the first body gesture sequence matches the particular body gesture sequence associated with the particular service category of the plurality of service categories: identifying the request for transportation service as a request for the particular service category, and providing, by the vehicle to the first person, the particular level of service associated with the particular service category.

21. A computer program product comprising a program that causes a computer to perform the method of any one of claims 16 to 20.

Images