Autonomous system termination support technology

Abstract

To provide systems and methods for providing assistance at the end of an autonomous system journey.SOLUTION: A system may comprise an autonomous system configured to transport a user, a first sensor coupled to the autonomous system and configured to provide a sensor stream of an environment between the autonomous system and a terminus, and a terminus assistance processor mechanically coupled to the autonomous system. In an example, the terminus assistance processor can receive and process the sensor stream, detect and track the user in the environment in response to the sensor stream, and provide status information to a first mobile device based on the sensor stream.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD Embodiments described herein relate generally to autonomous systems and, more particularly, to vehicle termination assistance techniques. [Background technology]

[0002] The proliferation of mobile communication devices has enabled users of such devices to retrieve and use information almost anytime, no matter where the user is located. Access to geographic and infrastructure information has made physical maps largely obsolete, as users can use mobile applications that can retrieve such geographic and infrastructure information and plan routes for traveling between two or more locations. Autonomous vehicles can also use various machine learning capabilities to guide the vehicle along a designated route for travel. [Brief explanation of the drawings]

[0003] In the drawings, which are not necessarily drawn to scale, like reference numbers may describe like components in different views. Like reference numbers with different subscripts may represent different instances of like components. Several embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings.

[0004] [Figure 1] 1 illustrates an environment applicable to a vehicle control system using an exemplary termination assistance tool.

[0005] [Figure 2] 1 illustrates generally an overview of the operation of a control system including an exemplary terminal assist capability within an environment.

[0006] [Figure 3]1 illustrates generally an exemplary screen display of an application with a landing assistance tool or a landing assistance tool screen display that may be launched on a navigation application of a first user or observer user's electronic device.

[0007] [Figure 4] 4 generally illustrates an alternative exemplary map display of a first region that may be used in conjunction with or in place of the display of FIG. 3;

[0008] [Figure 5] 1 generally illustrates a flowchart of an exemplary method of operation of an exemplary termination assist tool in accordance with the present subject matter.

[0009] [Figure 6] 1 is a block diagram illustrating an example machine that may implement any one or more of the techniques (e.g., methodologies) described herein, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of some example embodiments. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details.

[0011] For example, in the form of a vehicle navigation system, the vehicle's control system may also provide services to the user before the user uses the vehicle to travel from a first terminal (e.g., an endpoint on the route the user is traveling) and after the user completes the vehicle portion of the journey to a second terminal. Termination assistance tools provided by the control system may help provide the user with safe passage between each terminal and the vehicle. In addition, the termination assistance tools may monitor and record the user's passage between each terminal and the vehicle for post-event verification purposes. In certain examples, a second user may be designated as a watcher, and in addition to monitoring the journey in real time, the watcher may also monitor the user's passage between the vehicle and the corresponding terminal of the journey.

[0012] In certain examples, the terminal assistance technology may be employed at least in part within a vehicle's existing navigation system or in cooperation with a navigation application on a user's mobile device. The technology may be applicable to a wide variety of vehicles and is described herein with respect to autonomous vehicles. Other types of vehicles that may use the exemplary terminal assistance technology include, but are not limited to, skateboards, scooters, bicycles, motorcycles, boats, air vehicles such as airplanes, helicopters, or similar propeller- or jet-powered aircraft, and ground vehicles of all types, such as cars and trucks, and autonomous versions of such vehicles.

[0013] An autonomous vehicle (AV) is a vehicle that can operate without human assistance. AVs can operate in fully autonomous mode or partially autonomous mode. When in partially autonomous mode, the AV can provide some autonomous functions such as lane departure monitoring, speed control, collision avoidance, or the like while a human driver performs other aspects of driving, such as steering. An exemplary AV is provided in FIG. 1 and described further below.

[0014] An AV may include a variety of forward-, side-, and rear-facing sensors. The sensors may include radar, LiDAR (light imaging detection and ranging), cameras, ultrasonic, infrared, or other sensor systems. Forward-facing sensors may be used for adaptive cruise control, parking assistance, lane departure, collision avoidance, pedestrian detection, and the like. Rear-facing sensors may be used to warn the driver of potential obstacles (e.g., vehicles) when executing lane changes or reversing at low speeds (e.g., park distance monitors).

[0015] FIG. 1 illustrates an environment 100 applicable to a navigation system 104 using an exemplary end assistance tool 105. The environment 100 includes a user 101, a vehicle 102 that the user 101 has or will use for transportation, an end 103 of the user's journey, and a navigation system 104 using the end assistance tool 105. The vehicle 102 may include, for example, as part of the navigation system 104 or as part of the end assistance tool 105, one or more processors, an end assistance processor, and multiple sensors or transducers 106, 107 that assist in providing end assistance to the user 101. Such sensors and transducers may include a camera 106 and an illumination source 107. The camera 106 may capture and record a scene including a path 108 between the vehicle 102 and the end 103. As the user 101 traverses between the end 103 and the vehicle 102, the end assistance tool 105 may track the user 101 and monitor the scene for unknown movements or objects that may or may not pose a potential threat to the user's physical health. If the path 108 is dark, the illumination source 107 may be used to illuminate the path 108 .

[0016] In a particular example, user 101 may use a software application on personal mobile communication device 109 to interact with termination assistance tool 105. As user 101 traverses between vehicle 102 and termination 103, termination assistance tool 105 may establish a user communication link 122 with the user's personal communication device 109 and broadcast image information from camera 106 for user 101 to view. The visual feedback information provided by the image information may enable user 101 to more fully observe the environment 100 surrounding route 108, in addition to the environmental information available through the user's own view of the environment 100.

[0017] In particular examples, the navigation system 104 or the termination assistance tool 105 may include a back-end processor circuit 110. In some examples, the back-end processor circuit 110 may be implemented within the circuitry of the navigation system 104. In some examples, the back-end processing circuit 110 may be separate from the navigation system 104. In some examples, the back-end processing circuit 110 may be located remotely from the vehicle 102. In some examples, the back-end processing circuit 110 may optionally be used in a cloud computing environment 112. The back-end processing circuit 110 may receive image information provided by the camera 106 and store the information for later use, for example, to verify that the user 101 traversed the route 108 or to assist in investigating an incident that may have occurred while the user 101 was traversing the route 108.

[0018] In a particular example, user 101 may specify supervisory contact 114 using termination assistance tool 105 and may provide monitoring communication information to termination assistance tool 105. When user 101 invokes termination assistance at termination 103, back-end processing circuit 110 may establish monitoring communication link 116 using the monitoring communication information and may transmit image data from camera 106 to monitoring communication link 116.

[0019] In certain examples, the termination assistance tool 105 may use additional sensors 118 or transducers that may be available on the vehicle 102. For example, if the vehicle 102 includes additional sensors 118, such as additional cameras or lidar sensors, the termination assistance tool 105 may use the additional sensors 118 to scan the environment around the path 108 and detect other objects 120 or movement. In some examples, the vehicle may optionally deploy a drone 119 to provide additional sensors, such as cameras, to assist in tracking the user or perceived threats. Data from the additional sensors 118 may be processed using machine learning techniques to detect and classify potential objects 120 and movement, and then provide a warning to the user 101 via the user communication link 122. In some examples, the termination assistance tool 105 may provide a mapping image via the user communication link 122 such that the screen of the user personal communication device 109 shows a map of the environment 100 surrounding the vehicle 102 and the termination 103, as well as the location of the user 101 and any other potentially relevant objects 120 detected by the termination assistance tool 105. In some examples, an additional illumination source 124 on the vehicle 102 may be used to illuminate the area around the detected object 120 identified by the termination assistance tool 105 so that the user's 101's attention may be immediately brought to the area of ​​the detected object 120. In certain examples, the termination assistance tool 105 may track the detected object 120 using the additional illumination source 124, where the direction of the additional illumination source 124 may be manipulated by the termination assistance tool 105.

[0020] In a particular example, when preparing to use the termination assistance tool 105, the user 101 may provide connection information to use sensors and transducers 126 of the termination 103, such as sensors and transducers 126 of a security system of the termination 103. When the user 101 invokes termination assistance, the termination assistance tool 105 may detect that the vehicle 102 is at the termination 103 and may establish communication with the security system for the purpose of using the sensors or transducers 126 of the security system to provide termination assistance to the user 101.

[0021] 2 generally illustrates an overview of the operation of a navigation system including an exemplary terminal assistance capability within an environment 200. The environment 200 may include a vehicle 202 having a navigation system capable of providing terminal assistance, a first user 201 or more specifically the first user's computing device, back-end circuitry 210 for either the navigation system or the terminal assistance capability, and one or more optional supervisor users 214, 234 or more specifically the one or more optional supervisor users' computing devices. Operation of the terminal assistance may be initiated or requested 240 by the first user 201 immediately prior to using the vehicle 202, near or upon arrival of the vehicle 202 at the first user's 201 destination, or while setting up for use of the vehicle 202. In certain examples, terminal assistance may be initiated by the first user or automatically initiated by the navigation system computer of the vehicle 202 upon arrival at a destination, e.g., an initial destination for picking up the first user 201 or a final destination for dropping off the first user 201. Upon initialization, the vehicle 202 may perform multiple tasks, including, but not limited to, establishing a first user communication link, establishing a termination assistance communication link with a backend, etc. The vehicle may also initialize the use of predetermined resources 242 to begin tracking 244 the first user 201 between the vehicle 202 and the destination termination. Such resources may include tracking the user via a vehicle camera or other camera programmed within the termination assistance circuitry. A camera sensor stream or video feed may be communicated or streamed 246 to the first user 201, potentially allowing the user to see more of the environment than is visible from the first user's perspective. In some examples, if preprogrammed or selected 248 by the first user 201, the termination assistance circuitry may initialize a monitoring communication link to an optional second monitoring user 214, 234, and the sensor stream may be distributed 250 to the second user for monitoring 251 or analysis 270.In certain examples, the backend circuitry 210 may be part of the vehicle's navigation system, part of a termination assistance tool, or a cloud-based machine. In certain examples, one of the second monitor users 214, 234 may be a monitor user 214 selected by the first user 201, an employee of the navigation system provider, or the system's artificial intelligence (AI) 234. In certain examples, the backend 210 may record signals 252 captured by one or more resources used to monitor the first user's 201 movements 254 between the termination and the vehicle 202. Such recordings may be used to confirm to other users that the first user 201 safely crossed 254 between the vehicle and the termination, or to investigate incidents that occurred while the first user 201 was crossing 254 between the vehicle and the termination.

[0022] As the first user traverses 254 between the terminus and the vehicle, the first user 201 or one or more of the second users 214, 234, the camera's field of view, or optionally the vehicle's position may be adjusted 256 either automatically or in response to a request 258 from the first user 201. If the first user 201 encounters a situation while traversing 254 between the vehicle and the terminus, either the first user 201, the back-end circuitry 210, or one of the second users 214, 234 may initiate contact assistance for the first user 201, such as a first responder or someone else near the first user's location.

[0023] Upon completion of the crossing 254 between the first user's 201 termination and the vehicle, the termination assistance tool or the first user 201 may terminate 262 the termination assistance. In response to the termination 262 of the termination assistance, the termination assistance tool may terminate streaming 264 of any sensor signals and terminate or release 266 the communication link between the termination assistance tool and the first user 201 and the second user 214, 234. In certain examples, for example, when the first user 201 uses the termination assistance at the beginning of a journey or trip, the first user communication link may be transferred or reused for use by a navigation system. In certain examples, upon termination 262 of the termination assistance, the back-end circuitry 210 may record 268 the information stream. In certain examples, recording may include deleting the information stream, saving the information stream for a limited time, or saving the information stream to assist in investigating events that occurred during the termination assistance.

[0024] In certain examples, when the termination assistance tool is invoked when the area surrounding the path between the termination and the vehicle is dark, the termination assistance tool may use the illumination light source of the vehicle 202 or the illumination light source of the associated termination system to illuminate the path while the first user 201 traverses 254 the path. In some examples, if the focal direction of the illumination light source can be controlled, the first user 201 or the observer user 214 may control the direction of the illumination light source. In some examples, controlling the direction of the illumination light source may include moving the vehicle 202 to control the direction of the illumination light source.

[0025] A particular benefit of the terminal navigation tool becomes apparent when something unexpected occurs as the first user 201 traverses 254 the path between the vehicle and the terminal. For example, in certain instances, invoking 240 the terminal assistance capabilities of the terminal assistance tool may enable any one or more additional sensors on the vehicle or associated resources (e.g., a security system at the terminal, etc.) to scan or monitor the environment around the path between the terminal and the vehicle. Signals from the additional sensors, as well as signals from sensors tracking 244 the first user 201, may be processed at one or more levels to classify any detections. In some instances, the terminal assistance processor circuit or back-end circuit 210 of the vehicle 202 may be able to classify certain signal patterns as objects (e.g., trees, fire hydrants, curbs, bushes, etc.). A more sophisticated processor, back-end circuitry 210, or AI monitor 234 in the vehicle 202 may be able to analyze 270 the sensor information and classify the detection according to more detailed information (e.g., branches swaying in the wind, someone walking a pet, leaves blowing on the lawn, etc.), which may enable the more sophisticated processor or AI monitor 234 to classify such detection as a potential or actual threat to the first user 201 and take action to mitigate the threat or bring it to the attention of the first user 201 or monitor user 214.

[0026] In certain examples, the termination assistance tool may take action to direct the attention of the first user 201 or the observer user 214 to the threat. For example, the termination assistance tool may send a message to the electronic device of the first user 201 or the observer user 214. In some examples, the message may take the form of augmenting the sensor stream with either a message about the threat or an icon indicating the location of the threat relative to the first user 201. In certain examples, the termination assistance tool may use one or more additional resources to affect the threat. For example, if additional illumination sources are available that can illuminate the area around the detected threat, the termination assistance tool may enable the illumination sources, thereby directing the attention of the first user to the area of ​​the detected threat. In some examples, other lights on the vehicle, such as turn signals and tail lights, may be turned on to direct attention to the threat or to direct the attention of others in the area to the vehicle and the general vicinity of the threat. In some examples, the termination assistance tool may use an audio transducer, such as a speaker, to alert the first user to the threat or generate other sounds (e.g., a siren, etc.) to potentially alarm the threat and deter the threat.

[0027] In certain examples, a termination assistance tool, whether at the vehicle level, back-end level, or AI observer level, may detect an actual threat and may initiate a phone call, text message, or other signal to contact first responders 260 for assistance with first user 201. In certain examples, the termination assistance tool may provide a panic button type user input device on the screen of the first user 201 or observer user 214's electronic device, for example, via an application running on the respective electronic device. If first user 201 or observer user 214 suspects an imminent threat to first user 201 or some other imminent danger surrounding first user 201, the termination, or vehicle 202, first user 201 or observer user 214 may press or otherwise activate the panic button, and the termination assistance tool may contact 260 one or more first responders for assistance, and may include the location of first user 201 or vehicle 202.

[0028] 3 generally illustrates an example screen display 300 of an application with a termination assistance tool, or a screen display of a termination assistance tool, that may be launched on a navigation application of a first user or observer user's electronic device 301. The screen display 300 may include a first area 390 for viewing one or more video streams, such as a video stream 391 tracking the first user. In some examples, the first area may include an overview map showing the vehicle, termination, and first user's location as removed by the termination assistance tool. In some examples, if a second camera is available, a second video stream 392 may include a video stream of an area outside the field of view of the first camera tracking the first user, thereby allowing the first user to see more of the environment than is available from the first user's perspective. In certain instances, if something is detected, the aforementioned termination assistance tool, detection 393 from one or more video streams 391, 392, may be augmented with a location or descriptive text 394 of the detection 393 to draw the attention of a first user or a monitoring user to the detection.

[0029] An optional second area 395 of the screen display 300 may include user inputs 396, 397. In certain examples, the second area 395 of the screen display 300 may include controls 396 for directing a lighting source controllable via the termination assist tool or for directing or adjusting the field of view of a camera controllable via the termination assist tool. In certain examples, the second area 395 of the screen display 300 may include a panic button 397 that activates the termination assist tool to contact first responders. The second area 395 may also selectively display user inputs to activate the capabilities of the termination assist tool or to release the capabilities of the termination assist tool when, for example, the first user has completed traversing the path between the termination and the vehicle.

[0030] FIG. 4 generally illustrates an example map display 491 alternative to the first region 390 that may be used in conjunction with or in place of the displays 391, 392 of FIG. 3. Instead of displaying only image information, the first region 390 may also be used to display a map 491 of the environment surrounding the vehicle 402 and the terminus 403. The map 491 may be augmented with icons of the user 401, the terminus 403, the vehicle 402, and other objects 420 detected by the termination assistance tool. In certain examples, the termination assistance tool may use text 494 to augment the map display 491 to indicate what the object is and whether it appears to be a threat. In some examples, color may be used to indicate whether a detected object is classified, and whether the classification indicates whether the object is a threat, and possibly the level of threat the object represents.

[0031] FIG. 5 generally illustrates a flowchart of an exemplary method of operation of an exemplary termination assistance tool in accordance with the present protected subject matter. The operations of method 500 are performed by a hardware computer, such as that described above with respect to FIGS. 1-4 or below with respect to FIG. 6. At 501, after receiving a request from a first user, the termination assistance tool may capture image information from the environment that a second user may use to traverse between the termination and the vehicle. In some instances, as illustrated in this example, the first user and the second user are the same. An example of when the first user and the second user may be different users is when a parent requests that their child independently start or end a trip in an autonomous vehicle. In certain instances, the image information may include a video stream using a first camera on the vehicle or a second camera on a resource previously programmed into the termination assistance tool.

[0032] At 503, the camera or one or more additional sensors may scan the environment for objects other than the first user. In certain examples, the termination assistance tool may use the image information to detect and classify objects in the environment. In some examples, additional sensors on the vehicle or additional sensors on previously programmed resources may be used to scan the environment outside the field of view of the first camera for objects other than the first user.

[0033] At 505, a first communications link may be established between a controller of the termination assistance tool and a first user, or more precisely, the first user's electronic device. At 507, environmental status information may be transmitted to the first user as the first user traverses or travels between the termination and the vehicle. The status information may include one or more sensor streams, video streams, or other image information, allowing the first user to more fully recognize the environment around the path between the vehicle and the termination. In certain examples, the status information may include text or graphical representations of other objects detected in the environment.

[0034] At 509, a second communication link may optionally be established with a second user or the second user's electronic device. As previously mentioned, such a second user may be a supervisory user. In certain examples, the second user's contact information may be pre-programmed into the termination assistance tool or may be provided by the first user when requesting termination assistance. At 511, status information of the first user's environment may be transmitted to the second user as the first user traverses or travels between the termination and the vehicle. The status information may include one or more video streams or other image information, allowing the second user to be aware of the first user's health and the surrounding environment of the route between the first user and the vehicle and termination. In certain examples, the status information may include text or graphic representations of other objects detected in the environment.

[0035] At 513, if a threat is detected by the first user, the second user, or the termination assistance tool, such as by processing the image information, the controller of the termination assistance tool may contact a first responder for physical assistance. In certain examples, the first user or the second user may use user input on their respective electronic devices, such as a panic button provided on the display of an application associated with the termination assistance tool. In certain examples, the termination assistance tool may record status information for subsequent use. For example, the stored record may be used by a parent to verify that a first user, such as a child or other loved one, safely crossed between the vehicle and the termination. In other examples, the stored record may be used to assist the first user in investigating events that occurred in the environment while crossing between the vehicle and the termination.

[0036] Whether operating in conjunction with a navigation system or an autonomous vehicle system, the subject matter of this disclosure may extend the usefulness of such systems. The subject matter of this disclosure may also provide peace of mind to the user and others associated with the user by providing a means to more fully monitor the user's movements from the moment the user leaves the door of a first terminal to the moment the user passes through the door of a destination terminal, and even when the user crosses between intermediate terminals, such as when crossing between vehicles during a trip or using an intermediate terminal for a break during a trip.

[0037] Embodiments may be implemented in one or a combination of hardware, firmware, and software. Embodiments may also be implemented as instructions stored on a machine-readable storage device, which may be read and executed by at least one processor to perform the operations described herein. A machine-readable storage device may include any non-transitory mechanism for storing information in a form readable by a machine (e.g., a computer). For example, a machine-readable storage device may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, and other storage devices and media.

[0038] The processor subsystem may be used to execute instructions on a machine-readable medium. The processor subsystem may include one or more processors, each including one or more cores. Furthermore, the processor subsystem may be located on one or more physical devices. The processor subsystem may include one or more special-purpose processors, such as a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or a specific function processor.

[0039] The examples described herein may include or operate on logic or multiple components, modules, or mechanisms. A module may be hardware, software, or firmware communicatively coupled to one or more processors to perform the operations described herein. A module may be a hardware module, and such a module may be considered a tangible entity capable of performing specific operations and may be configured or arranged in a specific manner. In one example, a circuit may be arranged in a specific manner (e.g., internally or relative to external entities such as other circuits) as a module. In one example, one or more computer systems (e.g., standalone, client, or server computer systems) or all or a portion of one or more hardware processors may be configured by firmware or software (e.g., instructions, application portions, or applications) as modules that operate to perform specific operations. In one example, software may reside on a machine-readable medium. In one example, the software, when executed by the underlying hardware of a module, causes the hardware to perform specific operations. Thus, the term hardware module is understood to encompass a tangible entity, an entity that is physically configured, specifically configured (e.g., hardwired), or temporarily (e.g., transiently) configured (e.g., programmed) to operate in a particular manner or to perform some or all of the operations described herein. Given the multiple instances in which modules are temporarily configured, each module need not be instantiated at any one time. For example, if the modules comprise a general-purpose hardware processor configured using software, the general-purpose hardware processor may be configured as each different module at different times. The software may, for example, appropriately configure the hardware processor to configure a particular module at one instance of time and to configure a different module at a different instance of time.The modules may also be software or firmware modules that operate to perform the methodologies described herein.

[0040] Circuitry or circuit, as used herein, may comprise, for example, alone or in any combination, hardwired circuitry, programmable circuitry such as a computer processor with one or more individual instruction processing cores, state machine circuitry, and / or firmware that stores instructions executed by the programmable circuitry. Circuits, circuitry, or modules may collectively or individually be embodied as circuitry that forms part of a larger system, such as, for example, an integrated circuit (IC), a system-on-chip (SoC), a desktop computer, a laptop computer, a tablet computer, a server, a smartphone, etc.

[0041] As used in any embodiment herein, the term "logic" may refer to firmware and / or circuitry configured to perform any of the aforementioned operations. Firmware may be embodied as code, instructions or instruction sets, and / or hard-coded (e.g., non-volatile) data in memory devices and / or circuitry.

[0042] “Circuitry,” as used in any embodiment herein, may comprise, for example, hardwired circuitry, programmable circuitry, state machine circuitry, logic, and / or firmware that stores instructions executed by the programmable circuitry, alone or in any combination. Circuitry may be embodied as an integrated circuit, such as an integrated circuit chip. In some embodiments, circuitry may be formed, at least in part, by processor circuitry that executes code and / or instruction sets (e.g., software, firmware, etc.) corresponding to the functions described herein, thereby transforming a general-purpose processor into a specific-purpose processing environment to perform one or more operations described herein. In some embodiments, processor circuitry may be embodied as a standalone integrated circuit or may be incorporated as one of multiple components on an integrated circuit. In some embodiments, various components and circuitry of a node or other system may be integrated into a system-on-chip (SoC) architecture.

[0043] FIG. 6 is a block diagram illustrating a machine in the exemplary form of a computer system 600, according to one embodiment, on which a set or sequence of instructions may be executed to cause the machine to perform any one of the methodologies described herein. In alternative embodiments, the machine may operate as a standalone device and be connected (e.g., networked) to other machines. In a networked arrangement, the machine may operate in either the capacity of a server or a client machine in a server-client network environment, or may function as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a vehicle subsystem, a personal computer (PC), a tablet PC, a hybrid tablet, a personal digital assistant (PDA), a mobile phone, or any machine capable of executing instructions (sequential or otherwise) that specify operations to be performed by the machine. Furthermore, while only a single machine is illustrated, the term “machine” should also be interpreted to include any collection of machines individually or jointly executing a set (or sets) of instructions to perform any one or more methodologies described herein. Similarly, the term "processor-based system" should be interpreted to include any set of one or more machines controlled or operated by a processor (e.g., a computer) that individually or jointly execute instructions to perform any one or more methodologies described herein.

[0044] Exemplary computer system 600 includes at least one processor 602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both, a processor core, a compute node, etc.), a main memory 604, and a static memory 606, which communicate with each other via a link 608 (e.g., a bus). Computer system 600 may further include a video display unit 610, an alphanumeric input device 612 (e.g., a keyboard), and a user interface (UI) navigation device 614 (e.g., a mouse). In one embodiment, the video display unit 610, the input device 612, and the UI navigation device 614 are incorporated into a touchscreen display. Computer system 600 may further include a storage device 616 (e.g., a drive unit), a signal generating device 618 (e.g., a speaker), a network interface device 620, and one or more sensors (not shown), such as a global positioning system (GPS) sensor, a compass, an accelerometer, a gyrometer, a magnetometer, or other sensors. In one exemplary embodiment, computer system 600 may further include a navigation system 619, which may have functionality similar to navigation system 104 described with reference to FIG. 1 and elsewhere in this document.

[0045] Storage device 616 includes a machine-readable medium 622 on which is stored one or more sets of data structures and instructions 624 (e.g., software) embodied by or utilized in any one or more methodologies or functions described herein. Instructions 624 may also reside, completely or at least partially, within main memory 604, static memory 606, and / or within processor 602 during their execution by computer system 600, with main memory 604, static memory 606, and processor 602 also constituting machine-readable media.

[0046] While machine-readable medium 622 is illustrated as being a single medium in one exemplary embodiment, the term "machine-readable medium" may include a single medium or multiple media (e.g., centralized or distributed databases and / or associated caches and servers) that store one or more instructions 624. The term "machine-readable medium" should also be taken to include any tangible medium that can store, encode, or retain instructions that are executable by a machine and cause the machine to perform any one or more methodologies of the present disclosure, or that can store, encode, or retain data structures used by or associated with such instructions. The term "machine-readable medium" should accordingly be taken to include, but not be limited to, solid-state memory, and optical and magnetic media. Specific examples of machine-readable media include, by way of example only, semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM)) and flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and non-volatile memory including, but not limited to, CD-ROM disks and DVD-ROM disks.

[0047] The instructions 624 may further be transmitted or received over a communications network 626 using a transmission medium via the network interface device 620 utilizing any one of several well-known transfer protocols (e.g., HTTP). Examples of communications networks include a local area network (LAN), a wide area network (WAN), the Internet, a cellular network, a plain old telephone (POTS) network, and a wireless data network (e.g., a Bluetooth® network, a Wi-Fi® network, a 3G network and a 4G LTE / LTE-A network, a 5G network, a DSRC network, or a WiMAX network). The term “transmission medium” should be interpreted to include any intangible medium capable of being executed by a machine and storing, encoding, or retaining instructions, including digital or analog communications signals, or other intangible media that facilitate the communication of such software. [Additions and Examples]

[0048] (Example 1) A system for monitoring a user at an end of a journey, the system comprising: a vehicle configured to transport the user; a first sensor coupled to the vehicle and configured to provide a sensor stream of an environment between the vehicle and the end; and an end assistance processor mechanically coupled to the vehicle, the end assistance processor configured to receive and process the sensor stream, detect and track the user in the environment in response to the sensor stream, and provide status information to a first mobile device based on the sensor stream.

[0049] (Example 2) The protected subject matter of Example 1 includes the vehicle being an autonomous vehicle.

[0050] (Example 3) The protected object of Examples 1 and 2 includes a first illumination source, and the terminal support processor is configured to illuminate the environment with light from the first illumination source.

[0051] (Example 4) The subject matter of Example 3 includes the first illumination source being mechanically coupled to the vehicle.

[0052] (Example 5) The subject matter of protection in Examples 3 and 4 includes the first illumination light source being mechanically coupled to the terminal end.

[0053] Example 6 The protected object of Examples 1-5 includes a second sensor configured to scan the exterior of the environment.

[0054] (EXAMPLE 7) The protected subject matter of Example 6 includes the termination assistance processor being configured to receive a status signal from the second sensor, process the second sensor status signal to determine a potential threat to the user, and provide an alarm to the first mobile device in response to determining the potential threat from the second sensor status signal.

[0055] Example 8: The protected object of Example 7 includes a second illumination source, and the terminal support processor is configured to illuminate the potential threat with light from the second illumination source.

[0056] (Example 9) The protected object of Examples 7-8 includes the termination support processor being configured to contact a first responder in response to determining the potential threat from the second sensor status signal.

[0057] EXAMPLE 10 The protected subject matter of Examples 1-9 includes wherein the end support processor is configured to deliver the sensor stream to a cloud-based computer system.

[0058] (Example 11) The protected subject matter of Example 10 includes the termination assist processor being configured to establish communication with a second mobile device and route the sensor stream to the second mobile device in response to the request from the first mobile device.

[0059] (Example 12) A method for providing assistance to a first user at an end of a vehicle movement, the method comprising: capturing first information of the first user's movement in an environment between the vehicle and the end in response to a request from the first user; scanning the environment to detect objects other than the first user; establishing a first communication link between a controller and the first user in response to the request from the user; and transmitting status information of the environment to the first user via the first communication link while the first user moves between the end and the vehicle.

[0060] Example 13 The object of Example 12 includes illuminating at least a portion of a path between the vehicle and the terminus.

[0061] (Example 14) The protection objects of Examples 12 and 13 include steps of detecting objects other than the user in the environment, identifying that the objects can be moved, and illuminating the objects for viewing by the user.

[0062] (Example 15) The protected objects of Examples 12 to 14 include a step of receiving a monitor contact request at the controller from the first user via the first communication link, and a step of contacting a first responder via the controller in response to the monitor contact request.

[0063] (Example 16) The protection targets of Examples 12 to 15 include the end point being the location where the user ends the vehicle movement.

[0064] (Example 17) The protection targets of Examples 12 to 16 include the end point being the position where the user starts moving by vehicle.

[0065] (Example 18) The protected subject matter of Examples 12 to 17 includes that the step of capturing first information includes a step of tracking the movement of the first user in the environment between the vehicle and the terminal.

[0066] (Example 19) The protected objects of Examples 12 to 18 include the fact that the vehicle is an autonomous vehicle.

[0067] (Example 20) At least one non-transitory machine-readable medium including instructions for providing termination assistance from a vehicle to a first user, the instructions, when executed by a machine, causing the machine to perform operations including: capturing first information of the first user's movements in an environment between the vehicle and a termination in response to a request from the first user received at a controller of the vehicle; scanning the environment to detect objects other than the first user; establishing a first communications link between the controller and the first user in response to the request from the user; and transmitting the status information to the user while the user is traveling between the termination and the vehicle.

[0068] Example 21 The protected subject matter of Example 20 includes the action including illuminating at least a portion of a path between the vehicle and the terminus.

[0069] (Example 22) The protected objects of Examples 20 to 21 include those in which the actions include detecting a first object other than the first user in the environment, determining that the object can be moved, and illuminating the object for viewing by the first user.

[0070] (Example 23) The protected subject matter of Examples 20 to 22 includes the operations including receiving an emergency contact request at the controller from the first user via the first communications link, and contacting a first responder via the controller in response to the emergency contact request.

[0071] (Example 24) The protected subject matter of Examples 20 to 23 includes wherein the operation of capturing image information includes tracking the movement of the first user in the environment between the vehicle and the terminal.

[0072] (Example 25) The protected subject matter of Examples 20 to 24 includes the operations including establishing communication with a second mobile device and routing the status information to the second mobile device in response to the request from the first user.

[0073] (Example 26) At least one machine-readable medium including instructions that, when executed by a processing circuit configuration, cause the processing circuit configuration to perform operations to implement any of Examples 1 through 25.

[0074] (Example 27) An apparatus, the apparatus comprising means for implementing any of Examples 1 to 25.

[0075] (Example 28) A system, the system implementing any one of Examples 1 to 25.

[0076] (Example 29) A method, the method implementing any of Examples 1 to 25.

[0077] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific embodiments that may be practiced. These embodiments are also referred to herein as "examples." Such examples may include elements in addition to those shown or described. However, examples that include the elements shown or described are also contemplated. Furthermore, examples using any combination or arrangement of these shown or described elements (or one or more aspects thereof), either with respect to the specific example (or one or more aspects thereof) or other examples (or one or more aspects thereof) shown or described herein, are also contemplated.

[0078] The publications, patents, and patent documents referenced in this document are incorporated by reference herein in their entirety as if individually incorporated by reference. In the event of inconsistent usage between this document and those documents incorporated by reference, the usage in the incorporated references supplements the usage in this document, and to the extent of any irreconcilable conflict, the usage in this document will control.

[0079] In this document, the terms "a" or "an" are used, as is common in patent documents, to include one or more as well as one, independently of any other instance or use of "at least one" or "one or more." In this document, the term "or" is used to refer to a non-exclusive or, such that "A or B" includes "A but not B," "B but not A," and "A and B," unless otherwise indicated. In the appended claims, the terms "including" and "in which" are used as the plain English equivalents of the terms "comprising" and "wherein," respectively. Also, in the following claims, the terms "including" and "comprising" are open-ended, i.e., a system, device, component, or process that includes elements in addition to those listed after such a term in a claim is still considered to be within the scope of that claim. Furthermore, in the following claims, terms such as "first," "second," and "third" are used merely as labels and are not intended to suggest a numerical ordering of their objects.

[0080] The foregoing description is intended to be illustrative, not limiting. For example, the foregoing examples (or one or more aspects thereof) could be used in combination with one another. Upon reviewing the foregoing description, others skilled in the art may employ other embodiments. The Abstract will allow the reader to quickly ascertain the nature of the technical disclosure. The Abstract has been submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the foregoing Detailed Description, various features may be grouped together to streamline the disclosure. However, the claims may not recite all features disclosed herein, as embodiments may feature a subset of the foregoing features. Moreover, embodiments may include fewer features than those disclosed in a particular example. Accordingly, the following claims are hereby incorporated into the Detailed Description as separate embodiments, with each claim standing on its own as a separate embodiment. The scope of the embodiments disclosed herein is to be determined with reference to the appended claims, along with the full range of equivalents to which such claims are entitled.

Claims

1. 1. A system for monitoring a user at an end, comprising: an autonomous system configured to transport the user; a first sensor coupled to the autonomous system and configured to provide a sensor stream of an environment between the autonomous system and a terminal; an end assist processor coupled to the autonomous system, the end assist processor configured to receive and process the sensor stream, detect and track the user in the environment in response to the sensor stream, and provide first information to a first mobile device of the user based on the sensor stream; A system comprising:

2. The system of claim 1 , further comprising a first illumination source, wherein the end assistance processor is configured to illuminate the environment with light from the first illumination source.

3. The system of claim 2 , wherein the first illumination source is mechanically coupled to the autonomous system.

4. The system of claim 2 , wherein the first illumination source is mechanically coupled to the terminal end.

5. The system of claim 1 , further comprising a second sensor configured to scan the exterior of the environment.

6. 6. The system of claim 5, wherein the termination assistance processor is configured to receive a second sensor status signal from the second sensor, process the second sensor status signal to determine a potential threat to the user, and provide an alarm to the first mobile device in response to determining a potential threat from the second sensor status signal.

7. The system of claim 6 , further comprising a second illumination source, wherein the terminal assistance processor is configured to illuminate the potential threat with light from the second illumination source.

8. The system of claim 6 , wherein the terminal assistance processor is configured to contact a first responder in response to determining the potential threat from the second sensor status signal.

9. 9. The system of claim 1, wherein the end assist processor is configured to deliver the sensor stream to a cloud-based computer system, and wherein the end assist processor is configured to activate the first sensor in response to a request from the first mobile device.

10. 10. The system of claim 9, wherein the termination assist processor is configured to establish communication with a second mobile device and route the sensor stream to the second mobile device in response to the request from the first mobile device.

11. The system of claim 1 , wherein the autonomous system is an autonomous vehicle.

12. 1. A method for providing assistance to a user at a termination, comprising: capturing, in response to a request from the user, first information of the user's movements in an environment between an autonomous system and the termination; scanning the environment to detect objects other than the user; establishing a first communications link between a controller and the user in response to the request from the user; transmitting the first information of the environment to the user over the first communications link while the user is traveling between the termination and the autonomous system; A method comprising:

13. The method of claim 12 , further comprising illuminating at least a portion of a path between the autonomous system and the termination.

14. 14. The method of claim 12 or 13, further comprising detecting objects other than the user in the environment, identifying that the objects are capable of movement, and illuminating the objects for viewing by the user.

15. receiving a monitor contact request at the controller from the user via the first communication link; contacting a first responder via said controller in response to said monitor contact request; 15. The method of any one of claims 12 to 14, further comprising:

16. The method of any one of claims 12 to 15, wherein the endpoint is the location where the user ends their movement.

17. The method of any one of claims 12 to 16, wherein the end point is the location where the user starts moving.

18. 18. The method of claim 12, wherein capturing first information comprises tracking the movement of the user in the environment between the autonomous system and the termination.

19. A method for providing assistance to a user at a termination point, comprising: capturing, in response to a request from the user, first information of the user's movement in an environment between an autonomous system and the terminal using a first sensor at the terminal; receiving second information of the user's movement in the environment, the second information being information captured using a second sensor in the autonomous system; establishing a first communications link between a controller and the user in response to the request from the user; transmitting the first information and the second information to the user over the first communications link while the user is traveling between the termination and the autonomous system; A method comprising:

20. 20. The method of any one of claims 12 to 19, wherein the autonomous system is an autonomous vehicle.

21. 1. A program for providing termination assistance to a user from an autonomous system, the program comprising: capturing first information of the user's movements in an environment between the autonomous system and a termination in response to a request from the user received at a controller of the autonomous system; scanning the environment for objects other than the user; establishing a first communication link between the controller and the user in response to the request from the user; transmitting the first information to the user while the user is traveling between the termination and the autonomous system; A program that executes.

22. 22. The program of claim 21, further causing the machine to perform the steps of illuminating at least a portion of a path between the autonomous system and the termination.

23. 23. The program of claim 21 or 22, further causing the machine to perform the steps of: detecting objects other than the user in the environment; determining that the objects are movable; and illuminating the objects for viewing by the user.

24. The machine, receiving an emergency contact request at the controller from the user via the first communication link; contacting a first responder via the controller in response to the emergency contact request; The program according to any one of claims 21 to 23, further comprising:

25. 25. The program of claim 21, wherein the step of capturing the first information comprises capturing image information and tracking the movement of the user in the environment between the autonomous system and the terminal.

26. 26. The program of claim 21, further causing the machine to perform the steps of: establishing communication with a second mobile device; and routing the first information to the second mobile device in response to the request from the user.

27. A program for providing termination assistance to a user from an autonomous system, the program comprising: In response to a request from the user received at a controller of the autonomous system, using a first sensor at a terminal to capture first information of the user's movements in an environment between the autonomous system and the terminal; receiving second information of the user's movement in the environment, the second information being information captured using a second sensor in the autonomous system; establishing a first communication link between the controller and the user in response to the request from the user; transmitting the first information and the second information to the user while the user is traveling between the termination and the autonomous system; A program that executes.

28. At least one machine-readable medium storing a program according to any one of claims 21 to 27.

Images