Etiquette-based vehicle and control system having pair mode and smart behavior mode

Abstract

A self-powered personal vehicle 72 is provided that includes a mechanical drive system, a set of sensors, and a controller coupled to the mechanical drive system to move the vehicle. The self-powered vehicle 72 can operate in multiple modes, including a paired mode and a smart behavior mode. In the paired mode, the vehicle 72 follows the user's trajectory, and in the smart behavior mode, the vehicle 72 performs autonomous behavior. The self-powered vehicle 72 operates with hysteresis behavior so that the movement of the vehicle 72 is consistent with the ergonomic comfort of the user and the courtesy of third-party pedestrians. The self-powered vehicle 72 can operate in a platoon with other self-powered vehicles.

Description

Technical Field

[0001] The present invention relates to vehicles and vehicle control, and more particularly to a personal vehicle configured to operate in any of a plurality of modes, including a paired mode and a smart behavior mode, and a control system for the personal vehicle.

Background Art

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 061,897, filed Aug. 6, 2020, and U.S. Provisional Application No. 63 / 155,098, filed Mar. 1, 2021. Each of these applications is hereby incorporated by reference in its entirety.

Summary of the Invention

Means for Solving the Problems

[0003] According to one embodiment of the present invention, a self-powered personal vehicle is provided. The vehicle of this embodiment includes a mechanical drive system for moving the vehicle, a set of sensors, and a controller coupled to the mechanical drive system for managing the movement of the vehicle. The controller is configured to (i) follow the user's trajectory with hysteresis dynamics in a paired mode with the user such that the vehicle exhibits a delay in a manner consistent with the user's ergonomic comfort and the courtesy of a third-party pedestrian when following the user's movement, and (ii) operate the vehicle in a smart behavior mode such that the vehicle exits the paired mode and performs autonomous behavior.

[0004] In related embodiments, the controller is configured to operate the vehicle in pair mode so that the vehicle occupies a speed-dependent position behind the user. In further related embodiments, the controller is configured to operate the vehicle in pair mode so that, when there are no obstructions to the vehicle, the speed-dependent position is also initially set to the side of the user. Optionally, the controller is configured to operate the vehicle in pair mode so that, when there is an obstruction to the vehicle, the vehicle tucks towards a position immediately behind the user. Optionally, the tuck performed by the vehicle follows a curve. Optionally, the curve is a cubic spline curve based on parameters including the distance and angle between the vehicle and the user, the user's speed, the obstacle's speed, and the distance from the vehicle to the obstacle.

[0005] In another related embodiment, the controller is configured to operate the vehicle in pair mode such that, when the user occupies a position within an area of ​​a fixed region defined relative to the vehicle, the individual is considered stationary for following the user's movement. Optionally, the fixed region is teardrop-shaped. In yet another related embodiment, the controller is configured to operate the vehicle in pair mode such that, when the user is outside the fixed region but occupies a position within an area of ​​a tracking region surrounding the fixed region, the vehicle's orientation is adjusted so that the user is centered within the fixed region. Optionally, the controller is configured to operate the vehicle in pair mode such that, when the user leaves the tracking region at a speed exceeding a threshold, the vehicle begins to follow the user. Optionally, the threshold is preset to approximately 0.6 m / s.

[0006] In another related embodiment, the controller is configured to operate the vehicle in a smart action mode in which the vehicle performs autonomous actions as it passes through an entrance or exit. Optionally, the controller is further configured to cause the vehicle to perform a separate door-passing sequence of actions, which includes: a follow-approach moment in pair mode in which the vehicle follows the user until the moment the user reaches the vicinity of the entrance or exit; a disconnection moment in which the vehicle disconnects from pair mode with the user and initiates autonomous action when the recognized door opening angle reaches a threshold angle; a smart moment in which the vehicle performs an autonomous door-passing action and moves through the entrance or exit to a reconnection position; a reconnection moment in which the vehicle re-enters pair mode with the user; and a follow-depart moment in which the vehicle moves from a waiting position in the reconnection moment to a follow position in pair mode.

[0007] In another embodiment, the present invention provides a plurality of etiquette-based vehicles, each vehicle being: A mechanical drive system for moving such vehicles, A set of sensors, A controller that is coupled to a mechanical drive system and manages the movement of such a vehicle. The controller is configured to cause such vehicles to exhibit platoon behavior in order to join a convoy of multiple etiquette-based vehicles having a leader, the platoon behavior including joining a line and following in a line, and each etiquette-based vehicle other than any etiquette-based vehicle that is the leader and operates in smart behavior mode (i) enters pair mode with a separate object, which is selected from a group consisting of the leader and other vehicles among the multiple etiquette-based vehicles, and (ii) joins the convoy in turn as the leader performs trajectory following, directly or indirectly following the leader in hysteresis behavior.

[0008] Optionally, platoon behavior may further include a stage where the vehicles leave the platoon formation, exiting pair mode and entering smart behavior mode where the vehicles autonomously move out of the formation. Alternatively or additionally, when lining up, the vehicle closest to the leader is made to enter pair mode with the leader, and each vehicle moving further away from the leader is made to enter pair mode with the next closest vehicle to the leader. Again, alternative or additionally, the leader is an etiquette-based vehicle trained to autonomously proceed along a known route.

[0009] In a further optional choice from the embodiments described at the beginning of this summary of the invention, the vehicle, in pair mode, The starting point where the vehicle would have been located when the training exercise began, The vehicle would have followed the user's trajectory, and The location where the training activity should have ended and The system is configured to perform training behaviors of a path that it experiences, remembers, and learns, and after remembering such experiences, the vehicle autonomously follows the user's trajectory from a start to an end point in smart behavior mode. Optionally, the training behavior includes the vehicle remembering a velocity-to-position profile along the trajectory it would have followed the user, and the vehicle autonomously follows the trajectory using the remembered velocity-to-position profile in smart behavior mode. Further optional, the velocity-to-position profile is remembered as a set of position-dependent vectors.

[0010] The aforementioned features of the embodiment will be more readily understood by referring to the following detailed description with reference to the attached drawings. [Brief explanation of the drawing]

[0011] [Figure 1] This is a top view showing the relative positions of a user 12 and an etiquette-based vehicle in pair mode, where the vehicle is in continuous speed-dependent heel-following positions 13A, 13B, and 13C relative to the user, according to one embodiment of the present invention. [Figure 2] This figure shows the top-down relative positions of a user and an etiquette-based vehicle in pair mode according to one embodiment of the present invention, where the vehicle is initially in a series of speed-dependent heel-following positions 13A, 13B, and 13C relative to user 12 (similar to Figure 1), and then, to avoid an obstacle (to avoid a pedestrian at position 21), it is in a series of speed-dependent positions 23A, 23B, and 23C relative to user 22. [Figure 3] This is a graphic mapping diagram obtained from a study on the movements of individuals performing tasks in relatively stationary positions. [Figure 4A] This is a diagram of a stationary region according to an embodiment of the present invention. [Figure 4B] This is a diagram of the tracking region according to an embodiment of the present invention. [Figure 5] This is a top view showing the relative positions of a user and an etiquette-based vehicle in a continuous position when the vehicle decelerates in pair mode with a door-related follow-up approach moment (defined in paragraph 0012), according to an embodiment of the present invention. [Figure 6] This is a top view showing the relative position of the user and the etiquette-based vehicle in pair mode during a door separation moment (as defined in paragraph 0012), according to an embodiment of the present invention. [Figure 7] This is a top view showing the relative positions of the user and the etiquette-based vehicle when the vehicle autonomously moves through an entrance / exit during a smart moment related to the door (as defined in paragraph 0012), according to an embodiment of the present invention. [Figure 8] This is a top view according to one embodiment of the present invention, showing the relative position of the user and the etiquette-based vehicle at a reconnection moment (defined in paragraph 0012) with respect to the door, after the vehicle has autonomously moved through the doorway, the user has closed the door, and the vehicle is oriented in the direction most likely to be in the user's line of sight. [Figure 9]This is a top view showing the relative position of the user and the etiquette-based vehicle after the vehicle has re-entered pair mode with the user following the reconnection moment in the follow-away moment (as defined in paragraph 0012), according to one embodiment of the present invention. [Figure 10] This is a front view of a smart follow module according to another embodiment of the present invention, comprising a set of sensors and a controller configured to manage the movement of an autonomous host self-powered vehicle. [Figure 11] This is a rear view of a smart follow module according to another embodiment of the present invention, which includes a set of sensors and a controller configured to manage the movement of an autonomous host self-powered vehicle. [Figure 12] This is a top view showing the relative positions of a user and an etiquette-based vehicle at the start of a training action (as defined in paragraph 0012), before the user indicates a known route to the etiquette-based vehicle, according to one embodiment of the present invention. [Figure 13] This is a top view showing a known route from the start to the end of an etiquette-based vehicle, and the user, in a training behavior (defined in paragraph 0012), according to one embodiment of the present invention. [Figure 14] This is a top view showing an etiquette-based vehicle, according to one embodiment of the present invention, at the end of a training action, the vehicle being instructed to follow a known route (defined in paragraph 0012), and the user. [Figure 15] This is a top view showing an etiquette-based vehicle according to one embodiment of the present invention, which exits smart behavior mode and enters parking mode after traveling along a known route. [Figure 16] This is a top view illustrating an etiquette-based vehicle in smart behavior mode following a known route (as defined in paragraph 0012) and a user following a different route, according to one embodiment of the present invention. [Figure 17] This figure shows a group of etiquette-based vehicles forming a convoy according to one embodiment of the present invention. [Figure 18]A top - view diagram showing a plurality of ticket - based vehicles that follow a user in queue behavior (defined in paragraph 0012) according to an embodiment of the present invention. [Figure 19] A top - view diagram showing a plurality of ticket - based vehicles that follow a user with hysteresis behavior in queue behavior (defined in paragraph 0012) according to an embodiment of the present invention. [Figure 20] A top - view diagram showing a plurality of ticket - based vehicles when the queue behavior is switched off and the ticket - based vehicles move to the final position according to an embodiment of the present invention.

Mode for Carrying Out the Invention

[0012] Definitions The following terms used in this description and the appended claims shall have the meanings set forth below, unless the context requires otherwise. A "set" includes at least one element. A "vehicle" is a device for transporting a load on land or within a building, and the load is selected from the group consisting of goods, a set of people, and combinations thereof. A "personal vehicle" is a self - propelled vehicle equipped with a set of sensors and a controller for managing the movement of the vehicle, and is configured to operate in cooperation with a user. A "user" is an entity selected from the group consisting of a human pedestrian and a vehicle controlled by a human. A "vehicle controlled by a human" is a self - propelled vehicle equipped with a steering system and a drive system, and can be steered along a defined path by a human driver manually inputting directly or indirectly to the steering system and the drive system. Optionally, the human driver can steer the vehicle by remote control. A vehicle controlled by a human is considered to "autonomously" define a trajectory. A "leader" is an individual selected from a group consisting of the user and the leading etiquette-based vehicle, and this individual autonomously defines the trajectory of the next etiquette-based vehicle paired with it. An "etiquette-based vehicle" is a personal vehicle configured to operate in one of several modes, including pair mode and smart behavior mode. The "pair mode" of an etiquette-based vehicle is a mode in which the vehicle is configured to follow the user's trajectory using hysteresis behavior. A "formation" is a set of etiquette-based vehicles, where each vehicle in the set, except for the leader and any etiquette-based vehicle operating in smart behavior mode, enters pair mode with a separate object selected from a group consisting of (i) the leader and (ii) other vehicles in the set. The etiquette-based vehicles in a set are configured to move following the leader within the formation, with the leading etiquette-based vehicle in the set selected from a group consisting of the leader and the etiquette-based vehicles immediately behind the leader, and the subsequent etiquette-based vehicles in the set positioned after it. In a convoy of etiquette-based vehicles, the "platooning behavior" of a single etiquette-based vehicle is the behavior required of that single etiquette-based vehicle to participate in the convoy. The term "hysteresis behavior" refers to etiquette-based vehicle behavior that exhibits delays in a manner consistent with the user's ergonomic comfort and pedestrian courtesy when following the user's movement. An etiquette-based set of vehicles becomes a "convoy formation" when each consecutive vehicle directly or indirectly follows the leader's trajectory. A "known path" is a trajectory that a self-powered vehicle has been trained to follow autonomously. "Trained behavior" refers to etiquette-based vehicle behavior that is configured for the vehicle to learn its trajectory as a known path. The "Smart Behavior Mode" of an etiquette-based vehicle is a triggered mode in which the vehicle exits pair mode and performs autonomous actions. Once autonomous actions in Smart Behavior Mode are completed, the vehicle is configured to pause and wait in its environment for an event that will determine the vehicle's next mode, which by default will cause it to re-enter pair mode. In etiquette-based vehicles, "parking mode" refers to the mode in which the vehicle is stationary. A "trigger" is a set of conditions in the etiquette-based vehicle environment (which may include user behavior) under which a set of vehicle sensors works in conjunction with the vehicle's controller to exit pair mode and enter smart behavior mode. In various embodiments, the occurrence of a trigger is notified to the user by the etiquette-based vehicle, for example, by a set of vehicle light indicators (such as lighting), by an audible indicator (which may optionally be present in the vehicle), or by a display on the instrument panel of a device fixed to the vehicle or separate from the vehicle (such as via a smartphone) that communicates with the vehicle and runs a program that controls a set of vehicle behaviors. "Following behavior" refers to a set of standards for etiquette-based vehicle execution in pair mode, taking into account the user's perspective, the observer's perspective, and the technical requirements of the vehicle itself. Such execution includes follow position, starting and stopping behavior, cornering and pivoting behavior, and preparatory behavior. The "sensors" of an etiquette-based vehicle are devices configured to provide optical, wireless, or other information about a set of objects in the vehicle's environment, and this information is supplied as a set of electrical signals that are processed by a controller that manages the vehicle's movement. In this context, "sensors" include monocular cameras, stereoscopic cameras, radar imaging systems, and LiDAR systems. The "immobile zone" is defined in relation to the etiquette-based vehicle in pair mode with the user, and within the immobile zone, the user is considered stationary so that the vehicle can follow the user's movements. Door passage moments are five moments configured in which an etiquette-based vehicle performs separate actions in the course of a door passage sequence, according to embodiments of the present invention. Door passage moments collectively define classification criteria for separate actions that enable a practical analysis and implementation of vehicle door and exit passage, while each moment is still part of a larger door passage sequence. The five moments are as follows: 1. Follow-up approach moment refers to the behavior of an etiquette-based vehicle in pair mode, where it continuously follows the user until the moment the user reaches the door. The vehicle follows in a heel-follow position, adjusting the dynamic follow distance based on the user's speed and circling around obstacles as described below. The vehicle is configured to detect that the user has stopped when the user's speed is less than 0.6 m / s and to reach a stopping position 450 mm directly behind the user when the user's speed is less than 0.2 m / s. In the follow-up approach moment, embodiments of the present invention provide additional obstacle detection requirements so that the vehicle provides sufficient space for the user, and as a result the user can comfortably open the inward-swinging door. 2. The disconnection moment is the act of the vehicle disconnecting from the user pair mode and initiating autonomous door operation. In embodiments of the present invention, this action begins when the perceived door opening angle, which is the apparent angle between the door frame and the end of the open door calculated from the vehicle's stopping position, reaches 60°. 3. A smart moment is an autonomous door passage action performed by a vehicle, which includes planning a path for passing through an open doorway and proceeding through the doorway to a reconnection position determined for a particular door type. 4. The reconnection moment is the act of the vehicle re-entering the pair mode with the user. During the reconnection moment, the vehicle is directed in the most likely direction of the user's line of sight when the user closes the door. Line of sight is defined as the direction the user's head is facing. This initiates the pair mode with the user, and the vehicle follows with the hysteresis behavior described below. 5. The follow-away moment is the act of the vehicle transitioning from the standby position in the recombination moment to the heel-following position in pair mode. The vehicle resumes following when the user moves beyond the tracking area at a speed of at least 0.6 m / s. The vehicle is configured to draft behind the user and then transition to following, as described below. Inward-swinging / outward-swinging refers to the direction in which a hinged door swings. The direction is determined relative to the user's direction of travel. If the user needs to pull the door to open it, it is an inward-swinging door; on the other hand, if the user needs to push the door to open it, it is an outward-swinging door.

[0013] A dynamic behavioral model using etiquette-based vehicles in pair mode.

[0014] In the definition above, it was stated that etiquette-based vehicles, in pair mode, are configured to follow the user's trajectory with hysteresis behavior. Furthermore, "hysteresis behavior" refers to the operation of an etiquette-based vehicle that exhibits delays in a manner consistent with the user's ergonomic comfort and pedestrian courtesy when following the user's movement. Since both the user and the vehicle are self-powered and constitute a dynamic combination of components, the vehicle's controller is programmed to make the vehicle act, i.e., move, according to these principles from a variety of different dynamic perspectives. The vehicle's dynamic behavior from each of these perspectives is programmed to fit a set of models developed based on empirical measurements and relevant design considerations.

[0015] Regarding these design considerations, from the user's perspective, pair mode establishes a sense of trust. Not only the vehicle's field of view in the user's peripheral vision, but also the maneuvering and disengaging movements in response to obstacles, create operator awareness. The heel-follow position is inspired by the human-dog pairing and is based on already well-known pairing. In pair mode, the vehicle is programmed to be intuitive to operate, helping to foster a bond between the user and the vehicle. From a bystander's perspective, the user and vehicle are perceived as a social unit with a magical element. Following leverages social pedestrian behavior to strengthen the perception of pairing. The vehicle is visible to oncoming pedestrians, increasing their awareness. As bystanders approach the vehicle, the vehicle moves out of its path and around, modeling good pedestrian etiquette. A narrow heel-follow position is preferable to minimize the width occupied by the vehicle-user pair on the sidewalk. The decision of whether the vehicle's following position should be initially set on one side, either the user's left or right, should be made considering social norms and regional differences. By adhering to social norms, paired vehicles are perceived as passive and courteous, remaining in slow lanes and avoiding paths where oncoming bystanders are passing. Therefore, responsibility for vehicle movement rests with the user, who is the vehicle's leader, and the vehicle will follow cautiously. Specifically, this addresses the aspects of operation in paired mode, including heel-follow position, obstacle avoidance, and in-between-follow position.

[0016] Paired vehicle operation in heel-following position Figure 1 is a top view showing the relative positions of a user 12 and an etiquette-based vehicle in pair mode at continuous speed-dependent heel-following positions 13A, 13B, and 13C relative to the user, according to an embodiment of the present invention. The vehicle's operation is examined in terms of three different user walking speeds: low speed (0.6 m / sec), normal speed (1.2 m / sec), and high speed (2.1 m / sec). At a slow walking speed of 0.6 m / sec in an open space, the vehicle's position 13A (determined by the vehicle's center 131) is 800 mm and 21° relative to a reference point behind the centerline passing through the user at point 121. At a normal walking speed of 1.2 m / sec in an open space, the vehicle's position, measured at the vehicle's center 132, is 1000 mm and 17° relative to the same reference point 121. Finally, at a high speed of 2.1 m / s, the vehicle's position, measured at the vehicle's center 133, is 1450 mm and 11° relative to the reference point 121. The vehicle's follow position is intended to remain within a constant lateral offset of 300 mm (from the user's centerline to the vehicle's centerline) at three defined speeds. The angles defined above are along the 300 mm offset line and are intended to act as a guide. A dynamic stopping action is triggered when the walking speed falls below 0.6 m / s. The primary side of the user on which the vehicle maintains its follow position must be based on local norms; for example, in the United States, the vehicle should follow the user's right side by default, while in the United Kingdom, Japan, and Australia, the vehicle should follow the user's left side by default. This behavior must be specified during registration when mounting based on location and must also be set to user-controlled in the corresponding application for controlling the vehicle's behavior. If repeated failures occur and the vehicle fails to achieve the goal of following in the trailing position, it will be necessary to perform the follow operation in the advanced follow position discussed below.

[0017] Operation of paired vehicles during obstacle avoidance Figure 2 is a top view showing the relative positions of a user and an etiquette-based vehicle in pair mode according to an embodiment of the present invention, first (as in Figure 1) in continuous speed-dependent heel-following positions 13A, 13B, and 13C relative to user 12, and then, to avoid an obstacle (to avoid a pedestrian at position 21), the vehicle is in continuous speed-dependent positions 23A, 23B, and 23C relative to user 22. When the paired vehicle performs obstacle avoidance, it performs a circling action, which is a transition defined by a cubic spline curve. Specifically, when the vehicle encounters an obstacle such as a pedestrian 21, the vehicle circles behind user 22, transitioning to an angle of 0° measured at point 221 from the centerline of user 22, and maintaining the speed-dependent follow distance defined in the previous paragraph. The circling action during the transition period can be represented by a cubic spline curve based on the following parameters: the distance and angle between the vehicle and the user, the user's speed, the obstacle's speed, and the distance from the vehicle to the obstacle. The formula for a cubic spline is as follows: y=a(x-x1) 3 +b(x-x1) 2 Here, x1, a, and b are constants specific to the vehicle's speed.

[0018] Operation of the paired vehicle in the entry and follow position According to embodiments of the present invention, when the vehicle performs a circling maneuver, the vehicle maintains an in-follow position, following the user immediately behind. At a slow walking speed of 0.6 m / s in a crowded space, the vehicle's position is 800 mm and its angle is 0°. At a normal walking speed of 1.2 m / s in a crowded space, the vehicle's position is 1000 mm and its angle is 0°. At a fast walking speed of 2.1 m / s in a crowded space, the vehicle's position is 1450 mm and its angle is 0°. The vehicle is configured to return to a heel-follow position under appropriate circumstances. In one embodiment, the vehicle returns to the heel-follow position after a set time, i.e., 30 seconds of uninterrupted detection of the user following behind. In another embodiment, the vehicle is configured to have a peeping function, repeatedly moving to the side by a threshold distance sufficient to maintain a forward field of view while remaining within the in-follow threshold.

[0019] Immobile region and tracking region parameters According to some embodiments of the present invention, when an etiquette-based vehicle is in user-pair mode, the vehicle ignores minor movements by the user. However, determining when a movement is minor is not easy. For this reason, we conducted studies observing how much people move while performing simple everyday activities (such as searching for keys to open a door, selecting vegetables at a market, or tidying up groceries in the kitchen). These activities helped to define the areas in which people move during prolonged periods of stagnation. The data obtained from such studies is summarized in Figure 3, which is a graphic mapping derived from a study of the movements of individuals performing tasks in relatively stationary positions.

[0020] To determine the geometric distribution of the data using percentiles, the data summarized in Figure 3 was further analyzed. The findings are shown in Table 1. The data in Table 1 was used to define what is called the "fixed region" from the data at the 75th percentile and the "tracking region" from the data at the 90th percentile. In addition, it was found that 99% of the movement of stationary people occurred at a speed of ≤0.6 m / s. In embodiments of the present invention, this is defined as the user speed threshold at which a paired etiquette-based vehicle will resume following the user if it exceeds this threshold.

[0021] [Table 1]

[0022] From the user's perspective, the delay in the movement of the paired, etiquette-based vehicle facilitates simple decision-making when the user changes position within the stationary area. The user can navigate around the vehicle better because they know that the vehicle will remain where it was when the user first stopped. If the user moves beyond a defined area considered stationary, the vehicle turns in place to inform the user that the vehicle is still with them and keeps the user within the vehicle's field of view. This movement reaffirms to the user that the vehicle is still tracking them, even though the vehicle is not reacting to every movement of the user. The vehicle resumes following if the user moves beyond the tracking area.

[0023] The perspective of bystanders is also taken into consideration. By minimizing the vehicle's movement during periods of prolonged closure, the vehicle's footprint is reduced, making it less likely to disturb others. In addition, the vehicle appears calmer, more polite, and more patient. By turning around in place, the vehicle informs bystanders that it is paired with a user.

[0024] According to embodiments of the present invention, the definitions of the “stationary area” and the “tracking area” provide an indicator to the extent to which an etiquette-based vehicle in user-pair mode is likely to remain within a defined space or to continue walking. The vehicle tracks the user’s movement trajectory as the user performs an activity. The vehicle can determine whether the user is stationary or moving by constantly measuring the angle and distance to the user. Figures 4A and 4B illustrate the stationary area and the tracking area according to embodiments of the present invention, respectively. The “stationary area” is defined as a teardrop-shaped area with the following dimensions measured from the center of the vehicle: 850 mm in the +x direction, a 60° angle at the apex of the teardrop, 842 mm from the apex to where the straight edge intersects the circular edge, and a radius of 280 mm for the circular portion of the teardrop. The “tracking area” is defined as an area bounded by a parabolic curve and cutting angle with the following dimensions measured from the center of the vehicle. A parabola with its vertex at (1100,0), with boundaries of ±600mm in the ±y directions, an angle of 120° opening toward the parabola, and a distance of 660mm from the vertex to the intersection point of the parabola.

[0025] Pairing and Start The inventors' research shows that when a user pairs with an etiquette-based vehicle, the user tends to stand approximately 680 mm in front of the vehicle. The vehicle is configured to provide feedback to the user via light and sound when pairing is successful and the vehicle's movement remains stable. The inventors' analysis shows that the user will be within a teardrop-shaped stationary region drawn from the center of the vehicle 75% of the time. In embodiments of the present invention, if the user is outside the stationary region during pairing, the vehicle is configured to rotate itself to face the user and be centered. In further embodiments of the present invention, the vehicle will only begin following when the user leaves the tracking region. In embodiments of the present invention, if the user is within the stationary region during pairing, the vehicle remains stationary and provides feedback to the user of successful pairing via a user interface using light and sound. However, if the user is outside the stationary region but within the tracking region, the vehicle will adjust its orientation so that the user is centered within the stationary region. The vehicle is configured to remain stationary until the user leaves the tracking region.

[0026] The vehicle is configured to begin following when the user leaves the tracking area at a speed of at least approximately 0.6 m / s. If the user leaves the tracking area at a speed of less than approximately 0.6 m / s, the vehicle will move behind the user in accordance with the design specifications for stopping. As the vehicle adjusts its position and orientation to keep the user within its field of view, the stationary and tracking areas will change position along with the vehicle.

[0027] Retention When a person stops for several seconds at a time to perform stationary activities, this person tends to move around within a limited area. According to embodiments of the present invention, once a paired vehicle reaches a stopping position, it continues to track the user while remaining at the position where the vehicle was when the user first stopped. If the user moves beyond the stationary area within the tracking area, the vehicle turns in place in a manner similar to that configured for pairing and initiation, keeping the user within the vehicle's field of view. The vehicle resumes following only if the user moves beyond the tracking area. According to further embodiments of the present invention, the vehicle detects that the user is stopped when the user's speed is less than about 0.2 m / s and reaches a stopping position. In various embodiments, the vehicle's stopping position is 450 mm directly behind the user. The vehicle does not move forward from this position until the user moves beyond the tracking area. When the user moves beyond the stationary area into the tracking area, the vehicle rotates in place, centering itself according to the user. In various embodiments, the vehicle resumes following when the user moves beyond the tracking area at a speed of at least 0.6 m / s. If the user leaves the tracking area at a speed of less than 0.6 m / s, the vehicle will move behind the user in accordance with the vehicle's stopping design specifications. If the user stops too quickly and the vehicle fails to reach the stopping position (450 mm directly behind), the vehicle will remain in place and face the user. The vehicle will not move from this position until the user moves beyond the tracking area predicted by the vehicle's current orientation.

[0028] Smart behavior mode and access to entrances and exits In an etiquette-based vehicle, a "smart behavior mode" is defined as encompassing trigger-initiated actions in which the vehicle exits pair mode and performs autonomous actions. Once autonomous actions in smart behavior mode are completed, the vehicle is configured to pause and wait in its environment for an event that determines the vehicle's next mode, which in its initial configuration will re-enter pair mode. In a typical embodiment of the present invention, the vehicle performs a process that includes: temporarily dispairing from the user; autonomously passing through an entrance / exit while the user being followed holds the door open; moving to the opposite side of the entrance / exit and waiting for the user to pass through the door; and re-entering the pairing relationship with the user and following the user.

[0029] The design of autonomous behavior according to embodiments of the present invention is based on information from field observations and motion capture studies. The following six types of hinged swing doors were studied: left inward swing, left outward swing, right inward swing, right outward swing, double inward swing door, and double outward swing door. The direction of the hinge and swing is defined relative to the user's direction of travel. Doors that are not suspended from a frame, move hinged from one side, and swing to open along a vertical axis to a minimum of 90 degrees are not envisioned in this specification, but some of the design principles expressed herein may apply. Furthermore, the designs and design specifications studied are limited to approaching the door from the front and perpendicular to the door surface. Nevertheless, using a human-following robot has the advantage of leveraging human capabilities to open and keep the door open.

[0030] The movement of the machine through the doorway with the user while the user holds the door open involves a more sophisticated behavioral model of machine movement than a model in which the machine simply stays behind the user and adjusts to the user's forward movement. By configuring autonomous behavior from this perspective, the vehicle can assist the user in navigating challenging situations. The smart behaviors regarding doors described herein are designed to make the user's experience of passing through a doorway with a robot feel as if they were passing through with a person or other intelligent companion who understands the etiquette associated with opening, holding, and passing through doorways.

[0031] The inventors' field observations and motion capture studies focused on people moving in pairs or with rolling objects through the six types of doors listed above. A reconfigurable set of doors was specially created for motion capture. Twenty-four external participants were hired and instructed to pass through doors together, with one person holding the door open and the other following. Participants were not informed of the reason or use of the data to avoid bias in their behavior. Participants were asked to pass through the six types of doors in various roles, such as leader and follower. 2,240 passes of people passing through the six types of doors were collected.

[0032] The following set of questions facilitated the framework for analyzing inventors, etc. Where do people typically stop to open a door? Where do people or objects that are following a person typically stop when waiting for a door to be opened? How far does a person move when opening a door that swings inward? • From the follower's perspective, what is the degree to which the door is open when approaching it? At what point do followers become detached from their leader and begin to walk through the door? How long does it take from the moment the leader starts opening the door until the followers start walking through it? When passing through an inward-swinging door, how much space does a person leave between themselves and the person holding the door for the follower? Where do people or objects following a person typically wait to reconnect with the leader after passing through a door? Where do people typically stop to close a door? • Does the leader's gaze indicate a moment of reconnection before they leave together? How long does their gaze overlap? How much time elapses between the separation moment when the door opens and the reconnection moment when the door closes?

[0033] The insights that drove data-driven door behavior design focused on the spatial behavior, timing, gaze analysis, and perceived openness metrics of leaders and followers. No significant differences were found between single-leaf and double-leaf doors among the six types of doors studied. Therefore, the final door specifications focused on the following four types of doors: left inward swing, left outward swing, right inward swing, and right outward swing. Double-leaf door data was divided into left and right double-leaf doors, and classified along with insights for left and right single-leaf doors. Qualitative design considerations and technical specifications for etiquette-based vehicles guided the methodology for applying the insights that constitute the final design decisions.

[0034] Left-inward swing The left-inward swing door, relative to the user's direction of travel, is mounted on the left side of the frame and swings inward toward the user. The total time for a pair to pass through the left-inward swing door is 7.10 seconds.

[0035] Tracking approach moment Figure 5 is a top view showing the relative positions of a user 51 and an etiquette-based vehicle at consecutive positions 52 and 53 when the vehicle decelerates in pair mode with a follow-approach moment (defined in paragraph 0012) relative to the door 54, according to one embodiment of the present invention. The inventors' research has shown that when a person approaches a left inward swing door, there is considerable variation in the position of the leader toward the door handle, as shown in Figure 5. The leader reaches the handle at 400 mm from the centerline of the door toward the door handle and 821 mm perpendicular to the door. Time is defined as the moment (t=0) when the leader reaches the handle in the door passage sequence. The user's stopping position reflects these observations. The inventors' research has also shown that at the moment the leader reaches the handle of the left inward swing door, the follower's position is 383 mm from the centerline of the door and 1436 mm perpendicular to the door. The follower is aligned closely behind the leader. As a result of the inventors' understanding of how people approach the left-side inward swing door, they determined that the stopping position of the etiquette-based vehicle should be 1500mm perpendicular to the door and aligned behind the user, according to the stopping specifications.

[0036] Separation moment To determine when a person should open and hold a door open so that a robot can pass through autonomously, the moment a follower detaches from the leader was observed. The detachment scenario is defined by the perceived opening angle, i.e., the angle from the measured follower's position to the door frame and the edge of the door. In the case of a left inward swing door, detachment occurs when the follower begins to pass through the door, which the leader has held open. The perceived opening angle of the door at the time of detachment is 60°, while the actual door angle is 68°. The opening width of the door from the door frame to the edge of the door is 1000 mm. The time at which detachment occurs is t = 1.32 seconds. To understand the space occupied by the user while moving, opening, and holding the left inward swing door open, the user's movement is visualized as the open movement area. The open movement area is schematically represented as item 64 in Figure 6, which is a top view showing the relative position of the user 61 and the etiquette-based vehicle 62 in pair mode with respect to the door 63 in the separation moment (defined in paragraph

[0030] ), according to one embodiment of the present invention. The open movement area 64 is defined by the following four range values: a minimum of 390 mm and a maximum of 1216 mm perpendicular to the door, a minimum of 470 mm to the right and a maximum of 277 mm to the left in the direction of travel from the centerline of the door.

[0037] Smart Moments Figure 7 is a top view showing the relative positions of the user 71 and the etiquette-based vehicle 72 when the vehicle 72 autonomously moves along the path 73 through the doorway in a smart moment related to the door (as defined in paragraph

[0030] ) according to an embodiment of the present invention.

[0038] In the inventors' research, it was found that when a follower overtakes a leader, it maintains a personal space distance of 882 mm. Based on the recorded positions of the leader and follower, a recommended distance relative to the centerline of the door is given. The etiquette-based vehicle initiates a left inward swing autonomous path 73 through the door at the start of the smart moment, which begins at the moment of separation. In this action, the vehicle first turns to the right in the direction of travel with a radius of 650 mm perpendicular to the centerline of the door, and then turns back to align itself with the centerline of the door as the vehicle passes the threshold. As observed in the research, the waiting position of the follower on the opposite side of the door was 973 mm from the centerline of the door and 707 mm perpendicular to the door. This observation is used to set the following waiting position of the etiquette-based vehicle on the opposite side of the door: 1000 mm perpendicular to the centerline of the door and 500 mm perpendicular to the door. The smart moment begins when the door is detached (t=1.32 seconds) and ends when it is reconnected (t=4.27 seconds), allowing the vehicle to autonomously pass through the door in a total of 2.95 seconds.

[0039] Recombination moment Figure 8 is a top view showing the relative positions of user 81 and etiquette-based vehicle 82 in a recombination moment (defined in paragraph 0012) with respect to the door, after vehicle 82 has autonomously moved through the doorway, the user has closed the door, and the vehicle is oriented in the most likely direction of the user's line of sight, according to one embodiment of the present invention. The recombination moment begins when the user's line of sight coincides with the line of sight of the etiquette-based vehicle, confirming that they are ready to continue walking together again. The position of the vehicle in the recombination moment is determined from the observation that a person looks toward the open door for 74% of the time while closing the left inward swing door. The recombination moment occurs at t=4.27 seconds of the door passage sequence. The length of time the leader looks toward the follower is approximately 0.28 seconds. The vehicle is expected to re-enter pair mode with the user within 0.28 seconds.

[0040] Follow-away moment Figure 9 is a top view showing the relative positions of the user 91 and the etiquette-based vehicle 92 after the recombination moment, in a follow-away moment (defined in paragraph 0012), according to one embodiment of the present invention. When the user moves beyond the tracking area at a speed of at least 0.6 m / s, the vehicle is configured to operate in the entry-follow position, move to the center directly behind the user, and then transition to the heel-follow position. The period from the recombination moment to the follow-away moment is 2.55 seconds. The follow-away moment occurs at t=7.10 seconds of the door-passing sequence.

[0041] Right-side swing The right-outward swing door, relative to the user's direction of travel, is mounted on the right side of the frame and swings outward, away from the user. The total time for a pair to pass through the right-outward swing door is 7.09 seconds. Passing through the right-outward swing door involves five door passage moments, exactly the same as those described for the left-inward swing door. Compared to the left-inward swing door situation, the distances and timings are similar, although they are not identical due to the differing geometries in each case.

[0042] Figures 10 and 11 are front and rear views, respectively, of a smart follow module comprising a set of sensors and a controller configured to manage the movement of an autonomous host self-powered vehicle, according to another embodiment of the present invention.

[0043] Smart behavior mode and progression along known routes Non-profit owners of etiquette-based vehicles primarily use their vehicles outside their homes, whether they live in urban apartments, suburbs, or planned housing developments. After taking their vehicles out, owners typically need to store them in narrow, secluded, or difficult-to-access locations, often in follow-mode. Commercial owners of etiquette-based vehicles may similarly have a collection of vehicles that need to be stored and guided in a manner customized to their business environment. By teaching vehicles to follow known routes, vehicle owners can optimize vehicle accessibility in hard-to-navigate spaces and eliminate the burden of guiding vehicles into such spaces that would otherwise be necessary.

[0044] A known path is defined in paragraph 0012 as a trajectory that a self-powered vehicle is trained to move autonomously. As shown in Figures 12 to 14, an etiquette-based vehicle utilizing trained behavior learns a trajectory that is a known path from the user.

[0045] Figure 12 is a top view showing the relative positions of user 1201 and etiquette-based vehicle 1202 at the start of a training action (defined in paragraph 0012) and before user 1201 indicates a known path to etiquette-based vehicle 1202, according to one embodiment of the present invention. In some embodiments of the present invention, once user 1201 begins a training action, vehicle 1202 will perform a 360-degree scan of the environment around the vehicle's center 1204. In one embodiment of the present invention, vehicle 1202 creates a map of the area around the vehicle from the data collected during the 360-degree scan. After the scan, vehicle 1202 will enter a pair mode with the user. The vehicle has performed the scan and rotated to be behind user 1201, ready to follow such user, as shown in Figure 12. Since user 1201 and vehicle 1202 are larger than points in space, in some embodiments the vehicle will follow the trajectory of user's center 1203 at vehicle center 1204. In another embodiment, the vehicle 1202 follows the user's heel. In a further embodiment of the present invention, the vehicle can follow a wearable attachment.

[0046] In various embodiments of the present invention, a user can initiate and terminate training behavior in a vehicle through an app, a button on the vehicle, a body signal, a verbal signal, or other methods known to those skilled in the art.

[0047] Figure 13 is a top view showing an etiquette-based vehicle according to one embodiment of the present invention, which follows users 1301 and 1311 along a trajectory 1320 from a starting point 1304 to an ending point 1314. In Figure 12, user 1201 enters a training action and vehicle 1202 performs a scan. In Figure 13, user 1301 moves along a trajectory 1320 to be taught to vehicle 1302. In various embodiments, the center of the vehicle 1304 follows another point 1303 of user 1301. When user 1311 brings vehicle 1312 to the end of the trajectory 1320, user 1311 can end the training action, and as a result the vehicle will remember the ending point 1314.

[0048] In some embodiments of the present invention, the speed at which the user 1301 moves, i.e., the corresponding speed at which the etiquette-based vehicle 1302 moves, will be the speed at which the etiquette-based vehicle moves as it proceeds along a known path. The trajectory follows the user's center 1303 and the vehicle's center 1304.

[0049] Figure 14 is a top view showing an etiquette-based vehicle and user at the end of a training action according to one embodiment of the present invention, where vehicle 1412 followed user 1411. According to embodiments of the present invention, when user 1411 completes a training action, the vehicle marks the vehicle's location 1414 as the end point of the route. In some embodiments of the present invention, the vehicle remembers the area where the training action ended so that it can reproduce a known route from any point within that area. In some embodiments of the present invention, once the training mode is complete, the user or other operator remembers the known route by naming it. In a derivative embodiment, each direction of the route (i.e., start to end or end to start) may have its own name. This is advantageous because the user can select a route from multiple known routes. The routes may have a choice that is displayed in an app or other menu and instructs the vehicle to move along the selected route. In some embodiments of the present invention, the vehicle remembers the orientation of the vehicle at the end of the route and will reproduce this orientation after traveling along a known route in the future. Similarly, in some embodiments, after completing a training action, the vehicle will rotate 540 degrees. 540 degrees allows the vehicle to perform a 360-degree scan and then turn around to return along a known path.

[0050] In some embodiments, the robot must be within a radius, for example, one meter, of the path starting point in order to trigger the start of movement along a known path. In other embodiments, movement along a known path is triggered by entering a specific mode. For example, in one embodiment, when a vehicle exits parking mode and is in a specific area, for example, the vehicle's parking spot, the vehicle automatically moves along a known path and becomes ready to function.

[0051] In some embodiments of the present invention, an etiquette-based vehicle can adapt a known path to a smooth trajectory. In such embodiments, the vehicle can interpolate a smooth trajectory from a path walked by a user to determine a known path. In further embodiments, the vehicle can use sensors to determine if an obstacle is on a known path. In such embodiments, the vehicle can stop, wait for the obstacle to be removed, send an alarm, and locate a path around the obstacle.

[0052] Figure 15 is a top view showing an etiquette-based vehicle 1501 according to one embodiment of the present invention, which moves along a known path and then exits smart action mode to enter parking mode. In the embodiment shown in Figure 15, vehicle 1512 follows a trajectory from a starting location 1514 to an ending location 1504. In some embodiments, vehicle 1501 rotates around its axis and faces the direction of movement when moving in the reverse direction along the known path. This is advantageous because, in situations where the known path returns the vehicle to a storage location, the vehicle can exit the storage location more quickly when requested.

[0053] Figure 16 is a top view showing an etiquette-based vehicle 1612 in smart behavior mode following a known path (defined in paragraph 0012) 1622 according to one embodiment of the present invention. To illustrate that the vehicle moves independently of the user, users following different paths 1623 are shown. Furthermore, the embodiment in Figure 16 shows that the vehicle is still in smart behavior mode because it can move autonomously, but it is not in pair mode because the vehicle does not follow the user. Furthermore, Figure 16 shows that in some embodiments, the endpoint 1604 after traveling along the known path is a location 1642, which is illustrated here by a circle. Since location 1642 is larger than a point, the vehicle can better pass through obstacles or simply ignore obstacles that slightly obscure the location of the trajectory 1622 or the endpoint of the known path. Similarly, the starting location 1614 is defined by a circle 1641. In this embodiment, the user 1611 can begin traveling along the known path when the vehicle 1612 enters somewhere within the radius 1641. This reduces the need for accuracy, which can sometimes be cumbersome. When vehicle 1601 is parked, in some embodiments, the vehicle performs a scan to determine the area around the vehicle.

[0054] A dynamic behavioral model involving a convoy of etiquette-based vehicles. In the definition above, it was stated that in a procession, each of the etiquette-based vehicles in a set is paired with (i) a leader and (ii) a distinct object selected from a group of other vehicles in the set. The etiquette-based vehicles in a set are configured to move in the procession following the leader, with the leading etiquette-based vehicle in a set positioned immediately behind the leader, and subsequent etiquette-based vehicles in a set positioned behind it.

[0055] In some embodiments, etiquette-based vehicle platooning behavior has three stages. The first stage, shown in Figure 18, involves lining up. The vehicles are initially spaced apart and line up in a single file as user 1801 performs the trajectory follow. The second stage, shown in Figure 19, is following in a line. The vehicles proceed in a line, sequentially behind user 1905, in hysterical behavior. The third stage, shown in Figure 20, is breaking out of the line, with each vehicle performing smart behavior to exit pair mode and return to forming a group different from the formation of the line.

[0056] Furthermore, in some embodiments of the present invention, each vehicle in a convoy moves in hysteresis behavior. "Hysteresis behavior," as defined in paragraph 0012, refers to etiquette-based vehicle behavior in which, when following the user's movement, the vehicle exhibits a delay in a manner consistent with the user's ergonomic comfort and pedestrian courtesy. Since the user and paired vehicles are self-powered and constitute a dynamic combination of components, the vehicle controller is programmed to move the vehicles in a variety of distinct dynamic aspects.

[0057] Figure 17 shows a group of etiquette-based vehicles 1702, 1703, 1704, 1705, and 1706 forming a platoon according to one embodiment of the present invention. In some embodiments, the vehicles forming the platoon start from within a specific radius 1741, which is, for example, 1.9 meters. In further embodiments, each robot that the user wants to place in the platoon must also be within 1 meter of the nearest robot selected for the platoon. In one embodiment of the present invention, each robot enters a pair mode with a vehicle that will follow after the initialization of the platoon action. In some embodiments, user 1701 can select a vehicle 1702 to enter the platoon action with, which pairs with user 1701 and is therefore the leading vehicle behind the user. Vehicle 1702 then finds other nearby vehicles and communicates with them to similarly enter the platoon action. Then a second vehicle 1704 pairs with the first vehicle, and a third vehicle 1706 pairs with the second vehicle. This process continues until each vehicle is paired. In some embodiments, the order of the vehicles in the convoy is determined by their proximity to user 1701. In other embodiments, the order is determined randomly.

[0058] Figure 18 is a top view showing multiple etiquette-based vehicles following user 1801 in the first stage of a convoy action (as defined in paragraph 0012) according to one embodiment of the present invention. In one embodiment of the present invention, as the leader, user 1801, begins to walk in Figure 18, the leading vehicle follows the leader, followed by subsequent vehicles, and so on, so that the vehicles enter a convoy formation. In one embodiment, the leading vehicle 1802 follows the user after 1.8 seconds. In this embodiment, each successive etiquette-based vehicle (sometimes called a "robot") 1803, 1804, 1805, and then 1806 will be in its final following position 1.3 seconds after the previous robot. That is, robot 1803 will enter the convoy formation 3.1 seconds after the leader begins to walk, robot 1804 will enter the convoy formation 4.4 seconds after the leader begins to walk, and so on, until each vehicle enters the convoy formation. In another embodiment, the time it takes to enter the convoy varies depending on the user's walking speed and other factors. In the embodiment shown in Figure 18, the order was selected based on proximity to the leading follow position. As each vehicle enters the convoy, vehicle 1802 is at position 1812, vehicle 1803 is at position 1813, vehicle 1804 is at position 1814, vehicle 1805 is at position 1815, and vehicle 1806 is at position 1816.

[0059] Figure 19 is a top view showing multiple etiquette-based vehicles following a user in hysteresis behavior in a platoon (as defined in paragraph 0012) according to one embodiment of the present invention. Hysteresis behavior within a platoon helps to represent the platoon not as a collection of robotic vehicles, but as multiple individual vehicles functioning as a unified whole. This human-like atmosphere provides a sense of comfort to both the user and onlookers. In some embodiments, following in hysteresis behavior is achieved by having the vehicles follow at various distances based on the velocity of the object they are following. In one embodiment, the vehicles follow 600 mm behind an object moving at 0.6 m / s, 800 mm behind an object moving at 1.2 m / s, and 1250 mm behind an object moving at 2.1 m / s. Such vehicles remain 450 mm behind a stationary vehicle. Each distance is shown in Figure 19 by a group of vehicles 1901, 1902, and 1903. The lead vehicle in each group, represented by 1901a, 1902a, and 1903a, is positioned behind user 1905 at a distance corresponding to the user's speed. These follow distances mimic typical human behavior when following another person. The follow distance for group 1901 is shown in 1910, the follow distance for group 1902 is shown in 1920, and the follow distance for group 1903 is shown in 1930.

[0060] The path of a convoy of vehicles may deviate from the direct trajectory of the object the vehicle is following in order to navigate around obstacles in the path. In some embodiments, a certain amount of leeway may be allowed for a vehicle to deviate from the path of an object it is following, for example, 240 mm away.

[0061] Figure 20 is a top view showing multiple etiquette-based vehicles when platooning is switched off and the etiquette-based vehicles leave the platoon, according to one embodiment of the present invention. In various embodiments, platooning is switched off by an app, body signals, verbal signals, or a button press on the vehicle. In some embodiments, when platooning ends, etiquette-based vehicles 2002, 2003, 2004, 2005, and 2006 use smart behavior to join a group near where the platooning mode ended. In these embodiments, when the vehicles' platooning ends, the behavior includes a transitional phase in which the vehicles leave the platoon formation in the meantime and form a group with other vehicles that were previously in the platoon. In one embodiment, once all vehicles have joined the group, the grouped vehicles enter parking mode. As shown in Figure 20, vehicle 2002 did not need to move to form the group. In similar embodiments, other vehicles, including vehicle 2003, for example, did not need to move to form the group. In one embodiment, if a vehicle enters parking mode at the end of a platooning maneuver (for example, because the user has stopped moving and the platooning maneuver has ended), the vehicle will temporarily exit parking mode, perform a smart maneuver to move within a radius, and then re-enter parking mode. In another embodiment, as soon as the platooning maneuver ends, the vehicle enters parking mode. In yet another embodiment, as soon as the platooning maneuver ends, the vehicle will proceed along a known path. In some embodiments, the vehicles group themselves together so that each vehicle is at a resting place at roughly the same time. Thus, a vehicle at the back of the line will be at the back of the group, and vice versa.

[0062] Embodiment of a sensor system used by an etiquette-based vehicle in pair mode Since the etiquette-based vehicle is configured to follow the user's trajectory using hysteresis behavior in pair mode, it is essential for the etiquette-based vehicle to acquire information about the user's position over time, and it is also useful to acquire the user's orientation over time. For this purpose, the etiquette-based vehicle is equipped with a set of sensors.

[0063] Optical imaging systems using electronic image sensors provide high-resolution color and brightness information representing a 3D scene projected onto an image plane. Image representation is typically divided into pixels arranged as a rectangular array. Modern solid-state sensors can deliver resolutions of tens of megapixels. Typical consumer CMOS sensors range from 5 to 20 megapixels, while industrial-grade sensors range from 1 to 10 megapixels. Many applications require understanding the distance range to each element of the scene, which represents the "depth," a third dimension of the image.

[0064] There are two commonly used techniques for obtaining depth information: optical imaging, time-of-flight (TOF) imaging, and stereometric ranging. In systems using TOF imaging, a bright infrared light flashes along the same viewing axis from the point closest to the image sensor towards the scene. Each pixel of the sensor measures the time from the onset of the flash to the time the reflected light is received by the pixel, providing an estimated distance to the illuminated portion of the scene. This type of system has several drawbacks, including high power requirements, low resolution, and limited distance range.

[0065] In a stereoscopic ranging system, two identical cameras are fixed and positioned in a stable relationship. Two images are captured simultaneously, and the resulting images are compared pixel by pixel. A 2D image representation is provided, including an estimate of the pixel-by-pixel distance range determined by measuring the pixel parallax between the two cameras for features in the scene. Stereoscopic cameras can use any image sensor, allowing for the selection of a sensor according to the application and desired image quality. Stereoscopic cameras consume less power than TOF cameras because they do not require flashing bright lights. Furthermore, the distance range and accuracy can be optimized by selecting the imager resolution and the distance between cameras in the system.

[0066] While a monocular imaging system cannot provide distance range data on its own, it can provide azimuth information if the target object is identifiable. Generally, to optimize the performance of a ranging imager, the imager's field of view (FOV) should be limited to 90 degrees or less. In situations where the reader moves to the side of the 3D ranging system's FOV, the wide-FOV monocular camera can still determine the angular orientation, and the vehicle can be rotated to the right to re-enter the reader within the 3D sensor's FOV.

[0067] In a first embodiment of the set of sensors used by an etiquette-based vehicle in pair mode, a stereoscopic ranging optical imaging system is used in combination with a wide-FOV monocular imaging system. Both sensors supply information to a tracking system that the vehicle uses to determine the location of a user it is following. The tracking system monitors location information along with image information to calculate the user's location relative to the vehicle. Even if one or the other sensor is impaired by some external factor, such as sunlight, the other sensor can still provide enough information to maintain operation until the interference is removed. In situations where the user moves to the side of the 3D ranging system's FOV, the wide-FOV monocular camera can still determine the angular orientation, and the vehicle can rotate to the right to re-enter the reader within the 3D sensor's FOV.

[0068] In a second embodiment of the sensor set used by an etiquette-based vehicle in pair mode, 3D optical imaging is combined with 3D wireless imaging to provide user tracking information to the vehicle when in pair mode with the user. The optical imaging system employs machine vision techniques and a high-resolution multi-antenna radar imaging system to provide robust 3D point clouds under changing environmental conditions. The system comprises a wide-FOV monocular color camera and a 4D (x,y,z,v) radar ranging sensor. The camera and radar system are positioned close to each other on the vehicle, with overlapping fields of view (FOV).

[0069] Radar systems use the emission and reflection of RF signals to determine the distance range to a target. Multiple antenna systems can provide the distance range to a target along with X, Y, or elevation and azimuth information. Modern solid-state radar systems can achieve high-resolution imaging, providing x, y, z, and velocity information for each detected voxel. However, radar systems cannot operate at the same speed as optical systems, nor do they offer the same resolution, and they do not provide any information about the color of the scene.

[0070] In a second embodiment, a stereoscopic ranging optical imaging system is used in combination with a high-resolution radar system. Both sensors supply information to the tracking system. The tracking system monitors location information along with the image information and calculates the leader's location relative to the vehicle. Even if one or the other sensor is impaired by some external factor, such as sunlight, the other sensor can still provide enough information to maintain operation until the interference is removed.

[0071] In a third embodiment of the set of sensors used by an etiquette-based vehicle in pair mode, a wide-FOV monocular optical imaging system is combined with a high-resolution radar system. Both sensors provide information to the tracking system. The tracking system monitors location information along with the image information to calculate the user's location relative to the vehicle. Even if one or the other sensor is impaired by some external factor, such as sunlight, the other sensor can still provide enough information to maintain operation until the interference is removed. In situations where the user moves to the side of the 3D ranging system's FOV, the wide-FOV monocular camera can still determine the angular orientation, and the vehicle can be rotated to the right to re-enter the reader within the 3D sensor's FOV.

[0072] In a fourth embodiment of the sensor set used by an etiquette-based vehicle in pair mode, 3D optical imaging is combined with 3D radio imaging and wide-FOV monocular imaging to provide leader tracking information for the following vehicle. Specifically, the sensor set comprises an optical imaging system employing stereometric ranging techniques and a high-resolution multi-antenna radar imaging system to provide a robust 3D point cloud under changing environmental conditions. The sensor set further comprises a stereometric ranging camera system and a 4D (x,y,z,v) radar ranging sensor. The stereometric and radar systems are positioned close to each other on the vehicle, with similar and overlapping fields of view (FOV). An additional wide-FOV monocular imaging camera is provided to provide azimuth information when the leader steps outside the FOV of the 3D sensors.

[0073] Even if one sensor or another is impaired by some external factor, such as sunlight, another sensor can still provide enough information to maintain operation until the interference is removed. In situations where the user moves to the side of the 3D ranging system's field of view (FOV), the wide-FOV monocular camera can still determine the angle and direction, and the vehicle can be rotated to the right to re-enter the user within the 3D sensor's FOV.

[0074] Figures 10 and 11 are front and rear views, respectively, of a smart follow module, which includes a set of sensors and a controller configured to manage the movement of an autonomous host self-powered vehicle, according to another embodiment of the present invention. When the smart follow module is implemented in the host vehicle, the host vehicle operates as an etiquette-based vehicle in the manner described in relation to the previous figures, thereby being configured to operate in cooperation with the user, as previously described.

[0075] The module can use video and radar detection to identify target users and provide the host vehicle with distance range and direction information for tracking and following the user. As described in relation to other embodiments, the module enables intelligent trajectory planning and behavior control that is comfortable for pedestrians in terms of intuitive operation. The module is equipped with an Ethernet connection for integration with the host vehicle system. Module or vehicle-specific parameters, operating modes, data rate, and data format can be configured using an API. The API can also be used to control system operation, set up WiFi access, and update system software. During operation, all measurements and log data are communicated as real-time data via this port. Streaming of video and sensor images is available (at a low speed) via a separate WiFi connection. When using the module, the vehicle host supplies the module with the necessary operating settings, such as height from the ground, inclination angle, maximum target distance range, target type (person, machine), data rate (10-30 Hz), and data format (units for distance range and direction angle). In various embodiments, the user selects the following operating modes: Pure follow, trail, and custom offset. After the system starts, the user presses the "pairing" button to position the vehicle and the user in place. The module then begins processing the user's direction information and makes it available to the host using the specified protocol. The host then operates the vehicle based on the information coming from the module. If the module loses pairing, a message indicating that pairing has been lost is sent to the host. Once the task is complete, the operator can unpair by pressing the pairing button, or the host can send a stop command to the module.

[0076] The embodiments of the present invention described above are intended to be illustrative only, and numerous modifications and variations will be apparent to those skilled in the art. All such modifications and variations are intended to be within the scope of the present invention, as defined in the appended claims.

Claims

1. A mechanical drive system for moving the vehicle, A set of sensors configured to provide information about objects in the environment of the vehicle, A controller coupled to the mechanical drive system and the set of sensors, which manages the movement of the vehicle based at least partially on the information relating to objects in the vehicle's environment. A self-powered personal vehicle equipped with, The aforementioned controller (a) (i) a user pairing mode in which the vehicle follows the user's trajectory with a delay, and (ii) a smart action mode in which the vehicle autonomously exits the pairing mode and autonomously passes through the door opening, which is the entrance / exit, and the vehicle is operated in such a manner. (b) A self-propelled personal vehicle, characterized in that, in order for the vehicle to pass through the entrance, the vehicle is moved to a reconnection position through the entrance, based on the type of door corresponding to the entrance, which is one of the following: a left inward swing door, a left outward swing door, a right inward swing door, or a right outward swing door.

2. A self-powered personal vehicle according to claim 1, characterized in that the controller is configured to operate the vehicle in the pair mode such that the vehicle dynamically adjusts the distance between itself and the user based on the user's speed.

3. A self-powered personal vehicle according to claim 2, characterized in that the controller is configured to operate the vehicle in the pair mode so that it follows the user for a set distance when there are no obstacles that would interfere with the vehicle.

4. A self-powered personal vehicle according to claim 1, characterized in that when the vehicle is operated in the pair mode, and the user occupies a position within a fixed area defined relative to the vehicle, the controller is configured to consider the user to be stationary and keep the vehicle stationary in order to follow the user's trajectory.

5. A self-powered personal vehicle according to claim 4, characterized in that the immobile region is teardrop-shaped.

6. A self-powered personal vehicle according to claim 4, characterized in that the controller is configured to adjust the orientation of the vehicle when, in the pair mode, the user is outside the immobile area but occupies a position within the tracking area surrounding the immobile area, thereby operating the vehicle so that the user is positioned in the direction the vehicle is facing.

7. A self-powered personal vehicle according to claim 6, characterized in that the controller is configured to operate the vehicle such that, in pair mode, the vehicle begins to follow the user when the user leaves the tracking area at a speed exceeding a threshold.

8. A self-powered personal vehicle according to claim 7, characterized in that the threshold is preset and is approximately 0.6 m / second.

9. A self-powered personal vehicle according to claim 1, wherein, upon passage through the entrance / exit, the controller is configured to cause the vehicle to execute a door passage sequence of separate actions, and the door passage sequence is (a) In the pair mode, the vehicle follows the user until the moment the user reaches the vicinity of the entrance, and (b) A disconnection moment when the vehicle is disconnected from the paired mode with the user and the recognized door opening angle reaches a threshold angle, and the vehicle begins to pass through the entrance / exit. (c) The vehicle passes through the entrance and exit and proceeds through the entrance and exit to the reconnection position, smart moment, (d) The vehicle enters pair mode with the user again, and the recombination moment (e) A self-powered personal vehicle, characterized in that the vehicle includes a follow-away moment in which the vehicle transitions from a standby position in the recombination moment to a follow position in the pair mode.

Images