Mobile object and control method thereof

Abstract

To conduct travel control for a mobile object in a more appropriate following position relative to a user in accordance with target objects in surroundings, including the user.SOLUTION: A mobile object includes a sensor for detecting a target object in surroundings, recognizes and sets a user on the basis of an output from the sensor, generates a path of the mobile object so as to follow the user at a first position that is an obliquely rear side of the user in accordance with the movement of the set user and the output from the sensor, and causes the mobile object to travel in accordance with the generated path.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a moving object and a control method thereof. [Background technology]

[0002] Autonomous mobile objects, such as small mobility vehicles and robots, are known that travel close to a user to assist the user by guiding the user to a destination or carrying luggage. Patent Document 1 proposes a mobile object that moves and travels to an appropriate position relative to the user based on information about the user's torso and legs. Patent Document 2 also proposes a robot control system that carries the user's luggage and follows it at an appropriate distance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 115548 [Patent Document 2] Patent Publication No. 2021-64214 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, public places such as shopping malls, train stations, and airports often contain crowded areas. In such crowded places, it is extremely useful for a mobile body to assist a user by leading or following. On the other hand, crowded places are also places where surrounding objects can become obstacles for a mobile body assisting a specific user. Therefore, it is desirable to control the travel of a mobile body while taking into consideration not only the movement of the user to be assisted, but also surrounding objects and the positional relationship between the user to be assisted.

[0005] An object of the present invention is to perform travel control of a mobile body at a more appropriate tracking position for a user in accordance with surrounding targets including the user. [Means for solving the problem]

[0006] According to the present invention, for example, a mobile body includes a sensor that detects surrounding targets, a setting means that recognizes and sets a user based on an output from the sensor, a trajectory generation means that generates a trajectory of the mobile body so that the mobile body follows a first position diagonally behind the user in accordance with the movement of the user set by the setting means and the output from the sensor, and a travel control means that causes the mobile body to travel according to the generated trajectory. The first position is a position to the left rear or right rear of the user moving, and when the number of targets detected by the sensor in the surrounding environment where the user is moving exceeds a predetermined number, the trajectory generation means generates a trajectory to follow a second position between the first position to the left rear and the first position to the right rear instead of the first position. It is characterized by: [Effects of the Invention]

[0007] According to the present invention, it is possible to perform travel control of a mobile body at a more appropriate tracking position for a user in accordance with surrounding targets including the user. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a system configuration. [Figure 2] FIG. 1 is a diagram showing an example of the configuration of a moving body. [Figure 3] FIG. 1 is a diagram showing an example of a detailed configuration of the present system. [Figure 4] FIG. 2 is a diagram illustrating a driving mode of a moving body. [Figure 5] FIG. 1 is a diagram showing an overview of the services provided by this system. [Figure 6] 10 is a flowchart showing a processing procedure for controlling the trajectory of a moving object. [Figure 7] 10 is a flowchart showing a processing procedure for generating a trajectory of a moving object. [Figure 8] FIG. 2 is a diagram showing the positional relationship between a moving object and a user. [Figure 9] A diagram showing an example of the positional relationship between a moving object and a user depending on the surrounding environment. [Figure 10] A diagram showing an example of the positional relationship between a moving object and a user depending on the surrounding environment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Furthermore, the same reference numerals are used for the same or similar components, and redundant explanations will be omitted.

[0010] <System configuration example> 1 shows an example of the configuration of a system including a mobile object and a server according to one embodiment of the present invention. This system includes mobile objects 100a, 100b, and 100c, and a server 200. Because the mobile objects 100a, 100b, and 100c have similar configurations, the alphabets at the end of their reference numerals will be omitted below. Note that when describing a specific mobile object, an alphabet will be added to the end of the reference numeral.

[0011] The mobile object 100 is placed in various facilities such as shopping malls, parks, train stations, airports, and parking lots, and provides various services to a set user (hereinafter referred to as a "set user"). For example, the mobile object 100 can lead, follow, or guide a set user who is a support target, or make deliveries at the request of a pre-registered authenticated user. The services provided by the mobile object 100 can be switched depending on the driving mode of the mobile object, which will be described later with reference to FIG. 4. The set user refers to a user whose user identification has been performed using a vein sensor provided in the mobile object 100, which will be described later. Furthermore, it is not intended that the mobile object in the present invention be limited to the mobile object shown in FIG. 1. The present invention is applicable to various mobile objects, such as four-wheeled vehicles, two-wheeled vehicles, small mobility vehicles, and robots.

[0012] The server 200 monitors multiple mobile objects 100, moves the mobile objects to each area, and controls their locations, etc., to enhance user convenience. Specifically, the server 200 moves the mobile objects 100 to a location in an area, such as a building, where multiple mobile objects 100 are located, where the mobile objects 100 are more likely to be used. For example, the server 200 controls the mobile objects 100 by moving them to areas where crowds are occurring and increasing the number of mobile objects 100 in the area depending on the level of crowding. Furthermore, when registering a user, the server 200 may acquire information such as vein patterns from the mobile object 100 and register and authenticate the user. Whether authentication via the server 200 is required may be determined depending on the driving mode of the mobile object 100 used by the user. For example, when using the mobile object 100 in delivery mode, only users previously registered via the server 200 may be authenticated. On the other hand, in the leading, following, and guiding modes, the user does not need to be previously registered; the mobile object 100 may be able to use the modes simply by setting the user. The acquired identification information (vein information, feature information from captured images, etc.) is used for confirmation processing (re-authentication) when the user and the mobile body 100 are separated by a predetermined distance or more, i.e., when the user is lost and then reunited with the user.

[0013] The mobile object 100 and the server 200 can communicate bidirectionally via a network 300. More specifically, the mobile object 100 can access the network 300 via nearby access points 301 and 302 and communicate bidirectionally with the server 200 via the network 300. For example, if the mobile object 100 is located in a building such as a shopping mall or within its premises, the server 200 can determine its approximate location based on the access points 301 and 302 accessed by the mobile object 100. That is, the access points 301 and 302 each have location information about the location where they are installed, and the approximate location of the mobile object 100 can be determined based on this location information. Furthermore, the location information of the access points can easily identify which floor in the building the mobile object 100 is located on (altitude information). Furthermore, the server 200 can determine its precise location based on location information output from a GNSS (Global Navigation Satellite System) or the like provided in the mobile object 100, as described below. Furthermore, by combining these pieces of information, the server 200 can obtain location information, such as that the target mobile object 100 is located near an elevator in an underground parking lot. If the location information output from the GNSS includes altitude information, the altitude information may be used instead of the location information of the access point.

[0014] <Configuration of moving body> Next, an example of the configuration of the moving body 100 according to this embodiment will be described with reference to Fig. 2. Fig. 2(a) shows the internal configuration of the moving body 100, and Fig. 2(b) shows the back of the moving body 100 according to this embodiment. In the figure, arrow X indicates the front-to-rear direction of the moving body 100, with F indicating the front and R indicating the rear. Arrows Y and Z indicate the width direction (left-to-right direction) and up-to-down direction of the moving body 100, respectively. Since the moving bodies 100a, 100b, and 100c each have the same configuration, the following description will omit the letters at the end of their reference numbers.

[0015] As shown in FIG. 2(a), the mobile body 100 includes, as a traveling unit, front wheels 20, rear wheels 21a, 21b, motors 22, 23, a steering mechanism 24, a drive mechanism 25, and a storage section 26. The steering mechanism 24 is a mechanism that uses the motor 22 as a drive source to change the steering angle of the front wheels 20. By changing the steering angle of the front wheels 20, the traveling direction of the mobile body 100 can be changed. The drive mechanism 25 is a mechanism that uses the motor 23 as a drive source to rotate the pair of rear wheels 21a, 21b. By rotating the pair of rear wheels 21a, 21b, the mobile body 100 can move forward or backward.

[0016] The vehicle 100 is an autonomous electric vehicle that uses a battery 106 (described later) as its main power source. The propulsion unit 12 is a tricycle equipped with a front wheel 20 and a pair of rear wheels 21a, 21b. The propulsion unit 12 may be in other forms, such as a four-wheeled vehicle. The vehicle 100 may also be provided with a seat (not shown).

[0017] The storage unit 26 indicates a space in which the user's luggage and the like can be loaded. When vein authentication is performed by the vein sensor 107 (described later) and user settings are made, the door (not shown) of the storage unit 26 is unlocked, allowing the user to load luggage. Thereafter, the door is locked after a predetermined time has elapsed or when the set user leaves the mobile object 100. When user vein authentication is performed again, the door can be unlocked. Therefore, the vein information of the set user is stored in a memory or the like provided in the mobile object 100.

[0018] As shown in FIG. 2(b), the mobile object 100 further includes a vein sensor 107, a detection unit 108, and an operation panel 109. The vein sensor 107 is provided facing downward below the detection unit 108 and is a sensor that detects the veins of a user's hand inserted into the detection range. The user can perform user settings for the mobile object 100 by inserting their hand below the vein sensor 107. User registration can also be performed by notifying the server 200 of the vein information of the user that has been acquired by the vein sensor 107. A user who has been registered as a user in the server 200 can use more driving modes of the mobile object 100.

[0019] The detection unit 108 is a 360-degree camera that can simultaneously capture a 360-degree image in the horizontal direction centered on the moving body 100. Note that this is not intended to limit the present embodiment, and for example, a camera may be used in which the detection unit 108 is provided so as to be rotatable in the horizontal direction, and a 360-degree image is captured by combining images captured in multiple directions. Alternatively, a configuration may be adopted in which multiple detection units are provided, each capturing images in a different direction, and each individual image is analyzed. The moving body 100 can detect targets such as people and objects around the moving body 100 by analyzing the 360-degree captured images captured by the detection unit 108.

[0020] The operation panel 109 is a touch panel type liquid crystal display having a display unit and an operation unit. In the present invention, the display unit and the operation unit may be provided separately. The operation panel 109 displays various information such as a setting screen for setting the drive mode of the moving object 100 and map information for conveying the current location information to the user.

[0021] <Detailed system configuration> The detailed configuration of each device included in this system will be described with reference to Figure 3. Here, the configuration of each device will be described, but the configuration necessary for explaining the present invention will be mainly described, and explanation of other configurations will be omitted. In other words, the configuration of each device in the present invention is not limited to the configuration described below, and additional or alternative configurations are not excluded.

[0022] The server 200 is an information processing device such as a personal computer, and includes a control unit 210, a storage unit 220, and a communication unit 230. The control unit 210 includes a registration and authentication unit 211 and a monitoring unit 212. The control unit 210 performs various processes by reading and executing control programs stored in the storage unit 220. In addition to the control programs, the storage unit 220 stores various data, setting values, user registration information, and the like. The user registration information includes authentication information, including the user's vein information and characteristic information. The communication unit 230 controls communication with the mobile object 100 via the network 300.

[0023] The registration and authentication unit 211 registers users and authenticates pre-registered users. User registration may be performed via the mobile object 100 or via another device such as a smartphone or PC. When user registration is performed via the mobile object 100, authentication information is registered in association with the user's identification information, along with vein information acquired by the vein sensor 107 and user characteristic information extracted from an image acquired by the detection unit 108. The authentication information may also include information about a password set by the user. The identification information may include the user's name, registration number, or the like.

[0024] The monitoring unit 212 monitors multiple mobile objects 100 located in a predetermined area and controls the standby position and patrol area of ​​the mobile objects 100 according to the status of the facility where the mobile objects 100 are located. The status of the facility, such as the level of crowding, may be determined by acquiring captured images from each mobile object 100 and analyzing the images. In this case, for example, an image captured by a mobile object 100 in patrol mode is transmitted to the server 200 and used. The standby position is a predetermined location where a mobile object 100 without a user assigned to it stops. It should be noted that the mobile object 100 can temporarily stop at the standby position even if a user is assigned to it. For example, if the designated user enters a location where the mobile object 100 cannot accompany the designated user, the mobile object 100 can wait at a nearby standby position until the designated user re-authenticates. The patrol area refers to an area where a mobile object 100 without a designated user patrols in patrol mode, as described below. The monitoring unit 212 monitors the locations of the multiple mobile objects 100 and, for example, moves a mobile object 100 that is waiting in an area with few people to an area with many people. This makes it possible to provide a more convenient system. The monitoring unit 212 may also grasp the remaining battery power of each mobile object 100 and schedule charging of each mobile object 100 so that charging can be performed efficiently at the charging station.

[0025] 2, the mobile object 100 includes a control unit 101, a microphone 102, a speaker 103, a GNSS 104, a communication unit 105, a battery 106, and a storage unit 110. These components, the motors 22 and 23, the vein sensor 107, the detection unit 108, and the operation panel 109 are connected to each other via a system bus or the like so that signals can be transmitted between them. Note that, in the following, a description of the components already described using FIG. 2 will be omitted.

[0026] A control unit 101 such as an ECU (Electronic Control Unit) controls each device connected via a signal line. The control unit 101 performs various processes by reading and executing programs stored in a storage unit 110. In addition to the control programs, the storage unit 110 has an area for storing various data, setting values, etc., and a work area for the control unit 101. Note that the storage unit 110 does not need to be configured as a single device, and can be configured to include at least one memory device such as a ROM, a RAM, a HDD, or a SSD.

[0027] The operation panel 109 is a device having an operation unit and a display unit, and may be realized by, for example, a touch panel type liquid crystal display. Alternatively, the operation unit and the display unit may be provided separately. The operation panel 109 displays various operation screens, map information, notification information for the user, inquiry information, and the like. In addition to the operation panel 109, the mobile object 100 can also communicate with the user via the microphone 102 and speaker 103.

[0028] A GNSS (Global Navigation Satellite system) 104 receives GNSS signals to detect the current position of the mobile object 100. A communication unit 105 accesses a network 300 via access points 301 and 302 and communicates bidirectionally with a server 200, which is an external device. A battery 106 is a secondary battery such as a lithium-ion battery, and the mobile object 100 can propel itself using the propulsion unit with the power supplied from the battery 106. The power from the battery 106 is also supplied to each load.

[0029] The control configuration of the control unit 101 will be described. The control unit 101 includes, as its control configuration, a voice recognition unit 121, a dialogue unit 122, an image analysis unit 123, a user setting unit 124, a position determination unit 125, a trajectory generation unit 126, and a travel control unit 127. The voice recognition unit 121 receives ambient sounds of the moving object 100 via the microphone 102, and recognizes and interprets, for example, voice from the user. The dialogue unit 122 generates questions and answers when communicating with the user via voice, and outputs the questions and answers via the speaker 103. Note that, with regard to the communication with the user, the user's conversation recognized by voice, as well as answers, questions, warnings, and the like from the moving object 100 may be displayed on the operation panel 109 in accordance with the voice output or voice recognition.

[0030] The image analysis unit 123 analyzes images captured by the 360-degree camera, which is the detection unit 108. Specifically, the image analysis unit 123 recognizes targets, including people and objects, from the captured images, and extracts user characteristics by analyzing the images. User characteristics include various features such as the color of clothes, luggage, and behavioral habits.

[0031] The user setting unit 124 sets a user who uses the mobile object 100. Specifically, the user setting unit 124 sets the user by storing the vein information of the user acquired by the vein sensor 107 in the storage unit 110. The user setting unit 124 may also store characteristic information of the set user extracted by the image analysis unit 123 in association with the vein information. The vein information and characteristic information stored in the storage unit 110 are used to reconfirm the user when, for example, the user is lost after being set by the user setting unit 124. Here, "lost" refers to losing sight of the set user for a predetermined period of time or longer. When the user is lost, the mobile object 100 moves to, for example, a nearby location where it can stop, temporarily stops, and waits until the set user is confirmed by the vein sensor 107 or the detection unit 108.

[0032] The position determination unit 125 determines a position relative to the set user as a position at which the moving body 100 will travel. For example, when following a user, the position determination unit 125 determines at what position relative to the set user the moving body 100 will be followed, according to information on the user's movement and the surrounding environment. The following position is preferably a position where the user can easily confirm the moving body 100 and where there is a low possibility of contacting obstacles, including surrounding people. Details of the control for determining the following position will be described later.

[0033] The trajectory generation unit 126 generates a trajectory for the moving body 100 to move in accordance with the current drive mode of the moving body 100. The trajectory generated here is not a trajectory to the destination, but a trajectory of a short distance, such as 5 m. Therefore, the trajectory generation unit 126 repeatedly generates trajectories until the destination is reached or the user stops. Furthermore, if the user deviates from the trajectory, the trajectory generation unit 126 corrects the generated trajectory in accordance with the user's movement. Furthermore, the trajectory generation unit 126 predicts the movement of the set user based on the results of analysis of the captured image of the detection unit 108 by the image analysis unit 123, and generates a trajectory so as to maintain the tracking position and avoid targets that may become obstacles. Details of trajectory generation will be described later.

[0034] The travel control unit 127 controls the travel of the mobile object 100 so as to maintain the following position according to the trajectory generated by the trajectory generation unit 126. Specifically, the travel control unit 127 moves the mobile object 100 along the generated trajectory, and controls the movement while adjusting the positional relationship with the set user using the captured image of the detection unit 108. For example, the speed may be increased if the distance from the set user exceeds a predetermined distance, or if the set user deviates to the left from the trajectory, the mobile object 100 may similarly move to the left to maintain the following position. As described above, if the trajectory is completely deviated from the trajectory, the trajectory is regenerated.

[0035] <Drive mode> Next, the drive modes of the moving body 100 according to this embodiment will be described with reference to Fig. 4. Table 400 shown in Fig. 4 shows the drive modes and their characteristics of the moving body 100. The drive modes described below are merely examples, and are not intended to exclude other drive modes.

[0036] The mobile body 100 includes at least one driving mode, such as a leading mode, a following mode, a guidance mode, a delivery mode, a patrol mode, and an emergency mode. The leading mode is a mode in which the mobile body 100 is controlled to travel ahead of the user at a speed that matches the user's travel speed when no destination is set. The following mode is a mode in which the mobile body 100 is controlled to travel behind the user at a speed that matches the user's travel speed when no destination is set. The guidance mode is a mode in which the mobile body 100 is controlled to travel ahead of the user at a predetermined speed or the user's travel speed toward the destination when a destination is set by the user.

[0037] The delivery mode is a mode in which the mobile object 100 is controlled to travel at high speed toward a destination set by a user. The delivery mode is a mode in which any package is loaded into the storage unit 26 and delivered to the destination. The patrol mode is a mode in which the mobile object 100 is controlled to travel at low speed toward a predetermined station (a waiting station or a charging station). The patrol mode is a mode in which no user is set, and the mobile object 100 searches for a user. In the patrol mode, the mobile object 100 travels while monitoring the surrounding environment with the detection unit 108. For example, if the mobile object 100 detects a person approaching the mobile object 100 with their hand raised, the mobile object 100 determines that the user wishes to use the mobile object 100, and moves in front of the user and stops. The emergency mode is a mode in which the mobile object 100 is controlled to travel at high speed toward a predetermined station. The emergency mode is executed, for example, to move to a charging station when the charge level of the battery 106 falls below a predetermined value, or to deliver luggage to a lost and found station where lost items are kept if the set user forgets their luggage.

[0038] <System Operation Overview> Next, an overview of the operation of this system will be described with reference to FIG. 5. Multiple mobile objects 100 managed by server 200 are placed in various facilities such as shopping malls, parks, train stations, airports, and parking lots. Here, a case where multiple mobile objects 100 are placed in a shopping mall will be described as an example. A shopping mall has various bases such as parking lots, shops, restaurants, and restrooms. In addition, according to this system, stations such as a waiting station where mobile objects 100 wait, a charging station where mobile objects 100 are charged, and a lost and found station where items left behind by users of mobile objects 100 are returned are also installed. In this system, various services are provided to users in such facilities depending on the driving mode of the mobile objects 100.

[0039] For example, as shown in Fig. 5, mobile object 100a provides a follow-up service to set user A. Mobile object 100b provides a leading service or a guidance service to set user B. Mobile object 100c is temporarily stopped to wait for a set user (not shown) who has entered a nearby shop. For example, mobile object 100 stores map information of a shopping mall in storage unit 110, and waits at any nearby location if the set user enters a no-entry area set in the map information.

[0040] In such facilities, in addition to the set user, there are many targets that obstruct the travel of the mobile body 100. For example, obstacles may include a person crossing in front of the set user, a person passing by the set user, or a sign in front of a shop. The system according to this embodiment provides various services while avoiding such target obstacles. Among the various services provided by this system, services in the follow mode will be described below.

[0041] <Processing flow> The processing flow of the follow-up mode in the moving body according to this embodiment will be described below with reference to FIGS.

[0042] (Follow-up mode) First, a processing procedure in the tracking mode of the moving object 100 according to this embodiment will be described with reference to Fig. 6. The processing described below is realized by the CPU of the control unit 101 reading out a control program stored in the storage unit 110 into the RAM and executing it.

[0043] First, in S101, the control unit 101 sets a user. In a mobile object 100 that is patrolling or stopped and has no set user, the detection unit 108 constantly analyzes images of the surrounding environment. In such a situation, if the control unit 101 detects, for example, a person approaching with their hand raised, the control unit 101 approaches the target person and stops the mobile object 100 using the travel control unit 127. Thereafter, when the person inserts their hand into the detection range of the vein sensor 107, vein information is acquired, and the control unit 101 stores the acquired vein information in the storage unit 110 using the user setting unit 124, and sets the user as a set user.

[0044] Next, in S102, the control unit 101 displays a mode selection screen on the operation panel 109 for the user to select a drive mode for the moving object 100. In addition, in S103, the control unit 101 causes the detection unit 108 to capture an image of the user to extract feature points of the user using the image analysis unit 123. The extracted features include various features such as the color of clothes, luggage, and behavioral habits. These features are used to constantly recognize the user when following the user, etc. Note that S102 and S103 do not need to be executed in this processing order, and may be executed in the reverse order or in parallel.

[0045] Next, in S104, the control unit 101 starts controlling the movement of the moving object 100 in a drive mode selected by the user via the mode selection screen displayed on the operation panel 109. In this embodiment, a case where the tracking mode is selected will be described. When the movement control in the tracking mode is started, the control unit 101 starts monitoring the movement of the set user in accordance with the analysis results of the captured image by the image analysis unit 123. Specifically, the control unit 101 monitors at least the user's current position and orientation, and predicts the user's subsequent movement direction and movement speed. The current position may be acquired as a relative position, for example, based on the distance from the moving object 100. The user's orientation is determined, for example, by the orientation of the body (torso). This is because humans do not always face or gaze in the direction of movement, and the torso is more likely to be facing the direction of movement than the face or gaze. On the other hand, if the orientation of the torso cannot be detected, the orientation of the face may be used. The user's movement speed can be acquired based on time-series data regarding the user's current position.

[0046] Next, in S105, the control unit 101 executes a trajectory generation process for the moving body 100 using the position determination unit 125 and the trajectory generation unit 126. In the trajectory generation process, the position of the moving body 100 relative to the user (here, the following position) is determined, and further a trajectory indicating the movement route of the moving body 100 is generated. Details of the trajectory generation process will be described later with reference to FIG. 7. Subsequently, in S106, the control unit 101 controls the traveling of the moving body 100 according to the generated trajectory while maintaining the moving body 100 at the following position determined in S105 using the traveling control unit 127. The traveling control unit 127 controls the steering and speed of the moving body 100 according to the following position and trajectory.

[0047] Next, in S107, the control unit 101 determines whether or not trajectory regeneration is necessary during travel. If it is determined that trajectory regeneration is necessary, the process returns to S105; otherwise, the process proceeds to S108. Trajectory regeneration is necessary in two cases: a case in which the user corrects the trajectory when it deviates from the user's predicted trajectory; and a case in which the next trajectory is generated when the user approaches within a predetermined distance from the end point of the trajectory of the moving object 100 generated in S105. The user's predicted trajectory is a trajectory predicted based on the current position of the set user, not the following position, in the trajectory generated in S105. In other words, the control unit 101 does not individually generate the user's predicted trajectory and the trajectory of the moving object 100 in S105, but generates each trajectory based on the generated basic trajectory and the following position or the user's current position. The control unit 101 determines that the set user has deviated from the predicted trajectory when the user deviates from the predicted trajectory by more than a predetermined distance.

[0048] In S108, the control unit 101 determines whether the current driving mode (here, the tracking mode) has been ended by the setting user. For example, the user can issue an instruction to end the tracking mode via the operation panel 109. Alternatively, the user can issue an instruction to end the tracking mode by voice via the microphone 102. If it is determined that the mode has ended, the process of this flowchart ends; if not, the process returns to S106.

[0049] (Trajectory generation processing) Next, a detailed processing procedure of the trajectory generation process (S105) in the tracking mode of the moving body 100 according to this embodiment will be described with reference to Fig. 7. The processing described below is realized by the CPU of the control unit 101 reading out a control program stored in the storage unit 110 into the RAM and executing it.

[0050] First, in S201, the control unit 101 predicts the movement (trajectory) of the set user from the image captured by the detection unit 108. Specifically, the control unit 101 acquires the movement direction and movement speed of the set user from the time-series data based on the monitoring of the set user's current position and torso orientation started in S104 above. Furthermore, the control unit 101 predicts the set user's future trajectory based on the acquired movement direction, movement speed, and current position. In other words, here, the predicted trajectory of the set user described above is generated.

[0051] Next, in S202, the control unit 101 acquires user characteristic information as information to be used for tracking the user. For example, here, information such as the characteristics of the user's movements and information about items the user is carrying is acquired as needed. The acquired information is used to constantly recognize the user when tracking the set user and to determine the user's tracking position. The user's movement characteristics include, for example, characteristics related to the user's turning-around movement and characteristics of the movements of both arms. Regarding the user's turning-around movement, characteristic information is acquired as to whether the preceding set user, when visually recognizing the moving object 100, tends to turn around from the right or left side. This information is used to determine the user's tracking position. Regarding the movements of both arms, characteristic information is acquired as to which of the left and right arms moves more frequently. Regarding luggage, position information of the luggage, such as whether the luggage is being held in the left or right hand or whether it is being carried on the back, is acquired. The characteristic information of the movements of both arms and the position information of the luggage are used to recognize the user and to determine the user's tracking position.

[0052] Next, in S203, the control unit 101 acquires surrounding environment information from the captured image by the detection unit 108. The surrounding environment information is information about targets around the set user and the moving body 100 included in the captured image. That is, the control unit 101 extracts targets within a predetermined range from the captured image and, if the extracted targets are moving, also predicts their movement. The predetermined range is preferably, for example, the area in front of the set user. This is because, in order to generate a trajectory using the surrounding environment information, information about the area in which the set user is likely to move is required. For example, if the surrounding environment information is acquired about a person (target) crossing the area in front of the user from right to left, if the current tracking position for following the set user is to the rear left, it is recognized that there is a possibility of contact with a person crossing from right to left at the destination. Therefore, to avoid such contact, the tracking position of the moving body 100 can be corrected from the rear left to the rear right. When changing the tracking position, it is preferable to notify the set user of this by voice via the speaker 103.

[0053] Next, in S204, the control unit 101 determines whether the predetermined range is congested based on the acquired ambient environment information. For example, the determination here may be made if the number of targets recognized in the ambient environment information is greater than or equal to a predetermined number. Alternatively, the determination may be made if targets that may be in contact with the predetermined range ahead of the set user are detected on both the left and right sides. If it is determined that the range is congested, the process proceeds to S206; otherwise, the process proceeds to S205. In S206, the control unit 101 determines the tracking position of the moving object 100 relative to the set user to be between the left rear and right rear of the set user (second position), and proceeds to S207.

[0054] Meanwhile, in S205, the control unit 101 determines the tracking position of the moving object 100 relative to the set user to be a diagonally rearward position (first position), and proceeds to S207. Furthermore, here, whether the tracking position is the left rear or the right rear is determined based on various information. The control unit 101 determines the tracking position based on multiple criteria, such as which of the left rear and right rear positions is easier to recognize the user, which is less likely to come into contact with obstacles such as other targets at the destination, and which is more preferable for the user. The position that is easier to recognize the user may be selected, for example, as a position with many user feature points or a position where the user moves more frequently in a predetermined period of time. The position that is less likely to come into contact with obstacles such as other targets at the destination may be selected, for example, as a position with a larger area of ​​free space without targets in a predetermined range ahead. Furthermore, the position that is more preferable for the user may be selected as a position where the user looks back more frequently in a predetermined period of time.

[0055] In S207, the control unit 101 generates a trajectory of the moving body 100 based on the predicted trajectory of the set user. Specifically, the control unit 101 generates a trajectory of the moving body 100 based on the predicted trajectory and the determined follow-up position. Once the trajectory is generated, the process of this flowchart ends, and the process proceeds to S106 in FIG. 6.

[0056] <Following position> 8 is a diagram showing the following position of the moving body 100 according to this embodiment. When following set user A, the moving body 100 sets a predetermined position behind set user A as the following position.

[0057] 8, the predetermined rear position is divided into three regions by dashed lines centered on the set user A. That is, the following position is determined to be one of three positions: a position (first position) diagonally rearward including a left rear 801 and a right rear 803 of the user, and a position (second position) 802 directly behind the user.

[0058] As described above, the tracking position is basically adjusted between the left rear 801 and right rear 803 of the first position diagonally rearward. This is to position the moving body 100 at a position that is easier for the user to see when looking back. On the other hand, the position 802 directly behind is a tracking position that is used when the predetermined range 800 in front of the user is crowded. In this case, the moving body 100 moves to a position that is hidden by the set user A, thereby reducing the possibility of contact with other targets.

[0059] To acquire the ambient environment information for the predetermined range 800, images captured by the detection unit 108 of the moving body 100a that follows the set user A are used. Therefore, when the detection unit 108 of the moving body 100a acquires the ambient environment information for the predetermined range 800, there is a possibility that the set user A will become an obstacle and accurate information will not be acquired. Therefore, the moving body 100 according to this embodiment may move left and right to capture images as shown by arrow 804 in order to accurately capture the entire area of ​​the predetermined range 800 when acquiring the ambient environment information.

[0060] <Example of operation> Next, an example of the operation of the tracking position according to this embodiment will be described with reference to Fig. 9 and Fig. 10. First, with reference to Fig. 9, a case in which another target passes in front of setting user A will be described.

[0061] As shown in FIG. 9(a), the moving body 100a is following the left rear of the set user A. Meanwhile, in a predetermined range 900 in front of the set user A, targets X and Y, which are people, are moving in the direction of arrow 901. The control unit 101 can acquire the above-described surrounding environment information by analyzing images captured by the detection unit 108 using the image analysis unit 123. Note that if the set user A is an obstacle and part of the predetermined range 900 cannot be captured, the moving body 100a may move left and right as shown by arrow 902 to capture images. In such a situation, if the moving body 100a continues to follow the left rear, there is a possibility that the moving body 100a will come into contact with targets X and Y several meters ahead. Therefore, the moving body 100 changes its following position to reduce the possibility of such contact.

[0062] 9(b), the moving body 100 switches the following position to the right rear of the set user A and moves while traveling as indicated by arrow 911. As a result, when the moving body 100 passes the targets X and Y, it is moving to the right rear of the set user A, thereby reducing the possibility of contact.

[0063] FIG. 10 shows a situation in which a predetermined area ahead of set user A is congested. In this case, the moving body 100a travels with a position directly behind set user A as the follow-up position, as shown in FIG. 10. This allows the moving body 100a to avoid contact with other targets, even when passing through a congested area, by using set user A as a wall. Furthermore, when the moving body 100 is positioned directly behind set user A, it is desirable to adjust the distance to be shorter from set user A compared to when the moving body 100 is positioned diagonally behind. This further enables contact with other targets to be avoided.

[0064] <Summary of the embodiment> The above embodiment discloses at least the following embodiments.

[0065] 1. The moving body (100) of the above embodiment is A sensor (108) for detecting surrounding targets; setting means (101, 124) for recognizing and setting a user based on the output from the sensor; a trajectory generating means (125, 126, S104) for generating a trajectory of the moving object so as to follow the user at a first position diagonally behind the user according to the user movement set by the setting means and the output from the sensor; a travel control means (127) for causing the moving body to travel along the generated trajectory; Equipped with.

[0066] According to this embodiment, it is possible to control the travel of the mobile body at a more appropriate tracking position for the user in accordance with surrounding targets including the user.

[0067] 2. In the above embodiment, the first position is to the left or right rear of the moving user (801, 803).

[0068] According to this embodiment, the position of the moving body can be adjusted to a position that is preferable for the user and that avoids contact with other targets.

[0069] 3. In the above embodiment, when the number of targets detected by the sensor in the surrounding environment in which the user is moving exceeds a predetermined number, the trajectory generation means generates a trajectory to follow at a second position (802) between the left rear and right rear first positions instead of the first position.

[0070] According to this embodiment, in a crowded surrounding environment, it is possible to avoid contact with other targets by using the preceding user as a wall.

[0071] 4. In the above embodiment, the second location is closer to the user than the first location (802).

[0072] According to this embodiment, it is possible to more safely avoid contact with other targets in crowded surrounding environments.

[0073] 5. In the above embodiment, the trajectory generating means generates a trajectory to be followed at a position where the estimated area of ​​the free space detected by the sensor is the largest, either at the left rear or the right rear (S205).

[0074] According to this embodiment, it is possible to select a destination with fewer obstacles, and to avoid contact with other targets.

[0075] 6. In the above embodiment, the trajectory generation means generates a trajectory that follows the first position at either the left rear or right rear of the user, whichever rear position has the most recognizable feature points of the user (S205).

[0076] According to this embodiment, tracking can be performed at a position where the user can be more easily recognized.

[0077] 7. In the above embodiment, the trajectory generation means generates a trajectory as the first position that follows the position behind the arm of the user that is recognized as having the greater movement in a given period of time, either the left rear or right rear of the user (S205).

[0078] According to this embodiment, tracking can be performed at a position where the user can be more easily recognized.

[0079] 8. In the above embodiment, the trajectory generation means generates a trajectory as the first position that follows the position behind the user, either to the left or right, whichever position is recognized as being behind the user at which the user looks back more frequently in a specified period of time (S205).

[0080] According to this embodiment, tracking can be performed at a position more preferable to the user.

[0081] 9. In the above embodiment, the travel control means further adjusts the first position in accordance with the output from the sensor and controls the travel of the moving body (S106).

[0082] According to this embodiment, deviations from the user's predicted trajectory can be easily accommodated.

[0083] 10. In the above embodiment, the vehicle control system further includes a setting means (109, S102) for setting a travel mode related to trajectory control of the vehicle in accordance with a user input; When the setting means sets the following mode, the trajectory generating means generates a trajectory that follows the user.

[0084] According to this embodiment, various driving modes for the moving body can be provided.

[0085] 11. In the above embodiment, the sensor is a camera capable of capturing images horizontally over 360 degrees (108).

[0086] According to this embodiment, it is possible to acquire a wider range of surrounding images at once, and it is possible to reduce the processing load in tracking control that requires real-time control.

[0087] 12. In the above embodiment, the control method for a moving body (100) equipped with a sensor (108) for detecting surrounding targets includes: A setting step (S101) of recognizing and setting a user based on an output from the sensor; a trajectory generation step (S105) of generating a trajectory of the moving object so that the moving object follows a first position diagonally behind the user according to the movement of the user set in the setting step and an output from the sensor; a travel control step (S107) of causing the moving body to travel along the generated trajectory; Includes.

[0088] According to this embodiment, it is possible to control the travel of the mobile body at a more appropriate tracking position for the user in accordance with surrounding targets including the user.

[0089] Although the embodiments of the invention have been described above, the invention is not limited to the above-described embodiments, and various modifications and variations are possible within the scope of the gist of the invention. [Explanation of symbols]

[0090] 100a, 100b, 100c: mobile units, 200: server, 300: network

Claims

1. A mobile object, a sensor for detecting surrounding targets; a setting means for recognizing and setting a user based on an output from the sensor; a trajectory generating means for generating a trajectory of the moving object so as to follow the user at a first position diagonally behind the user, in accordance with the movement of the user set by the setting means and an output from the sensor; a travel control means for causing the moving body to travel along the generated trajectory; Equipped with the first position is located to the left or right rear of the user who is moving, The trajectory generation means generates a trajectory that follows a second position between the left rear and the right rear first position instead of the first position when the number of targets detected by the sensor in the surrounding environment in which the user is moving exceeds a predetermined number.

2. The moving body according to claim 1 , wherein the second location is closer to the user than the first location.

3. 3. The moving body according to claim 1, wherein the trajectory generating means generates a trajectory that follows a position at which the estimated area of ​​the free space detected by the sensor is largest, either the left rear or the right rear.

4. The moving body according to claim 1 or 2, characterized in that the trajectory generation means generates a trajectory that follows the first position at a rear position between the left rear and right rear of the user, whichever is rearward and where many recognizable feature points of the user are present.

5. The moving body according to claim 1 or 2, characterized in that the trajectory generation means generates a trajectory that follows the first position at a position behind the arm of the user that is recognized as having more movement in a given period of time, either at the left rear or right rear of the user.

6. The moving body according to claim 1 or 2, characterized in that the trajectory generation means generates a trajectory that follows the first position at either the left rear or right rear of the user, whichever position is recognized as being behind the user at which the user looks back more frequently in a given period of time.

7. 7. The moving body according to claim 1, wherein the travel control means further adjusts the first position in accordance with the output from the sensor and controls the travel of the moving body.

8. a setting unit for setting a running mode relating to trajectory control of the moving object in accordance with a user input; 8. The moving body according to claim 1, wherein the trajectory generating means generates a trajectory that follows the user when the following mode is set by the setting means.

9. 9. The moving body according to claim 1, wherein the sensor is a camera capable of capturing images over 360 degrees in the horizontal direction.

10. A method for controlling a moving body equipped with a sensor for detecting surrounding targets, comprising: a setting step of recognizing and setting a user based on an output from the sensor; a trajectory generation step of generating a trajectory of the moving object so as to follow the user at a first position diagonally behind the user, in accordance with the movement of the user set in the setting step and an output from the sensor; a travel control step of causing the moving body to travel along the generated trajectory; Including, the first position is located to the left or right rear of the user who is moving, In the trajectory generation step, when the number of targets detected by the sensor in the surrounding environment in which the user is moving exceeds a predetermined number, a trajectory is generated that follows a second position between the left rear and the right rear first positions instead of the first position.

Images