MOBILE BODY CONTROL DEVICE, MOBILE BODY CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM

Abstract

To provide a technique capable of more appropriately assisting the movement of a person in a site.SOLUTION: A mobile body control device comprises: acquisition means that acquires sensor information, including a captured image obtained by capturing the periphery of a mobile body, for recognizing an object around the mobile body; setting means that sets one control mode of a plurality of control modes; and control means that controls traveling of the mobile body on the basis of the set control mode and a result of detection of a specific person based on the sensor information. The control means controls the traveling of the mobile body so that the mobile body travels on the basis of the traveling speed of the mobile body, generation of a traveling trajectory of the mobile body, and a positional relationship between the specific person and the mobile body, corresponding to the control mode.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control device for a mobile object, a control method for a mobile object, a program, and a storage medium. [Background technology]

[0002] In recent years, robots that transport users' luggage and self-propelled vehicles that lead users through airports have become known (Patent Documents 1 and 2). Patent Document 1 proposes a technology in which a transport robot carrying a user's luggage follows the user while maintaining an appropriate distance from the user. Patent Document 2 proposes a technology in which a self-propelled vehicle derives a travel route to a boarding location where the user will board an aircraft, and leads the user to the boarding location along the travel route while ensuring that the distance between the user and the self-propelled vehicle does not exceed a predetermined distance. Furthermore, Patent Document 3 discloses a technology in which a moving body moves to an appropriate position, such as in front of the user or diagonally forward with respect to the user, to follow the user. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-64214 [Patent Document 2] Patent Publication No. 2021-22108 [Patent Document 3] International Publication WO2017 / 115548 Summary of the Invention [Problem to be solved by the invention]

[0004] Meanwhile, many different people move around large areas of amusement facilities, large commercial facilities, airports, parks, parking lots, etc. Some people want to move around without having to carry their luggage, while others want to be guided to their desired location, and still others want someone to move ahead of them to reduce the burden of moving through a crowd.

[0005] There is a growing demand for mobile bodies such as mobile robots that assist people in their mobility to be able to support the various mobility modes described above, as well as to provide mobility support with improved ease of use, safety, and the like.

[0006] The present invention has been made in view of the above-mentioned problems, and its purpose is to realize a technology that can more appropriately support the movement of people within a premises. [Means for solving the problem]

[0007] According to the present invention, A control device for a moving body, an acquisition means for acquiring sensor information including an image of the surroundings of the moving body, for recognizing objects around the moving body; a setting means for setting one of a plurality of control modes; a control unit that controls the travel of the moving object based on a set control mode and a detection result of a specific person based on the sensor information, The control means controls the traveling of the moving body so that the moving body travels at a traveling speed of the moving body, a traveling trajectory of the moving body, and a positional relationship between the specific person and the moving body, according to a control mode. death, the plurality of control modes include a first mode in which the moving object travels ahead of the specific person based on a behavior prediction for the specific person without a destination specified by the specific person, When the first mode is set, the control means controls the traveling of the moving body so that the moving body travels ahead of the specific person based on a behavior prediction for the specific person without a destination specified by the specific person. A control device for a moving object is provided. [Effects of the Invention]

[0008] According to the present invention, it is possible to more appropriately support the movement of people within a premises. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a mobility assistance system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of the external configuration of a robot as an example of a moving body according to an embodiment; [Figure 3A] FIG. 1 is a block diagram illustrating an example of the functional configuration of a moving body according to an embodiment; [Figure 3B] FIG. 1 is a block diagram showing an example of the functional configuration of a server as an example of an information processing apparatus according to an embodiment; [Figure 4A] FIG. 1 is a diagram showing an example of transition of a control mode of a moving body according to an embodiment; [Figure 4B] FIG. 1 is a diagram showing an example of using a mobile object in a large commercial facility according to an embodiment. [Figure 4C] 1 is a diagram illustrating several control modes of a moving object according to an embodiment; [Figure 5A] Table (1) explaining the characteristics of each control mode of the moving body according to the embodiment [Figure 5B] Table (2) explaining the characteristics of each control mode of the moving body according to the embodiment [Figure 6A] 1 is a flowchart illustrating an example of a process related to state control of a moving body according to an embodiment; [Figure 6B] 1 is a flowchart illustrating an example of a process related to a leading mode of a moving object according to an embodiment; [Figure 6C] 1 is a flowchart illustrating an example of a process related to a moving object tracking mode according to an embodiment; [Figure 6D] 1 is a flowchart illustrating an example of a process related to a guidance mode of a moving body according to an embodiment; [Figure 6E] 1 is a flowchart illustrating an example of a process related to a delivery mode of a mobile object according to an embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Furthermore, the same reference numerals are used for the same or similar components, and redundant explanations will be omitted.

[0011] <Configuration of mobility support system> The configuration of a mobility assistance system 10 according to this embodiment will be described with reference to Fig. 1. The mobile object 100 is, for example, a robot capable of autonomous travel. For example, the mobile object 100 is equipped with a battery and moves mainly by motor power. The mobile object 100 travels within the premises of an amusement facility, a large commercial facility, an airport, a park, a sidewalk, a parking lot, or the like.

[0012] The mobile object 100 can guide a specific person (also referred to as a user) to a specific location in a space, store the user's luggage in a housing and follow the user, or deliver luggage from a specific location in the space to the user's location (or from the user's location to a specific location). Within the space, multiple mobile objects 100 operate autonomously. Note that when describing the mobile objects in a manner that distinguishes them from one another, they will be described with different reference numerals such as 100a, 100b, etc. However, when there is no need to distinguish between the individual mobile objects, they will be described simply as mobile objects 100.

[0013] In this embodiment, an example is described in which no person rides on the moving body 100, but other people may ride on the moving body when it runs alongside a walking user. Also, in this embodiment, an example is described in which the moving body moves by driving wheels, but other autonomously moving moving bodies (e.g., walking robots) that can walk on two or more legs may also be included.

[0014] The mobile object 100 can connect to the network 130 via wireless communication, such as fifth-generation mobile communication, wireless LAN, or communication between mobile objects. In response to instructions from the management server 120, the mobile object 100 autonomously moves between stations (described later) or moves to a location specified by the user 110. The mobile object 100 can measure the internal and external conditions of the mobile object (such as the mobile object's position, running state, and surrounding object targets) using various sensors (described later) and accumulate the measured data. The mobile object 100 may transmit at least a portion of the accumulated data to the management server 120. When information about the mobile object 100 is transmitted to the management server 120, the information about the mobile object 100 is transmitted at regular intervals or in response to the occurrence of a specific event.

[0015] The communication terminal 140 is, for example, a smartphone, but is not limited to this and may be an earphone-type communication terminal, a personal computer, a tablet terminal, a game console, smart glasses, a smart watch, etc. The communication terminal 140 connects to the network 130 via wireless communication such as fifth-generation mobile communication or wireless LAN.

[0016] The network 130 includes a communication network such as the Internet or a mobile phone network, and transmits information between the mobile object 100, the management server 120, the communication terminal 140, and the like.

[0017] <External configuration of the moving object> Next, an example of the external configuration of the moving body according to this embodiment will be described with reference to Fig. 2. In Fig. 2, arrow X indicates the front-to-rear direction of moving body 100, with F indicating the front and R indicating the rear. Arrows Y and Z indicate the width direction (left-to-right direction) and up-down direction of moving body 100.

[0018] The moving body 100 includes a pair of front wheels 201 and a rear wheel 202, which are included in a propulsion unit 304 described below. The propulsion unit 304 may have other forms, such as a four-wheeled vehicle or a two-wheeled vehicle.

[0019] The mobile object 100 has a housing 210 capable of storing luggage. A front surface 211 of the housing is provided with an openable lid for storing luggage, and the lid is equipped with a locking mechanism. The locking mechanism is controlled by the mobile object 100. For example, the mobile object 100 releases the lock state when user authentication is successful. A monitor 220 equipped with a touch panel is disposed on the top surface 212 of the housing, and the user can, for example, select a control mode for the mobile object 100, specify a desired destination, and check information about facilities.

[0020] The sensor box 230 includes a detection unit 306 therein that generates data for recognizing objects and users present around the mobile object 100 through the front surface 231, side surface, etc. of the sensor box 230. A vein sensor is also disposed on the bottom surface 232 of the sensor box 230 for authenticating a user (specific person) who uses the mobile object 100. In this embodiment, the vein sensor is used to detect and recognize the feature amounts of veins in the user's palm, thereby identifying the user.

[0021] <Example of functional configuration of a mobile unit> Next, an example of the functional configuration of the moving body 100 of this embodiment will be described with reference to FIG. 3A.

[0022] The mobile body 100 is an electrically powered autonomous mobile body that includes a propulsion unit 304 and uses a battery 305 as its main power source. The battery 305 is a secondary battery such as a lithium ion battery, and the propulsion unit 304 uses the power supplied from the battery 305 to propel the mobile body 100 by itself.

[0023] The propulsion unit 304 includes a steering mechanism. The steering mechanism uses a first motor as a drive source to change the steering angle of the pair of front wheels 201. By changing the steering angle of the pair of front wheels 201, the traveling direction of the mobile body 100 can be changed. The propulsion unit 304 also includes a drive mechanism. The drive mechanism uses a second motor as a drive source to rotate the rear wheels 202. By rotating the rear wheels 202, the mobile body 100 can move forward or backward. The propulsion unit 304 can detect and output physical quantities that represent the motion of the mobile body 100, such as the traveling speed, acceleration, steering angle, and rotational acceleration of the body of the mobile body 100.

[0024] The mobile object 100 includes a detection unit 306. The detection unit 306 generates data for recognizing objects (including objects and people) present around the mobile object 100. The detection unit 306 includes sensors such as an imaging device, a radar device, a lidar (Light Detection and Ranging), and an ultrasonic sensor, each of which has a detection range covering the periphery of the mobile object 100, and outputs sensor information. The imaging device may be configured using a fisheye lens, or a set of multiple imaging devices capable of stereoscopic photography may be arranged facing multiple directions. Furthermore, the detection unit 306 includes a Global Navigation Satellite system (GNSS) sensor and receives GNSS signals to detect the current location of the mobile object 100. The detection unit 306 may also detect the current location using signals from a wireless LAN or Bluetooth (registered trademark).

[0025] The mobile object 100 includes a control unit (ECU) 301. The control unit 301 functions as a control device for the mobile object (mobile object). The control unit 301 includes one or more processors 302, such as a CPU, and a memory 303, which is a storage device such as a semiconductor memory. The memory 303 stores programs executed by the processor 302, data used in processing by the processor 302, and the like. A plurality of sets of processors 302 and memories 303 may be provided for different functions of the mobile object 100, and may be configured to be able to communicate with each other.

[0026] The control unit 301 acquires physical quantities representing the motion output by the propulsion unit 304, the detection results of the detection unit 306, input information from the operation panel 31, audio information input from the audio input device 307, and the like, and executes corresponding processing. For example, the control unit 301 controls the motor of the propulsion unit (controls the movement of the propulsion unit 304), controls the display on the operation panel 31, alerts nearby people by audio, and transmits information to the management server 120. The control unit 301 may further include, as a processor, a GPU, or dedicated hardware suitable for executing processing of a machine learning model such as a neural network, in addition to a CPU. In addition, the control unit 301 executes processing related to state control according to this embodiment, which will be described later.

[0027] The voice input device 307 collects voices around the mobile object 100. The control unit 301 can recognize the input voices and execute corresponding processing. The storage device 308 is a non-volatile large-capacity storage device that stores map information including information on routes that the mobile object 100 can travel, areas where entry is restricted, landmarks, stores, etc. The storage device 35 may also store programs executed by the processor 302, data used by the processor 302 for processing, etc. The storage device 308 may store various parameters of machine learning models for voice recognition and image recognition executed by the control unit 301 (for example, trained parameters and hyperparameters of a deep neural network, etc.).

[0028] The communication device 309 is a communication device that can connect to the network 130 via wireless communication such as fifth generation mobile communication or wireless LAN.

[0029] The presentation device 310 displays (presents) a user interface screen for the user on the monitor 220, and outputs (presents) speech to the surroundings of the mobile object 100 via a microphone. When the mobile object 100 has information to present to the user, the presentation device 310 may transmit the information to be presented to the communication terminal 140 carried by the user (specific person) via the communication device 309, instead of or in addition to outputting the information from the presentation device 310. The communication terminal 140 that receives the information to be presented can output the presentation information received, for example, via an application on the communication terminal 140. To achieve such presentation, for example, the user may pair their own communication terminal 140 with the mobile object 100 when starting to use the mobile object 100, or may set the communication terminal 140 to be capable of mutual communication via a network. The input device 311 may include, for example, a touch panel and be configured as an integral part of the monitor 220. The input device 311 accepts operational input from the user via the touch panel.

[0030] The user authentication unit 312 is equipped with a vein sensor that authenticates the user without contact, and when the user holds their palm over the bottom surface 232 of the sensor box 230, it extracts the feature amount of the veins in the user's palm to identify the user. It determines whether the feature amount matches the vein feature amount registered when the mobile object 100 was first used, and if the feature amount matches, it determines that the user is the same as the user registered when the mobile object 100 was first used.

[0031] The control unit 301 executes a program stored in the memory 303 or the storage device 308 to realize the functions of a user identification section 321 , a detection information processing section 322 , a voice information processing section 323 , and a mode control section 324 .

[0032] The detection information processing unit 322 recognizes objects (including a specific person (user) using the mobile object) present around the mobile object 100 based on information input from the detection unit 306. The detection information processing unit 322 includes a machine learning model that processes sensor information including images, and the trained machine learning model performs inference stage processing. The machine learning model of the detection information processing unit 322 performs processing to recognize objects from sensor information, for example, by performing calculations of a deep learning algorithm using a deep neural network (DNN). The objects may include users, other people, signs, road signs, equipment, building components such as windows and entrances, roads, mobile objects, motorcycles, etc., included in the image. In addition, the machine learning model of the detection information processing unit 322 can recognize human faces, human gestures, etc. included in image information. Note that, in this embodiment, an example is described in which the mobile object 100 processes sensor information, but the sensor information processing may be performed by an external server (not shown), and the recognition results may be received from the server.

[0033] The detection information processing unit 322 also recognizes the object's status, such as its position, speed, and acceleration. The object's position may be recognized as a relative position in the coordinate system of the mobile object 100, for example, and then converted to an absolute position in the coordinate system used for map information, as necessary. Furthermore, the detection information processing unit 322 calculates the predicted positions of surrounding objects by behavior prediction using a machine learning model (e.g., a deep learning algorithm) based on the recognition result of the current object's position. For example, the predicted position of the object is the predicted position of the object at each future time point from the current time t + Δt*n (n = 1, . . . 3). Here, increasing the value of n lengthens the prediction period, and decreasing the value of n shortens the prediction period. Note that a known method can be used to generate the traveling trajectory of the mobile object using the object's behavior prediction. For example, the traveling trajectory can be generated by evaluating the predicted positions of surrounding objects and a risk potential representing the degree of interference between the mobile object 100 and the objects.

[0034] The user identification unit 321 continuously determines whether the user can be detected from the sensor information. For example, if the user cannot be detected for a predetermined time since the last time the user was detected, the user is determined to be lost. Furthermore, if the user approaches the moving object 100 again, the user is determined to have been detected again from the sensor information.

[0035] The voice information processing unit 323 generates speech information according to the positional relationship between the mobile object 100 and surrounding objects (e.g., the user or other people). The voice information processing unit 323 includes a machine learning model that processes voice information and executes the inference stage processing of the machine learning model. The machine learning model of the voice information processing unit 323 performs calculations of a deep learning algorithm using, for example, a deep neural network (DNN) to recognize the content of a user's speech and generate speech information for people around the mobile object 100. Separate machine learning algorithms may be used for recognizing the content of a user's speech and generating the speech information.

[0036] In recognizing the content of a user's utterance, for example, if the user's utterance includes an instruction for a location, the place name, etc. included in the utterance information may be identified. In recognizing the content of a user's utterance, for example, the place name, landmark name such as a building, store name, object name, etc. included in the utterance information are recognized. The DNN becomes trained by performing the learning stage processing, and by inputting the utterance information into the trained DNN, recognition processing (inference stage processing) for the utterance information can be performed. Note that, in this embodiment, an example is described in which the mobile object 100 performs the speech recognition processing, but the speech recognition processing may also be performed by an external server (not shown), and the recognition result may be received from the server.

[0037] The mode control unit 324 executes processes relating to state control, which will be described later, and processes for each control mode (for example, a following mode, a guidance mode, etc.), which will be described later, to make the mobile object travel in accordance with the characteristics of the control mode.

[0038] <Configuration of information processing device> Furthermore, the configuration of the management server 120 as an example of an information processing device according to this embodiment will be described with reference to FIG. 3B. The management server 120 is configured with one or more server devices. Each of the functional blocks described may be integrated or separated, and the functions described may be realized by different blocks. What is described as hardware may be realized by software, and vice versa.

[0039] The control unit 351 includes one or more processors 352, such as a CPU, and a memory 353, which is a storage device such as a semiconductor memory. The memory 353 stores programs to be executed by the processor 352, data used for processing by the processor 352, and the like. The control unit 351 loads programs stored in the memory 353 or the storage unit 356 into the memory 353 and executes them using the processor, thereby controlling the operation of each unit within the control unit 351 and the operation of each unit of the management server 120.

[0040] The power supply unit 354 is a power supply that supplies power to each unit of the management server 120. The communication unit 355 includes a communication circuit that communicates with the communication terminal 140 and the mobile object 100 via the network 130. The storage unit 356 includes a non-volatile storage medium such as a semiconductor memory, and stores setting values ​​and programs required for the operation of the management server 120. The storage unit 356 stores information about the mobile objects received from multiple mobile objects via the communication unit 355. The information about the mobile objects includes, for example, data about the movement of the mobile objects, data about the remaining battery power, the position of the mobile objects, data about the current control mode, etc.

[0041] The mobile body data acquisition unit 371 acquires information about the mobile body from each of the multiple mobile bodies and stores it in the storage unit 356. The traveling data acquisition unit 371 associates the information about the mobile body with information that identifies the mobile body and stores it.

[0042] The reservation control unit 372 accepts a reservation from the communication terminal 140 to use the mobile object 100 at a specified location. The reservation control unit 372 identifies mobile objects 100 that are in an available state, such as an idle state described below, among the mobile objects 100 that exist near the specified location based on the location information of each mobile object 100. The reservation control unit 372 sets a reservation including the specified location for the identified mobile object 100. The reservation is, for example, periodically queried by the mobile object 100, and when a reservation is set, the mobile object 100 sets the control mode to a reserved state and moves to the specified location.

[0043] The reservation setting unit 371 receives a request to cancel a set reservation from the communication terminal 140. When a request to cancel a reservation is received, information regarding the cancellation of the reservation is transmitted to the mobile object 100, and the control mode of the mobile object 100 is changed to the patrol mode.

[0044] If there are multiple stations (described separately below) within the facility where the mobile objects 100 can be charged, the mobile object allocation unit 373 moves the mobile objects 100 between the multiple stations. The mobile object allocation unit 373 predicts the demand for mobile objects at each station (e.g., the demand ratio between stations) according to the number of people near each station, and adjusts the number of mobile objects 100 staying at each station according to the prediction results. Alternatively, if a certain number of mobile objects 100 are periodically moved between stations in a patrol mode, the mobile object allocation unit 373 calculates the density of the number of users in the area associated with the station, and reduces the number of mobile objects 100 traveling in a patrol mode in areas with a higher density.

[0045] The management information generation unit 374 generates a management screen that enables a system administrator to check the operating status of the mobile object 100, provide the location of the mobile object 100 in response to an inquiry from a user searching for the mobile object 100, and allocate tasks to the mobile object 100 (for example, manually set a reservation). The management information generation unit 374 uses various data about the mobile object 100 acquired by the mobile object data acquisition unit 371 or stored in the memory unit 356 to display information such as the remaining amount of power, location information, and the currently operating control mode on the management screen.

[0046] <Communication terminal configuration> The communication terminal 140 of this embodiment includes a processor and a memory. In accordance with a user input entered via an operation unit, the communication terminal 140 calls the mobile object 100 and specifies a meeting point. The communication terminal 140 includes a communication device including, for example, a communication circuit, and transmits and receives necessary data to and from the management server 120 via mobile communication such as LTE.

[0047] <State transition of a moving object and usage example of a mobility support system> Next, state transitions of the moving body 100 and examples of use of the mobility assistance system will be described with reference to Figures 4A, 4B, and 4C. Figure 4A shows state transitions of the moving body 100. Note that control of the state transitions described in Figure 4A can be realized by the processor 302 executing a program. For example, the moving body 100 becomes operational when it is placed in a station and powered on.

[0048] First, the control state of the mobile object 100 starts from startup 401. In startup 401, the mobile object 100 downloads data such as a travel route, map information including area definitions (including information on areas where entry is restricted), and a list of registered users from the management server 120. The mobile object 100 also loads the above-mentioned machine learning model, etc. The state of the mobile object 100 then transitions to idle 402.

[0049] In the idle state 402, the mobile object 100 periodically communicates with the management server 120, for example, every 10 seconds, to check whether a reservation has been set and wait for an instruction to transition to the patrol mode. The mobile object 100 is stopped in the idle state 402. If a reservation has been set, the mobile object 100 transitions to the reservation mode 405. If the mobile object 100 receives an instruction to transition to the patrol mode 404 from the management server 120 while in the idle state 402, the mobile object 100 transitions to the patrol mode 404. The patrol mode 404 is a mode in which the mobile object 100 travels between stations. The mobile object 100 determines a travel route to the destination station and travels autonomously. In the idle state 402, when a user holds their palm over the bottom surface 232 of the sensor box 230, the mobile object 100 transitions to the authentication state 403, where the mobile object 100 performs authentication to register the user or determine whether the user is the same person as the registered user. In authentication 403, when user registration is completed or user authentication is successful, the traveling mode specified by the user (guided mode, leading mode, etc.) is started, or the traveling mode before transitioning to authentication 403 is resumed. In reservation mode, if a predetermined time has passed after the mobile object 100 arrives at a specified location without a user appearing to use it, the mobile object 100 transitions to idle mode 402, for example. The mobile object 100 may transition to patrol mode 404 to return to the station.

[0050] 4B shows an example of implementing this mobility support system in a large commercial facility. Within the facility, multiple stations 401a and 401b are installed where mobile objects 100 can be charged, and multiple mobile objects are located within the stations.

[0051] For example, when the mobile object 100a located at the station 401a transitions to the reservation mode, it moves to the location 403 designated by the reservation. When a person approaches the mobile object 100a, the mobile object 100a stops and performs user authentication of the specific person (user 110a) by hand vein authentication. When the mobile object 100a starts using the mobile object 100a, palm vein information of the user 110a does not exist, so the mobile object 100a temporarily registers the vein information for this use. Alternatively, if the vein information can be registered in advance on the server side and the user has registered their own vein information on the server, the mobile object 100a may perform user authentication based on the vein information registered on the server. Alternatively, the mobile object 100a may scan the vein information and transmit it to the server, and user authentication itself may be performed on the server side. The user 110a selects the control mode they wish to use via the touch panel (input device 311). The mobile object 100a then travels near the user 110a in the selected mode.

[0052] In the example shown in FIG. 4B, the mobile object 100c is set to the patrol mode. Therefore, the mobile object 100c is traveling to move between stations. The mobile object 100 traveling in the patrol mode accepts user authentication (user registration) by waving a hand. A user 110 can use a mobile object 100 in the patrol mode that happens to be nearby. In this way, the mobile object 100 can increase opportunities for people to use the mobile object 100. Furthermore, the mobile object 100 slows down its traveling speed in the patrol mode compared to its traveling speed in other control modes so that users can more easily access the mobile object 100 in the patrol mode.

[0053] Referring again to Figure 4A, when registration is completed or authentication is successful in authentication 403, operation in each designated mode is started or resumed. The control mode transitions according to the mode selected by the user via the touch panel (input device 311), for example, and becomes one of guide mode 407, delivery mode 408, follow mode 409, and lead mode 410.

[0054] Furthermore, while traveling in these modes, if the moving body 100 arrives at a destination (in a mode in which a destination is set) or if the moving body 100 is unable to detect the user for a predetermined period of time or longer (in a mode in which the moving body 100 travels alongside the user), the moving body 100 transitions to standby mode 411. The moving body 100 also transitions to standby mode 411 when the user 110 moves into an area where entry by moving bodies is restricted (such as a store area). Standby mode 411 is a mode in which the moving body 100 waits at a specific point until it starts or resumes a traveling mode in which it travels near the user 110.

[0055] For example, in the example shown in FIG. 4B, when the mobile object 100b is traveling near the user 110b and the user 110b enters a store space 402b (an area where the entry of the mobile object 100b is restricted), the mobile object 100b transitions to a standby mode and moves to a specific location (standby space 403) to wait. If the user 110b subsequently visits the standby space 403 and is successfully authenticated by the mobile object 100b, the user 110b can set a new mode or resume the previous traveling mode. If a predetermined time has passed since the previous user authentication without user authentication, the mobile object 100b may transition to a patrol mode in which the mobile object 100b moves to a station.

[0056] When the remaining battery charge of the mobile object 100 falls below a certain value, the mobile object 100 switches from any mode to emergency mode. Emergency mode 406 is a mode in which the mobile object 100 returns to the station. The mobile object 100 also switches to emergency mode if a predetermined time has passed and luggage is left inside the housing, in order to move to the station (or a predetermined place for leaving lost items).

[0057] Next, referring to FIG. 4C, the leading mode 410, the guiding mode 407, the following mode 409, and the delivery mode 408 will be outlined.

[0058] In the leading mode 410, the moving body 100 moves ahead of the user 110 while adjusting its speed and maintaining an appropriate distance from the user 110 based on predicted behavior of the user 110. In the leading mode, the moving body 100 does not know the user's destination. The moving body 100 autonomously avoids collisions in crowds. The moving body 100 may recognize the user's gestures and accept directional instructions based on the gestures. The moving body 100 may adjust its direction of travel in accordance with the recognized gestures.

[0059] In the follow mode 409, the moving object 100 follows the user 110 behind the user 110. Even in this mode, the moving object 100 does not know the user's destination. The moving object 100 autonomously avoids collisions in a crowd.

[0060] In guidance mode 407, the mobile object 100 travels ahead of the user 110 along a travel route toward a destination specified by the user 110. The mobile object 100 first sets the travel speed of the mobile object to a predetermined speed, and then adjusts the travel speed according to changes in the distance to the user 110. The mobile object 100 autonomously avoids collisions in crowds. Delivery mode 408 is a mode in which the mobile object 100 transports the package 150 to a specified destination, and travels alone.

[0061] <Characteristics of each control mode> 5A and 5B show, in table form, the characteristics of the control modes according to this embodiment, namely, the leading mode, the following mode, the guidance mode, the delivery mode, the patrol mode, and the emergency mode. Each mode is characterized by, for example, attributes such as the traveling speed of the mobile object, the traveling trajectory generation of the mobile object, the positional relationship between the user and the mobile object, speech control to people around the mobile object, and the manner of presenting information to people around the mobile object. In the leading mode, the following mode, and the guidance mode shown in FIG. 5A, the mobile object travels near the user. On the other hand, in the delivery mode, the patrol mode, and the emergency mode shown in FIG. 5B, the mobile object travels independently. The following describes the descriptions in FIGS. 5A and 5B that deserve particular explanation.

[0062] Regarding the running speed of the moving body, in the leading mode and the following mode, the moving body 100 controls the running speed of the moving body so that the distance to the user 110 is maintained within a certain range, that is, so that the running speed of the moving body matches the moving speed of the user. On the other hand, in the guidance mode, the running speed of the moving body is set to a predetermined speed, and then the running speed is adjusted according to changes in the distance to the user 110.

[0063] In the leading mode and the guiding mode, the positional relationship between the user and the moving body restricts movement (offset) in the left and right directions perpendicular to the direction of travel. This is to prevent the moving body 100 traveling in front of the user from swaying left and right, making it difficult for the user walking behind. The offset restriction in the leading mode is larger than the offset restriction in the guiding mode (the amount of movement in the left and right directions within a given time is smaller).

[0064] In the leading mode, the mobile body 100 generates a traveling trajectory for the mobile body in accordance with the user's predicted behavior, in which the acceleration and speed in the left-right direction perpendicular to the traveling direction are more restricted than in other control modes. For example, in the leading mode, the acceleration and speed in the left-right direction perpendicular to the traveling direction are more restricted than in the guided mode, which follows a traveling route toward a specified destination. Because there is no rapid left-right swing, the user can walk a short distance and take a traveling trajectory that allows surrounding pedestrians and others to give way whenever possible. Users of the leading mode are expected to be elderly people and those who have difficulty walking, and suppressing left-right swing can assist such users in walking.

[0065] In addition, in the following mode, the target period or target distance for behavior prediction is set shorter than the value used in the leading mode, and a traveling trajectory of the moving body 100 according to the user's behavior prediction is generated. In the following mode, since the moving body 100 travels behind the user, the predicted period or distance can be shorter than in the leading mode in which the moving body travels in front of the user, and the amount of calculation related to the behavior prediction can be reduced.

[0066] In the case of failure to detect the user (lost) while traveling, the mobile body 100 stops in the leading mode and the guiding mode if it temporarily fails to detect the user, and requires user authentication again. On the other hand, in the following mode, for example, if the mobile body 100 again captures the user walking in front from sensor information, it resumes traveling without performing user authentication.

[0067] In the speech control for people around the moving body, in the following mode, speech information requesting the user to stop is output when the distance between the moving body and the user becomes equal to or greater than a predetermined first distance. This allows a user who is not in the field of view of the moving body to be aware of the moving body's delay. In the leading mode and the guiding mode, the moving body is traveling in front of the user, so there is no need to output such speech information. In the leading mode and the guiding mode, if there are people on or near the movement trajectory of the moving body traveling in front of the user, the moving body 100 outputs speech information requesting those people to move.

[0068] Regarding the presentation of information to people around the moving body, when the leading mode is set, the moving body 100 outputs information about the specific store to the user as speech or displays it on a display device in response to the behavior prediction result for the user indicating that the user will enter the area of ​​the specific store.

[0069] Furthermore, when the leading mode is set, with regard to presenting information to people around the moving body, in response to the user's behavior prediction result indicating that the moving body 100 will enter an entry restricted area, the moving body 100 stops at a position a predetermined second distance away from the area and outputs speech information informing the user to stop.

[0070] In contrast, in the follow mode, when the user's behavior prediction result indicates that the moving body 100 will enter an entry restricted area, the moving body 100 stops at a position a predetermined third distance away from the area, which is longer than the second distance, and outputs speech information informing the user of the stop.

[0071] Next, the delivery mode, patrol mode, and emergency mode will be described with reference to Fig. 5B. Regarding the traveling speed of the mobile object, in the patrol mode, the traveling speed of the mobile object 100 is slower than the traveling speed in other control modes. This makes it easier for the user to access the mobile object 100.

[0072] Regarding trajectory generation, in delivery mode and emergency mode, the optimal route to the destination can be set based on map information. In patrol mode, the mobile unit travels within a certain distance from the edge of the drivable area so as to obstruct the movement of pedestrians and other mobile units as little as possible.

[0073] The mobile object 100 accepts user authentication while traveling in patrol mode. For example, if a person approaches the mobile object 100 while traveling in patrol mode (when the mobile object recognizes a person approaching the mobile object within a predetermined distance, or when the predicted movement trajectory of the person approaching the mobile object is determined to intersect with the movement trajectory of the mobile object), the mobile object may stop with its front facing the approaching person. On the other hand, the mobile object 100 does not accept user authentication while traveling in reservation mode or emergency mode because the mobile object 100 is traveling toward a location designated in the reservation or its remaining battery power has dropped below a certain level. While traveling in such reservation mode or emergency mode, the mobile object 100 displays on the presentation device 310 that it does not accept user authentication or that the user cannot use the mobile object. This allows the user to easily understand which mobile objects are available for use while traveling.

[0074] <State control processing> Next, the operation of processing related to state control of a moving object will be described with reference to Fig. 6A. This processing is realized by processor 302 of control unit 301 of the moving object executing a program stored in memory 303 or storage device 308, and by operating each component included in control unit 301. In the following description, the subject of operation of this processing will be described as control unit 301, but in reality, each component of control unit 301 executes the processing.

[0075] In S601, the control unit 301 executes startup processing. The startup processing is as described above with reference to FIG. 4A. In S602, the control unit 301 executes idle state processing. The control unit 301, for example, periodically communicates with the management server 120 to check whether a reservation has been set. In S603, the control unit 301 determines whether a reservation has been made based on the response from the management server 120, and if a reservation has been made, the control unit 301 proceeds to S604; if not, S603 is repeated.

[0076] In S604, the control unit 301 transitions to reservation mode and acquires information about the reservation from the management server 120. The information about the reservation includes, for example, the location of the specified point expressed in a map coordinate system. The information about the reservation may further include the start time of use of the mobile object 100. The mobile object 100 determines an optimal route to the specified point using the information about the specified point and map information including the travel route and area definition. The control unit 301 moves to the specified point while controlling the travel of its own device along the determined optimal route.

[0077] In S605, upon arrival at the designated location, the control unit 301 determines whether authentication registration will be performed, for example, within a predetermined time after arrival. The authentication is, for example, registration of the user's palm vein information. If the user holds their palm over the bottom surface 232 of the sensor box 230 and registers the vein information within the predetermined time, the control unit 301 proceeds to S606; otherwise, the control unit 301 terminates this process. Note that, since the user's palm vein information does not exist at the start of use of the mobile object 100, the mobile object 100a temporarily registers the vein information for this use. Note that the registered information may be stored in, for example, the storage device 308 and may be erased at a predetermined timing, for example, in response to a user's instruction to end use, a change in the date and time, or upon shutdown. As described above, user authentication may use vein information pre-registered on the server side, or user authentication may be performed on the server side.

[0078] In S606, the control unit 301 accepts the selection of the control mode to be used by the user via the touch panel (input device 311). In S607, the control unit 301 transitions the state to the selected control mode and travels near the user in the selected mode. The operation in each mode will be described later.

[0079] As a result of the moving object 100 continuing to travel near the user, in S608, the control unit 301 determines whether the moving object 100 has arrived at the destination or whether a predetermined waiting criterion has been met. If the determination is met, the control unit 301 proceeds to S609; if not, the control unit 301 repeats the process. If the moving object 100 is operating in the guidance mode, the moving object 100 may arrive at the destination. Furthermore, as an example, the control unit 301 determines that the user's movement meets the predetermined criterion for the moving object to wait away from the user when the user approaches a group of people with a predetermined density or more within a predetermined distance. That is, when the user approaches a crowd of people with a predetermined number or more within a predetermined distance, the control unit 301 determines that the waiting criterion has been met to prevent a collision between the moving object 100 and the people. Alternatively, the control unit 301 may determine that the user's movement meets the predetermined criterion when the user approaches a group of people with a predetermined density or more within a predetermined distance and / or when the reliability of the user's predicted trajectory is equal to or less than a threshold. Furthermore, the control unit 301 may determine that the moving object's movement meets the predetermined criterion when speech information including an instruction to wait is received from the user. Also, when the moving body 100 is traveling in the guidance mode and a certain distance or more separates from the user, it may be determined that the user's movement satisfies a predetermined criterion for the moving body to wait away from the user. Furthermore, if the user cannot be detected from the sensor information for a predetermined time (a time corresponding to the user being completely lost) since the last time the user was detected, the control mode of the moving body is shifted to a standby mode.

[0080] In S609, the control unit 301 transitions to a standby mode, searches for a standby space in map data, and calculates a travel route to the standby space. The control unit 301 moves to the standby space along the travel route. Upon arrival at the standby space, the control unit 301 may wait with the front of the mobile object facing in the direction away from the user (the point at which the standby mode was entered). Note that the control unit 301 may transition from the standby mode to a patrol mode and move to the station upon the passage of a predetermined time since starting standby in the standby space.

[0081] In S610, the control unit 301 then transitions to emergency mode in response to a decrease in the remaining power (i.e., when the remaining power falls below a threshold), and moves to a station. Note that when the remaining power of the mobile object falls below a threshold, if there is no station within a range that can be moved due to the remaining power, the control unit 301 may transition to standby mode. The control unit 301 then terminates this process.

[0082] <Processing related to leading mode of moving body> Next, the process related to the leading mode will be described with reference to Fig. 6B. This process is started when the leading mode is selected in S606 of Fig. 6A and the leading mode is executed in S607. Therefore, this process is realized by the processor 302 of the control unit 301 of the moving object executing a program stored in the memory 303 or the storage device 308, and the operation of each component included in the control unit 301.

[0083] In S621, the control unit 301 acquires sensor information for recognizing objects around the mobile unit. In S622, the control unit 301 predicts the behavior of the object using a machine learning model based on the sensor information using the above-mentioned detection information processing unit 322. In S623, the control unit 301 determines whether the detection of the user has failed in the processing of the detection information processing unit 322 over a predetermined period of time, and proceeds to S626 if it has failed, or to S624 if it has not failed.

[0084] In S624, the control unit 301 generates a running trajectory for the mobile object based on the behavior prediction. In generating the running trajectory for the mobile object, a running trajectory is generated that runs in front of the mobile object so as to have a substantially constant positional relationship with the predicted position of the user. In addition, in the leading mode, the running trajectory for the mobile object is generated by adding constraints so that the acceleration and speed in a direction perpendicular to the traveling direction are more limited than in other control modes. In S625, the control unit 301 runs within the facility according to the generated running trajectory.

[0085] In S626, if the control unit 301 fails to detect the user for a predetermined period of time, it stops and waits for user authentication. In S627, if the user approaches the moving object 100 again and authentication is performed, and the authentication is successful, the control unit 301 returns the process to S621 and resumes traveling.

[0086] If the control unit 301 cannot detect the user for the predetermined period of time, the control unit 301 may output speech information calling on the user to return to the location of the moving object. Furthermore, if a predetermined time has passed since the output of the speech information without detecting the user from the sensor information, the control unit 301 may proceed to S629 or may transition the control mode of the moving object to the patrol mode.

[0087] In S628, the control unit 301 determines whether a predetermined time for transitioning to standby mode has elapsed without successful user authentication after failing to detect the user, and if so, transitions to standby mode in S629. The process then ends.

[0088] <Processing related to moving object tracking mode> Next, processing related to the follow-up mode will be described with reference to Fig. 6C. This processing is started when the follow-up mode is selected in S606 of Fig. 6A and the follow-up mode is executed in S607. Therefore, this processing is realized by the processor 302 of the control unit 301 of the moving object executing a program stored in the memory 303 or the storage device 308, and the operation of each component included in the control unit 301.

[0089] In S641, the control unit 301 acquires sensor information for recognizing objects around the mobile unit. In S642, the control unit 301 uses the above-mentioned detection information processing unit 322 to predict the behavior of the object using a machine learning model based on the sensor information. In S643, the control unit 301 determines whether the detection of the user has failed in the processing of the detection information processing unit 322 over a predetermined period of time, and proceeds to S646 if it has failed, or to S644 if it has not failed.

[0090] In S644, the control unit 301 generates a traveling trajectory of the mobile object based on the behavior prediction. In generating the traveling trajectory of the mobile object, a traveling trajectory is generated that travels behind the user so as to have a substantially constant positional relationship with the predicted position of the user. In addition, in the following mode, the target period or target distance of the behavior prediction is set shorter than the value used in the leading mode, and a traveling trajectory of the mobile object 100 is generated according to the behavior prediction of the user. In S645, the control unit 301 travels within the facility according to the generated traveling trajectory.

[0091] In S646, if the control unit 301 fails to detect the user for a predetermined period of time, it temporarily stops. At this time, it continues to recognize surrounding objects. In S647, if it detects a user approaching the moving object 100 again, it returns the process to S641 and resumes running.

[0092] In S648, the control unit 301 determines whether a predetermined time for transitioning to standby mode has elapsed without redetecting the user after failing to redetect the user, and if so, transitions to standby mode in S649. The process then ends.

[0093] <Processing related to guidance mode of moving object> Next, the process related to the guidance mode will be described with reference to Fig. 6D. This process is started when the guidance mode is selected in S606 of Fig. 6A and the guidance mode is executed in S607. Therefore, this process is realized by the processor 302 of the control unit 301 of the moving object executing a program stored in the memory 303 or the storage device 308, and the operation of each component included in the control unit 301.

[0094] At S661, the control unit 301 accepts input of a destination and sets a route to the destination. At S662, the control unit 301 starts traveling along a travel path based on the route. At this time, the traveling speed is set to a speed predetermined as the traveling speed in the guidance mode. At S663, sensor information for recognizing objects around the mobile object is acquired. At S664, the control unit 301 acquires the position of the object based on the sensor information using the above-mentioned detection information processing unit 322. This allows the control unit 301 to acquire changes in the distance to the user.

[0095] In S665, the control unit 301 determines whether the detection of the user has failed in the processing of the detection information processing unit 322 over a predetermined period of time, and if so, proceeds to S666, and if not, proceeds to S664.

[0096] In S666, the control unit 301 adjusts the traveling trajectory (traveling speed) of the mobile object based on the change in the acquired distance. In generating the traveling trajectory of the mobile object, the traveling trajectory of the mobile object is adjusted so that the distance to the user is approximately constant. In S667, the control unit 301 travels within the facility according to the generated traveling trajectory. In S668, the control unit 301 determines whether the destination has been reached based on the current position, and if it has been determined that the destination has been reached, the process proceeds to S629, and if it has not been reached, the process returns to S663.

[0097] In S669, if the control unit 301 fails to detect the user for a predetermined period of time, it stops temporarily and waits for user authentication. In S670, if the user approaches the moving object 100 again and authentication is performed, and the authentication is successful, the control unit 301 returns the process to S663 and resumes traveling. Thereafter, similar to the process shown in Fig. 6B, the control unit 301 executes the processes of S628 and S629 to transition to standby mode and terminates this process.

[0098] <Processing related to delivery modes of mobile devices> Next, the process related to the delivery mode will be described with reference to Figure 6E. This process is started when the delivery mode is selected in S606 of Figure 6A and the delivery mode is executed in S607. Therefore, this process is realized by the processor 302 of the control unit 301 of the mobile object executing a program stored in the memory 303 or storage device 308, and the operation of each component included in the control unit 301.

[0099] At S681, the control unit 301 detects that a package has been placed in the vehicle. The detection of the placement of a package may be performed by detecting an increase in the weight of the housing or by detecting that the door has been closed. At S682, the control unit 301 accepts input of a destination and sets a route to the destination. At S683, the control unit 301 travels along a travel trajectory based on the route. At S684, sensor information for recognizing objects around the mobile body is acquired. At S685, the control unit 301 detects the object based on the sensor information using the above-mentioned detection information processing unit 322. In delivery mode, there is no object traveling parallel to the vehicle, so the position of the object is used simply to avoid collisions. The control unit 301 adjusts the travel trajectory of the mobile body according to the position of the object. At S686, the control unit 301 travels along the adjusted travel trajectory.

[0100] In S687, the control unit 301 determines whether the mobile object 100 has arrived at the destination based on the current location. If it is determined that the mobile object 100 has arrived, the process proceeds to S688. If it is determined that the mobile object 100 has not arrived, the process returns to S684. In S688, the control unit 301 transitions the mobile object 100 to a standby mode and ends this process. Note that the control unit 301 may also determine whether the destination is within a predetermined distance from a station where the mobile object 100 can be charged. If the destination is within the predetermined distance from the station, the control unit 301 may transition to a patrol mode in which the mobile object 100 moves to the station without transitioning to the standby mode.

[0101] As described above, in the above-described embodiment, the control device of the mobile body is configured to acquire sensor information for recognizing objects around the mobile body, present predetermined information to people around the mobile body, and set one of a plurality of control modes. The control device of the mobile body further controls the travel of the mobile body and the presentation of the predetermined information based on the set control mode and the detection result of a specific person based on the sensor information. In this case, the control controls the travel of the mobile body and the presentation of the predetermined information by changing at least one of the travel speed of the mobile body, the generation of the travel trajectory of the mobile body, the positional relationship between the specific person and the mobile body, and the presentation mode of information to people around the mobile body, for each control mode.

[0102] In this way, various modes of assistance can be provided, making it possible to more appropriately assist people in moving around the site.

[0103] In the above-described embodiment, the control device of the mobile body acquires sensor information for recognizing objects around the mobile body, sets one of a plurality of control modes, and controls the travel of the mobile body based on the set control mode and the detection result of a specific person based on the sensor information. The plurality of control modes include a plurality of travel modes in which the mobile body travels near the specific person walking, and a standby mode in which the mobile body waits at a specific location until one of the travel modes is started or resumed. Furthermore, in the above control, the travel of the mobile body is controlled to travel near the specific person walking in the plurality of travel modes, and when it is determined that the movement of the specific person or the movement of the mobile body meets a predetermined criterion for the mobile body to wait away from the specific person, the mobile body transitions from one of the travel modes to a standby mode.

[0104] This makes it possible to more appropriately support people's movement within the premises, while also increasing safety by preventing mobile objects from entering crowded areas or places where they should not be.

[0105] Furthermore, in the above-described embodiment, the control device of the mobile body acquires sensor information for recognizing objects around the mobile body, sets one of a plurality of control modes, and controls the travel of the mobile body based on the set control mode and the detection result of a specific person based on the sensor information. In this case, the control controls the travel of the mobile body in a travel mode among the plurality of control modes so that the mobile body travels near a specific person walking, and the plurality of control modes include a reservation mode. Furthermore, in the reservation mode, the control controls the travel of the mobile body so that the specific person travels to a point designated by the specific person before starting the travel mode.

[0106] This makes it possible to more appropriately support people's movement within the premises while improving usability when starting to use the mobile object.

[0107] <Summary of the embodiment> 1-1. The control device (e.g., 301) of the moving body (e.g., 100) in the above embodiment is an acquisition means (e.g., 306) for acquiring sensor information including an image captured around the mobile body in order to recognize objects around the mobile body; A setting means (e.g., 311, 301) for setting one of a plurality of control modes; a control means (e.g., 301, 322, 324) for controlling the travel of the moving body based on a set control mode and a result of detecting a specific person based on sensor information; The control means controls the movement of the moving body so that the moving body moves at a running speed, a running trajectory of the moving body, and a positional relationship between the specific person and the moving body that correspond to the control mode.

[0108] According to this embodiment, by providing various modes of assistance, it becomes possible to more appropriately assist people in moving around the premises.

[0109] 1-2. (Providing information to people around a moving object) In the control device for a moving body according to the above embodiment, Further, the mobile device has a presentation means (e.g., 310) for presenting predetermined information to people around the mobile device, The control means controls the presentation of the predetermined information so that the presentation mode of the information to people around the moving object corresponds to the control mode.

[0110] 1-3. (Leading mode, Following mode, Guidance mode) In the control device for a moving body according to the above embodiment, The multiple control modes include a first mode (leading mode) in which the mobile body travels ahead of a specific person based on a behavior prediction for the specific person, a second mode (following mode) in which the mobile body follows the specific person from behind the specific person, and a third mode (guiding mode) in which the mobile body travels ahead of the specific person according to a travel route toward a destination specified by the specific person.

[0111] 1-4. (Travel speed in guidance mode) In the control device for a moving body according to the above embodiment, When the third mode (guidance mode) is set, the control means sets the running speed of the moving body to a predetermined speed, and then adjusts the running speed according to changes in the distance to a specific person.

[0112] 1-5. (Leading and Guidance Mode Offset) In the control device for a moving body according to the above embodiment, When a mode for traveling ahead of a specific person is set, the control means controls the positional relationship between the specific person and the moving body so that the range of movement in the left and right directions perpendicular to the traveling direction of the moving body within a specified time is smaller than in a mode for traveling behind the specific person.

[0113] 1-6. (Differences in offset limits between leading mode and guidance mode) In the control device for a moving body according to the above embodiment, When the first mode is set, the control means limits the range of movement of a specific person and the moving body in the left and right directions perpendicular to the moving body's direction of travel within a specified time period to a range that is more limited than the range of movement within a specified time period in the third mode, with respect to the positional relationship between the specific person and the moving body.

[0114] 1-7. (Trajectory generation in following mode) In the control device for a moving body according to the above embodiment, When a mode for traveling behind a specific person is set, the control means sets the target period or target distance for behavior prediction in generating the traveling trajectory of the moving body to be shorter than the value used in the mode for traveling behind a specific person, and generates the traveling trajectory of the moving body according to the behavior prediction of the specific person.

[0115] 1-8. (Control when lost) In the control device for a moving body according to the above embodiment, If the control means cannot detect a specific person from the sensor information for a predetermined period of time since the last detection, it stops the mobile body temporarily and resumes the movement of the mobile body in response to successful authentication of the specific person to start or resume use of the mobile body by the specific person.

[0116] 1-9. (Control when lost 2) In the control device for a moving body according to the above embodiment, When the control means cannot detect a specific person from the sensor information for a predetermined time period since the last detection, the control means stops the mobile body once, and restarts the running of the mobile body in response to detecting the specific person again from the sensor information.

[0117] 1-10. (Speech output in follow-up mode) In the control device for a moving body according to the above embodiment, When a mode for traveling behind a specific person is set, the control means outputs speech information requesting the specific person to stop when the distance between the moving body and the specific person becomes greater than a predetermined first distance, in relation to presenting information to people around the moving body.

[0118] 1-11. (Speech output in leading and guiding modes) The control means is set to a mode in which the mobile body travels in front of a specific person, and when a person is present around the mobile body in the travel path of the mobile body, outputs speech information requesting the person to move.

[0119] 1-12. (Communication in the leading mode) In the control device for a moving body according to the above embodiment, When the first mode (leading mode) is set, the control means, in relation to presenting information to people around the moving body, presents information about a specific store to a specific person in response to the behavior prediction result for the specific person indicating that the person will enter the area of ​​the specific store.

[0120] 1-13. (Stopping in leading mode) In the control device for a moving body according to the above embodiment, When the first mode (leading mode) is set, the control means, in relation to presenting information to people around the moving body, stops at a position a predetermined second distance away from the predetermined area in response to the behavior prediction result of a specific person indicating that the moving body will enter a predetermined area where entry is restricted, and outputs speech information informing the specific person to stop.

[0121] 1-14. (Stopping in follow mode) In the control device for a moving body according to the above embodiment, When the second mode (following mode) is set, the control means, in relation to presenting information to people around the moving body, stops at a position a predetermined third distance away from the predetermined area that is longer than the second distance in response to the behavior prediction result of a specific person indicating that the moving body will enter a predetermined area where entry is restricted, and outputs speech information informing the specific person to stop.

[0122] The invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the invention. [Explanation of symbols]

[0123] 100... Mobile object, 120... Management server, 301... Control unit, 304... Travel unit, 306... Detection unit, 310... Presentation device

Claims

1. A control device for a moving body, an acquisition means for acquiring sensor information including an image of the surroundings of the moving body, for recognizing objects around the moving body; a setting means for setting one of a plurality of control modes; a control unit that controls the travel of the moving object based on a set control mode and a detection result of a specific person based on the sensor information, the control means controls the traveling of the moving body so that the moving body travels at a traveling speed of the moving body, at a traveling trajectory generated for the moving body, and at a positional relationship between the specific person and the moving body, according to a control mode; the plurality of control modes include a first mode in which the moving object travels ahead of the specific person based on a behavior prediction for the specific person without a destination specified by the specific person, The control device for a mobile body is characterized in that, when the first mode is set, the control means controls the movement of the mobile body so that the mobile body moves in front of the specific person based on a behavior prediction for the specific person without a destination specified by the specific person.

2. a presentation means for presenting predetermined information to people around the moving object; The control device for a moving body according to claim 1 , wherein the control means controls the presentation of the predetermined information so that the presentation mode of the information to people around the moving body corresponds to a control mode.

3. 3. The control device for a mobile body according to claim 1, wherein the plurality of control modes further include a second mode in which the mobile body follows the specific person behind the specific person, and a third mode in which the mobile body travels ahead of the specific person along a travel route toward a destination specified by the specific person.

4. 4. The control device for a moving body according to claim 3, wherein when the third mode is set, the control means sets the traveling speed of the moving body to a predetermined speed and then adjusts the traveling speed in accordance with a change in the distance to the specific person.

5. The control device for a mobile body as described in claim 3 or 4, characterized in that when a mode in which the mobile body is traveling in front of the specific person is set, the control means controls the positional relationship between the specific person and the mobile body so that the range of movement of the mobile body in the left and right directions perpendicular to the direction of travel within a specified time is smaller than in a mode in which the mobile body is traveling behind the specific person.

6. A control device for a moving body described in any one of claims 3 to 5, characterized in that when the first mode is set, the control means limits the range of movement in the left and right directions perpendicular to the direction of travel of the moving body within a predetermined time period, with respect to the positional relationship between the specific person and the moving body, to a range of movement within the predetermined time period in the third mode.

7. A control device for a mobile body described in any one of claims 3 to 6, characterized in that when a mode in which the mobile body is traveling behind the specific person is set, the control means sets the target period or target distance for behavior prediction for generating the traveling trajectory of the mobile body to be shorter than the value used in a mode in which the mobile body is traveling in front of the specific person, and generates a traveling trajectory of the mobile body according to the behavior prediction of the specific person.

8. A control device for a mobile body described in any one of claims 3 to 7, characterized in that if the control means cannot detect the specific person from the sensor information for a predetermined time since the last detection, it stops the mobile body temporarily, and resumes the movement of the mobile body upon successful authentication of the specific person to start or resume use of the mobile body by the specific person.

9. A control device for a mobile body described in any one of claims 3 to 7, characterized in that the control means stops the mobile body temporarily when the specific person cannot be detected from the sensor information for a predetermined time since the last time the specific person was detected from the sensor information, and resumes the movement of the mobile body when the specific person is detected again from the sensor information.

10. A control device for a mobile body as described in any one of claims 3 to 9, characterized in that when a mode in which the mobile body is traveling behind the specific person is set, the control means, in relation to presenting information to people around the mobile body, outputs speech information requesting the specific person to stop when the distance between the mobile body and the specific person becomes greater than a predetermined first distance.

11. The control device for a mobile body described in any one of claims 3 to 9, characterized in that the control means is set to a mode in which the mobile body travels in front of the specific person, and when a person is present around the mobile body on the travel path of the mobile body, outputs speech information requesting the person to move.

12. A control device for a mobile body described in any one of claims 3 to 11, characterized in that when the first mode is set, the control means, with regard to presenting information to people around the mobile body, presents information about a specific store to the specific person in response to the behavior prediction result for the specific person indicating that the specific person will enter the area of ​​the specific store.

13. A control device for a mobile body described in any one of claims 3 to 9, characterized in that when the first mode is set, the control means, with regard to presenting information to people around the mobile body, stops at a position a predetermined second distance away from the predetermined area in response to the behavior prediction result of the specific person indicating that the mobile body will enter a predetermined area where entry is restricted, and outputs speech information informing the specific person to stop.

14. The control device of a mobile body as described in claim 13, characterized in that when the second mode is set, with regard to presenting information to people around the mobile body, in response to the behavior prediction result of the specific person indicating that the mobile body will enter a predetermined area where entry is restricted, the control means stops at a position that is a predetermined third distance from the predetermined area and is longer than the predetermined second distance, and outputs speech information informing the specific person to stop.

15. A control device for a moving body, an acquisition means for acquiring sensor information including an image of the surroundings of the moving body, for recognizing objects around the moving body; a setting means for setting one of a plurality of control modes; a control unit that controls the travel of the moving object based on a set control mode and a detection result of a specific person based on the sensor information, the control means controls the traveling of the moving body so that the moving body travels at a traveling speed of the moving body, at a traveling trajectory generated for the moving body, and at a positional relationship between the specific person and the moving body, according to a control mode; the plurality of control modes include a first mode in which the moving body travels ahead of the specific person based on a behavior prediction for the specific person, a second mode in which the moving body follows the specific person from behind the specific person, and a third mode in which the moving body travels ahead of the specific person according to a travel route toward a destination designated by the specific person, A control device for a moving body, characterized in that when a mode in which the moving body is traveling in front of the specific person is set, the control means controls the positional relationship between the specific person and the moving body so that the range of movement of the moving body in the left and right directions perpendicular to the direction of travel within a specified time is smaller than in a mode in which the moving body is traveling behind the specific person.

16. A control device for a moving body, an acquisition means for acquiring sensor information including an image of the surroundings of the moving body, for recognizing objects around the moving body; a setting means for setting one of a plurality of control modes; a control unit that controls the travel of the moving object based on a set control mode and a detection result of a specific person based on the sensor information, the control means controls the traveling of the moving body so that the moving body travels at a traveling speed of the moving body, at a traveling trajectory generated for the moving body, and at a positional relationship between the specific person and the moving body, according to a control mode; the plurality of control modes include a first mode in which the moving body travels ahead of the specific person based on a behavior prediction for the specific person, a second mode in which the moving body follows the specific person from behind the specific person, and a third mode in which the moving body travels ahead of the specific person according to a travel route toward a destination designated by the specific person, The control means is characterized in that, when the first mode is set, the range of movement in the left and right directions perpendicular to the direction of travel of the moving body within a specified time period, with respect to the positional relationship between the specific person and the moving body, is more limited than the range of movement within the specified time period in the third mode.

17. A method for controlling a moving object, comprising: an acquisition step of acquiring sensor information including an image of the surroundings of the moving body for recognizing objects around the moving body; a setting step of setting one control mode out of a plurality of control modes; a control step of controlling the travel of the moving object based on a set control mode and a detection result of a specific person based on the sensor information, the control step includes controlling the traveling of the moving body so that the moving body travels at a traveling speed of the moving body, at a traveling trajectory of the moving body, and at a positional relationship between the specific person and the moving body, according to a control mode; the plurality of control modes include a first mode in which the moving object travels ahead of the specific person based on a behavior prediction for the specific person without a destination specified by the specific person, In the control process, when the first mode is set, the method controls the movement of the moving body so that the moving body moves in front of the specific person based on a behavior prediction for the specific person without a destination specified by the specific person.

18. A method for controlling a moving object, comprising: an acquisition step of acquiring sensor information including an image of the surroundings of the moving body for recognizing objects around the moving body; a setting step of setting one control mode out of a plurality of control modes; a control step of controlling the travel of the moving object based on a set control mode and a detection result of a specific person based on the sensor information, In the control step, the traveling of the moving body is controlled so that the moving body travels at a traveling speed of the moving body, a traveling trajectory of the moving body, and a positional relationship between the specific person and the moving body, which are in accordance with a control mode; the plurality of control modes include a first mode in which the moving body travels ahead of the specific person based on a behavior prediction for the specific person, a second mode in which the moving body follows the specific person from behind the specific person, and a third mode in which the moving body travels ahead of the specific person according to a travel route toward a destination designated by the specific person, A method for controlling a moving body, characterized in that, in the control process, when a mode in which the moving body is traveling in front of the specific person is set, the control step controls the positional relationship between the specific person and the moving body so that the range of movement of the moving body in the left and right directions perpendicular to the direction of travel within a specified time is smaller than in a mode in which the moving body is traveling behind the specific person.

19. A method for controlling a moving object, comprising: an acquisition step of acquiring sensor information including an image of the surroundings of the moving body for recognizing objects around the moving body; a setting step of setting one control mode out of a plurality of control modes; a control step of controlling the travel of the moving object based on a set control mode and a detection result of a specific person based on the sensor information, In the control step, the traveling of the moving body is controlled so that the moving body travels at a traveling speed of the moving body, a traveling trajectory of the moving body, and a positional relationship between the specific person and the moving body, which are in accordance with a control mode; the plurality of control modes include a first mode in which the moving body travels ahead of the specific person based on a behavior prediction for the specific person, a second mode in which the moving body follows the specific person from behind the specific person, and a third mode in which the moving body travels ahead of the specific person according to a travel route toward a destination designated by the specific person, A method for controlling a moving body, characterized in that, in the control process, when the first mode is set, the range of movement in the left and right directions perpendicular to the direction of travel of the moving body within a specified time period, with respect to the positional relationship between the specific person and the moving body, is limited more than the range of movement within the specified time period in the third mode.

20. A program for causing a computer to function as each of the means of the control device for a moving body according to any one of claims 1 to 16.

21. A storage medium storing a program for causing a computer to function as each of the means of the control device for a moving body according to any one of claims 1 to 16.

Images