Method for controlling a mobile object, control device, mobile object, program, and storage medium

Abstract

This technology provides the ability to ensure that the movement of a moving object and a specific person is in an appropriate state. [Solution] The method for controlling a moving object according to the present disclosure includes: recognizing the translational state of a specific person relative to the moving object using an image taken of the area around the moving object; determining whether the translational state of the specific person is a predetermined state as an appropriate state; and, if it is determined that the translational state of the specific person is not an appropriate state, outputting a request for the specific person to take action to bring the translational state of the specific person to an appropriate state.

Description

Technical Field

[0001] The present invention relates to a moving body control method, a control device, a moving body, a program, and a storage medium.

Background Art

[0002] Moving bodies such as robots that travel autonomously by following a user (also referred to as a specific person) or leading the user are known. Patent Document 1 discloses a technology for, in a moving body that travels in front of a specific person while predicting the behavior of the specific person, uttering or displaying information about the specific store when the behavior prediction result for the specific person indicates entry into a specific store area.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0004] By the way, when a moving body moves together with a specific person, the distance between the moving body and the specific person may be kept constant in some cases. However, for example, when the relative speed of the specific person in a hurry is higher than that of the moving body, the distance may not be kept constant. As a result, the distance between the specific person moving ahead and the following moving body may widen, or the specific person behind may approach the moving body traveling ahead. When such a situation occurs, adverse effects such as the specific person feeling uncomfortable or reduced safety may arise.

[0005] The present invention has been made in view of the above problems, and an object thereof is to provide a technology capable of bringing the movement of a moving body and a specific person into an appropriate state.

Means for Solving the Problems

[0006] According to the present invention, A method for controlling a mobile body that moves autonomously with a specific person, Using images taken around the moving object, the translational state of the specific person relative to the moving object is recognized. Determining whether the translational state of the aforementioned specific person is a predetermined state which is an appropriate state, A method for controlling a moving object is provided, which includes, if it is determined that the translational state of the specified person is not the appropriate state, outputting a request for the specified person to take action to bring the translational state of the specified person to the appropriate state. [Effects of the Invention]

[0007] According to the present invention, it becomes possible to maintain the proper state of movement between a moving object and a specific person. [Brief explanation of the drawing]

[0008] [Figure 1] A perspective view showing an example of the overall configuration of the mobile body of this embodiment. [Figure 2] A schematic plan view illustrating an example of a mobile unit for the mobile body of this embodiment. [Figure 3] A block diagram showing an example of the control system for the mobile body of this embodiment. [Figure 4] A block diagram showing an example of the functional configuration of the control unit in this embodiment. [Figure 5A] Figure (1) illustrates the state optimization in this embodiment. [Figure 5B] Figure (2) illustrates the state optimization in this embodiment. [Figure 5C] Figure (3) illustrates the optimization of the state in this embodiment. [Figure 5D] Figure (4) illustrates the state optimization in this embodiment. [Figure 5E] Figure (5) illustrates the optimization of the state in this embodiment. [Figure 6] A flowchart illustrating the sequence of operations for intent acquisition processing in this embodiment. [Modes for carrying out the invention]

[0009] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more features from the multiple features described in the embodiments may be combined arbitrarily. Furthermore, identical or similar configurations will be given the same reference numeral, and redundant descriptions will be omitted.

[0010] <Embodiment> The following embodiments will be described using a micromobility as an example of a mobile body. A micromobility is, for example, a small electric vehicle that can carry luggage and follow, lead, or guide a user (also referred to as a specific person). However, this embodiment may be applied not only to vehicles that carry luggage but also to vehicles that can carry people, and not only to vehicles but also to autonomously mobile robots. For example, it can be applied to a robot that can walk autonomously using its feet without having wheels. Furthermore, the following description will be based on an example of a mobile body having one driven wheel, but the mobile body does not necessarily have to have a driven wheel, and more than one driven wheel may be used. This embodiment is not limited to mobile bodies that are electrically powered but can also be applied to mobile bodies that move using other power sources.

[0011] <Mobile Unit Configuration> Referring to Figure 1, the configuration of the mobile body 100 of the embodiment will be described. Figure 1 is a perspective view showing the overall configuration of the mobile body. As shown in Figure 1, the mobile body 100 of the embodiment comprises a lower housing 111, an upper housing 112, a front display unit 113, a plurality of detection units 114, an audio output unit 115, a light-emitting unit 116, two drive wheels 120, and a driven wheel 121. The side on which the drive wheels 120 are located is considered the front.

[0012] The lower housing 111 rotatably holds the drive wheel 120 and the driven wheel 121, and accommodates a traveling unit including the drive wheel 120 and the driven wheel 121. The upper portion of the lower housing 111 has a loading space 118 in which a user's luggage can be loaded. The lower housing 111 accommodates the light emitting unit 116. The region of the lower housing 111 that accommodates the light emitting unit 116 is made of a material that can transmit light.

[0013] The upper housing 112 is configured to extend upward from the rear portion of the lower housing 111. The upper housing 112 supports the front display unit 113, the detection unit 114, and the audio output unit 115 at the upper end portion.

[0014] The front display unit 113 notifies the user of information through images. The front display unit 113 may be a display device that displays images, such as a liquid crystal display device and an organic EL (Electro Luminescence) display device. The front display unit 113 is provided on the front side of the upper end of the upper housing 112. The front display unit 113 is oriented forward. The front display unit 113 notifies the user of information by displaying an image imitating human eyes or a face, characters, and the like.

[0015] The plurality of detection units 114 detect targets such as people and objects around the mobile body 100. The detection unit 114 may be an imaging apparatus having an optical system such as a lens, an image sensor, and the like. The detection unit 114 captures images of the surroundings and outputs image data. Instead of or in addition to the imaging apparatus, the detection unit 114 may include a device capable of detecting an object, such as a radar and a LiDAR (Light Detection and Ranging). The plurality of detection units 114 are provided, for example, on the front, rear, left, and right of the upper end portion of the upper housing 112. Thereby, the plurality of detection units 114 can capture images in all directions (that is, 360° around) of the mobile body 100 to generate images. Note that the installation location and the number of the detection units 114 are not particularly limited, and may be changed as appropriate.

[0016] The audio output unit 115 notifies surrounding people such as the user of information by audio. The audio output unit 115 may be an audio output device such as a speaker.

[0017] The light emitting unit 116 notifies surrounding people such as the user of information by light. The light emitting unit 116 includes, for example, one or a plurality of light emitting elements such as LEDs (Light Emitting Diodes). The light emitting unit 116 is provided so as to surround the lower housing 111. The light emitting unit 116 may emit light in a plurality of colors. The light emitting unit 116 can emit light of different colors for each region or direction. That is, the light emitting unit 116 emits light of different colors in each of, for example, the front region 116F, the left region 116L, and the right region. Further, the light emitting unit 116 can switch between light emission and light extinction for each region. Note that the light emission of the regions may be controlled in finer regions.

[0018] The two drive wheels 120 are provided on the left and right of the front portion and the lower portion of the lower housing 111. The drive wheels 120 rotate around the axle when driving force is transmitted from the traveling unit.

[0019] The driven wheel 121 is provided at the center of the rear portion and the lower portion of the lower housing 111. The driven wheel 121 is held so as to be capable of turning around a vertical axis extending in the up-down direction.

[0020] FIG. 2 is a schematic plan view for explaining the traveling unit 125 of the moving body 100. The traveling unit 125 of the moving body 100 will be described with reference to FIG. 2.

[0021] The traveling unit 125 is provided at the lower portion of the lower housing 111. The traveling unit 125 has two drive wheels 120, a driven wheel 121, a drive mechanism 122, a motor 123a, a motor 123b, and a battery 124.

[0022] When power is supplied from the battery 124, motors 123a and 123b function as a drive source and supply driving force to the drive mechanism 122. The drive mechanism 122 transmits the driving force supplied from motors 123a and 123b to the drive wheels 120. This causes the drive mechanism 122 to rotate the drive wheels 120, moving the mobile body 100 forward or backward. The drive mechanism 122 causes the direction of travel of the mobile body 100 to be either left or right by creating a rotational difference between motors 123a and 123b. By changing the magnitude of the rotational difference, the drive mechanism 122 enables various turns of the mobile body 100, such as pivot turns, slow turns, and super-pivot turns.

[0023] Figure 3 is a block diagram of the control system of the mobile unit 100. The control system of the mobile unit 100 will be described with reference to Figure 3. As shown in Figure 3, the mobile unit 100 further comprises a control unit 130, an operation unit 132, a top display unit 131, an audio input unit 133, a GNSS sensor 134, a storage device 135, and a communication unit 136.

[0024] The control unit 130 is an example of a control device, also called an ECU (Electronic Control Unit), and may be a computer. The control unit 130 is responsible for the overall control of the mobile body 100. The control unit 130 includes one or more processors, such as a CPU (Central Processing Unit), memory devices such as semiconductor memory, and interfaces with external devices.

[0025] The processor may be an MPU (Micro Processing Unit), GPU (Graphics Processing Unit), NPU (Neural Processing Unit), or QPU (Quantum Processing Unit) in addition to or instead of a CPU. The processor realizes various functions of the control unit 130 by reading and executing computer programs stored in memory devices or the like.

[0026] Some or all of the functions of the control unit 130 may be implemented by one or more circuits, such as an ASIC (Application Specific Integrated Circuit) and a PLD (Programmable Logic Device) including an FPGA (Field Programmable Gate Array).

[0027] The memory device may be, for example, RAM (Random Access Memory). The memory device stores programs executed by the processor and data used by the processor for processing. Multiple sets of processors, memory devices, and interfaces may be provided according to the function of the mobile unit 100, and each set may be configured to communicate with one another.

[0028] The control unit 130 acquires image information from the detection unit 114, user input information acquired by the operation unit 132, and voice information from the voice input unit 133, and performs processing according to each piece of information. The control unit 130 performs behavioral control of the mobile body 100, including its movement, by controlling the motors 123a and 123b of the driving unit 125. The control unit 130 notifies the user or people in the vicinity of information. For example, the control unit 130 notifies the user of information by controlling the display of images displayed on the front display unit 113 and the top display unit 131. The control unit 130 may also notify the user of information by emitting light from the light-emitting unit 116. The control unit 130 may also notify the user of information by outputting voice from the voice output unit 115. The control unit 130 may perform processing on the image information from the detection unit 114 using a machine learning model for image recognition (e.g., a deep neural network). Furthermore, the control unit 130 may perform processing on the audio information from the audio input unit 133 using a machine learning model for speech recognition (e.g., a deep neural network).

[0029] The control unit 130 can recognize feature points in images captured by the detection unit 114 using a machine learning model. Furthermore, the control unit 130 can estimate the self-position of the mobile body 100 using the recognized feature points and map information pre-stored in the memory device 135. Known self-position estimation techniques can be used to estimate the self-position of the mobile body on the map. For example, the surrounding scenery can be photographed in advance, and the photographed results or feature points recognized from the photographed results can be associated with a map to create map information. The control unit 130 can estimate its self-position by determining the match between feature points in images taken during travel and feature points in the map information, thereby identifying the drivable area and the location of obstacles in the surrounding environment. The map information stored in the memory device 135 may be represented, for example, in the form of an occupied grid map. Each grid in the occupied grid map may represent an area where the mobile body 100 can travel or an area of ​​obstacles. If the current position of the mobile body 100 is placed in the center of the occupied grid map, the location of obstacles relative to the current position of the mobile body 100 can be identified.

[0030] Furthermore, the control unit 130 can use a machine learning model to recognize people (e.g., pedestrians) and other traffic participants in an image. The control unit 130 can recognize pedestrians and assign a unique identifier (ID) to each pedestrian. Then, in subsequent images, the control unit 130 can track the pedestrians who have been assigned the identifiers within the image.

[0031] The top display unit 131 may be an image display device. The top display unit 131 is provided on the upper surface of the upper end of the upper housing 112. Based on image information acquired from the control unit 130, the top display unit 131 displays images and characters related to information to be notified to the user. For example, the top display unit 131 displays an operation screen for accepting user input.

[0032] The operation unit 132 may be an input device such as a touch panel or keyboard. The operation unit 132 is, for example, provided on the display surface of the top display unit 131. The operation unit 132 receives input from the user and outputs it to the control unit 130.

[0033] The audio input unit 133 may be a microphone or the like. The audio input unit 133 picks up the voice of the user and the surroundings of the mobile unit 100 and outputs it to the control unit 130.

[0034] The GNSS sensor 134 receives GNSS signals from satellites to detect the current position of the mobile object 100. GNSS stands for Global Navigation Satellite System.

[0035] The storage device 135 includes a non-volatile recording medium for storing various types of data. The storage device 135 may be an HDD (Hard Disk Drive) or an SSD (Solid State Drive), etc. The storage device 135 may store programs executed by the processor, parameters necessary for program execution, and data to be processed by the processor. The storage device 135 may also store various parameters of machine learning models for speech recognition and image recognition executed by the control unit 130 (e.g., trained parameters and hyperparameters of a deep neural network). The storage device 135 can store map information of the space in which the mobile body 100 travels.

[0036] The communication unit 136 communicates with external devices such as the user's communication terminal 140 via wireless communication, such as Wi-Fi and fifth-generation mobile communication.

[0037] Figure 4 is a functional block diagram showing the functions of the control unit 130. The functions of the control unit 130 will be explained with reference to Figure 4.

[0038] The control unit 130 includes a person recognition unit 401, an environment recognition unit 402, a dialogue control unit 403, an intent estimation unit 404, a path generation unit 405, and a drive control unit 406. The control unit 130 may, for example, realize the functions of each part of the control unit 130 by reading a program from the storage device 135 and executing it.

[0039] The person recognition unit 401 recognizes pedestrians around the moving object 100 from images taken by the detection unit 114 that capture the area around the moving object 100. If the face and entire body of a user (also called a specific person) have been photographed in advance and the features of that specific person have been registered, the person recognition unit 401 can use a machine learning model to recognize pedestrians in the images captured by the detection unit 114, and distinguish between the specific person and other pedestrians.

[0040] Furthermore, the person recognition unit 401 uses the image captured by the detection unit 114 to predict the future position of a specific person (and other pedestrians) relative to the moving object 100. Known techniques can be used for predicting the future position of pedestrians using images. For example, the person recognition unit 401 outputs information indicating that pedestrian A will be located at a position (x, y) on the plane on which the moving object 100 is traveling (for example, with the position of the moving object 100 as the origin) after a time Δt has elapsed from the current time. In other words, the person recognition unit 401 can predict the position, speed of movement, and future position of a specific person and other pedestrians.

[0041] In this embodiment, the person recognition unit 401 recognizes the translational state of a specific person relative to a moving object. The translational state of a specific person relative to a moving object is recognized by the positional state between the moving object and the specific person, and the movement speed of both the moving object and the specific person.

[0042] The environment recognition unit 402 may recognize the surrounding environment from multiple images taken of the moving object 100 from various directions using one or more trained machine learning models. From the captured images, the environment recognition unit 402 recognizes the position, size, and shape of roads, pathways, buildings, walls, shops, signs, markers, obstacles, and roads and landmarks such as vehicles. The environment recognition unit 402 can recognize (acquire) information about the surrounding environment based on the recognition results from the images.

[0043] The environment recognition unit 402 may recognize (acquire) information about the surrounding environment, such as the location, size, and shape of roads and landmarks around the mobile body 100, based on the mobile body 100's own position and map information.

[0044] The dialogue control unit 403 generates a notification for a specific person based on the recognition results of the person recognition unit 401 and information about the surrounding environment from the environment recognition unit 402. The generated notification is output, for example, from the voice output unit 115. In addition to or instead of voice notification, the dialogue control unit 403 may also output a notification to a specific person by emitting light from the front display unit 113 or the light-emitting unit 116. Furthermore, the front display unit 113 can display an image representing a specific facial expression or various expressions representing the state of a moving object.

[0045] The dialogue control unit 403 may pre-store notifications associated with the recognition results of the person recognition unit 401 and information about the surrounding environment from the environment recognition unit 402, and select a notification according to the recognition results, etc. Alternatively, the dialogue control unit 403 may use a language model that outputs notifications according to the recognition results of the person recognition unit 401 and information about the surrounding environment from the environment recognition unit 402.

[0046] In this embodiment, the dialogue control unit 403 generates a notification depending on whether the translational state of a specific person relative to a moving object is in an appropriate state (e.g., within an appropriate range). In the following description, an example of an appropriate state is described when the translational state of a specific person relative to a moving object has a range, but the appropriate state may also apply when the translational state does not have a range. Details will be described later.

[0047] If the dialogue control unit 403 recognizes the actions of a specific person in response to a notification using the person recognition unit 401, it can generate and output further notifications according to the recognition result.

[0048] The intent estimation unit 404 estimates a specific person's intent by recognizing their gestures or the orientation of a part of their body based on images taken around the moving object 100. Alternatively, the intent estimation unit 404 estimates a specific person's intent by recognizing the content of their speech based on audio recorded from the surroundings of the moving object 100. For example, the intent estimation unit 404 can input the captured images into a machine learning model to recognize the gestures or the orientation of a part of a specific person's body. The intent estimation unit 404 can also input the recorded audio into a machine learning model to recognize the content of a specific person's speech. The intent estimation unit 404 may also estimate a specific person's intent to a notification by recognizing their actions in response to the notification.

[0049] The path generation unit 405 generates a path for the mobile body 100 to travel. If a specific person is present in the travel area, the path generation unit 405 generates a path to follow the specific person or to lead (guide) the specific person. In the following mode, the path generation unit 405 generates a path so that the mobile body 100 travels a short distance behind the specific person. In the leading mode, the path generation unit 405 generates a path so that the mobile body travels a short distance in front of the specific person, while predicting the movement of the specific person. In the leading mode, the path generation unit 405 may generate a path so that other pedestrians who might obstruct the movement of the specific person do not get too close to the specific person.

[0050] The drive control unit 406 causes the mobile body 100 to travel along the path generated by the path generation unit 405 by controlling, for example, motor 123a and motor 123b. The drive control unit 406 may also cause motor 123a and motor 123b to rotate in opposite directions to perform a pivot turn or a pivot turn (also called a turn in place) without moving the mobile body 100. The drive control unit 406 may also turn the mobile body 100 by a turn other than a pivot turn or a pivot turn.

[0051] <Overview of State Optimization> Next, the state optimization according to this embodiment will be explained with reference to Figures 5A to 5E.

[0052] Figure 5A schematically shows the presence of the mobile body 100 and a specific person 501 on the plane on which the mobile body 100 travels. The plane on which the mobile body 100 travels is, for example, an area where pedestrians walk, and includes sidewalks, public open spaces, floors inside buildings, etc., but may also include roadways. The mobile body 100 autonomously moves through the area where pedestrians walk, for example, while leading (guiding) the specific person at least temporarily.

[0053] While leading the specific person 501, the person recognition unit 401 recognizes the specific person and predicts the specific person's movement speed and direction. The person recognition unit 401 also recognizes the translational state of the specific person 501 relative to the moving object 100. In the example shown in Figure 5A, the distance between the moving object 100 and the specific person 501 is 502. Distance 502 falls within the range of appropriate distance for movement between the moving object 100 and the specific person. Furthermore, the moving object 100 and the specific person 501 are moving forward at a speed below a predetermined threshold. The difference in movement speed between the moving object 100 and the specific person 501 is assumed to be approximately zero.

[0054] In this case, for example, the dialogue control unit 403 determines that the translational state of the specific person 501 relative to the moving object 100 is appropriate because the movement speed of the specific person 501 is below a predetermined threshold and the distance 502 between the moving object 100 and the specific person 501 is within an appropriate distance range.

[0055] Figure 5B shows how the movement speed of the specific person 501 increased after Figure 5A. In the example shown in Figure 5B, the distance between the moving object 100 and the specific person 501 is shorter than the distance 502, and falls outside the range of the appropriate distance between the moving object 100 and the specific person 501. The moving object 100 is moving forward at a speed below the first threshold (lower limit of the appropriate range), but the specific person 501 is moving at a speed above the second threshold (upper limit of the appropriate range).

[0056] In this case, for example, the dialogue control unit 403 determines that the translational state of the specific person 501 relative to the moving object 100 is not appropriate because the movement speed of the specific person 501 exceeds the second threshold and the distance between the moving object 100 and the specific person 501 is outside the appropriate distance range. The dialogue control unit 403 outputs a request for the specific person 501 to take action in order to bring the translational state of the specific person 501 within the appropriate range.

[0057] For example, the dialogue control unit 403 outputs an utterance 503 requesting a specific person 501 to "walk slowly, as you cannot move at a speed greater than your current speed." In this utterance 503, the request "walk slowly" is a request to reduce the movement speed of the specific person 501. In addition to the request to the specific person, the request may also include an expression indicating the reason why the translational state cannot be maintained (for example, "you cannot move at a speed greater than your current speed"). When the safety of the specific person and the moving object decreases due to their proximity, or when the moving object cannot catch up to the specific person, an intuitive request to improve the translational state can be conveyed to optimize the translational state. Furthermore, by indicating the reason why the translational state cannot be maintained, the specific person can gain a deeper understanding of how to maintain the translational state.

[0058] Furthermore, if the specific person 501 moves away from the mobile object 100, the dialogue control unit 403 can output a request to the specific person 501 to move closer to the mobile object 100.

[0059] Figure 5C shows the operation when the translational state does not fall within the appropriate range even after a request is output from the mobile body 100.

[0060] The dialogue control unit 403 determines, within a predetermined time after outputting a request for the actions of a specific person, that the translational state of the specific person 501 has not improved from being within an appropriate range. This determination may be based on the fact that the distance between the moving body 100 and the specific person 501 is shorter than the distance 502. After this determination, the path generation unit 405 generates a path that moves the moving body 100 to a travel position where the translational state of the specific person becomes within an appropriate range. For example, the path generation unit 405 determines a travel position to the right of the specific person 501 where the distance 502 can be secured, and moves the moving body 100 there.

[0061] At this time, the dialogue control unit 403 notifies the person 501 of the current position of the moving object 100 before moving the moving object 100 to the right of the person 501. For example, the dialogue control unit 403 outputs a notification (utterance 504) such as, "I will move to the side because I am obstructing the person's walking." Here, the expression "I will move to the side" indicates the relative position of the moving object 100 from the person 501. The content of the voice notification may be a linguistic expression such as "I will move to the side," or it may be a non-linguistic expression such as a sound notification that does not depend on words. In this way, when the situation does not improve through requests to the person, the moving object can improve its translation state to an appropriate range on its own, thereby avoiding the continuation of a state that is unsafe or unpleasant. In addition, by notifying the person before the moving object changes its position, the person will not be confused by the change in the position of the moving object.

[0062] Figure 5D shows the case where, after the translational state of a specific person 501 has improved to an appropriate range, the movement speed of the specific person 501 increases, and the translational state is once again outside the appropriate range.

[0063] In this case, the dialogue control unit 403 again outputs an utterance 505 requesting the specific person 501 to "walk slowly." The mobile body 100 may also display an expression 506 indicating that the mobile body 100 is in a high-power state by illuminating the front display unit 113 or the light-emitting unit 116. The expression (expression 506) indicating that the mobile body 100 is in a high-power state may include, for example, an expression that the translation state is not appropriate or is difficult, such as an expression that the mobile body 100 is sweating or breathing heavily. By having the mobile body 100 express the state of a person, a sense of familiarity with the mobile body 100 can be fostered.

[0064] Figure 5E shows the case where the translational state of the specific person 501 improves to an appropriate range after a request from the dialogue control unit 403. When the translational state is optimized, the dialogue control unit 403 notifies the specific person 501 of the moving position of the mobile body 100 and an expression of gratitude (utterance 507). By notifying the moving position of the mobile body, the specific person 501 becomes more strongly aware of the presence of the mobile body 100 and maintains the translational state.

[0065] In this way, if the mobile object 100 determines that the translational state of the specific person 501 is not within the appropriate range, it outputs a request for the specific person to take action to bring the translational state of the specific person within the appropriate range. In this way, it becomes possible to bring the movement of the mobile object and the specific person into an appropriate state.

[0066] Referring to Figure 6, a series of state optimization operations performed by the control unit 130 will be described. This series of operations is achieved by the control unit 130 executing a program stored in the memory device 135, and each part within the control unit 130 executing its own processing. This series of operations can be performed when the mobile body 100 is autonomously driving.

[0067] In S601, the person recognition unit 401 acquires an image of the area around the moving object 100 from the detection unit 114. At this time, the environment recognition unit 402 may also acquire the same image.

[0068] In S602, the person recognition unit 401 recognizes the translational state of a specific person relative to the moving object 100. As described above, the translational state of a specific person relative to the moving object is recognized by the positional state between the moving object and the specific person, and the movement speed of the moving object and the specific person.

[0069] In S603, the dialogue control unit 403 determines whether the recognized translational state is within an appropriate range. The dialogue control unit 403 determines that the translational state of a specific person is not within an appropriate range if, for example, the difference between the movement speed of the specific person and the movement speed of the moving object is greater than a speed threshold, or if the distance between the specific person and the moving object is not included in a certain distance range. If the dialogue control unit 403 determines that the translational state of the specific person is not within an appropriate range, it proceeds to S604; otherwise, it returns to S603.

[0070] This type of determination allows us to determine, for example, that the translational state of a specific person is inappropriate if the distance between the moving object and the specific person is too close or too far. In other words, it allows us to optimize the translational state when the safety decreases due to the proximity of the specific person and the moving object, or when the moving object fails to catch up to the specific person.

[0071] In S604, the dialogue control unit 403 outputs a request for an action to bring the translation state within an appropriate range. As described above, the dialogue control unit 403 outputs an utterance 503 requesting the specific person 501 to "walk slowly, as you cannot move at a speed greater than your current speed."

[0072] In S605, the dialogue control unit 403 determines whether the translational state has improved. For example, if the distance between the moving object 100 and the specific person 501 falls outside the appropriate range, the dialogue control unit 403 determines that the translational state has not improved and proceeds to S606. If the dialogue control unit 403 determines that the translational state has improved, it proceeds to S609.

[0073] In the example shown in Figure 6, the process proceeds to S609 when it is determined that the translational state has improved. However, as described above in Figure 5E, the dialogue control unit 403 may notify the specific person 501 of the moving position of the mobile body 100 and an expression of gratitude once the translational state is optimized.

[0074] In S606, the path generation unit 405 searches for a travel position to bring the translational state of the specific person within an appropriate range. In S607, the dialogue control unit 403 notifies the future position of the moving body 100 before moving to the searched travel position. For example, the dialogue control unit 403 outputs a notification (utterance 504) such as, "I will move to the side because I am obstructing the walking of the specific person."

[0075] In S608, the path generation unit 405 sets the searched travel position as the target position, and the drive control unit 406 moves the mobile body 100 to the target position.

[0076] In S609, the control unit 130 determines whether to continue moving the mobile body 100. If it determines to continue moving, it returns to S603; otherwise, it terminates the process.

[0077] In the above-described embodiment, the control device was explained as being the control unit 130 of the mobile body 100. However, the control device may be included in, for example, an information processing server located outside the mobile body 100. In this case, the control device may perform some or all of the processing performed by the control unit 130 while communicating with the mobile body 100. In other words, this embodiment is also applicable when the mobile body 100 is remotely controlled from a device separate from the mobile body 100.

[0078] As described above, in the above embodiment, the person recognition unit 401 recognizes the translational state of a specific person 501 relative to the moving object 100 using an image taken around the moving object 100, and the dialogue control unit 403 determines whether the translational state of the specific person is within an appropriate range. If the dialogue control unit 403 determines that the translational state of the specific person 501 is not within an appropriate range, it outputs a request for the specific person 501 to take action to bring the translational state of the specific person 501 within an appropriate range. In this way, it becomes possible to bring the movement of the moving object and the specific person into an appropriate state.

[0079] <Summary of Embodiments> (Composition 1) A method for controlling a mobile body that moves autonomously with a specific person, Using images taken around the moving object, the translational state of the specific person relative to the moving object is recognized. Determining whether the translational state of the aforementioned specific person is a predetermined state as an appropriate state, A method for controlling a moving object, which includes, if it is determined that the translational state of the specified person is not the appropriate state, outputting a request for the specified person to take action to bring the translational state of the specified person to the appropriate state.

[0080] In this embodiment, it becomes possible to ensure that the movement of the moving object and the movement of a specific person are in an appropriate state.

[0081] (Configuration 2) The control method according to configuration 1, wherein the determination is made if the magnitude of the difference between the movement speed of the specified person and the movement speed of the moving object is greater than a speed threshold, or if the distance between the specified person and the moving object is not included in a certain distance range, then it is determined that the translational state of the specified person is not the appropriate state.

[0082] In this embodiment, the translational state can be optimized when safety is reduced due to the proximity of a specific person and a moving object, or when the moving object cannot catch up to a specific person.

[0083] (Composition 3) The control method according to configuration 1, wherein the request for the actions of the specified person includes at least one of a request to reduce the movement speed of the specified person and a request to move closer to the moving object.

[0084] In this embodiment, when a specific person and a moving object approach each other and safety is reduced, or when the moving object fails to catch up to a specific person, an intuitive request for improvement of the translational state can be communicated, thereby optimizing the translational state.

[0085] (Composition 4) The control method according to configuration 3, wherein the request for the actions of the specified person further includes an expression indicating the reason why the translational state cannot be maintained.

[0086] In this embodiment, by showing the reasons why the translational state cannot be maintained, a particular person can gain a better understanding of how to maintain the translational state.

[0087] (Composition 5) The control method according to configuration 3, wherein when outputting a request for the actions of the specified person, the display device is made to display an expression indicating that the moving object is in a high-output state.

[0088] In this embodiment, by having the mobile body 100 express the state of a person, it is possible to create a sense of familiarity with the mobile body 100.

[0089] (Composition 6) The control method according to configuration 1, further comprising: if the specified person is moving behind the moving body, and within a predetermined time after outputting a request for the specified person's actions, the translational state of the specified person is not in the appropriate state, the body moves to a driving position where the translational state of the specified person becomes appropriate.

[0090] In this embodiment, if the situation does not improve through requests to a specific person, the moving object can improve its translational state to an appropriate state on its own, thereby avoiding the continuation of conditions that are unsafe or unpleasant.

[0091] (Composition 7) The control method according to configuration 6, further comprising notifying the specified person of the moving body of the moving body's moving position before moving to a moving position where the translational state of the specified person becomes appropriate.

[0092] In this embodiment, by notifying the person before the moving object changes its position, the person will not be confused by the change in the object's location.

[0093] (Composition 8) The control method according to configuration 7, wherein notifying the moving position of the moving body includes notifying it using an expression that indicates the relative position of the moving body from the specific person.

[0094] In this embodiment, a specific person can intuitively understand the movement of the moving object.

[0095] (Composition 9) The control method according to Configuration 1, which, if the translational state of the specified person is not in the appropriate state within a predetermined time after outputting a request for the specified person's actions, repeats the process of outputting a request for the specified person's actions.

[0096] In this embodiment, repetition can increase the chances of a particular person understanding the request.

[0097] (Composition 10) The control method according to Configuration 1, further comprising notifying the specified person of the movement position of the moving body and an expression of gratitude if, within a predetermined time after outputting a request for the actions of the specified person, the translational state of the specified person improves to the appropriate state.

[0098] In this embodiment, by notifying the moving position of the moving object, a specific person becomes more strongly aware of the object's presence and maintains its translational state.

[0099] (Composition 11) The control method according to Configuration 1, wherein outputting the aforementioned request includes, at least temporarily, outputting a request for the actions of the specified person while autonomously moving through an area where pedestrians are walking, while leading the specified person.

[0100] In this embodiment, when a moving object leads a specific person, the movement of the moving object and the specific person can be kept in an appropriate state.

[0101] The invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of the gist of the invention. [Explanation of Symbols]

[0102] 100 ···Mobile unit, 113 ···Front display unit, 115 ···Audio output unit, 116 ···Light-emitting unit, 130 ···Control unit, 140 ···Communication terminal, 401 ···Person recognition unit, 402 ···Environment recognition unit, 403 ···Dialogue control unit, 404 ···Intention estimation unit, 405 ···Route generation unit, 406 ···Drive control unit

Claims

1. A method for controlling a mobile body that moves autonomously with a specific person, Using images taken around the moving object, the translational state of the specific person relative to the moving object is recognized. Determining whether the translational state of the aforementioned specific person is a predetermined state as an appropriate state, A method for controlling a moving object, which includes, if it is determined that the translational state of the specified person is not the appropriate state, outputting a request for the specified person to take action to bring the translational state of the specified person to the appropriate state.

2. The control method according to claim 1, wherein the determination is made in which case the translational state of the specific person is not in the appropriate state if the magnitude of the difference between the movement speed of the specific person and the movement speed of the moving object is greater than a speed threshold, or if the distance between the specific person and the moving object is not included in a certain distance range.

3. The control method according to claim 1, wherein the request for the actions of the specified person includes at least one of a request to reduce the movement speed of the specified person and a request to move closer to the moving object.

4. The control method according to claim 3, wherein the request for the actions of the specified person further includes an expression indicating the reason why the translational state cannot be maintained.

5. The control method according to claim 3, wherein when outputting a request for the actions of the specified person, the display device is made to display an expression indicating that the moving object is in a high-output state.

6. The control method according to claim 1, further comprising: if the specified person is moving behind the moving body, and within a predetermined time after outputting a request for the action of the specified person, the translational state of the specified person is not in the appropriate state, the body moves to a driving position where the translational state of the specified person becomes appropriate.

7. The control method according to claim 6, further comprising notifying the specified person of the moving body of the moving body's moving position before moving to a moving position where the translational state of the specified person becomes appropriate.

8. The control method according to claim 7, wherein notifying the moving position of the moving body includes notifying it using an expression that indicates the relative position of the moving body from the specific person.

9. The control method according to claim 1, wherein if the translational state of the specified person is not in the appropriate state within a predetermined time after outputting a request for the action of the specified person, the process of outputting a request for the action of the specified person is repeated.

10. The control method according to claim 1, further comprising notifying the specified person of the movement position of the moving body and an expression of gratitude if, within a predetermined time after outputting a request for the actions of the specified person, the translational state of the specified person improves to the appropriate state.

11. The control method according to claim 1, wherein outputting the request includes outputting a request for the actions of the specific person while autonomously moving through an area where a pedestrian walks, at least temporarily, following, leading, or guiding the specific person.

12. A control device for a mobile body that moves autonomously with a specific person, A recognition means that recognizes the translational state of the specific person relative to the moving object using an image taken of the area around the moving object, A determination means for determining whether the translational state of the aforementioned specific person is a predetermined state as an appropriate state, A control device for a moving body, which includes an output means that outputs a request for an action by the specified person to bring the translational state of the specified person to an appropriate state, if it is determined that the translational state of the specified person is not the appropriate state.

13. A mobile entity that moves autonomously with a specific person, A recognition means that recognizes the translational state of the specific person relative to the moving object using an image taken of the area around the moving object, A determination means for determining whether the translational state of the aforementioned specific person is a predetermined state as an appropriate state, A mobile body including, if it is determined that the translational state of the specified person is not the appropriate state, an output means that outputs a request for the specified person to take action to bring the translational state of the specified person to the appropriate state.

14. A program for causing a computer to perform each step of the control method described in any one of claims 1 to 11.

15. A storage medium storing a program for causing a computer to perform each step of the control method described in any one of claims 1 to 11.

Images