Control method, control device, mobile device and program

Abstract

This provides new technologies for controlling mobile objects. [Solution] A control method for controlling a moving object that leads or guides a user, comprising: a first step of acquiring the state of each of a plurality of pedestrians present in the surrounding environment of the moving object, including a first pedestrian who is a follower and is following the moving object; and a second step of controlling the behavior of the moving object so that the predetermined state of the first pedestrian is resolved when the state of the first pedestrian acquired in the first step is in a predetermined state.

Description

Technical Field

[0001] The present invention relates to a control method, a control device, a mobile object, and a program.

Background Art

[0002] In recent years, research and development of mobile objects (self-propelled mobile objects) having a function of autonomously traveling (moving) without requiring operation (driving) by a user (user) have been progressing. As such a mobile object, for example, a so-called micro mobility, which is a robot that supports the walking of a user, represented by a follow-up type mobile object that travels following a user and a leading type mobile object that travels while leading (guiding) the user, is known.

[0003] Considering the actual road traffic environment in which micro mobility (and its user) participates, a crowded scene in which a large number of pedestrians exist as traffic participants in addition to the user is assumed as one of such scenes. In a crowded scene, in order to make the micro mobility travel according to the flow of surrounding pedestrians, making the micro mobility follow the surrounding (for example, forward) pedestrians is preferable as a cooperative action for smoothing the traffic of traffic participants.

[0004] As described above, the mobile object is required to have a technology for controlling the action and behavior of the mobile object in accordance with the road traffic environment in which the mobile object participates. Patent Document 1 discloses a technology that is not related to a self-propelled mobile object, but presents traveling situation information indicating the situation in which the mobile object is placed, based on the traveling information of the mobile object, in a method corresponding to the adaptation state of the user (driver) who operates the mobile object to the mobile object.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0006] However, when micromobility vehicles are used as a coordinated action in response to the road traffic environment, allowing them to follow surrounding pedestrians, there is a risk of causing psychological stress to the pedestrians being followed. This is because being arbitrarily targeted and tracked by a micromobility vehicle, as well as having the vehicle travel close by, can be factors that cause psychological stress to pedestrians. Thus, there is room for improvement in the practice of having micromobility vehicles follow surrounding pedestrians in order to mitigate the psychological burden on the pedestrians being followed.

[0007] The present invention aims to provide a novel technique for controlling a moving object. [Means for solving the problem]

[0008] One aspect of the present invention is a control method for controlling a moving body that is leading or guiding a user, comprising: a first step of acquiring the state of each of a plurality of pedestrians present in the surrounding environment of the moving body, including a first pedestrian who is a follower and is following the moving body; and a second step of controlling the behavior of the moving body so that the predetermined state of the first pedestrian is resolved when the state of the first pedestrian acquired in the first step is in a predetermined state.

[0009] Another aspect of the present invention is a control device for controlling a mobile body that travels leading or guiding a user, characterized by comprising: an acquisition unit that acquires the state of each of a plurality of pedestrians present in the surrounding environment of the mobile body, including a first pedestrian who is a follower and is following the mobile body; and a control unit that controls the behavior of the mobile body so that the predetermined state of the first pedestrian is resolved when the state of the first pedestrian acquired by the acquisition unit is in a predetermined state.

[0010] A mobile body, as yet another aspect of the present invention, is a mobile body that travels leading or guiding a user, and is characterized by comprising: an acquisition unit that acquires the state of each of a plurality of pedestrians present in the surrounding environment of the mobile body, including a first pedestrian who is a follower and is following the mobile body; and a control unit that controls the behavior of the mobile body so that when the state of the first pedestrian acquired by the acquisition unit is in a predetermined state, the predetermined state of the first pedestrian is resolved.

[0011] Another aspect of the present invention is a program that causes a computer to execute the control method described above.

[0012] Further objects or other aspects of the present invention will be revealed by embodiments described below with reference to the accompanying drawings. [Effects of the Invention]

[0013] According to the present invention, for example, a new technology relating to the control of a moving object can be provided. [Brief explanation of the drawing]

[0014] [Figure 1] This is a perspective view showing the overall configuration of the mobile body as one aspect of the present invention. [Figure 2] This is a plan view illustrating the configuration of the running unit. [Figure 3] This is a block diagram showing the control system of a mobile object. [Figure 4] This diagram shows the configuration of the functional blocks realized by the control unit of the mobile body. [Figure 5] This is a flowchart illustrating the control process in a mobile device. [Figure 6] This is a diagram illustrating the control process in a mobile device. [Figure 7] This is a diagram illustrating the control process in a mobile device. [Figure 8] This is a diagram illustrating the control process in a mobile device. [Figure 9]This is a diagram for explaining control processing in a moving object. [Figure 10] This is a diagram for explaining control processing in a moving object. [Figure 11] This is a diagram illustrating an example of a method for realizing a moving object overtaking a pedestrian. [Modes for Carrying Out the Invention]

[0015] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the scope of claims, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. In addition, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0016] In the present embodiment, a micro-mobility, specifically a small electric vehicle that can load cargo and travels while leading or guiding a user (user) will be described as an example of the moving object. However, the moving object may be a micro-mobility that can carry not only cargo but also a person. Further, the present embodiment does not limit the moving object to an electric vehicle, and is applicable to moving objects other than electric vehicles. In the following description, a moving object having one driven wheel will be described as an example, but the moving object may have two or more driven wheels. Note that the driven wheels are not necessarily required, and the moving object may not have driven wheels.

[0017] In the present embodiment, the moving object preferably travels autonomously (realizes autonomous traveling) when leading or guiding the user. In addition, even when the moving object is not traveling while leading or guiding the user, it is preferable to transmit that it recognizes the user via an HMI (Human Machine Interface) or the like, suppress the user's anxiety, and give a sense of security.

[0018] Referring to Figure 1, the configuration of the mobile body 100 in this embodiment will be described. Figure 1 is a perspective view showing the overall configuration of the mobile body 100. As shown in Figure 1, the mobile body 100 has a lower housing 111, an upper housing 112, a front display unit 113, a plurality of detection units 114, an output unit 115, a light-emitting unit 116, two drive wheels 120, and a driven wheel 121. In the mobile body 100, the side on which the drive wheels 120 are provided is considered the front.

[0019] The lower housing 111 rotatably holds the drive wheels 120 and the driven wheels 121, and houses the running unit including the drive wheels 120 and the driven wheels 121. The lower housing 111 includes a loading space 118 on its upper part on which the user's luggage can be loaded. The lower housing 111 houses the light-emitting unit 116. The area of ​​the lower housing 111 that houses the light-emitting unit 116 is made of a material that can transmit light.

[0020] The upper housing 112 is configured to extend upward from the rear of the lower housing 111. At its upper end, the upper housing 112 supports the front display unit 113, the detection unit 114, and the output unit 115.

[0021] The front display unit 113 displays images and characters, and notifies (provides) various information to the user and those around them through such images and characters. The front display unit 113 includes a display device that displays images, such as a liquid crystal display device or an organic EL (Electro Luminescence) display device. The front display unit 113 is provided facing forward at the upper front end of the upper housing 112. The front display unit 113 notifies the user of information by displaying images or characters that resemble human faces.

[0022] The multiple detection units 114 have the function of detecting targets in the surrounding environment of the mobile body 100, such as people (pedestrians, etc.) and objects present around the mobile body 100. For example, the detection unit 114 includes a camera having an optical system such as a lens and an image sensor, and captures images of the area around the mobile body 100 and outputs image information. Alternatively, the detection unit 114 may include radar or lidar (Light Detection and Ranging) in place of, or in addition to, a camera, to detect targets present around the mobile body 100. The multiple detection units 114 are installed (arranged) for example on the front, back, left, and right sides of the upper end of the upper housing 112. Therefore, the multiple detection units 114 can detect targets in all directions around the mobile body 100 (i.e., 360 degrees around the mobile body 100). Note that the installation position and number of the detection units 114 are not particularly limited and can be changed as appropriate.

[0023] The output unit 115 outputs sound and notifies (provides) various information to the user and those around them through this sound. The output unit 115 includes an audio output device such as a speaker.

[0024] The light-emitting unit 116 notifies (provides) various information to the user and those around them by emitting light. The light-emitting unit 116 includes, for example, one or more light-emitting elements such as LEDs (Light Emitting Diodes). In this embodiment, the light-emitting unit 116 is provided so as to surround the lower housing 111. The light-emitting unit 116 may emit light of multiple colors. The light-emitting unit 116 can also emit light of different colors depending on the light-emitting region or direction. For example, the light-emitting unit 116 emits light of different colors in the front region 116F, the left region 116L, and the right region (not shown). In addition, the light-emitting unit 116 can switch between illumination and extinguishing for each light-emitting region. Note that the light-emitting region of the light-emitting unit 116 may be divided into even finer regions than the region shown in Figure 1.

[0025] The drive wheels 120 are located at the front and lower left and right sides of the lower housing 111. The drive wheels 120 rotate around the axle when driving force is transmitted from the running unit.

[0026] The driven wheel 121 is located at the rear of the lower housing 111 and in the center of the lower part. The driven wheel 121 is held so as to be able to pivot around a vertical axis that extends in the vertical direction.

[0027] Referring to Figure 2, the travel unit 125 of the mobile body 100 will be described. Figure 2 is a schematic plan view showing the configuration of the travel unit 125.

[0028] The driving unit 125 is located at the bottom of the lower housing 111. In addition to two drive wheels 120 and a driven wheel 121, the driving unit 125 includes a drive mechanism 122, motors 123a and 123b, and a battery 124.

[0029] When power is supplied from the battery 124, motors 123a and 123b function as a drive source and supply driving force to the drive mechanism 122. The drive mechanism 122 transmits the driving force supplied from motors 123a and 123b to the drive wheels 120. This causes the drive mechanism 122 to rotate the drive wheels 120, making the mobile body 100 move (forward or backward). The drive mechanism 122 can change the direction in which the mobile body 100 moves (travels) to the left or right by creating a rotational difference between motors 123a and 123b. By changing the rotational difference between motors 123a and 123b, the drive mechanism 122 can achieve various turns of the mobile body 100, such as pivot turns, slow turns, and super-pivot turns.

[0030] The control system of the mobile unit 100 will be described with reference to Figure 3. Figure 3 is a block diagram of the control system of the mobile unit 100. As shown in Figure 3, the mobile unit 100 further includes a control unit 130, an operation unit 132, a top display unit 131, an input unit 133, a GNSS sensor 134, a storage unit 135, and a communication unit 136.

[0031] The control unit 130 is a control device (control unit) that controls the entire mobile body 100, and is also called an ECU (Electronic Control Unit). The control unit 130 is composed of, for example, a computer and comprehensively controls each part of the mobile body 100 to operate it. The control unit 130 includes one or more processors, such as a CPU (Central Processing Unit), memory devices such as semiconductor memory, and interfaces with external devices.

[0032] The control unit 130 may include, in addition to or instead of a CPU, an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an NPU (Neural Processing Unit), a QPU (Quantum Processing Unit), etc. The processor realizes various functions of the control unit 130 by reading and executing computer programs stored in memory devices or the like.

[0033] Some or all of the functions of the control unit 130 may be implemented by one or more circuits, such as an ASIC (Application Specific Integrated Circuit) and a PLD (Programmable Logic Device) including an FPGA (Field Programmable Gate Array).

[0034] The memory device includes, for example, RAM (Random Access Memory). The memory device stores programs executed by the processor and data used by the processor for processing. Multiple combinations of processors, memory devices, and interfaces are provided for each function of the mobile unit 100, and each combination may be configured to communicate with one another.

[0035] The control unit 130 controls the mobile body 100 based on outputs from multiple detection units 114 (detection information such as the detection results of targets present around the mobile body 100), user input information acquired via the operation unit 132, and user voice information acquired via the input unit 133. The control unit 130 performs behavioral control, including the movement of the mobile body 100, by controlling the driving unit 125 (motors 123a and 123b). The control unit 130 also notifies (provides) various information to the user and people in the surrounding area via the front display unit 113, top display unit 131, light-emitting unit 116, and output unit 115. The control unit 130 may perform processing on image information, which is one of the detection information acquired by the detection unit 114, using a machine learning model for image recognition (e.g., a deep neural network). The control unit 130 may also perform processing on voice information acquired by the input unit 133 using a machine learning model for speech recognition (e.g., a deep neural network).

[0036] The top display unit 131 is provided on the upper surface of the upper end of the upper housing 112 and includes a display device that displays images and characters. Based on image information acquired from the control unit 130, the top display unit 131 displays images and characters related to information to be notified to the user and those around them. The top display unit 131 also displays, for example, an operation screen for accepting user operations (inputs).

[0037] The operation unit 132 functions as an input device such as a touch panel or keyboard, and in this embodiment, it is provided via an operation screen displayed on the top display unit 131. The operation unit 132 receives operations from the user and outputs the operation information to the control unit 130.

[0038] The input unit 133 functions as a sound collection device such as a microphone. The input unit 133 collects the voices of the user and surroundings of the mobile unit 100 and outputs them as audio information to the control unit 130.

[0039] The GNSS sensor 134 receives GNSS (Global Navigation Satellite System) signals from satellites to detect (acquire) the current position of the mobile object 100.

[0040] The memory unit 135 includes a non-volatile storage medium for storing various information and data. The memory unit 135 includes HDDs (Hard Disk Drives) and SSDs (Solid State Drives), etc. The memory unit 135 may store programs executed by the processor, parameters necessary for program execution, and data to be processed by the processor. The memory unit 135 may also store various parameters of machine learning models executed by the control unit 130 (for example, trained parameters and hyperparameters of a deep neural network).

[0041] The communication unit 136 communicates with external devices such as a communication terminal 140 owned by the user, for example, via wireless communication such as Wi-Fi or fifth-generation mobile communication.

[0042] In a road traffic environment in which a mobile body 100 with such a configuration participates, there are generally many other traffic participants besides the user of the mobile body 100. In particular, since it is assumed that there are multiple pedestrians around the mobile body 100, it is necessary to make the mobile body 100 move in accordance with the flow of surrounding pedestrians. Therefore, in this embodiment, as a coordinated action with respect to the road traffic environment in which the mobile body 100 participates, the mobile body 100 is made to follow the surrounding pedestrians. However, making the mobile body 100 follow the pedestrians around it may cause psychological burden on the pedestrians being followed. Therefore, this embodiment provides a new technology related to the control of the mobile body 100 (its actions and behavior) to alleviate the psychological burden on the pedestrians being followed by the mobile body 100.

[0043] In this embodiment, in order to realize a technology for alleviating the psychological burden on pedestrians who are followed by the mobile body 100, the control unit 130 of the mobile body 100 includes a functional block, as shown in Figure 4. Specifically, the control unit 130 includes an acquisition unit 401, a determination unit 402, and an action control unit 403 as functional blocks. Figure 4 is a diagram showing the configuration of the functional blocks realized by the control unit 130 of the mobile body 100.

[0044] The acquisition unit 401 acquires the status of each of the multiple pedestrians present around the mobile body 100 (the surrounding environment). The area around the mobile body 100 is defined as an area within a predetermined distance from the mobile body 100, for example, the area inside a circle with a predetermined radius centered on the mobile body 100. The multiple pedestrians present around the mobile body 100 include pedestrians that the mobile body 100 follows (hereinafter sometimes referred to as "first pedestrians") and pedestrians that are not followed.

[0045] In this embodiment, the acquisition unit 401 acquires the psychological load state representing the stress on the mobile body 100 of multiple pedestrians present around the mobile body 100, as the state of each of the pedestrians. First, the acquisition unit 401 identifies multiple pedestrians present around the mobile body 100 from the detection information (image information) acquired by the multiple detection units 114, and acquires pedestrian information by recognizing at least one of each pedestrian's facial expression and behavior. For example, a pedestrian's facial expression can be recognized by detecting feature points related to the pedestrian's facial expression (such as the eyes, the ends of the eyebrows, the tip of the nose, and the ends of the mouth) from the image information, and analyzing the movement of these feature points to analyze the pedestrian's facial expression. For example, Active Shape Models (ASM), Active Appearance Models (AAM), and Constrained Local Models (CLM) can be used to detect feature points. On the other hand, a pedestrian's behavior can be recognized by learning the positions of the pedestrian's joints from the image information to estimate the pedestrian's skeleton, and analyzing the movement of this skeleton to analyze the pedestrian's behavior. For skeletal estimation, for example, Human Pose Estimation is used. From this pedestrian information, the acquisition unit 401 can estimate the pedestrian's psychological stress level by determining the pedestrian's facial expressions (changes) and pedestrian behavior (changes).

[0046] The estimation of the pedestrian's psychological burden state in the acquisition unit 401 will be explained in detail. It is preferable to quantify the pedestrian's psychological burden state, and for example, it is preferable that the numerical value increases as the psychological burden increases from a state of light to a state of heavy psychological burden.

[0047] For example, if a pedestrian's facial expression is positive (e.g., happy, natural), the acquisition unit 401 estimates that the pedestrian's stress on the moving object 100 is low, and therefore estimates that the pedestrian's psychological burden is light. On the other hand, if a pedestrian's facial expression is negative (e.g., angry, disgusted, afraid, surprised, sad), the acquisition unit 401 estimates that the pedestrian's stress on the moving object 100 is high, and therefore estimates that the pedestrian's psychological burden is heavy.

[0048] Furthermore, if pedestrians rarely turn around, the acquisition unit 401 estimates that the pedestrian's psychological burden is low, as it is considered that the pedestrian's stress on the moving object 100 is low. On the other hand, if pedestrians frequently turn around, the acquisition unit 401 estimates that the pedestrian's psychological burden is high, as it is considered that the pedestrian's stress on the moving object 100 is high.

[0049] Furthermore, if pedestrians rarely accelerate or decelerate (accelerating or decelerating), the acquisition unit 401 estimates that the pedestrian's psychological burden is light, as it is considered that the pedestrian's stress on the moving object 100 is low. On the other hand, if pedestrians frequently accelerate or decelerate (accelerating or decelerating), the acquisition unit 401 estimates that the pedestrian's psychological burden is heavy, as it is considered that the pedestrian's stress on the moving object 100 is high.

[0050] Furthermore, if a pedestrian rarely performs predefined gestures (for example, hand gestures to encourage the moving object 100), the pedestrian's stress on the moving object 100 is considered to be low, and the acquisition unit 401 estimates that the pedestrian's psychological burden is light. On the other hand, if a pedestrian frequently performs predefined gestures, the pedestrian's stress on the moving object 100 is considered to be high, and the acquisition unit 401 estimates that the pedestrian's psychological burden is heavy.

[0051] The determination unit 402 extracts the state of the first pedestrian, which is the target of the moving body 100's tracking, from the states of each of the multiple pedestrians acquired by the acquisition unit 401, and determines whether the state of the first pedestrian is in a predetermined state. In this embodiment, the determination unit 402 determines whether the psychological load state indicating the state of the first pedestrian acquired by the acquisition unit 401 exceeds a threshold.

[0052] In this way, the determination unit 402 determines whether the psychological burden on the first pedestrian towards the moving object 100 is heavy, that is, whether the first pedestrian is experiencing strong stress from being followed by the moving object 100. Therefore, it is preferable to set the threshold to correspond to a heavy psychological burden on the pedestrian. For example, when estimating the psychological burden on the pedestrian from the pedestrian's facial expression, the threshold is set to a negative facial expression (or a corresponding numerical value). When estimating the psychological burden on the pedestrian from the pedestrian turning around, the threshold is set to the number of times the pedestrian turns around when the psychological burden is heavy (or a corresponding numerical value). When estimating the psychological burden on the pedestrian from the pedestrian's acceleration and deceleration, the threshold is set to the number of times the pedestrian accelerates or decelerates when the psychological burden is heavy (or a corresponding numerical value). When estimating the psychological burden on the pedestrian from the pedestrian's predefined gestures, the threshold is set to the number of times the predefined gesture was performed (or a corresponding numerical value).

[0053] The behavior control unit 403 controls the behavior of the mobile body 100 via the travel unit 125 (performs behavior control). Here, the behavior of the mobile body 100 includes the movement and stopping of the mobile body 100, and the following and stopping of the first pedestrian, which is the target of the mobile body 100's follow.

[0054] In this embodiment, the behavior control unit 403 controls the behavior of the mobile body 100 so as to resolve the state of the first pedestrian, which is followed by the mobile body 100, when the determination unit 402 determines that the state of the first pedestrian is in a predetermined state, that is, the psychological load state of the first pedestrian exceeds a threshold. The behavior of the mobile body 100 to resolve the state of the first pedestrian's psychological load state being in a predetermined state, that is, exceeding a threshold, will be explained in detail later in relation to the control processing in the mobile body 100. On the other hand, when the determination unit 402 determines that the state of the first pedestrian, which is followed by the mobile body 100, is not in a predetermined state, that is, the psychological load state of the first pedestrian is below a threshold, the behavior control unit 403 controls the behavior of the mobile body 100 so as to maintain (continue) following the first pedestrian.

[0055] The control process in the mobile body 100 will be explained with reference to Figure 5. Figure 5 is a flowchart illustrating the control process in the mobile body 100, in particular the control process that causes the mobile body 100 to follow pedestrians around it. This control process is performed by the control unit 130 comprehensively controlling each part of the mobile body 100.

[0056] In this embodiment, the road traffic environment in which the mobile body 100 participates will be described using the example shown in Figure 6, where multiple pedestrians PD1, PD2, PD3, PD4, PD5, PD5, PD6, and PD7 are present around the mobile body 100. Furthermore, the user of the mobile body 100 (the pedestrian led or guided by the mobile body 100) is pedestrian PD1, and the target of the mobile body 100's follow (the pedestrian who causes the mobile body 100 to follow) is pedestrian PD2 (the first pedestrian). It will be assumed that the direction in which the mobile body 100 travels coincides with the direction in which the multiple pedestrians PD1 to PD7 walk. Figure 6 is a diagram for explaining the control process in the mobile body 100 and shows an example of a road traffic environment in which the mobile body 100 participates.

[0057] In S502, the behavior control unit 403 controls the behavior of the mobile body 100 so that it follows the first pedestrian, i.e., pedestrian PD2, which is the target of the follow operation. In this way, in S502, the mobile body 100 is made to follow the first pedestrian (pedestrian PD2), which is the target of the follow operation.

[0058] In S504, the acquisition unit 401 acquires the psychological load state of each of the multiple pedestrians PD1 to PD7 present around the moving body 100. The method by which the acquisition unit 401 acquires the psychological load state of each pedestrian PD1 to PD7 is as described above, so a detailed explanation is omitted here.

[0059] In S506, the determination unit 402 determines whether the state of the first pedestrian, pedestrian PD2, acquired in S504, is in a predetermined state, that is, whether the psychological load state of pedestrian PD2 exceeds a threshold. If the state of the first pedestrian, pedestrian PD2, is not in a predetermined state (the psychological load state does not exceed a threshold), the process proceeds to S502, and the behavior of the mobile body 100 is controlled to maintain (continue) following pedestrian PD2. On the other hand, if the state of the first pedestrian, pedestrian PD2, is in a predetermined state (the psychological load state exceeds a threshold), the process proceeds to S508.

[0060] In S508, the behavior control unit 403 controls the behavior of the mobile body 100 so that a predetermined state of the first pedestrian, pedestrian PD2, is resolved, that is, so that the state in which pedestrian PD2's psychological load exceeds a threshold is resolved. In this way, S508 causes the mobile body 100 to take action to resolve the predetermined state of the first pedestrian.

[0061] In this embodiment, when the state of the first pedestrian (pedestrian PD2), which is the target of the moving body 100, is in a predetermined state, the behavior of the moving body 100 is controlled so that the predetermined state of the first pedestrian is resolved. This makes it possible to alleviate the psychological burden on the first pedestrian towards the moving body 100 according to the state of the first pedestrian. Furthermore, by setting the state of the first pedestrian as a psychologically burdened state, it is advantageous to alleviate the psychological burden on the first pedestrian towards the moving body 100 in a manner appropriate to the psychological burden state of the first pedestrian. In addition, by estimating the psychological burden state from pedestrian information obtained by recognizing at least one of the pedestrian's facial expression and behavior, it is possible to easily and accurately obtain the psychological burden state of the pedestrian. Furthermore, by setting the predetermined state of the first pedestrian as a state in which the psychological burden state exceeds a threshold, it is possible to alleviate the psychological burden on the first pedestrian towards the moving body 100 when the first pedestrian is experiencing strong stress.

[0062] The following describes in detail the actions (resolution actions) of the mobile body 100 to resolve the predetermined state of the first pedestrian, who is the target of the mobile body 100's tracking, in S508, that is, the state in which the psychological burden exceeds a threshold.

[0063] One specific example of the mobile body 100's elimination behavior is to remove pedestrian PD2, the first pedestrian, from its tracking target and cease tracking pedestrian PD2. As a result, the distance DT between pedestrian PD2 and the mobile body 100 will widen as pedestrian PD2 continues walking (see Figure 6). Therefore, the psychological burden on pedestrian PD2 towards the mobile body 100 can be reduced. Furthermore, from the perspective of reducing the psychological burden on pedestrian PD2 towards the mobile body 100 in a shorter time, it is preferable to stop tracking pedestrian PD2 and stop the mobile body 100 from moving.

[0064] Another specific example of the mobile body 100's disengagement behavior is shown in Figure 7, where it selects a second pedestrian, different from the first pedestrian PD2, among a group of pedestrians PD3 to PD7, for example, pedestrian PD3, as its target for following, and begins following pedestrian PD3. In this way, by changing the first pedestrian that the mobile body 100 follows from pedestrian PD2 to pedestrian PD3, the mobile body 100 moves from a position behind pedestrian PD2 to a position behind pedestrian PD3 and follows pedestrian PD3. Therefore, the psychological burden on pedestrian PD2 towards the mobile body 100 can be reduced. Figure 7 is a diagram illustrating the control process in the mobile body 100 and shows an example of the mobile body 100's disengagement behavior.

[0065] Furthermore, as shown in Figure 7, when changing the first pedestrian who is followed by the mobile body 100, it is preferable to select a pedestrian from among the multiple pedestrians PD3 to PD7 whose psychological stress level is below the threshold as the new second pedestrian to be followed. Even if the target of the mobile body 100 to be followed is changed from the first pedestrian to the second pedestrian, if the psychological stress level of the new target, the second pedestrian, exceeds the threshold, the target of the mobile body 100 to be followed must be changed again to a pedestrian other than the second pedestrian. Therefore, by selecting a pedestrian whose psychological stress level is below the threshold as the new second pedestrian to be followed, it is possible to avoid repeatedly changing the target of the mobile body 100 to be followed. It is preferable to select the pedestrian from among the multiple pedestrians PD3 to PD7 whose difference between the psychological stress level and the threshold is the largest, that is, the pedestrian who causes the least stress to the mobile body 100, as the new second pedestrian to be followed. This is expected to lengthen the time it takes for the psychological stress level of the second pedestrian, the new target of the moving body, to exceed a threshold, thereby suppressing an increase in the number of times the moving body 100 changes its target.

[0066] Furthermore, as shown in Figure 7, when changing the first pedestrian who is followed by the mobile body 100, the pedestrian closest to the mobile body 100 among the multiple pedestrians PD3 to PD7 may be designated as the new second pedestrian to be followed. This allows the mobile body 100 to quickly follow the new second pedestrian when changing the target being followed by the mobile body 100. This also contributes to improving usability for pedestrian PD1, who is the user being led or guided by the mobile body 100.

[0067] Another specific example of the movement of the moving object 100 is to obtain the density (population density) of pedestrians PD1 to PD7 present around the moving object 100 and move the moving object 100 to an area where this density is below a predetermined value. This allows the moving object 100 to move quickly into an open space and to create distance (distance DT) from the first pedestrian, pedestrian PD2, thereby mitigating the psychological burden on pedestrian PD2 towards the moving object 100. The density of pedestrians PD1 to PD7 present around the moving object 100 can be determined from detection information such as the detection results of targets present around the moving object 100 obtained by the detection unit 114.

[0068] Another specific example of a resolving action by the moving object 100 is to change its relative position to the first pedestrian, pedestrian PD2. For example, by moving the moving object 100 from directly behind pedestrian PD2 to the left rear or right rear, the stress on pedestrian PD2 towards the moving object 100 can be reduced, and the psychological burden on pedestrian PD2 towards the moving object 100 can be alleviated.

[0069] Specifically, as shown in Figure 8, a virtual follow lane VLN and a virtual adjacent lane VAL adjacent to the virtual follow lane are set up to make the mobile body 100 follow the first pedestrian, pedestrian PD2. Then, the lane in which the mobile body 100 is traveling is changed from the virtual follow lane VLN to the virtual adjacent lane VAL. This effectively changes the lane of the mobile body 100 (changes the relative position of the mobile body 100 with respect to pedestrian PD2), thereby mitigating the psychological burden on pedestrian PD2 towards the mobile body 100. Figure 8 is a diagram illustrating the control process in the mobile body 100 and shows an example of the mobile body 100's resolution behavior.

[0070] Furthermore, as shown in Figure 9, the mobile body 100 may be moved away from the first pedestrian, pedestrian PD2. This can be achieved, for example, by slowing down the mobile body 100's speed in response to the actions of pedestrian PD2, or by generating a virtual obstacle at pedestrian PD2's position that the mobile body 100 should avoid colliding with. In this way, by moving the mobile body 100 away from pedestrian PD2, the stress on pedestrian PD2 towards the mobile body 100 is reduced, and the psychological burden on pedestrian PD2 towards the mobile body 100 can be alleviated. Figure 9 is a diagram illustrating the control process in the mobile body 100 and shows an example of the mobile body 100's elimination behavior.

[0071] Furthermore, as shown in Figure 9, when moving the mobile body 100 away from the first pedestrian, pedestrian PD2, the acceleration (running speed) of the mobile body 100 may be changed according to the acceleration (walking speed) of the first pedestrian, pedestrian PD2. The acceleration of each pedestrian can be directly detected as detection information by multiple detection units 114 from the actions of each pedestrian around the mobile body 100. Therefore, the pedestrian information acquired from the detection information by the acquisition unit 401 also includes information on the acceleration of each pedestrian around the mobile body 100 and its changes. Changing the acceleration of the mobile body 100 according to the acceleration of the first pedestrian, pedestrian PD2 is particularly useful when the acceleration of the first pedestrian, pedestrian PD2 is increasing. This is because when the acceleration of the first pedestrian, pedestrian PD2 is increasing, it is thought that pedestrian PD2 is feeling a very strong stress from the mobile body 100, and it is necessary to quickly move the mobile body 100 away from pedestrian PD2. Therefore, when the acceleration of the first pedestrian, pedestrian PD2, increases, it is preferable to move the moving body 100 away from pedestrian PD2 by changing the acceleration of the moving body 100 in accordance with the acceleration of pedestrian PD2. This makes it possible to quickly alleviate the psychological burden on pedestrian PD2 towards the moving body 100.

[0072] Another specific example of the mobile body 100's elimination behavior is, as shown in Figure 10, overtaking pedestrian PD2 by changing the acceleration (running speed) of the mobile body 100 in accordance with the acceleration (walking speed) of pedestrian PD2, who is the first pedestrian. Overtaking pedestrian PD2 by the mobile body 100 can be achieved by sequentially generating virtual obstacles (VOBs) that the mobile body 100 should avoid colliding with, as the position of pedestrian PD2 changes as pedestrian PD2 walks, as shown in Figure 11. Overtaking pedestrian PD2 by the mobile body 100 is particularly useful when pedestrian PD2's acceleration is increasing. This is because, as mentioned above, when pedestrian PD2's acceleration is increasing, it is thought that pedestrian PD2 is experiencing very strong stress from the mobile body 100, and it is necessary to quickly remove pedestrian PD2 from the tracking target. Overtaking pedestrian PD2 by the mobile body 100 also effectively notifies pedestrian PD2 that it has been removed from the tracking target of the mobile body 100. Therefore, the psychological burden on pedestrian PD2 towards the moving object 100 can be quickly alleviated. Figure 10 is a diagram illustrating the control process in the moving object 100 and shows an example of the object's resolution behavior. Figure 11 is a diagram illustrating an example of a method for enabling the moving object 100 to overtake pedestrian PD2.

[0073] In the embodiment described above, the control device was described as a control unit 130 of the mobile body 100. However, the control device may be included in a device located outside the mobile body 100, for example, an information processing server. In this case, the control device may perform some or all of the processing performed by the control unit 130 while communicating with the mobile body 100. Thus, this embodiment is also applicable when the mobile body 100 is remotely controlled from a device separate from the mobile body 100.

[0074] <Summary of Embodiments> 1. The control method of the above embodiment is as follows: A control method for controlling a mobile body (e.g., 100) that drives while leading or guiding a user (e.g., PD1), A first step (e.g., S504) is to acquire the state of each of the multiple pedestrians (e.g., PD1 to PD7) present in the surrounding environment of the moving object, including the first pedestrian (e.g., PD2) who is the object being followed and causing the moving object to follow it, If the state of the first pedestrian obtained in the first step is in a predetermined state, a second step (for example, S508) controls the behavior of the moving body so that the predetermined state of the first pedestrian is resolved, It is characterized by having the following:

[0075] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be mitigated depending on the state of the first pedestrian.

[0076] 2. In the control method described above, The state of each of the multiple pedestrians (e.g., PD1 to PD7) acquired in the first step (e.g., S504) is characterized in that it includes the psychological load state of the pedestrian.

[0077] This embodiment is advantageous in mitigating the psychological burden on the first pedestrian towards the moving object in a manner appropriate to the psychological burden state of the first pedestrian.

[0078] 3. In the control method described above, The first step (for example, S504) is characterized by recognizing at least one of the facial expressions and behaviors of each of the plurality of pedestrians (for example, PD1 to PD7) to acquire pedestrian information, and estimating the psychological load state from the pedestrian information.

[0079] According to this embodiment, it becomes possible to easily and accurately acquire the psychological stress level of pedestrians.

[0080] 4. In the control method described above, The predetermined state is characterized in that it includes a state in which the psychological load state exceeds a threshold.

[0081] According to this embodiment, when the first pedestrian is experiencing high levels of stress, the psychological burden on the first pedestrian towards the moving object can be mitigated.

[0082] 5. In the control method described above, The second step (for example, S508) is characterized by controlling the behavior of the moving body (for example, 100) so as to remove the first pedestrian (for example, PD2) from the tracking target and to stop tracking the first pedestrian.

[0083] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be reduced.

[0084] 6. In the control method described above, The second step (for example, S508) is characterized in that, among the plurality of pedestrians (for example, PD1 to PD7), a second pedestrian (for example, PD3) different from the first pedestrian (for example, PD2) is selected as the target for following, and the behavior of the moving body (for example, 100) is controlled to start following the second pedestrian.

[0085] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be reduced.

[0086] 7. In the control method described above, The second step (for example, S508) is characterized in that, among the plurality of pedestrians (for example, PD1 to PD7), the pedestrian whose psychological load state acquired in the first step (for example, S504) is below the threshold is designated as the second pedestrian (for example, PD3).

[0087] According to this embodiment, it is possible to avoid repeatedly changing the target being followed by the moving object.

[0088] 8. In the control method described above, The second step (for example, S508) is characterized in that, among the plurality of pedestrians (for example, PD1 to PD7), the pedestrian with the largest difference between the psychological load state acquired in the first step (for example, S504) and the threshold is designated as the second pedestrian (for example, PD3).

[0089] According to this embodiment, it is possible to suppress an increase in the number of times the target being followed by the moving object is changed.

[0090] 9. In the control method described above, The second step (for example, S508) is characterized in that, among the plurality of pedestrians (for example, PD1 to PD7), the pedestrian closest in distance from the moving object (for example, 100) is designated as the second pedestrian (for example, PD3).

[0091] According to this embodiment, the moving object can be made to follow a second pedestrian, which is a new target for tracking, in a short amount of time.

[0092] 10. In the control method described above, The second step (for example, S508) is characterized by obtaining the density of pedestrians (for example, PD1 to PD7) present in the surrounding environment and controlling the behavior of the moving body (for example, 100) so that it moves to an area where the density is less than or equal to a predetermined value.

[0093] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be reduced.

[0094] 11. In the control method described above, The second step (for example, S508) is characterized by controlling the behavior of the moving body (for example, 100) so as to change the relative position of the moving body (for example, 100) with respect to the first pedestrian (for example, PD2).

[0095] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be reduced.

[0096] 12. In the control method described above, The second step (for example, S508) is characterized by setting a virtual adjacent lane (VAL) adjacent to a virtual follow lane (for example, VLN) for causing the moving body (for example, 100) to follow the first pedestrian (for example, PD2), and controlling the behavior of the moving body so that the lane in which the moving body travels is changed from the virtual follow lane to the virtual adjacent lane.

[0097] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be reduced.

[0098] 13. In the control method described above, The second step (for example, S508) is characterized by controlling the behavior of the moving body (for example, 100) so that it moves away from the first pedestrian (for example, PD2).

[0099] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be reduced.

[0100] 14. In the control method described above, The pedestrian information includes information regarding changes in the acceleration of the pedestrian (for example, PD1 to PD7), The present invention is characterized in that, when the acceleration of the first pedestrian (e.g., PD2) is increasing, the behavior of the moving body (e.g., 100) is controlled so that the moving body moves away from the first pedestrian by changing the acceleration of the moving body in accordance with the acceleration of the first pedestrian.

[0101] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be quickly alleviated.

[0102] 15. In the control method described above, The pedestrian information includes information regarding changes in the pedestrian's acceleration (e.g., PD1 to PD7), The present invention is characterized in that, when the acceleration of the first pedestrian (e.g., PD2) is increasing, the behavior of the moving body is controlled so that the moving body overtakes the first pedestrian by changing the acceleration of the moving body in accordance with the acceleration of the first pedestrian (e.g., 100).

[0103] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be quickly alleviated.

[0104] 16. The control device of the above-described embodiment is A control device (e.g., 130) that controls a mobile body (e.g., 100) that travels leading or guiding a user (e.g., PD1), An acquisition unit (e.g., 401) acquires the state of each of the multiple pedestrians (e.g., PD1 to PD7), including the first pedestrian (e.g., PD2), who is the target of following the moving object and is present in the surrounding environment of the moving object, When the state of the first pedestrian acquired by the acquisition unit is in a predetermined state, a control unit (for example, 403) controls the behavior of the moving body so that the predetermined state of the first pedestrian is resolved, It is characterized by having the following:

[0105] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be mitigated depending on the state of the first pedestrian.

[0106] 17. The mobile body of the above-described embodiment is A mobile body (e.g., 100) that leads or guides a user (e.g., PD1) while traveling, An acquisition unit (e.g., 130, 401) acquires the state of each of the multiple pedestrians (e.g., PD1 to PD7), including the first pedestrian (e.g., PD2), who is the target of following the moving object and is present in the surrounding environment of the moving object, When the state of the first pedestrian acquired by the acquisition unit is in a predetermined state, a control unit (for example, 130, 403) controls the behavior of the moving body so that the predetermined state of the first pedestrian is resolved, It is characterized by having the following:

[0107] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be mitigated depending on the state of the first pedestrian.

[0108] 18. The program of the above-described embodiment is: The computer is instructed to execute the control method described above.

[0109] According to this embodiment, the psychological burden on the first pedestrian towards the moving object can be mitigated depending on the state of the first pedestrian.

[0110] The invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of the gist of the invention. [Explanation of symbols]

[0111] 100: Mobile object 114: Detection unit 130: Control system 401: Acquisition unit 402: Determination unit 403: Action control unit

Claims

1. A control method for controlling a mobile body that drives while leading or guiding a user, The first step is to acquire the state of each of the multiple pedestrians present in the surrounding environment of the moving object, including the first pedestrian who is the object being followed and causing the moving object to follow it. If the state of the first pedestrian obtained in the first step is in a predetermined state, the second step involves controlling the behavior of the moving body so that the predetermined state of the first pedestrian is resolved. A control method characterized by having the following:

2. The control method according to claim 1, characterized in that the state of each of the plurality of pedestrians acquired in the first step includes the psychological load state of the pedestrian.

3. The control method according to claim 2, characterized in that, in the first step, for each of the plurality of pedestrians, at least one of the pedestrian's facial expression and behavior is recognized to acquire pedestrian information, and the psychological load state is estimated from the pedestrian information.

4. The control method according to claim 2, characterized in that the predetermined state includes a state in which the psychological load state exceeds a threshold.

5. The control method according to claim 4, characterized in that, in the second step, the behavior of the moving body is controlled so as to remove the first pedestrian from the tracking target and stop tracking the first pedestrian.

6. The control method according to claim 5, characterized in that, in the second step, the behavior of the moving body is controlled so that a second pedestrian, different from the first pedestrian, is selected as the target for following, and the movement of the moving body begins to follow the second pedestrian.

7. The control method according to claim 6, characterized in that, in the second step, among the plurality of pedestrians, the pedestrian whose psychological load state acquired in the first step is below the threshold is designated as the second pedestrian.

8. The control method according to claim 7, characterized in that, in the second step, the pedestrian with the largest difference between the psychological load state acquired in the first step and the threshold is designated as the second pedestrian.

9. The control method according to claim 7, characterized in that, in the second step, the pedestrian who is closest in distance from the moving body among the plurality of pedestrians is designated as the second pedestrian.

10. The control method according to claim 5, characterized in that the second step involves obtaining the density of pedestrians present in the surrounding environment and controlling the behavior of the moving body so that it moves to an area where the density is less than or equal to a predetermined value.

11. The control method according to claim 4, characterized in that the second step involves controlling the behavior of the moving body so as to change the relative position of the moving body with respect to the first pedestrian.

12. The control method according to claim 11, characterized in that, in the second step, a virtual adjacent lane is set adjacent to the virtual following lane for causing the moving body to follow the first pedestrian, and the behavior of the moving body is controlled to change the lane in which the moving body travels from the virtual following lane to the virtual adjacent lane.

13. The control method according to claim 11, characterized in that the second step involves controlling the movement of the moving body so that it moves away from the first pedestrian.

14. The pedestrian information includes information regarding changes in the pedestrian's acceleration, The control method according to claim 3, characterized in that, when the acceleration of the first pedestrian is increasing, the movement of the moving body is controlled so that the moving body moves away from the first pedestrian by changing the acceleration of the moving body in accordance with the acceleration of the first pedestrian.

15. The pedestrian information includes information regarding changes in the pedestrian's acceleration, The control method according to claim 3, characterized in that, when the acceleration of the first pedestrian is increasing, the behavior of the moving body is controlled so that the moving body overtakes the first pedestrian by changing the acceleration of the moving body in accordance with the acceleration of the first pedestrian.

16. A control device for controlling a mobile body that drives while leading or guiding a user, An acquisition unit that acquires the status of each of the multiple pedestrians present in the surrounding environment of the moving object, including the first pedestrian who is the target of following the moving object, When the state of the first pedestrian acquired by the acquisition unit is in a predetermined state, the control unit controls the behavior of the moving body so that the predetermined state of the first pedestrian is resolved, A control device characterized by having

17. A mobile vehicle that drives while leading or guiding the user, An acquisition unit that acquires the status of each of the multiple pedestrians present in the surrounding environment of the moving object, including the first pedestrian who is the target of following the moving object, When the state of the first pedestrian acquired by the acquisition unit is in a predetermined state, the control unit controls the behavior of the moving body so that the predetermined state of the first pedestrian is resolved, A mobile body characterized by having

18. A program for causing a computer to execute the control method described in any one of claims 1 to 15.

Images