Control method, control device, mobile device and program

Abstract

This provides new technology for controlling moving objects that are advantageous for avoiding obstacles. [Solution] A control method for controlling a moving object, comprising: a first step of identifying at least one pedestrian present in the surrounding environment of the moving object; a second step of acquiring a predicted trajectory that the pedestrian is expected to move along and a movement trajectory that the pedestrian actually moves along relative to the predicted trajectory, for the pedestrian identified in the first step; and a third step of controlling the behavior of the moving object so that the moving object does not travel along the predicted trajectory if the difference between the predicted trajectory and the movement trajectory is greater than or equal to a threshold when a predetermined number of the predicted trajectories and movement trajectories have been acquired in the second step.

Description

Technical Field

[0001] The present invention relates to a control method, a control device, a moving body, and a program.

Background Art

[0002] In recent years, research and development of moving bodies (self-propelled moving bodies) having a function of autonomously traveling (moving) without requiring operation (driving) by a user (user) have been progressing. As such a moving body, for example, a so-called micro-mobility, which is a robot that assists the walking of a user, represented by a follow-type moving body that travels following the user and a leading-type moving body that travels ahead of the user, is known.

[0003] In an actual road traffic environment where micro-mobility (and its user) participates, obstacles such as road surface steps (unevenness), puddles, mud, dust, and broken glass exist. Since such obstacles become factors that affect the moving body such as damage, it is necessary to cause the moving body to travel so as to avoid the obstacles. Patent Documents 1 and 2 disclose technologies for detecting pedestrians and pedestrian traffic lights (danger areas) around a moving body, and controlling the behavior and traveling of the moving body in response thereto.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of Invention

Problems to be Solved by the Invention

[0005] However, in mobile vehicles, recognizing obstacles such as road surface irregularities, puddles, mud, dust, and glass shards generally requires expensive sensors with high specifications, leading to high costs. Therefore, there is a need for technologies that allow mobile vehicles to avoid obstacles without increasing costs.

[0006] An exemplary objective of this invention is to provide a novel technique for controlling a moving object that is advantageous for avoiding obstacles. [Means for solving the problem]

[0007] One aspect of the present invention is a control method for controlling a moving object, characterized by comprising: a first step of identifying at least one pedestrian present in the surrounding environment of the moving object; a second step of acquiring a predicted trajectory that the pedestrian is expected to move along and a movement trajectory that the pedestrian actually moves along relative to the predicted trajectory, for the pedestrian identified in the first step; and a third step of controlling the behavior of the moving object so that, when a predetermined number of the predicted trajectories and movement trajectories have been acquired in the second step, the difference between the predicted trajectory and the movement trajectory is greater than or equal to a threshold; and when the difference between the predicted trajectory and the movement trajectory is greater than or equal to a threshold.

[0008] Another aspect of the present invention is a control device for controlling a moving body, comprising: an identification unit that identifies at least one pedestrian present in the surrounding environment of the moving body; an acquisition unit that acquires, for each pedestrian identified by the identification unit, a predicted trajectory that the pedestrian is expected to move along, and a movement trajectory that the pedestrian actually moves along relative to the predicted trajectory; and a control unit that, when the acquisition unit has acquired a predetermined number of the predicted trajectories and movement trajectories, controls the behavior of the moving body so that the moving body does not travel along the predicted trajectory if the difference between the predicted trajectory and the movement trajectory is greater than or equal to a threshold.

[0009] A mobile body, as yet another aspect of the present invention, is a mobile body comprising: an identification unit that identifies at least one pedestrian present in the surrounding environment of the mobile body; an acquisition unit that acquires, for each pedestrian identified by the identification unit, a predicted trajectory that the pedestrian is expected to move along, and a movement trajectory that the pedestrian actually moves along relative to the predicted trajectory; and a control unit that, when the acquisition unit has acquired a predetermined number of the predicted trajectories and movement trajectories, controls the behavior of the mobile body so that, if the difference between the predicted trajectory and the movement trajectory is greater than or equal to a threshold, the mobile body does not travel along the predicted trajectory.

[0010] Another aspect of the present invention is a program that causes a computer to execute the control method described above.

[0011] Further objects or other aspects of the present invention will be revealed by embodiments described below with reference to the accompanying drawings. [Effects of the Invention]

[0012] According to the present invention, for example, it is possible to provide a new technique for controlling a moving object that is advantageous for avoiding obstacles. [Brief explanation of the drawing]

[0013] [Figure 1] This is a perspective view showing the overall configuration of the mobile body as one aspect of the present invention. [Figure 2] This is a plan view illustrating the configuration of the running unit. [Figure 3] This is a block diagram showing the control system of a mobile object. [Figure 4] This diagram shows the configuration of the functional blocks realized by the control unit of the mobile body. [Figure 5] This is a flowchart illustrating the control process in a mobile device. [Figure 6] This diagram illustrates a method for driving a moving object while avoiding designated avoidance zones. [Figure 7]This diagram illustrates a method for driving a moving object while avoiding the first and second obstacles. [Modes for carrying out the invention]

[0014] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more features from the multiple features described in the embodiments may be combined arbitrarily. Furthermore, identical or similar configurations will be given the same reference numeral, and redundant descriptions will be omitted.

[0015] In this embodiment, the mobile vehicle is described using micromobility as an example, specifically a small electric vehicle capable of carrying luggage and leading (guiding) a user, or a small electric vehicle that follows a user. However, the mobile vehicle may be a micromobility vehicle capable of carrying not only luggage but also a person. Furthermore, this embodiment is not limited to electric vehicles and can be applied to mobile vehicles other than electric vehicles. In the following description, a mobile vehicle having one driven wheel is described as an example, but the mobile vehicle may have two or more driven wheels. Note that driven wheels are not necessarily required, and the mobile vehicle may not have driven wheels.

[0016] In this embodiment, it is preferable that the mobile vehicle autonomously drives (achieves autonomous driving) when leading (guiding) or following the user. Furthermore, even when the mobile vehicle is not leading or following the user, it is preferable that it communicates that it recognizes the user via an HMI (Human Machine Interface) or the like, thereby reducing the user's anxiety and providing a sense of security.

[0017] With reference to Figure 1, the configuration of the moving body 100 in the present embodiment will be described. Figure 1 is a perspective view showing the overall configuration of the moving body 100. As shown in Figure 1, the moving body 100 includes a lower housing 111, an upper housing 112, a front display unit 113, a plurality of detection units 114, an output unit 115, a light emitting unit 116, two drive wheels 120, and a driven wheel 121. Note that in the moving body 100, the side where the drive wheels 120 are provided is defined as the front side.

[0018] The lower housing 111 rotatably holds the drive wheels 120 and the driven wheel 121, and accommodates a traveling unit including the drive wheels 120 and the driven wheel 121. The lower housing 111 includes a loading space 118 on the upper portion thereof, which can carry a user's luggage. The lower housing 111 accommodates the light emitting unit 116. The region of the lower housing 111 that accommodates the light emitting unit 116 is made of a material capable of transmitting light.

[0019] The upper housing 112 is configured to extend upward from the rear portion of the lower housing 111. The upper housing 112 supports the front display unit 113, the detection units 114, and the output unit 115 at its upper end portion.

[0020] The front display unit 113 displays images and characters, and notifies (provides) various information to the user and surrounding people via such images and characters. The front display unit 113 includes a display device that displays images, such as a liquid crystal display device and an organic EL (Electro Luminescence) display device. The front display unit 113 is provided facing forward on the front side of the upper end of the upper housing 112. The front display unit 113 notifies the user of information by displaying an image imitating a human face, characters, and the like.

[0021] The multiple detection units 114 have the function of detecting targets in the surrounding environment of the mobile body 100, such as people (pedestrians, etc.) and objects present around the mobile body 100. For example, the detection unit 114 includes a camera having an optical system such as a lens and an image sensor, and captures images of the area around the mobile body 100 and outputs image information. Alternatively, the detection unit 114 may include radar or lidar (Light Detection and Ranging) in place of, or in addition to, a camera, to detect targets present around the mobile body 100. The multiple detection units 114 are installed (arranged) for example on the front, back, left, and right sides of the upper end of the upper housing 112. Therefore, the multiple detection units 114 can detect targets in all directions around the mobile body 100 (i.e., 360 degrees around the mobile body 100). Note that the installation position and number of the detection units 114 are not particularly limited and can be changed as appropriate.

[0022] The output unit 115 outputs sound and notifies (provides) various information to the user and those around them through this sound. The output unit 115 includes an audio output device such as a speaker.

[0023] The light-emitting unit 116 notifies (provides) various information to the user and those around them by emitting light. The light-emitting unit 116 includes, for example, one or more light-emitting elements such as LEDs (Light Emitting Diodes). In this embodiment, the light-emitting unit 116 is provided so as to surround the lower housing 111. The light-emitting unit 116 may emit light of multiple colors. The light-emitting unit 116 can also emit light of different colors depending on the light-emitting region or direction. For example, the light-emitting unit 116 emits light of different colors in the front region 116F, the left region 116L, and the right region (not shown). In addition, the light-emitting unit 116 can switch between illumination and extinguishing for each light-emitting region. Note that the light-emitting region of the light-emitting unit 116 may be divided into even finer regions than the region shown in Figure 1.

[0024] The drive wheels 120 are located at the front and lower left and right sides of the lower housing 111. The drive wheels 120 rotate around the axle when driving force is transmitted from the running unit.

[0025] The driven wheel 121 is located at the rear of the lower housing 111 and in the center of the lower part. The driven wheel 121 is held so as to be able to pivot around a vertical axis that extends in the vertical direction.

[0026] Referring to Figure 2, the travel unit 125 of the mobile body 100 will be described. Figure 2 is a schematic plan view showing the configuration of the travel unit 125.

[0027] The driving unit 125 is located at the bottom of the lower housing 111. In addition to two drive wheels 120 and a driven wheel 121, the driving unit 125 includes a drive mechanism 122, motors 123a and 123b, and a battery 124.

[0028] When power is supplied from the battery 124, motors 123a and 123b function as a drive source and supply driving force to the drive mechanism 122. The drive mechanism 122 transmits the driving force supplied from motors 123a and 123b to the drive wheels 120. This causes the drive mechanism 122 to rotate the drive wheels 120, making the mobile body 100 move (forward or backward). The drive mechanism 122 can change the direction in which the mobile body 100 moves (travels) to the left or right by creating a rotational difference between motors 123a and 123b. By changing the rotational difference between motors 123a and 123b, the drive mechanism 122 can achieve various turns of the mobile body 100, such as pivot turns, slow turns, and super-pivot turns.

[0029] The control system of the mobile unit 100 will be described with reference to Figure 3. Figure 3 is a block diagram of the control system of the mobile unit 100. As shown in Figure 3, the mobile unit 100 further includes a control unit 130, an operation unit 132, a top display unit 131, an input unit 133, a GNSS sensor 134, a storage unit 135, and a communication unit 136.

[0030] The control unit 130 is a control device (control unit) that controls the entire mobile body 100, and is also called an ECU (Electronic Control Unit). The control unit 130 is composed of, for example, a computer and comprehensively controls each part of the mobile body 100 to operate it. The control unit 130 includes one or more processors, such as a CPU (Central Processing Unit), memory devices such as semiconductor memory, and interfaces with external devices.

[0031] The control unit 130 may include, in addition to or instead of a CPU, an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an NPU (Neural Processing Unit), a QPU (Quantum Processing Unit), etc. The processor realizes various functions of the control unit 130 by reading and executing computer programs stored in memory devices or the like.

[0032] Some or all of the functions of the control unit 130 may be implemented by one or more circuits, such as an ASIC (Application Specific Integrated Circuit) and a PLD (Programmable Logic Device) including an FPGA (Field Programmable Gate Array).

[0033] The memory device includes, for example, RAM (Random Access Memory). The memory device stores programs executed by the processor and data used by the processor for processing. Multiple combinations of processors, memory devices, and interfaces are provided for each function of the mobile unit 100, and each combination may be configured to communicate with one another.

[0034] The control unit 130 controls the mobile body 100 based on outputs from multiple detection units 114 (detection information such as the detection results of targets present around the mobile body 100), user input information acquired via the operation unit 132, and user voice information acquired via the input unit 133. The control unit 130 performs behavioral control, including the movement of the mobile body 100, by controlling the driving unit 125 (motors 123a and 123b). The control unit 130 also notifies (provides) various information to the user and people in the surrounding area via the front display unit 113, top display unit 131, light-emitting unit 116, and output unit 115. The control unit 130 may perform processing on image information, which is one of the detection information acquired by the detection unit 114, using a machine learning model for image recognition (e.g., a deep neural network). The control unit 130 may also perform processing on voice information acquired by the input unit 133 using a machine learning model for speech recognition (e.g., a deep neural network).

[0035] The top display unit 131 is provided on the upper surface of the upper end of the upper housing 112 and includes a display device that displays images and characters. Based on image information acquired from the control unit 130, the top display unit 131 displays images and characters related to information to be notified to the user and those around them. The top display unit 131 also displays, for example, an operation screen for accepting user operations (inputs).

[0036] The operation unit 132 functions as an input device such as a touch panel or keyboard, and in this embodiment, it is provided via an operation screen displayed on the top display unit 131. The operation unit 132 receives operations from the user and outputs the operation information to the control unit 130.

[0037] The input unit 133 functions as a sound collection device such as a microphone. The input unit 133 collects the voices of the user and surroundings of the mobile unit 100 and outputs them as audio information to the control unit 130.

[0038] The GNSS sensor 134 receives GNSS (Global Navigation Satellite System) signals from satellites to detect (acquire) the current position of the mobile object 100.

[0039] The memory unit 135 includes a non-volatile storage medium for storing various information and data. The memory unit 135 includes HDDs (Hard Disk Drives) and SSDs (Solid State Drives), etc. The memory unit 135 may store programs executed by the processor, parameters necessary for program execution, and data to be processed by the processor. The memory unit 135 may also store various parameters of machine learning models executed by the control unit 130 (for example, trained parameters and hyperparameters of a deep neural network).

[0040] The communication unit 136 communicates with external devices such as a communication terminal 140 owned by the user, for example, via wireless communication such as Wi-Fi or fifth-generation mobile communication.

[0041] In the road traffic environment in which the mobile body 100, having such a configuration, participates, there are obstacles that can cause damage to the mobile body 100, such as uneven road surfaces, puddles, mud, dust, and glass fragments. Therefore, in controlling the mobile body 100, it is necessary to control the behavior of the mobile body 100 so as to avoid obstacles such as uneven road surfaces, puddles, mud, dust, and glass fragments. Such obstacles are difficult to detect (recognize) with the multiple detection units 114 that are generally provided on the mobile body 100, and expensive sensors with high specifications are required. However, providing such sensors separately from the detection units 114, or incorporating them into the detection units 114, would lead to a high cost for the mobile body 100 and is not practical.

[0042] Therefore, in this embodiment, a new technology for controlling a mobile body 100 that can avoid obstacles such as road surface steps, puddles, mud, dust, and glass fragments without using expensive sensors with high specifications is provided. In the following, obstacles such as road surface steps, puddles, mud, dust, and glass fragments, that is, obstacles that are difficult to detect with the detection unit 114 that the mobile body 100 generally has, will be referred to as first obstacles.

[0043] In this embodiment, in order to realize a technology for the mobile body 100 to avoid a first obstacle, the control unit 130 of the mobile body 100 includes a functional block, as shown in Figure 4. Specifically, the control unit 130 includes, as a functional block, a specific unit 401, an acquisition unit 402, a counter unit 403, a determination unit 404, and an action control unit 405. Figure 4 is a diagram showing the configuration of the functional block realized by the control unit 130 of the mobile body 100.

[0044] The identification unit 401 identifies at least one pedestrian present in the vicinity (surrounding environment) of the moving body 100 from the detection information (image information) acquired by the plurality of detection units 114. The vicinity of the moving body 100 is defined as an area within a predetermined distance from the moving body 100, for example, the area inside a circle with a predetermined radius centered on the moving body 100.

[0045] The acquisition unit 402 acquires, for pedestrians identified by the identification unit 401, the predicted trajectory that the pedestrian is expected to move on, and the actual movement trajectory of the pedestrian relative to the predicted trajectory. Any technology known in the industry can be applied to the method of acquiring the predicted trajectory in the acquisition unit 402. For example, the pedestrian's predicted trajectory can be acquired (predicted) by learning the positions of the pedestrian's joints from detection information (image information) acquired by multiple detection units 114 to estimate the pedestrian's skeleton, and analyzing the movement of this skeleton to analyze the pedestrian's behavior. For example, Human Pose Estimation can be used to estimate the skeleton. Furthermore, the movement trajectory can be acquired directly from the detection information acquired by the multiple detection units 114.

[0046] The counter unit 403 has the function of counting the output from the acquisition unit 402. In this embodiment, the counter unit 403 counts the acquisition of predicted trajectories and moving trajectories by the acquisition unit 402, that is, it counts the number of predicted trajectories and moving trajectories acquired by the acquisition unit 402 (number of acquisitions).

[0047] The determination unit 404 determines whether the number of times the predicted trajectory and movement trajectory have been acquired, as counted by the counter unit 403, has reached a predetermined number.

[0048] Furthermore, the determination unit 404 determines whether the difference between the pedestrian's predicted trajectory and its movement trajectory, acquired by the acquisition unit 402, is greater than or equal to a threshold. If the difference between the pedestrian's predicted trajectory and its movement trajectory is less than the threshold, it is considered that the pedestrian is moving along the predicted trajectory, and therefore no first obstacle exists on or near the predicted trajectory. On the other hand, if the difference between the pedestrian's predicted trajectory and its movement trajectory is greater than or equal to the threshold, it is considered that the pedestrian is moving significantly off the predicted trajectory, and therefore a first obstacle exists on or near the predicted trajectory. This is because pedestrians generally tend to change their direction of movement (i.e., deviate from the predicted drive) to avoid the first obstacle when it is present in the direction they are moving (direction of travel). Therefore, in this embodiment, the determination unit 404 can be said to substantially recognize (detect) the first obstacle present around the moving body 100 by determining whether the difference between the pedestrian's predicted trajectory and its movement trajectory is greater than or equal to a threshold.

[0049] The behavior control unit 405 controls the behavior of the mobile body 100 via the travel unit 125 (performs behavior control). Here, the behavior of the mobile body 100 includes traveling and stopping the mobile body 100, and changing the direction in which the mobile body 100 travels (travel direction).

[0050] In this embodiment, the behavior control unit 405 controls the behavior of the mobile body 100 so that the mobile body 100 does not travel along the pedestrian's predicted trajectory when the determination unit 404 determines that the difference between the pedestrian's predicted trajectory and the mobile body's trajectory acquired by the acquisition unit 402 is greater than or equal to a threshold. As described above, if the difference between the pedestrian's predicted trajectory and the mobile body's trajectory is greater than or equal to a threshold, it is assumed that a first obstacle exists on or near the predicted trajectory, so the mobile body 100 is made to travel in a way that avoids the first obstacle. This makes it possible to make the mobile body 100 travel in a way that avoids the first obstacle even if the mobile body 100 (detection unit 114) does not recognize the first obstacle.

[0051] The control process in the mobile body 100 will be explained with reference to Figure 5. Figure 5 is a flowchart illustrating the control process in the mobile body 100, in particular, the control process for moving the mobile body 100 while avoiding the first obstacle present around the mobile body 100. This control process is performed by the control unit 130 comprehensively controlling each part of the mobile body 100. It is assumed that the mobile body 100 is moving autonomously in a straight line.

[0052] In S502, the identification unit 401 identifies at least one pedestrian present around the moving body 100 from the detection information acquired by the multiple detection units 114.

[0053] In S504, the acquisition unit 402 acquires, for the pedestrian identified by the identification unit 401, the predicted trajectory that the pedestrian is expected to move on, and the actual movement trajectory of the pedestrian relative to the predicted trajectory. The method for acquiring the predicted trajectory and movement trajectory in the acquisition unit 402 is as described above, so a detailed explanation is omitted here.

[0054] In S506, the counter unit 403 counts the acquisition of the predicted trajectory and the moving trajectory by the acquisition unit 402. Specifically, the counter unit 403 increments the number of times the predicted trajectory and the moving trajectory have been acquired (increases the value of the number of acquisitions by 1).

[0055] In S508, the determination unit 404 determines whether the number of times the predicted trajectory and moving trajectory have been acquired has reached a predetermined number. If the number of times the predicted trajectory and moving trajectory have been acquired has not reached the predetermined number, the process proceeds to S504 to continue acquiring the predicted trajectory and moving trajectory. On the other hand, if the number of times the predicted trajectory and moving trajectory have been acquired has reached the predetermined number, the process proceeds to S510.

[0056] In S510, the determination unit 404 determines whether the difference between the predicted trajectory and the moving trajectory of the pedestrian, obtained in S504, is greater than or equal to a threshold. Since a predetermined number of predicted trajectories and moving trajectories are obtained, a predetermined number of differences are also obtained. Therefore, the determination unit 404 may compare one difference arbitrarily selected from these differences with the threshold, or it may compare the threshold with a difference obtained by statistically analyzing these differences (such as the average difference), or it may compare the threshold with the largest difference among these differences. If the difference between the predicted trajectory and the moving trajectory of the pedestrian is not greater than or equal to the threshold, that is, if it is considered that there is no first obstacle around the moving body 100, the process proceeds to S512. On the other hand, if the difference between the predicted trajectory and the moving trajectory of the pedestrian is greater than or equal to the threshold, that is, if it is considered that there is a first obstacle around the moving body 100, the process proceeds to S514.

[0057] In S512, the action control unit 405 controls the actions of the mobile body 100 so as to maintain (continue) its straight-line movement.

[0058] In S514, the action control unit 405 sets the difference region between the predicted trajectory and the actual trajectory of the moving body 100 as the avoidance region.

[0059] In S515, the action control unit 405 controls the actions of the mobile body 100 so that the mobile body 100 avoids the avoidance area set in S514.

[0060] In this embodiment, when a predetermined number of predicted trajectories and movement trajectories of a pedestrian are acquired, if the difference between the predicted trajectory and the movement trajectory of the pedestrian is greater than or equal to a threshold, the movement of the mobile body 100 is controlled so that it does not travel along the predicted trajectory of the pedestrian. As a result, even if the mobile body 100 does not recognize the first obstacle, it is possible to make the mobile body 100 travel while avoiding the first obstacle.

[0061] In the following section S514, a method for preventing the mobile body 100 from traveling along the predicted trajectory of a pedestrian will be specifically described.

[0062] One specific example of a method to make the mobile object 100 travel in a way that avoids following the predicted trajectory of a pedestrian is to generate a travel path DR for the mobile object 100 on top of the pedestrian P's travel path MT, as shown in Figure 6. Specifically, lines connecting the positions PP1, PP2, PP3, PP4, PP5, and PP6 where the pedestrian P has moved are generated as the travel path DR for the mobile object 100. Figure 6 is a diagram illustrating a method for making the mobile object 100 travel while avoiding the avoidance area.

[0063] As another specific example, a method for making the mobile object 100 not travel along the predicted trajectory of a pedestrian is to set an avoidance area AR that the mobile object 100 should avoid, as shown in Figure 6. The avoidance area AR is defined by the predicted trajectory PT of the pedestrian P and the movement MT of the pedestrian P. Then, the behavior of the mobile object 100 is controlled so that it avoids traveling through the avoidance area AR. This makes it possible to make the mobile object 100 travel while avoiding the avoidance area AR.

[0064] Preferably, the avoidance area AR is the region enclosed by the predicted trajectory PT of the pedestrian P and the movement trajectory MT of the pedestrian P. This makes it possible to more reliably avoid the avoidance area AR when moving the moving object 100.

[0065] Furthermore, as shown in Figure 6, a virtual obstacle VOB is set within the area of ​​the avoidance region AR, and the behavior of the mobile body 100 is controlled to avoid collision with the virtual obstacle VOB. This makes it possible to drive the mobile body 100 while avoiding the avoidance region AR.

[0066] In the road traffic environment in which the mobile body 100 participates, as shown in Figure 7, there are not only first obstacles FOB, which are obstacles that are difficult to detect with the detection unit 114 that the mobile body 100 generally possesses, but also second obstacles SOB, which can be detected by the detection unit 114. The second obstacles SOB are actual obstacles that exist around the mobile body 100, including, for example, walls and pillars. In such cases, as described above, a virtual obstacle VOB is set within the area of ​​the avoidance region AR, and a first collision cost is set as the collision cost for the virtual obstacle VOB, and a second collision cost is set as the collision cost for the second obstacle SOB. Here, the collision cost is a numerical representation of the degree to which the mobile body 100 should avoid colliding with an obstacle. The behavior of the mobile body 100 is then controlled so that the collision costs, including the first collision cost for the virtual obstacle VOB and the second collision cost for the second obstacle SOB, fall within an acceptable range. This makes it possible to drive the mobile body 100 while avoiding the first obstacle and the second obstacle, even if the mobile body 100 does not recognize the first obstacle. Figure 7 is a diagram illustrating a method for driving the mobile body 100 while avoiding the first obstacle FOB and the second obstacle SOB.

[0067] Furthermore, it is preferable that the first collision cost related to the virtual obstacle VOB be smaller than the second collision cost related to the second obstacle SOB. The second obstacle SOB is generally considered to be more likely to be damaged when the mobile body 100 collides with it compared to the first obstacle FOB corresponding to the virtual obstacle VOB. Therefore, by making the first collision cost smaller than the second collision cost, it becomes possible to drive the mobile body 100 in a way that avoids the first obstacle FOB while preferentially avoiding the second obstacle SOB.

[0068] In the embodiment described above, the control device was described as a control unit 130 of the mobile body 100. However, the control device may be included in a device located outside the mobile body 100, for example, an information processing server. In this case, the control device may perform some or all of the processing performed by the control unit 130 while communicating with the mobile body 100. Thus, this embodiment is also applicable when the mobile body 100 is remotely controlled from a device separate from the mobile body 100.

[0069] <Summary of Embodiments> 1. The control method of the above embodiment is as follows: A control method for controlling a moving object (for example, 100), The first step (for example, S502) is to identify at least one pedestrian present in the surrounding environment of the moving object, A second step (e.g., S504) is performed to obtain, for the pedestrian identified in the first step, a predicted trajectory (e.g., PT) that the pedestrian is expected to move, and a movement trajectory (e.g., MT) that the pedestrian actually moved on relative to the predicted trajectory. When a predetermined number of the predicted trajectories and the moving trajectories have been acquired in the second step, if the difference between the predicted trajectories and the moving trajectories is greater than or equal to a threshold, a third step (for example, S514) is taken to control the behavior of the moving body so that it does not travel along the predicted trajectories. It is characterized by having the following:

[0070] According to this embodiment, even if the moving object does not recognize obstacles that are difficult to detect, it is possible to make the moving object move while avoiding obstacles.

[0071] 2. In the control method described above, In the third step (for example, S514), if the difference between the predicted trajectory (for example, PT) and the moving trajectory (for example, MT) is greater than or equal to a threshold, the region defined by the predicted trajectory and the moving trajectory is set as an avoidance region (for example, AR), and the behavior of the moving body (for example, 100) is controlled so as to avoid the moving body traveling through the avoidance region.

[0072] According to this embodiment, the moving body can be made to travel while avoiding the avoidance area.

[0073] 3. In the control method described above, The avoidance area (e.g., AR) is characterized in that it includes the area enclosed by the predicted trajectory (e.g., PT) and the moving trajectory.

[0074] According to this embodiment, it becomes possible to drive the moving object while more reliably avoiding the avoidance area.

[0075] 4. In the control method described above, The third step (for example, S514) is characterized by setting a virtual obstacle (for example, VOB) within the area of ​​the avoidance region (for example, AR), and controlling the behavior of the moving body (for example, 100) so as to avoid the moving body colliding with the virtual obstacle.

[0076] According to this embodiment, it becomes possible to drive the moving object while avoiding the avoidance area.

[0077] 5. In the control method described above, The third step (for example, S514) is characterized by setting a virtual obstacle (for example, VOB) within the area of ​​the avoidance region (for example, AR), and controlling the behavior of the moving body (for example, 100) so that the collision cost, including a first collision cost related to the virtual obstacle and a second collision cost related to actual obstacles (for example, SOB) present in the surrounding environment, falls within an acceptable range.

[0078] According to this embodiment, even if the moving body does not recognize obstacles that are difficult to detect, it is possible to drive the moving body while avoiding obstacles that are difficult to detect and actual obstacles present in the surrounding environment.

[0079] 6. In the control method described above, The first collision cost is characterized by being smaller than the second collision cost.

[0080] According to this embodiment, it is possible to drive the mobile body in a manner that prioritizes avoiding actual obstacles present in the surrounding environment while also avoiding obstacles that are difficult to detect from the mobile body.

[0081] 7. In the control method described above, The second step (for example, S504) is characterized by predicting the predicted trajectory (for example, PT) from the pedestrian's behavior identified in the first step (for example, S502).

[0082] According to this embodiment, the predicted trajectory of a pedestrian can be predicted with high accuracy.

[0083] 8. The control device of the above-described embodiment is A control device (e.g., 130) for controlling a mobile object (e.g., 100), A identifying unit (e.g., 401) that identifies at least one pedestrian present in the surrounding environment of the moving object, For the pedestrian identified by the identification unit, an acquisition unit (e.g., 402) acquires a predicted trajectory (e.g., PT) that the pedestrian is predicted to move along, and a movement trajectory (e.g., MT) that the pedestrian actually moved along relative to the predicted trajectory. When the acquisition unit acquires a predetermined number of predicted trajectories and moving trajectories, if the difference between the predicted trajectory and the moving trajectory is greater than or equal to a threshold, a control unit (for example, 405) controls the behavior of the moving body so that the moving body does not travel along the predicted trajectory. It is characterized by having the following:

[0084] According to this embodiment, even if the moving object does not recognize obstacles that are difficult to detect, it is possible to make the moving object move while avoiding obstacles.

[0085] 9. The mobile body of the above-described embodiment is A moving object (for example, 100), A identifying unit (e.g., 401) that identifies at least one pedestrian present in the surrounding environment of the moving object, For the pedestrian identified by the identification unit, an acquisition unit (e.g., 402) acquires a predicted trajectory (e.g., PT) that the pedestrian is predicted to move along, and a movement trajectory (e.g., MT) that the pedestrian actually moved along relative to the predicted trajectory. When the acquisition unit acquires a predetermined number of predicted trajectories and moving trajectories, if the difference between the predicted trajectory and the moving trajectory is greater than or equal to a threshold, a control unit (for example, 405) controls the behavior of the moving body so that the moving body does not travel along the predicted trajectory. It is characterized by having the following:

[0086] According to this embodiment, even if the moving object does not recognize obstacles that are difficult to detect, it is possible to make the moving object move while avoiding obstacles.

[0087] 10. The program of the above-described embodiment is: The computer is instructed to execute the control method described above.

[0088] According to this embodiment, even if the moving object does not recognize obstacles that are difficult to detect, it is possible to make the moving object move while avoiding obstacles.

[0089] The invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of the gist of the invention. [Explanation of Symbols]

[0090] 100: Mobile unit 114: Detection unit 130: Control system 401: Identification unit 402: Acquisition unit 403: Counter unit 404: Determination unit 405: Action control unit

Claims

1. A control method for controlling a mobile object, A first step is to identify at least one pedestrian present in the surrounding environment of the moving object, A second step involves obtaining, for the pedestrian identified in the first step, the predicted trajectory that the pedestrian is expected to move, and the actual trajectory that the pedestrian moved in relation to the predicted trajectory. When a predetermined number of the predicted trajectories and the moving trajectories have been acquired in the second step, if the difference between the predicted trajectories and the moving trajectories is greater than or equal to a threshold, a third step is taken to control the behavior of the moving body so that it does not travel along the predicted trajectories. A control method characterized by having the following:

2. The control method according to claim 1, characterized in that, in the third step, if the difference between the predicted trajectory and the moving trajectory is greater than or equal to a threshold, the region defined by the predicted trajectory and the moving trajectory is set as an avoidance region, and the behavior of the moving body is controlled so as to avoid the moving body traveling through the avoidance region.

3. The control method according to claim 2, characterized in that the avoidance area includes the area enclosed by the predicted trajectory and the moving trajectory.

4. The control method according to claim 2, characterized in that the third step involves setting a virtual obstacle within the area of ​​the avoidance region and controlling the behavior of the moving body so as to avoid collision with the virtual obstacle.

5. The control method according to claim 2, characterized in that the third step involves setting a virtual obstacle within the area of ​​the avoidance region, and controlling the behavior of the moving body so that the collision cost, which includes a first collision cost related to the virtual obstacle and a second collision cost related to actual obstacles present in the surrounding environment, falls within an acceptable range.

6. The control method according to claim 5, characterized in that the first collision cost is smaller than the second collision cost.

7. The control method according to claim 1, characterized in that the second step involves predicting the predicted trajectory from the pedestrian's behavior identified in the first step.

8. A control device for controlling a moving object, A unit that identifies at least one pedestrian present in the surrounding environment of the moving object, The acquisition unit acquires, for the pedestrian identified by the identification unit, a predicted trajectory that the pedestrian is expected to move, and a movement trajectory that the pedestrian actually moved in relation to the predicted trajectory. When the acquisition unit has acquired a predetermined number of predicted trajectories and movement trajectories, if the difference between the predicted trajectory and the movement trajectory is greater than or equal to a threshold, the control unit controls the behavior of the moving body so that the moving body does not travel along the predicted trajectory. A control device characterized by having

9. It is a mobile object, A unit that identifies at least one pedestrian present in the surrounding environment of the moving object, The acquisition unit acquires, for the pedestrian identified by the identification unit, a predicted trajectory that the pedestrian is expected to move, and a movement trajectory that the pedestrian actually moved in relation to the predicted trajectory. When the acquisition unit has acquired a predetermined number of predicted trajectories and movement trajectories, if the difference between the predicted trajectory and the movement trajectory is greater than or equal to a threshold, the control unit controls the behavior of the moving body so that the moving body does not travel along the predicted trajectory. A mobile body characterized by having

10. A program for causing a computer to execute the control method described in any one of claims 1 to 7.

Images