Control device for a moving body, control method for a moving body, and storage medium

Abstract

A mobile body control device according to the present invention can move both on a roadway and in a prescribed area different from a roadway, the mobile body control device comprising: a road type recognition unit that recognizes whether a mobile body is moving on a roadway or in the prescribed area on the basis of an output from a detection device that detects an external condition of the mobile body; an acquisition unit that acquires auxiliary information for recognizing whether a signal of a pedestrian signal indicates that a pedestrian may advance or is instructed to stop, the auxiliary information different from traffic information indicating that the signal displayed by the pedestrian signal on which the attention of the mobile body is focused indicates that the pedestrian may advance or is instructed to stop; a signal recognition unit that uses the auxiliary information acquired by the acquisition unit to recognize the traffic information, which indicates that the signal displayed by the pedestrian signal indicates that the pedestrian may advance or is instructed to stop; and a control unit which, when the mobile body is positioned in the prescribed area, controls the mobile body on the basis of the traffic information of the pedestrian signal recognized by the signal recognition unit.

Description

Technical Field

[0001] The present invention relates to a control device for a moving body, a control method for a moving body, and a storage medium.

Background Art

[0002] Conventionally, a system has been disclosed that respectively collects the left and right sounds of personal mobility and detects from the collected sounds which side of the left and right of the personal mobility is the direction of the roadway (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above system, since the state of the pedestrian traffic signal is not considered, there are cases where the moving body cannot be controlled according to the surrounding situation.

[0005] The present invention has been made in consideration of such circumstances, and one of the objects is to provide a control device for a moving body, a control method for a moving body, and a storage medium that can realize control of the moving body according to the surrounding situation.

Means for Solving the Problems

[0006] The control device for a moving body, the control method for a moving body, and the storage medium according to this invention employ the following configuration. (1) The control device for a moving body according to one aspect of the present invention is a control device for a moving body capable of moving on both a roadway and a predetermined area different from the roadway. Based on the output of a detection device that detects the external situation of the moving body, a road type recognition unit that recognizes whether the moving body is moving on the roadway or moving on the predetermined area, and a pedestrian signal that the moving body focuses on. An acquisition unit that acquires auxiliary information, which is information different from the traffic information indicating the travel permission and stop instruction indicated by the signal of the pedestrian signal and is for recognizing the travel permission or stop instruction indicated by the signal of the pedestrian signal, and using the auxiliary information acquired by the acquisition unit, a signal recognition unit that recognizes traffic information that is the travel permission or stop instruction indicated by the signal of the pedestrian signal, and when the moving body is located in the predetermined area, a control unit that controls the moving body based on the traffic information of the pedestrian signal recognized by the signal recognition unit.

[0007] (2) In the aspect of (1) above, the auxiliary information is an audible signal for visually impaired persons output according to the traffic information of the pedestrian signal.

[0008] (3) In the aspect of (1) or (2) above, the auxiliary information is the traffic information of a signal that permits the travel of traffic participants in a direction different from the direction in which the pedestrian signal being focused on permits the travel of traffic participants.

[0009] (4) In any of the aspects of (1) to (3) above, the auxiliary information is information indicating the movement of a person estimated to move based on the traffic information indicated by the pedestrian signal being focused on or information indicating the movement of people around the moving body.

[0010] (5) In any of the aspects (1) to (4) above, the auxiliary information is an audible notification for visually impaired persons output according to the traffic information of the pedestrian traffic signal, and information indicating the movement of a person estimated to move based on the traffic information indicated by the pedestrian traffic signal of interest or information indicating the movement of a person around the moving body. The signal recognition unit uses the audible notification for visually impaired persons and the information indicating the movement of the person to recognize whether the pedestrian traffic signal of interest permits progress. The control unit controls the moving body based on the recognition result of the signal recognition unit.

[0011] (6) In any of the aspects (1) to (5) above, the auxiliary information is an audible notification for visually impaired persons output according to the traffic information of the pedestrian traffic signal, and the traffic information of a traffic signal that permits progress in a direction different from the direction in which the pedestrian traffic signal of interest permits progress. The signal recognition unit uses the audible notification for visually impaired persons and the traffic information of the traffic signal that permits progress in a different direction to recognize whether the pedestrian traffic signal of interest permits progress. The control unit controls the moving body based on the recognition result of the signal recognition unit.

[0012] (7) In any of the aspects (1) to (6) above, the auxiliary information is information indicating the movement of a person estimated to move based on the traffic information indicated by the pedestrian traffic signal of interest or information indicating the movement of a person around the moving body, and the traffic information of a traffic signal that permits progress in a direction different from the direction in which the pedestrian traffic signal of interest permits progress. The signal recognition unit uses the information indicating the movement of the person and the traffic information of the traffic signal that permits progress in a different direction to recognize whether the pedestrian traffic signal of interest permits progress. The control unit controls the moving body based on the recognition result of the signal recognition unit.

[0013] (8) In any one of the aspects (1) to (7) above, the auxiliary information is: (1) an audible notification for visually impaired persons output according to the traffic information of the pedestrian traffic signal; (2) information indicating the movement of a person estimated to move based on the traffic information indicated by the pedestrian traffic signal being focused on, or information indicating the movement of a person around the moving body; or (3) the traffic information of a traffic signal that permits travel in a direction different from the direction in which the pedestrian traffic signal being focused on permits travel. The signal recognition unit recognizes, based on one or more pieces of information among (1), (2), or (3), that the pedestrian traffic signal around the moving body has switched from a first state of a stop instruction to a second state of permitting travel, and based on information different from the information used when recognizing the switch to the second state among (1), (2), or (3), recognizes whether the pedestrian traffic signal being focused on has changed to the second state or whether a pedestrian traffic signal different from the pedestrian traffic signal being focused on has changed to the second state.

[0014] (9) In any one of the aspects (1) to (8) above, the auxiliary information is an audible notification for visually impaired persons output according to the traffic information of the pedestrian traffic signal. The signal recognition unit refers to information in which the type of the audible notification stored in the storage device is associated with the direction in which the pedestrian traffic signal permits the travel of traffic participants, and recognizes the traffic information of the pedestrian traffic signal based on the audible notification.

[0015] (10) In any one of the aspects (1) to (9) above, the control unit limits the speed when the moving body moves on the roadway to a first speed, limits the speed when the moving body moves in the predetermined area to a second speed lower than the first speed, is scheduled to move according to the traffic information of the pedestrian traffic signal being focused on, and when the signal recognition unit recognizes that the pedestrian traffic signal being focused on permits travel, starts an operation to move the moving body in the direction permitted by the pedestrian traffic signal.

[0016] (11): In any one of the aspects (1) to (10) above, in an intersection including a first pedestrian signal that regulates traffic in a first direction and a second pedestrian signal that regulates traffic in a second direction different from the first direction, when the moving body moves according to the traffic information of the first pedestrian signal, based on one or more of the following information: an audible signal for visually impaired persons output according to the traffic information of the pedestrian signal, which is the auxiliary information; the traffic information of a signal different from the first pedestrian signal; or information indicating the movement of a person estimated to move based on the traffic information indicated by the first pedestrian signal or information indicating the movement of a person around the moving body, the signal recognition unit recognizes the state of the first pedestrian signal, and when the control unit recognizes that the signal recognition unit has recognized that the first pedestrian signal has changed to a state permitting progress, the control unit moves the moving body according to the state of the first pedestrian signal.

[0017] (12): In any one of the aspects (1) to (11) above, the acquisition unit acquires the traffic information of the target pedestrian signal based on the output of the detection device, and when the acquisition unit cannot acquire the traffic information, the signal recognition unit acquires the traffic information of the pedestrian signal using the auxiliary information.

[0018] (13): In the aspect (12) above, the control unit controls the moving body based on the recognition result of the signal recognition unit. Then, when the acquisition unit acquires the traffic information of the target pedestrian signal and recognizes that the target pedestrian signal permits progress, the control unit causes the moving body to enter a road or a crosswalk where the traffic information permits progress from the sidewalk where the moving body is located and cross the road or the crosswalk.

[0019] (14) A control method for a moving body according to another aspect of the present invention is such that a control device of a moving body capable of moving in both a roadway and a predetermined area different from the roadway recognizes whether the moving body is moving in the roadway or moving in the predetermined area based on an output of a detection device that detects an external situation of the moving body, and acquires auxiliary information that is different from traffic information indicating a travel permission and a stop instruction indicated by a signal of a pedestrian signal displayed by the pedestrian signal that the moving body is paying attention to and that recognizes a travel permission or a stop instruction indicated by the signal of the pedestrian signal, recognizes a travel permission or a stop instruction indicated by the signal of the pedestrian signal using the acquired auxiliary information, and controls the moving body based on the recognized state of the pedestrian signal when the moving body is located in the predetermined area.

[0020] (15) A storage medium storing a program according to another aspect of the present invention causes a control device of a moving body capable of moving in both a roadway and a predetermined area different from the roadway to recognize whether the moving body is moving in the roadway or moving in the predetermined area based on an output of a detection device that detects an external situation of the moving body, acquire auxiliary information that is different from traffic information indicating a travel permission and a stop instruction indicated by a signal of a pedestrian signal displayed by the pedestrian signal that the moving body is paying attention to and that recognizes a travel permission or a stop instruction indicated by the signal of the pedestrian signal, recognize a travel permission or a stop instruction indicated by the signal of the pedestrian signal using the acquired auxiliary information, and control the moving body based on the recognized state of the pedestrian signal when the moving body is located in the predetermined area.

Advantages of the Invention

[0021] According to the aspects (1) to (15) above, it is possible to realize control of a moving body according to the surrounding situation.

[0022] According to the aspect (8) above, after the control device recognizes that the traffic information of the traffic signal has changed, by specifying the changed traffic signal, it is possible to quickly prepare for moving the moving body and quickly move the moving body.

[0023] According to the above aspect (9), the control device can identify the traffic lights whose traffic information has changed with higher accuracy by using the type of the notifier.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Modes for Carrying Out the Invention

[0025] Hereinafter, with reference to the drawings, embodiments of a control device for a moving body, a control method for a moving body, and a storage medium according to the present invention will be described.

[0026] Hereinafter, with reference to the drawings, embodiments of a control device for a moving body, a control method for a moving body, and a storage medium according to the present invention will be described. The moving body moves on both a roadway and a predetermined area different from the roadway. The moving body may be referred to as micromobility. An electric kick scooter is a type of micromobility. The predetermined area is, for example, a sidewalk. Also, the moving body may be a vehicle that can carry passengers, or an autonomous moving body that can perform autonomous driving without a driver. The latter autonomous moving body is used, for example, for transporting luggage and the like. The moving body may include small mobility such as running parallel to a walking user and carrying luggage or leading a person, and may also include other moving bodies capable of autonomous movement (such as a walking robot, etc.). The predetermined area may be part or all of a roadside strip, a bicycle lane, an open space, etc., or may include all of a sidewalk, a roadside strip, a bicycle lane, an open space, etc. In the following description, it is assumed that the predetermined area is a sidewalk. In the following description, the part described as "sidewalk" can be read as "predetermined area" as appropriate.

[0027] FIG. 1 is a diagram showing an example of the configuration of a moving body 1 and a control device 100 according to an embodiment. The moving body 1 is equipped with, for example, an external detection device 10, a moving body sensor 12, an operator 14, an internal camera 16, a positioning device 18, a microphone 20, a mode changeover switch 22, a moving mechanism 30, a drive device 40, an external notification device 50, a storage device 70, and a control device 100. Some of these configurations that are not essential for realizing the functions of the present invention may be omitted.

[0028] The external detection device 10 is various devices that use the traveling direction of the moving body 1 as the detection range. The external detection device 10 includes an external camera, a radar device, LIDAR (Light Detection and Ranging), a sensor fusion device, and the like. The external detection device 10 outputs information (such as an image, the position of an object, etc.) indicating the detection result to the control device 100.

[0029] The moving body sensor 12 includes, for example, a speed sensor, an acceleration sensor, a yaw rate (angular velocity) sensor, a direction sensor, and an operation amount detection sensor attached to the operator 14. The operator 14 includes, for example, an operator for instructing acceleration and deceleration (such as an accelerator pedal or a brake pedal) and an operator for instructing steering (such as a steering wheel). In this case, the moving body sensor 12 may include an accelerator opening sensor, a brake depression amount sensor, a steering torque sensor, and the like. The moving body 1 may be provided with an operator in a form other than the above as the operator 14 (for example, a non-circular rotary operator, a joystick, a button, etc.).

[0030] The internal camera 16 images at least the head of the passenger of the moving body 1 from the front. The internal camera 16 is a digital camera that uses an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The internal camera 16 outputs the captured image to the control device 100.

[0031] The positioning device 18 is a device that positions the position of the moving body 1. The positioning device 18 is, for example, a GNSS (Global Navigation Satellite System) receiver. Based on the signals received from GNSS satellites, the positioning device 18 identifies the position of the moving body 1 and outputs it as position information. The position information of the moving body 1 may be estimated from the position of the Wi-Fi base station to which the communication device described later is connected. The microphone 20 picks up the sounds around the moving body 1. The microphone 20 may include a plurality of microphones.

[0032] The mode switching switch 22 is a switch operated by the occupant. The mode switching switch 22 may be a mechanical switch or a GUI (Graphical User Interface) switch set on a touch panel. The mode switching switch 22 receives an operation to switch the driving mode to, for example, Mode A: an assist mode in which either a steering operation or acceleration / deceleration control is performed by the occupant and the other is automatically performed, including Mode A-1 in which a steering operation is performed by the occupant and acceleration / deceleration control is automatically performed, and Mode A-2 in which an acceleration / deceleration operation is performed by the occupant and steering control is automatically performed; Mode B: a manual driving mode in which a steering operation and an acceleration / deceleration operation are performed by the occupant; or Mode C: an automatic driving mode in which operation control and acceleration / deceleration control are automatically performed.

[0033] The moving mechanism 30 is a mechanism for moving the moving body 1 on a road. The moving mechanism 30 is, for example, a wheel group including a steering wheel and drive wheels. The moving mechanism 30 may be legs for multi-legged walking.

[0034] The driving device 40 outputs force to the moving mechanism 30 to move the moving body 1. For example, the driving device 40 includes a motor that drives the drive wheels, a battery that stores electric power supplied to the motor, a steering device that adjusts the steering angle of the steering wheel, and the like. The driving device 40 may be provided with an internal combustion engine, a fuel cell, or the like as a driving force output means or a power generation means. The driving device 40 may further be provided with a braking device due to frictional force or air resistance.

[0035] The external notification device 50 is provided, for example, on the outer plate portion of the moving body 1, and is a lamp, a display device, a speaker, etc. for notifying information toward the outside of the moving body 1. The external notification device 50 performs different operations when the moving body 1 is moving on a sidewalk and when it is moving on a roadway. For example, the external notification device 50 is controlled such that the lamp emits light when the moving body 1 is moving on a sidewalk, and does not emit light when the moving body 1 is moving on a roadway. It is preferable that the emission color of this lamp is a color defined by regulations. The external notification device 50 may be controlled such that the lamp emits light in green when the moving body 1 is moving on a sidewalk, and emits light in blue when the moving body 1 is moving on a roadway. When the external notification device 50 is a display device, the external notification device 50 displays, in text or graphics, that "the vehicle is traveling on a sidewalk" when the moving body 1 is traveling on a sidewalk.

[0036] Figure 2 is a perspective view of the moving body 1 seen from above. In the figure, FW is a steering wheel, RW is a drive wheel, SD is a steering device, MT is a motor, and BT is a battery. The steering device SD, the motor MT, and the battery BT are included in the drive device 40. AP is an accelerator pedal, BP is a brake pedal, WH is a steering wheel, SP is a speaker, and MC is a microphone. The illustrated moving body 1 is a single-seater moving body, and the passenger P is seated in the driver's seat DS and wearing a seat belt SB. Arrow D1 is the traveling direction (velocity vector) of the moving body 1. The external detection device 10 is provided near the front end of the moving body 1, the internal camera 16 is at a position where it can image the head of the passenger P from in front of the passenger P, and the mode switch 22 is provided on the boss portion of the steering wheel WH, respectively. Near the front end of the moving body 1, an external notification device 50 as a display device is provided.

[0037] Returning to FIG. 1, the storage device 70 is a non-transitory storage device such as, for example, a HDD (Hard Disk Drive), a flash memory, or a RAM (Random Access Memory). The storage device 70 stores map information 72, a program 74 executed by the control device 100, and the like. In the figure, the storage device 70 is shown outside the frame of the control device 100, but the storage device 70 may be included in the control device 100. Further, the storage device 70 may be provided on a server (not shown).

[0038] [Control Device] The control device 100 includes, for example, a road type recognition unit 120, a recognition unit 130, a voice recognition unit 140, a traffic signal recognition unit 150, and a control unit 160. These components are realized, for example, when a hardware processor such as a CPU (Central Processing Unit) executes a program (software) 74. Some or all of these components may be realized by hardware (including a circuit unit; circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in the storage device 70 in advance, or may be stored in a removable storage medium (non-transitory storage medium) such as a DVD or a CD-ROM, and may be installed in the storage device 70 when the storage medium is mounted on a drive device.

[0039] The road type recognition unit 120 recognizes whether the moving body 1 is moving on a roadway or a sidewalk. The road type recognition unit 120 recognizes whether the moving body 1 is moving on a roadway or a sidewalk, for example, by analyzing an image captured by an external camera of the external detection device 10. The outputs of a radar device, LIDAR, a sensor fusion device, etc. may be used as auxiliary.

[0040] Based on a plurality of events indicating that the moving body 1 is moving on a road lane in the image captured by the external camera, the road type recognition unit 120 recognizes whether the moving body 1 is moving on a road lane or a sidewalk. For example, the road type recognition unit 120 classifies each pixel in the frame of the image into a class by semantic segmentation and assigns a label, and assumes a plurality of virtual regions and their boundary lines. The road type recognition unit 120 classifies and processes at least the region of the image captured by the external camera into a region where the moving body 1 exists (hereinafter, the own region) and a region adjacent to the own region where the moving body 1 does not exist (hereinafter, the adjacent region).

[0041] For example, when a predetermined number or more of the following events are included, it may be determined that the moving body 1 is moving on a road lane. For example, that there are no static obstacles other than the vehicle 203 in the own region 200 (such as standing signs, postal posts, utility poles, trash cans, etc.), that the vehicle 203 is moving in the own region 200, that there is a road marking 205 on the road surface of the own region 200, that there is a crosswalk 206 in the own region 200, and that the own region 200 is on the lower side with respect to the step 202. FIG. 3 is a diagram showing an example of a scene where the moving body 1 is moving on a road lane. Note that in FIG. 3 (and subsequent figures), for the sake of explanation, it is represented as a bird's-eye view seen from above, but each process may be performed on the image plane.

[0042] Based on the output of the external detection device 10, the recognition unit 130 recognizes objects existing around the moving body 1. The objects include some or all of moving bodies such as vehicles, bicycles, and pedestrians, road markings, steps, guardrails, road shoulders, median strips as road boundaries, structures installed on the road such as road signs and billboards, and obstacles such as falling objects (lying on the road). For example, when an image captured by the external camera of the external detection device 10 is input, the recognition unit 130 inputs the image captured by the external camera into a learned model that has been trained to output information such as the presence, position, and type of an object, thereby obtaining information such as the presence, position, and type of other moving bodies. The type of other moving bodies can also be estimated based on the size in the image, the intensity of the reflected wave received by the radar device of the external detection device 10, etc. The recognition unit 130 obtains, for example, the speed of other moving bodies detected by the radar device using Doppler shift, etc.

[0043] The recognition unit 130 recognizes traffic lights including traffic lights for vehicles and traffic lights for pedestrians. The recognition unit 130 recognizes the traffic information of the signals of the traffic lights. The traffic information of the traffic lights is information indicating permission to proceed (move) in a predetermined direction, information indicating a stop instruction, etc. The traffic light recognition unit 150 recognizes the color of the lamp and the direction of the arrow of the traffic light to recognize the traffic information.

[0044] The sound recognition unit 140 obtains the sound picked up by the microphone 20 and recognizes the obtained sound. For example, the sound recognition unit 140 recognizes whether the picked-up sound is the same as or similar to a predetermined sound stored in the storage device 70. The predetermined sound is, for example, the sound output by the traffic light for pedestrians. The sound output by the traffic light for pedestrians is, for example, the sound output when the traffic light for pedestrians indicates go. This sound is output, for example, so that visually impaired persons can recognize the traffic information of the traffic light for pedestrians. The sound recognition unit 140 may use audio-visual localization technology to recognize the direction and position where the sound is output.

[0045] The traffic signal recognition unit 150 recognizes the traffic information of the pedestrian traffic signal using the auxiliary information described later. When the recognition unit 130 cannot recognize the traffic information of the pedestrian traffic signal or the confidence level of the recognition result of the traffic information of the pedestrian traffic signal is less than the threshold value, the traffic signal recognition unit 150 recognizes (or estimates) the traffic information of the pedestrian traffic signal using the auxiliary information. The confidence level being less than the threshold value means, for example, a state where the traffic information of the pedestrian traffic signal cannot be recognized or cannot be recognized sufficiently. For example, when the camera of the external detection device 10 cannot clearly image the pedestrian traffic signal or when the traffic information of the traffic signal cannot be obtained from the image (or when the accuracy of image processing is low), the traffic signal recognition unit 150 cannot recognize the traffic information of the pedestrian traffic signal from the image. In this case, it is determined that the confidence level is less than the threshold value.

[0046] The control unit 160 controls the drive device 40 according to, for example, the set driving mode. The mobile body 1 may execute only some of the following driving modes, but in any case, the control unit 160 sets different speed limit values when the mobile body 1 moves on a roadway and when it moves on a sidewalk. In that case, the mode changeover switch 22 may be omitted.

[0047] In mode A-1, the control unit 160 refers to the information on the travel path and the object based on the output of the recognition unit 130. When the mobile body 1 moves on a roadway, it maintains a certain distance or more from the object existing in front of the mobile body 1. When the distance from the object existing in front of the mobile body 1 is sufficiently long, the control unit 160 controls the motor MT of the drive device 40 so that the mobile body 1 moves at the first speed V1 (for example, a speed of 10 [km / h] or more and less than several tens [km / h]).

[0048] When the mobile body 1 moves on a sidewalk, the control unit 160 maintains the distance from an object existing in front of the mobile body 1 to be a certain value or more, and when the distance from the object existing in front of the mobile body 1 is sufficiently long, controls the motor MT of the drive device 40 so that the mobile body 1 moves at a second speed V2 (for example, a speed less than ten [km / h]). Such a function is similar to the ACC (Adaptive Cruise Control) function of a vehicle with the first speed V1 or the second speed V2 as a set speed, and the technology used in ACC can be utilized. In mode A-1, the control unit 160 controls the steering device SD so as to change the steering angle of the steered wheels based on the operation amount of the operator 14 such as a steering wheel. Such a function is similar to the function of a power steering device, and the technology used in the power steering device can be utilized. Note that the mobile body 1 may have a steering device in which the operator 14 and the steering mechanism are mechanically connected without performing electronic control for steering.

[0049] In mode A-2, the control unit 160 refers to the information on the walking path and the object based on the output of the recognition unit 130, generates a target trajectory that can avoid the object within the walking path, and controls the steering device SD of the drive device 40 so that the mobile body 1 moves along the target trajectory. Regarding acceleration and deceleration, the control unit 160 controls the motor MT of the drive device 40 based on the speed of the mobile body 1 and the operation amount of the accelerator pedal or the brake pedal. When the mobile body 1 is moving on a road lane, the control unit 160 controls the motor MT of the drive device 40 with the first speed V1 as the upper limit speed (in the case of mode A-2, it means that even if there is a further acceleration instruction when the upper limit speed is reached, the mobile body 1 is not accelerated), and when the mobile body 1 is moving on a sidewalk, controls the drive device 40 with the second speed V2 as the upper limit speed.

[0050] In Mode B, the control unit 160 controls the motor MT of the drive device 40 based on the speed of the moving body 1 and the operation amount of the accelerator pedal or the brake pedal. When the moving body 1 is moving on a roadway, the control unit 160 controls the motor MT of the drive device 40 with the first speed V1 as the upper limit speed (in the case of Mode B, this means that even if there is a further acceleration instruction when the upper limit speed is reached, the moving body 1 will not be accelerated). When the moving body 1 is moving on a sidewalk, the control unit 160 controls the motor MT of the drive device 40 with the second speed V2 as the upper limit speed. Regarding steering, it is the same as in Mode A-1. Hereinafter, the mode when traveling on a roadway may be referred to as Mode B1 (the second moving mode), and the mode when traveling on a sidewalk may be referred to as Mode B2 (the first moving mode).

[0051] In Mode C, the control unit 160 refers to the information on the roadway and the object based on the output of the recognition unit 130, generates a target trajectory that can avoid the object within the roadway and move, and controls the drive device 40 so that the moving body 1 moves along the target trajectory. Also in Mode C, when the moving body 1 is moving on a roadway, the control unit 160 controls the drive device 40 with the first speed V1 as the upper limit speed, and when the moving body 1 is moving on a sidewalk, the control unit 160 controls the drive device 40 with the second speed V2 as the upper limit speed.

[0052] [Processing when crossing a road] The following describes the process when the moving body 1 crosses a road in mode C. FIG. 4 is a diagram showing an example of a scene where traffic information of a pedestrian signal cannot be recognized. For example, assume that the moving body 1 has stopped on a sidewalk in order to cross the road. In this case, there are a plurality of traffic participants around the moving body 1, and the moving body 1 cannot recognize the traffic information of the pedestrian signal SP, or even if it can be temporarily recognized, it cannot be surely recognized. In such a case, the moving body 1 determines that the confidence level of the recognized traffic information of the pedestrian signal SP is less than the threshold value. For example, when the vehicle height of the moving body 1 is about the height of a person's shoulder or lower, even if it can recognize the traffic information of the vehicle signal SV attached at a high position, it may not be able to recognize the traffic information of the pedestrian signal SP. Therefore, the moving body 1 performs the following process. For example, the following process is performed when moving on the sidewalk in mode C.

[0053] Based on the output of a detection device that detects the external situation of the moving body 1, the signal recognition unit 150 recognizes the traffic information of the pedestrian signal using auxiliary information that is different from the traffic information indicated by the signal of the pedestrian signal that the moving body 1 is paying attention to and that is for recognizing the traffic information indicated by the signal of the pedestrian signal.

[0054] The auxiliary information includes, for example, one or more pieces of information among (1)-(3). (1) is an audible notification for visually impaired persons output according to the traffic information of the pedestrian signal. (2) is the traffic information of a signal (another signal) that permits the progress of traffic participants in a direction different from the direction in which the pedestrian signal that the moving body 1 is paying attention to permits the progress of traffic participants. (3) is information indicating the movement of the person around the pedestrian signal or the moving body 1 that is being paid attention to.

[0055] (1) is acquired by the sound recognition unit 140. (2) and (3) are acquired by the recognition unit 130. The traffic signal recognition unit 150 recognizes the traffic information of the pedestrian traffic signal of interest based on one or more pieces of auxiliary information among the above information. For example, the traffic signal recognition unit 150 recognizes that the pedestrian traffic signal of interest permits progress or recognizes that it gives a stop instruction.

[0056] When the mobile body 1 is located in a predetermined area, the control unit 160 controls the movement of the mobile body 1 based on the traffic information of the pedestrian traffic signal recognized by the traffic signal recognition unit 150. The predetermined area is, for example, a sidewalk. Controlling the mobile body 1 means starting the movement of the mobile body or maintaining a state of keeping it stopped. Starting the movement means, for example, moving the mobile body 1 within the sidewalk. For example, moving the mobile body 1 forward until it reaches in front of the road. Controlling the mobile body 1 may mean, for example, performing control to quickly start the movement of the mobile body 1. Control for quickly starting the movement of the mobile body 1 means putting it in a ready state for starting the movement. The ready state means, for example, supplying power to the motor MT, increasing the output of the motor MT, reducing the braking degree of the braking device, releasing the parking brake, and the like.

[0057] FIG. 5 is a diagram for explaining the processing of the traffic signal recognition unit 150 using the auxiliary information. The traffic signal recognition unit 150 determines, for example, whether the above-described auxiliary information (1)-(3) satisfies the conditions corresponding to each piece of auxiliary information, and recognizes the traffic information of the pedestrian traffic signal based on the determination result. This will be specifically described below.

[0058] FIG. 6 is a diagram for explaining processing using a warning sound or other traffic signal for visually impaired persons. When the traffic signal recognition unit 150 cannot recognize the traffic information of the pedestrian signal SP and recognizes the sound output from the pedestrian signal SP, it recognizes that the traffic information of the pedestrian signal SP has changed from a state indicating stop to a state indicating go. For example, the traffic signal recognition unit 150 may determine whether the pedestrian signal SP is outputting a sound by referring to the position and direction where the sound recognized by the sound recognition unit 140 is output. When the sound output when the pedestrian signal SP indicates go, which is stored in the storage device 70 in advance, is output from the pedestrian signal SP, the traffic signal recognition unit 150 may recognize that the pedestrian signal indicates go.

[0059] The traffic signal recognition unit 150 may recognize the traffic information of the pedestrian signal SP based on the state of the vehicle signal SV (a signal that permits the progress of a traffic participant in a direction different from the direction in which the pedestrian signal that the moving body 1 focuses on permits the progress of the traffic participant). For example, when the recognition unit 130 cannot recognize the traffic information of the pedestrian signal SP and recognizes the traffic information of the vehicle signal SV, the traffic signal recognition unit 150 recognizes the traffic information of the pedestrian signal SP based on the traffic information of the vehicle signal SV. For example, when the vehicle signal SV indicates stop, the traffic signal recognition unit 150 may recognize that the pedestrian signal SP is likely to indicate go.

[0060] FIG. 7 is a diagram for explaining processing using human movement. When the traffic signal recognition unit 150 cannot recognize the traffic information of the pedestrian traffic signal SP, based on the movement of traffic participants around the moving body 1 or the movement of traffic participants estimated to move based on the information indicated by the pedestrian traffic signal SP, it is recognized that the pedestrian traffic signal SP has changed from a state indicating stop to a state indicating go. The traffic participants around the moving body 1 are traffic participants H (such as people) within a predetermined distance from the moving body 1 or existing in front of the moving body 1. The traffic participants estimated to move based on the information indicated by the pedestrian traffic signal SP are, for example, traffic participants located around the pedestrian traffic signal SP. The traffic participants located around the pedestrian traffic signal SP are, for example, traffic participants H (traffic participants located on the sidewalk on the opposite side of the road from the sidewalk where the pedestrian traffic signal SP is installed) located on the sidewalk on the opposite side of the road from the sidewalk where the moving body 1 is located. For example, when the traffic participant H starts moving in the direction of the pedestrian traffic signal SP, the traffic signal recognition unit 150 recognizes that the pedestrian traffic signal SP has changed from a state indicating stop to a state indicating go. The traffic participants located around the pedestrian traffic signal SP may be, for example, (instead of or in addition to) traffic participants located on the sidewalk where the pedestrian traffic signal SP is installed (traffic participants not shown located on the sidewalk on the opposite side of the road from the sidewalk where the moving body 1 exists).

[0061] As described above, when the traffic signal recognition unit 150 satisfies a predetermined number of the following (Condition 1) - (Condition 3), it recognizes that the pedestrian traffic signal SP is in a state indicating go. (Condition 1) A notification sound has been output. (Condition 2) Another traffic signal has changed to a stop state. (Condition 3) The surrounding traffic participants have started moving in accordance with the traffic information indicating the progress of the pedestrian traffic signal they are paying attention to.

[0062] The traffic signal recognition unit 150 may calculate a score as follows. When the calculated score is equal to or higher than a threshold value, it may recognize that the pedestrian traffic signal SP indicates "go". For example, when the above conditions are satisfied, the traffic signal recognition unit 150 assigns points according to the satisfied conditions and weights the assigned points. The weights are set for each condition. The above score may be added to the above-described confidence level. When the confidence level including the calculated score is equal to or higher than the threshold value, it may be recognized that the pedestrian traffic signal indicates "go".

[0063] As described above, when the traffic signal recognition unit 150 recognizes that the pedestrian traffic signal SP indicates "go", the control unit 160 starts preparing to cross the road according to the pedestrian traffic signal SP or starts an operation for crossing. For example, the control unit 160 moves the moving body 1 closer to the crosswalk or moves the moving body 1 to a position in front of the crosswalk. For example, after starting this operation, when the recognition unit 130 can confirm traffic information indicating that the pedestrian traffic signal SP actually indicates "go", the control unit 160 causes the moving body 1 to cross the road. The moving body 1 moves on the sidewalk and the crosswalk with the second speed V2 as the upper limit speed.

[0064] In this way, by controlling the moving body 1 using the recognition result of the traffic signal recognition unit 150, the control unit 160 can control the moving body 1 according to the surrounding situation even when traffic information of the pedestrian traffic signal SP cannot be recognized.

[0065] For example, when there is a traffic participant in front of the moving body 1 or when it is impossible to recognize that the pedestrian signal SP indicates go, even if the traffic participant in front starts to move, the moving body 1 cannot move from its position and may interfere with the behavior of the traffic participants behind. In contrast, in the present embodiment, since the moving body 1 starts control according to the recognition result of the signal recognition unit 150 as described above, it can move quickly or start moving to a position where it is easy to recognize the pedestrian signal SP while ensuring safety. As a result, it can be expected that the moving body 1 recognizes the traffic information of the pedestrian signal SP while adapting to the surrounding behavior and can cross the road while adapting to the surrounding situation.

[0066] [Flowchart] FIG. 8 is a flowchart showing an example of the flow of processing executed by the control device 100. First, the recognition unit 130 determines whether it can recognize the traffic information of the target pedestrian signal (step S100). If it cannot recognize the traffic information of the target pedestrian signal, the signal recognition unit 150 recognizes the sound output by the pedestrian signal, the traffic information of other signals, or the movement of traffic participants (steps S102, S104, S106).

[0067] Next, the signal recognition unit 150 derives a score based on the recognition results in steps S102 - S106 and determines whether the score is equal to or greater than a threshold value (step S108). If the score is less than the threshold value, the processing of one routine of this flowchart ends.

[0068] If the score is equal to or greater than the threshold value, the signal recognition unit 150 recognizes that the target pedestrian signal indicates go (step S110). Next, the control unit 160 controls the moving body 1 based on the recognition result of the signal recognition unit 150 (step S112). For example, the control device 100 recognizes the traffic information of the pedestrian signal while moving the moving body 1 closer to the crosswalk, and when the confidence level of the traffic information of the pedestrian signal is equal to or greater than the threshold value, causes the moving body 1 to cross the crosswalk. Thereby, the processing of one routine of this flowchart ends.

[0069] As described above, the control device 100 can recognize the traffic information of the pedestrian signal using the auxiliary information and control the moving body 1 according to the recognition result, thereby realizing the control of the moving body according to the surrounding situation.

[0070] [Processing at intersections] In the following description, an example in which the moving body 1 crosses a crosswalk at an intersection will be described. FIG. 9 is a diagram showing an example of a scene where a crosswalk is crossed at an intersection. For example, at the intersection, it is assumed that four crosswalks C1 - C4 are provided, and pedestrian signals are provided for each crosswalk. For crosswalk C1, SP1A and SP1B are provided, ··· for crosswalk C4, SP4A and SP4B are provided. At the intersection, vehicle signals SV1 and SV3 for regulating traffic in the first direction and vehicle signals SV2 and SV4 for regulating traffic in the second direction intersecting the first direction are provided. For example, it is assumed that the moving body 1 has stopped near the pedestrian signal SP4A in order to cross the crosswalk C4. The moving body 1 crosses the crosswalk C4 according to the traffic information of the pedestrian signals SP4A and SP4B (pedestrian signals corresponding to the crosswalk C4), but other traffic participants cannot recognize the traffic information of the pedestrian signals SP4A and SP4B.

[0071] In such a case, the signal recognition unit 150 recognizes the traffic information of the target pedestrian signal SPB based on the sound output by the pedestrian signals SP4A and SP4B (or one of the sounds output by the pedestrian signal SP4A and the sound output by the pedestrian signal SP4B), the traffic information of the vehicle signals SV1 - SV4, or the state of the traffic participant attempting to cross the crosswalk C4.

[0072] As described above, even at an intersection with a plurality of pedestrian signals, the control device 100 can recognize the traffic information of the pedestrian signals using the auxiliary information and control the moving body 1 according to the recognition result, thereby realizing the control of the moving body according to the surrounding situation.

[0073] [Timing when traffic information of a traffic signal changes and recognition of the traffic signal that has changed] Based on one or more of the above (1), (2), or (3) information, the traffic signal recognition unit 150 recognizes that the pedestrian traffic signal around the moving body 1 has switched from the first state of a stop instruction to the second state that permits progress. When recognizing that the state has switched to the second state among the above (1), (2), or (3), based on information different from the information used when recognizing the switch to the second state, it recognizes whether the pedestrian traffic signal of interest has changed to the second state or whether a pedestrian traffic signal different from the pedestrian traffic signal of interest has changed to the second state. For example, as shown in FIG. 10, the traffic signal recognition unit 150 recognizes the timing when the traffic information of the pedestrian traffic signal has switched based on an alarm sound, traffic information of other signals, or human movement, and uses elements different from the elements used for the determination of the above timing to recognize the switched pedestrian traffic signal. The different elements only need to have some elements different, and the elements may overlap.

[0074] For example, in FIG. 9 described above, when the traffic signal recognition unit 150 recognizes a sound, it recognizes that the traffic information of one of the traffic signals at the intersection has changed. When the movement of a traffic participant starts, it recognizes that the traffic information of the pedestrian traffic signal corresponding to the direction in which the traffic participant moves has changed. In FIG. 9, when a traffic participant moves according to the traffic information of the pedestrian traffic signal SP4B, the traffic signal recognition unit 150 recognizes that the traffic information of the pedestrian traffic signal SP4B has changed. When other traffic participants move according to the traffic information of the pedestrian traffic signals SP3A and SP3B, the traffic signal recognition unit 150 recognizes that the traffic information of the pedestrian traffic signal SP4B has not changed. For example, when the vehicle traffic signal SV3 changes to a state indicating go or the vehicle traffic signal SV4 changes to traffic information indicating stop, the traffic signal recognition unit 150 may recognize that the traffic information of the pedestrian traffic signal SP4B has changed.

[0075] For example, when the traffic signal recognition unit 150 recognizes that the traffic information of the pedestrian signal has changed, the mobile body 1 performs a first operation. The first operation is a preparation for the mobile body 1 to move. The preparation means that the mode of the mobile body 1 transitions from the stopped state mode to the ready state mode. For example, it is a mode for operating the motor MT, a mode for relaxing the braking degree of the braking device, or a mode for releasing the parking brake (or preparing for release).

[0076] For example, when the traffic signal recognition unit 150 recognizes that the traffic information of the pedestrian signal (the pedestrian signal corresponding to the crosswalk to cross) of interest has changed, the mobile body 1 performs a second operation. The second operation is, for example, an operation to move the mobile body 1 or an operation to transition the mode from the ready state mode to the stopped state mode. For example, when the signal for which the traffic information has changed is the pedestrian signal SP4B, the control unit 160 moves the mobile body 1 so as to cross the crosswalk C4. For example, the control device 100 moves the mobile body 1 closer to the crosswalk while following the movement of the traffic participants.

[0077] For example, when the traffic signal for which the traffic information has changed is the pedestrian signal SP3A, the control unit 160 changes the mode from the ready state mode to the stopped state mode or moves the mobile body 1 to a position that does not obstruct the traffic of the traffic participants crossing the crosswalk C3.

[0078] [Flowchart] FIG. 11 is a flowchart showing an example of the flow of processing executed by the control device 100. First, the recognition unit 130 determines whether it can recognize the traffic information of the pedestrian signal of interest (step S200). If it cannot recognize the traffic information of the pedestrian signal of interest, the traffic signal recognition unit 150 determines whether it satisfies the conditions regarding the timing of the change in the traffic information of the pedestrian signal (step S202).

[0079] When the conditions regarding the timing of the change in traffic information of the pedestrian signal are satisfied, the control unit 160 recognizes that the traffic information of the pedestrian signal has changed and performs a first operation (step S204). Next, the signal recognition unit 150 determines whether the conditions for specifying the pedestrian signal are satisfied (step S206). When the conditions for specifying the pedestrian signal are satisfied, the control unit 160 starts a second operation based on the recognition result of the signal recognition unit 150 (step S208). Thereby, the processing of one routine of this flowchart is completed.

[0080] By the above-described processing, the control device 100 can shift to an operation for smoothly moving the moving body 1 by the first operation, and can realize control of the moving body according to the surrounding situation by the second operation.

[0081] <Modification Example> The signal recognition unit 150 may recognize the traffic information of the signal based on the signal information 76 stored in the storage device 70. The signal information 76 is information in which information regarding the pedestrian signal at the intersection and information on the sound output by the pedestrian signal are associated with each other. The information regarding the pedestrian signal is, for example, the position of the pedestrian signal, information on the moving direction restricted by the pedestrian signal, the moving direction allowed when the sound is being output, and the like. The sound output by the pedestrian signal is, for example, the sound output when the pedestrian signal indicates "go ahead".

[0082] FIG. 12 is a diagram showing an example of the signal information 76. For example, the pedestrian signal 1 at intersection 1 outputs sound 1, and the pedestrian signal 2 does not output sound 2. For example, the pedestrian signal SW3 at intersection 3 outputs sound 5, and the pedestrian signal SW4 outputs sound 6. The signal recognition unit 150 refers to the signal information 76 and specifies the pedestrian signal corresponding to the sound output by the pedestrian signal recognized by the sound recognition unit 140 and the moving direction allowed by the pedestrian signal.

[0083] FIG. 13 is a diagram showing an example of a scene where the traffic signal recognition unit 150 controls the moving body 1 using the traffic signal information 76. Descriptions similar to those in FIG. 9 are omitted. At an intersection, when the pedestrian traffic signal SW4B indicates go, the moving body 1 is planned to cross the crosswalk C4 corresponding to the pedestrian traffic signal SW4. At this time, assume that the pedestrian traffic signal SW3A changes to a state indicating go and the pedestrian traffic signal SW3A outputs sound 5. The traffic signal recognition unit 150 refers to the traffic signal information 76 to specify the position of the pedestrian traffic signal associated with the sound 5 recognized by the sound recognition unit 140 and the moving direction allowed by the pedestrian traffic signal. The traffic signal recognition unit 150 recognizes that the pedestrian traffic signal SW3A has changed to a state where it is possible to cross the crosswalk C3. The traffic signal recognition unit 150 recognizes that the traffic information of the pedestrian traffic signal SW4B corresponding to the crosswalk C4 that the moving body 1 is planned to cross has not changed.

[0084] For example, after sound 5 stops, when the traffic signal recognition unit 150 recognizes the sound 6 associated with the pedestrian traffic signal SW4A, it recognizes that the pedestrian traffic signal SW4B has changed to a state where it can cross the crosswalk C4. Instead of recognizing, the confidence level that the pedestrian traffic signal SW4B has changed to a state where it can cross the crosswalk C4 may be increased, or the above-mentioned score may be increased.

[0085] As described above, by using the traffic signal information 76, the traffic signal recognition unit 150 can recognize the traffic information of the pedestrian traffic signal with higher accuracy.

[0086] In the above example, in the case of mode C, it has been described that the moving body 1 is controlled according to the recognition result of the traffic information of the above-mentioned pedestrian traffic signal. Instead (or in addition), the process of controlling the moving body 1 according to the recognition result of the traffic information of the above-mentioned pedestrian traffic signal may also be applied in other modes. For example, the occupant may be notified that the traffic information of the pedestrian traffic signal has changed, or the operation or acceleration / deceleration may be controlled according to the traffic information of the pedestrian traffic signal.

[0087] In addition to (or instead of) the auxiliary information, the control device 100 may recognize the traffic information of the traffic signal based on the experience information or the provided information. The experience information is information generated based on information about the traffic signal obtained when the moving body 1 or another moving body 1 passes through the area where the traffic signal is installed. The provided information is information about the traffic signal provided by a predetermined institution or server device. Information about the traffic signal is, for example, among a plurality of traffic signals, the timing at which the traffic signal indicates go or stop, information about the sound output by the traffic signal, and the like. For example, the experience information or the provided information includes information such as a schedule in which the traffic information of the traffic signal changes, such as after the first traffic signal stops, after a predetermined time, the second traffic signal indicates go.

[0088] As described above, the control device 100 can use the auxiliary information to control the moving body 1, thereby realizing control of the moving body according to the surrounding situation.

[0089] The above-described embodiment can be expressed as follows. A storage medium storing computer-readable instructions, A processor connected to the storage medium, and the processor, by executing the computer-readable instructions (the processor executing the computer-readable instructions to:) controls a moving body capable of moving in both a lane and a predetermined area different from the lane, recognizes whether the moving body is moving in the lane or moving in the predetermined area based on the output of a detection device that detects the external situation of the moving body, acquires auxiliary information that is different from the traffic information indicating the go-ahead permission and stop instruction indicated by the signal of the pedestrian traffic signal and that recognizes the go-ahead permission or stop instruction indicated by the signal of the pedestrian traffic signal that the moving body is focusing on, Using the acquired auxiliary information, recognize traffic information indicating permission to proceed or a stop instruction indicated by the signal of the pedestrian traffic signal, When the moving body is located in the predetermined area, control the moving body based on the recognized traffic information of the pedestrian traffic signal, A control device for a moving body.

[0090] As described above, the embodiments for carrying out the present invention have been described using the embodiments. However, the present invention is not limited to such embodiments, and various modifications and substitutions can be made without departing from the gist of the present invention.

Explanation of Reference Numerals

[0091] 10 External detection device 12 Moving body sensor 14 Operator 16 Internal camera 18 Positioning device 20 Microphone 22 Mode switch 30 Moving mechanism 40 Driving device 50 External notification device 70 Storage device 100 Control device 120 Road type recognition unit 130 Recognition unit 140 Voice recognition unit 150 Traffic signal recognition unit 160 Control unit

Claims

1. A control device for a moving body capable of moving in both a lane and a predetermined area different from the lane, comprising: a road type recognition unit that recognizes whether the moving body is moving in the lane or in the predetermined area based on the output of a detection device that detects the external situation of the moving body; an acquisition unit that acquires auxiliary information, which is information different from the traffic information indicating the travel permission and stop instruction indicated by the signal of the pedestrian signal that the moving body is focusing on, and is for recognizing the travel permission or stop instruction indicated by the signal of the pedestrian signal; a signal recognition unit that recognizes the traffic information indicating the travel permission or stop instruction indicated by the signal of the pedestrian signal by using the auxiliary information acquired by the acquisition unit; a control unit that controls the moving body based on the traffic information of the pedestrian signal recognized by the signal recognition unit when the moving body is located in the predetermined area; A control device for a moving body, comprising the above components.

2. The auxiliary information is an audible notification for visually impaired persons output according to the traffic information of the pedestrian signal. The control device for a moving body according to Claim 1.

3. The auxiliary information is the traffic information of a signal that permits the travel of traffic participants in a direction different from the direction in which the pedestrian signal being focused on permits the travel of traffic participants. The control device for a moving body according to Claim 1.

4. The auxiliary information is information indicating the movement of a person estimated to move based on the traffic information indicated by the pedestrian signal being focused on or information indicating the movement of people around the moving body. The control device for a moving body according to Claim 1.

5. The auxiliary information is an audible notification for visually impaired persons output according to the traffic information of the pedestrian signal, and information indicating the movement of a person estimated to move based on the traffic information indicated by the pedestrian signal being focused on or information indicating the movement of people around the moving body. The signal recognition unit recognizes whether the pedestrian signal being focused on permits travel by using the audible notification for visually impaired persons and the information indicating the movement of people. The control unit controls the moving body based on the recognition result of the signal recognition unit. The control device for a moving body according to Claim 1.

6. The auxiliary information is an audible notification for visually impaired persons output according to the traffic information of the pedestrian signal, and the traffic information of a signal that permits the travel of traffic participants in a direction different from the direction in which the pedestrian signal being focused on permits the travel of traffic participants. The signal recognition unit uses the traffic information of the audible notification for visually impaired persons and the traffic signal that permits travel in different directions to recognize whether the pedestrian signal being focused on permits travel. The control unit controls the moving body based on the result of the recognition by the signal recognition unit. The control device for a moving body according to claim 1.

7. The auxiliary information is information indicating the movement of a person estimated to move based on the traffic information indicated by the pedestrian signal being focused on or information indicating the movement of a person around the moving body, and the traffic information of a traffic signal that permits travel in a direction different from the direction in which the pedestrian signal being focused on permits travel. The signal recognition unit uses the information indicating the movement of the person and the traffic information of the traffic signal that permits travel in different directions to recognize whether the pedestrian signal being focused on permits travel. The control unit controls the moving body based on the result of the recognition by the signal recognition unit. The control device for a moving body according to claim 1.

8. The auxiliary information is (1) an audible notification for visually impaired persons output according to the traffic information of the pedestrian signal, (2) information indicating the movement of a person estimated to move based on the traffic information indicated by the pedestrian signal being focused on or information indicating the movement of a person around the moving body, or (3) the traffic information of a traffic signal that permits travel in a direction different from the direction in which the pedestrian signal being focused on permits travel. The signal recognition unit recognizes that the pedestrian signal around the moving body has switched from the first state of a stop instruction to the second state of permitting travel based on one or more pieces of information among (1), (2), or (3), and based on information different from the information used when recognizing the switch to the second state among (1), (2), or (3), recognizes whether the pedestrian signal being focused on has changed to the second state or whether a pedestrian signal different from the pedestrian signal being focused on has changed to the second state. The control device for a moving body according to claim 1.

9. The auxiliary information is an audible notification for visually impaired persons output according to the traffic information of the pedestrian signal. The signal recognition unit refers to information in which the type of the audible notification stored in the storage device is associated with the direction in which the pedestrian signal permits the travel of traffic participants, and recognizes the traffic information of the pedestrian signal based on the audible notification. The control device for a moving body according to claim 1.

10. The control unit: limits the speed of the moving body to a first speed when the moving body moves on a roadway, and limits the speed of the moving body to a second speed lower than the first speed when the moving body moves in the predetermined area; when it is planned to move according to the traffic information of the pedestrian signal being focused on and the signal recognition unit recognizes that the pedestrian signal being focused on permits progress, starts an operation to move the moving body in the direction permitted by the pedestrian signal; The control device for a moving body according to claim 1.

11. In an intersection including a first pedestrian signal that regulates traffic in a first direction and a second pedestrian signal that regulates traffic in a second direction different from the first direction, when the moving body moves according to the traffic information of the first pedestrian signal, the signal recognition unit, based on one or more pieces of information among the auxiliary information, an auditory notification sound for visually impaired persons output according to the traffic information of the pedestrian signal, the traffic information of a signal different from the first pedestrian signal, or information indicating the movement of a person estimated to move based on the traffic information indicated by the first pedestrian signal or information indicating the movement of a person around the moving body, recognizes the state of the first pedestrian signal; when the signal recognition unit recognizes that the state of the first pedestrian signal has changed to a state permitting progress, the control unit moves the moving body according to the state of the first pedestrian signal; The control device for a moving body according to claim 1.

12. The acquisition unit acquires the traffic information of the pedestrian signal being focused on based on the output of the detection device; when the acquisition unit cannot acquire the traffic information, the signal recognition unit acquires the traffic information of the pedestrian signal using the auxiliary information; The control device for a moving body according to claim 1.

13. Based on the recognition result of the signal recognition unit, the control unit controls the moving body, and then, when the acquisition unit acquires the traffic information of the pedestrian signal being focused on and recognizes that the pedestrian signal being focused on permits progress, causes the moving body to enter from the sidewalk where the moving body is located into a road or crosswalk permitted by the traffic information and cross the road or the crosswalk; The control device for a moving body according to claim 1.

14. A control device for a moving body capable of moving in both a roadway and a predetermined area different from the roadway Based on the output of a detection device that detects the external situation of the moving body, recognize whether the moving body is moving on a roadway or moving in the predetermined area. Obtain auxiliary information that is different from the traffic information indicating the go-ahead and stop instructions indicated by the signal of the pedestrian signal that the moving body is paying attention to, and that recognizes the go-ahead or stop instruction indicated by the signal of the pedestrian signal. Using the obtained auxiliary information, recognize the go-ahead or stop instruction indicated by the signal of the pedestrian signal. When the moving body is located in the predetermined area, control the moving body based on the recognized state of the pedestrian signal. A method for controlling a moving body.

15. In a control device for a moving body capable of moving in both a roadway and a predetermined area different from the roadway, Based on the output of a detection device that detects the external situation of the moving body, cause recognition of whether the moving body is moving on a roadway or moving in the predetermined area. Cause acquisition of auxiliary information that is different from the traffic information indicating the go-ahead and stop instructions indicated by the signal of the pedestrian signal that the moving body is paying attention to, and that recognizes the go-ahead or stop instruction indicated by the signal of the pedestrian signal. Using the obtained auxiliary information, cause recognition of the go-ahead or stop instruction indicated by the signal of the pedestrian signal. When the moving body is located in the predetermined area, cause control of the moving body based on the recognized state of the pedestrian signal. A storage medium in which a program is stored.

Images