Mobile body control device, mobile body control method, and storage medium

By combining road type identification and signal recognition, using auxiliary information and reporting sound, the problem that the existing system failed to consider the status of pedestrian signal is solved, and the flexible control and safety response of mobile bodies between lanes and specified areas is realized. .

CN118946919APending Publication Date: 2024-11-12HONDA MOTOR CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202280094202.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-03-29
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing mobile body control system fails to effectively consider the status of pedestrian signal machines, resulting in the inability to properly control according to the surrounding conditions in some cases.

Method used

A mobile body control device is designed to identify and control the travel direction and speed of the mobile body through road type identification, auxiliary information acquisition and signal recognition, combined with the report sound used by visually impaired persons and traffic information of pedestrian signal. , ensure that it is in line with the surrounding conditions.

Benefits of technology

It realizes flexible control of the mobile body between the lane and the specified area, can quickly respond to changes in traffic information of the signal, and improves safety and accuracy in complex environments such as intersections.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118946919A_ABST
    Figure CN118946919A_ABST
Patent Text Reader

Abstract

A control device for a moving body capable of moving both in a lane and in a predetermined region different from the lane, the control device being provided with: a road type recognition unit that recognizes a road type of the moving body on the basis of an output from a detection device that detects an external condition of the moving body; whether the moving body moves on the lane or in the specified area is identified by the identification unit; an acquisition unit that acquires assistance information, which is different from traffic information indicating a travel permission or stop instruction indicated by a signal of a pedestrian traffic light displayed by the pedestrian traffic light to which the moving body focuses, for identifying the travel permission or stop instruction indicated by the signal of the pedestrian traffic light, the assistance information being information for identifying the travel permission or stop instruction indicated by the signal of the pedestrian traffic light; a signal recognition unit that uses the assist information acquired by the acquisition unit to recognize traffic information as a travel permission or stop instruction indicated by the signal of the pedestrian traffic light; and a control unit that, when the moving body is located in the predetermined region, controls the moving body on the basis of the traffic information of the pedestrian traffic light recognized by the signal recognition unit.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a control device for a mobile object, a control method for a mobile object and a storage medium. Background Art

[0002] Conventionally, there has been disclosed a system that receives sounds from the left and right sides of a personal moving body and detects which side of the personal moving body is a lane based on the received sounds (see, for example, Patent Document 1).

[0003] Prior Art Literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-197387 Summary of the invention

[0006] Problems to be solved by the invention

[0007] However, in the above-mentioned system, the state of the pedestrian signal is not taken into consideration, and therefore the moving body may not be controlled according to the surrounding conditions.

[0008] The present invention has been made in consideration of such circumstances, and one of its objects is to provide a control device for a mobile body, a control method for a mobile body, and a storage medium that can realize control of the mobile body according to surrounding conditions.

[0009] Solutions to Solve Problems

[0010] The control device for a mobile object, the control method for a mobile object, and the storage medium according to the present invention employ the following configurations.

[0011] (1): One aspect of the present invention relates to a control device for a mobile body, wherein the mobile body is capable of moving in both a lane and a specified area different from the lane, wherein the control device for the mobile body comprises: a road type recognition unit, which recognizes whether the mobile body is moving in the lane or in the specified area based on the output of a detection device that detects the external condition of the mobile body; an acquisition unit, which acquires auxiliary information, the auxiliary information being information different from the traffic information indicating permission to proceed and stop instructions indicated by the signal of the pedestrian signal that the mobile body is focusing on, and used to recognize the permission to proceed or stop instructions indicated by the signal of the pedestrian signal; a signal recognition unit, which uses the auxiliary information acquired by the acquisition unit to recognize the traffic information indicated by the signal of the pedestrian signal as permission to proceed or stop instructions; and a control unit, which controls the mobile body based on the traffic information of the pedestrian signal recognized by the signal recognition unit when the mobile body is located in the specified area.

[0012] (2) In the above-mentioned aspect (1), the auxiliary information is a notification sound for the visually impaired outputted based on the traffic information of the pedestrian signal.

[0013] (3) Based on the scheme of (1) or (2) above, the auxiliary information is the traffic information of a signal that permits traffic participants to travel in a direction different from the direction permitted by the pedestrian signal.

[0014] (4): Based on any one of the above schemes (1) to (3), the auxiliary information is information indicating the behavior of a person who is estimated to be moving based on the traffic information shown by the pedestrian signal of interest or information indicating the behavior of people around the moving body.

[0015] (5): Based on any one of the above schemes (1) to (4), the auxiliary information is a report sound for the visually impaired output based on the traffic information shown by the pedestrian signal, and information indicating the movement of a person presumed to be moving based on the traffic information shown by the pedestrian signal or information indicating the movement of people around the moving body, and the signal recognition unit uses the report sound for the visually impaired and the information indicating the movement of the person to identify whether the pedestrian signal is allowed to proceed, and the control unit controls the moving body based on the recognition result of the signal recognition unit.

[0016] (6): Based on any one of the above schemes (1) to (5), the auxiliary information is a report sound for the visually impaired output based on the traffic information of the pedestrian signal, and the traffic information of the signal that permits travel in a direction different from the direction permitted by the pedestrian signal of interest, and the signal recognition unit uses the report sound for the visually impaired and the traffic information of the signal that permits travel in the different direction to identify whether the pedestrian signal of interest permits travel, and the control unit controls the moving body based on the recognition result of the signal recognition unit.

[0017] (7): Based on any one of the above schemes (1) to (6), the auxiliary information is information indicating the movement of a person presumed to be moving based on the traffic information shown by the pedestrian signal of interest or information indicating the movement of people around the moving body, and the traffic information of the signal that permits travel in a direction different from the direction permitted by the pedestrian signal of interest, and the signal recognition unit uses the information indicating the movement of the person and the traffic information of the signal that permits travel in the different direction to identify whether the pedestrian signal of interest permits travel, and the control unit controls the moving body based on the recognition result of the signal recognition unit.

[0018] (8) Based on any one of the above schemes (1) to (7), the auxiliary information is (1) a report sound for the visually impaired output based on the traffic information of the pedestrian signal, (2) information indicating the movement of a person who is presumed to be moving based on the traffic information shown by the pedestrian signal or information indicating the movement of people around the moving body, or (3) the traffic information of a signal that permits travel in a direction different from the direction permitted by the pedestrian signal, and the signal recognition unit recognizes that the pedestrian signal around the moving body has switched from a first state indicating a stop to a second state permitting travel based on one or more of the information (1), (2) and (3), and recognizes whether the pedestrian signal being targeted has changed to the second state or a pedestrian signal different from the pedestrian signal being targeted has changed to the second state based on information from (1), (2) and (3) that is different from the information used when recognizing the switch to the second state.

[0019] (9): Based on any one of the above schemes (1) to (8), the auxiliary information is a report sound for the visually impaired output according to the traffic information of the pedestrian signal, and the signal recognition unit refers to the information stored in the storage device that establishes a correspondence between the category of the report sound and the direction in which the pedestrian signal allows traffic participants to travel, and recognizes the traffic information of the pedestrian signal based on the report sound.

[0020] (10): Based on any one of the above schemes (1) to (9), the control unit limits the speed of the moving body when moving in the lane to a first speed, and limits the speed of the moving body when moving in the specified area to a second speed lower than the first speed. When the moving body is scheduled to move according to the traffic information of the pedestrian signal of interest and the signal recognition unit recognizes that the pedestrian signal of interest permits travel, the control unit starts an action for moving the moving body in the direction permitted by the pedestrian signal.

[0021] (11): Based on any one of the above schemes (1) to (10), the signal recognition unit identifies the state of the first pedestrian signal based on one or more of the following three information as the auxiliary information at an intersection including a first pedestrian signal that restricts traffic in a first direction and a second pedestrian signal that restricts traffic in a second direction different from the first direction, when the moving body moves according to the traffic information of the first pedestrian signal. The three information are: a report sound for the visually impaired output based on the traffic information of the pedestrian signal; the traffic information of a signal different from the first pedestrian signal; and information indicating the actions of a person who is presumed to be moving based on the traffic information shown by the first pedestrian signal or information indicating the actions of people around the moving body. When the signal recognition unit recognizes that the first pedestrian signal has changed to a state where travel is permitted, the control unit moves the moving body according to the state of the first pedestrian signal.

[0022] (12): Based on any one of the above schemes (1) to (11), the acquisition unit acquires the traffic information of the pedestrian signal of interest based on the output of the detection device, and the signal recognition unit uses the auxiliary information to acquire the traffic information of the pedestrian signal when the acquisition unit cannot acquire the traffic information.

[0023] (13): Based on the scheme of (12) above, the control unit controls the moving body based on the recognition result of the signal recognition unit, and then, when the acquisition unit acquires the traffic information of the pedestrian signal of interest and recognizes that the pedestrian signal of interest permits travel, the moving body is caused to enter the road or crosswalk where the traffic information permits travel from the sidewalk where the moving body is located and cross the road or crosswalk.

[0024] (14): Another scheme of the present invention relates to a control method for a mobile body, wherein the mobile body is capable of moving in both a lane and a specified area different from the lane, wherein the control device of the mobile body performs the following processing: based on the output of a detection device that detects the external condition of the mobile body, identifying whether the mobile body is moving in the lane or in the specified area; obtaining auxiliary information, which is information different from the traffic information indicating permission to proceed and stop instructions indicated by the signal of the pedestrian signal that the mobile body is focusing on, and is used to identify the permission to proceed or the stop instruction indicated by the signal of the pedestrian signal; using the obtained auxiliary information to identify the permission to proceed or the stop instruction indicated by the signal of the pedestrian signal; and when the mobile body is located in the specified area, controlling the mobile body based on the identified state of the pedestrian signal.

[0025] (15): Another embodiment of the present invention relates to a storage medium storing a program, wherein a mobile body can move in both a lane and a specified area different from the lane, and the program causes a control device of the mobile body to perform the following processing: based on the output of a detection device that detects the external condition of the mobile body, identify whether the mobile body is moving in the lane or in the specified area; obtain auxiliary information, which is information different from the traffic information indicating permission to proceed and stop instructions shown by the signal of the pedestrian signal that the mobile body is focusing on, and is used to identify the permission to proceed or the stop instruction shown by the signal of the pedestrian signal; use the obtained auxiliary information to identify the permission to proceed or the stop instruction shown by the signal of the pedestrian signal; and when the mobile body is located in the specified area, control the mobile body based on the identified state of the pedestrian signal.

[0026] Effects of the Invention

[0027] According to the above-mentioned aspects (1) to (15), it is possible to realize control of the moving object according to the surrounding conditions.

[0028] According to the above-mentioned aspect (8), the control device can quickly prepare for moving the moving body and quickly move the moving body by identifying the changed signal after recognizing that the traffic information of the signal has changed.

[0029] According to the above-mentioned aspect (9), the control device can more accurately identify the traffic signal whose traffic information has changed by using the type of the reporting sound. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1It is a diagram showing an example of the configuration of the moving object 1 and the control device 100 according to the embodiment.

[0031] Figure 2 This is a perspective view of the moving body 1 as viewed from above.

[0032] Figure 3 This is a diagram showing an example of a scene in which a moving object 1 is moving on a lane.

[0033] Figure 4 This is a diagram showing an example of a scene in which traffic information of a pedestrian signal cannot be recognized.

[0034] Figure 5 This is a diagram for explaining the processing of the signal recognition unit 150 using the auxiliary information.

[0035] Figure 6 This is a diagram for explaining processing using a warning sound or other signal for the visually impaired.

[0036] Figure 7 This is a diagram for explaining processing using human behavior.

[0037] Figure 8 This is a flowchart showing an example of the flow of processing executed by the control device 100 .

[0038] Fig. 9 This is a diagram showing an example of a scene of crossing a crosswalk at an intersection.

[0039] Fig.10 This is a diagram showing an example of identifying a signal switched to the second state and a condition for switching the signal to the second state.

[0040] Fig.11 This is a flowchart showing an example of the flow of processing executed by the control device 100 .

[0041] Fig.12 It is a diagram showing an example of the signal information 76 .

[0042] Fig.13 This is a diagram showing an example of a scene in which the signal recognition unit 150 controls the moving object 1 using the signal information 76 . DETAILED DESCRIPTION

[0043] Hereinafter, embodiments of a control device for a moving object, a control method for a moving object, and a storage medium according to the present invention will be described with reference to the accompanying drawings.

[0044] Hereinafter, the embodiments of the control device of the mobile body, the control method of the mobile body, and the storage medium of the present invention will be described with reference to the accompanying drawings. The mobile body moves in both a lane and a prescribed area different from the lane. The mobile body is sometimes referred to as a micro mobile body. An electric scooter is a type of micro mobile body. The so-called prescribed area is, for example, a sidewalk. In addition, the mobile body may be a means of transportation that passengers can take, or an autonomous mobile body that can travel autonomously in an unmanned manner. The latter autonomous mobile body is used, for example, for transporting goods. The mobile body may include a small mobile body that travels side by side with a walking user to carry goods or guide people, and may also include other mobile bodies that can move autonomously (such as walking robots, etc.). The so-called prescribed area may also be part or all of a roadside area, a bicycle lane, an open space, etc., or may include all of a sidewalk, a roadside area, a bicycle lane, an open space, etc. In the following description, the prescribed area is set to be a sidewalk. The part recorded as "sidewalk" in the following description can be appropriately read as "prescribed area".

[0045] Figure 1 1 is a diagram showing an example of the structure of a moving body 1 and a control device 100 according to the embodiment. The moving body 1 is equipped with, for example, an external detection device 10, a moving body sensor 12, an operating member 14, an internal camera 16, a positioning device 18, a microphone 20, a mode switching switch 22, a moving mechanism 30, a driving device 40, an external reporting device 50, a storage device 70, and the control device 100. Some of these structures that are not necessary for realizing the functions of the present invention may be omitted.

[0046] The external detection device 10 is a variety of devices that set the travel direction of the mobile body 1 as a detection range. The external detection device 10 includes an external camera, a radar device, LIDAR (Light Detection and Ranging), a sensor fusion device, etc. The external detection device 10 outputs information (image, position of an object, etc.) indicating the detection result to the control device 100.

[0047] The moving body sensor 12 includes, for example, a speed sensor, an acceleration sensor, a yaw rate (angular velocity) sensor, an azimuth sensor, and an operation amount detection sensor mounted on the operating member 14. The operating member 14 includes, for example, an operating member for indicating acceleration and deceleration (for example, an accelerator pedal, a brake pedal), and an operating member for indicating steering (for example, a steering wheel). In this case, the moving body sensor 12 may include an accelerator opening sensor, a brake pedal amount sensor, a steering torque sensor, and the like. The moving body 1 may also include an operating member other than the above as the operating member 14 (for example, a non-circular rotating operating member, a joystick, a button, etc.).

[0048] The interior camera 16 captures at least the head of the passenger of the moving object 1 from the front. The interior camera 16 is a digital camera using an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The interior camera 16 outputs the captured image to the control device 100 .

[0049] The positioning device 18 is a device for detecting the position of the mobile body 1. The positioning device 18 is, for example, a GNSS (Global Navigation Satellite System) receiver, which determines the position of the mobile body 1 based on the signal received from the GNSS satellite and outputs it as position information. The position information of the mobile body 1 can also be estimated based on the position of the Wi-Fi base station to which the communication device described later is connected. The microphone 20 receives the surrounding sound of the mobile body 1. The microphone 20 can also include a plurality of microphones.

[0050] The mode switching switch 22 is a switch operated by the passenger. The mode switching switch 22 may be a mechanical switch or a GUI (Graphical User Interface) switch set on a touch panel. The mode switching switch 22, for example, accepts an operation to switch the driving mode to any one of Mode A, Mode B, and Mode C. Mode A is an assist mode in which the passenger performs one of steering operation and acceleration / deceleration control, and the other of steering operation and acceleration / deceleration control is automatically performed. There may be Mode A-1 in which the passenger performs steering operation and acceleration / deceleration control is automatically performed, and Mode A-2 in which the passenger performs acceleration / deceleration operation and steering control is automatically performed. Mode B is a manual driving mode in which the passenger performs steering operation and acceleration / deceleration operation. Mode C is an automatic driving mode in which operation control and acceleration / deceleration control are automatically performed.

[0051] The moving mechanism 30 is a mechanism for moving the moving body 1 on the road. The moving mechanism 30 is, for example, a group of wheels including a steering wheel and a driving wheel. The moving mechanism 30 may also be a leg for multi-legged walking.

[0052] The drive device 40 outputs force to the moving mechanism 30 to move the moving body 1. For example, the drive device 40 includes a motor that drives the driving wheels, a battery that stores power supplied to the motor, a steering device that adjusts the steering angle of the steering wheel, etc. The drive device 40 may also include an internal combustion engine, a fuel cell, etc. as a driving force output unit or a power generation unit. The drive device 40 may also include a braking device that brakes by friction or air resistance.

[0053] The external reporting device 50 is, for example, a lamp, a display device, a speaker, etc., which is provided on the outer plate of the moving body 1 and is used to report information to the outside of the moving body 1. The external reporting device 50 performs different actions when the moving body 1 is moving on the sidewalk and when the moving body 1 is moving on the roadway. For example, the external reporting device 50 is controlled in the following manner: the lamp is lit when the moving body 1 is moving on the sidewalk, and the lamp is not lit when the moving body 1 is moving on the roadway. The light color of the lamp is preferably a color prescribed by the law. The external reporting device 50 may also be controlled in the following manner: the lamp is lit green when the moving body 1 is moving on the sidewalk, and the lamp is lit blue when the moving body 1 is moving on the roadway. When the external reporting device 50 is a display device, the external reporting device 50 displays the meaning of "driving on the sidewalk" in the form of text or graphics when the moving body 1 is driving on the sidewalk.

[0054] Figure 2 It is a perspective view obtained by observing the mobile body 1 from above. In the figure, FW is a steering wheel, RW is a driving wheel, SD is a steering device, MT is a motor, and BT is a battery. The steering device SD, the motor MT, and the battery BT are included in the driving device 40. AP is an accelerator pedal, BP is a brake pedal, WH is a steering wheel, SP is a speaker, and MC is a microphone. The mobile body 1 shown in the figure is a mobile body for one person to ride, and the passenger P sits in the driver's seat DS and wears a seat belt SB. The arrow D1 is the direction of travel (speed vector) of the mobile body 1. The external detection device 10 is arranged near the front end of the mobile body 1, the internal camera 16 is arranged at a position that can photograph the head of the passenger P from the front of the passenger P, and the mode switching switch 22 is arranged on the hub of the steering wheel WH. An external reporting device 50 as a display device is arranged near the front end of the mobile body 1.

[0055] return Figure 1 The storage device 70 is a non-temporary storage device such as a HDD (Hard Disk Drive), a flash memory, or a RAM (Random Access Memory). The storage device 70 stores map information 72, a program 74 for execution by the control device 100, and the like. Although the storage device 70 is shown outside the frame of the control device 100 in the figure, the storage device 70 may be included in the control device 100. In addition, the storage device 70 may also be set on a server not shown in the figure.

[0056] [Control device]

[0057] The control device 100 includes, for example, a road type recognition unit 120, a recognition unit 130, a sound recognition unit 140, a traffic light recognition unit 150, and a control unit 160. These components are implemented by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software) 74. Some or all of these components may be implemented by hardware (including a circuit unit: circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), and a GPU (Graphics Processing Unit), or may be implemented by the cooperation of software and hardware. The program may be stored in advance in the storage device 70, or may be stored in a removable storage medium (non-temporary storage medium) such as a DVD or CD-ROM, and installed in the storage device 70 by assembling the storage medium in a drive device.

[0058] The road type recognition unit 120 recognizes whether the moving object 1 is moving in a lane or on a sidewalk. The road type recognition unit 120 recognizes whether the moving object 1 is moving in a lane or on a sidewalk by, for example, analyzing an image captured by an external camera of the external detection device 10. Outputs of a radar device, LIDAR, a sensor fusion device, etc. may also be used as an auxiliary.

[0059] The road type recognition unit 120 recognizes whether the moving body 1 is moving in a lane or on a sidewalk based on multiple phenomena indicating that the moving body 1 is moving in a lane in the image captured by the external camera. For example, the road type recognition unit 120 classifies and labels each pixel in the frame of the image by class through semantic segmentation, and assumes multiple hypothetical areas and their boundaries. The road type recognition unit 120 classifies at least the area of ​​the image captured by the external camera into an area where the moving body 1 exists (hereinafter referred to as the current area) and an area adjacent to the current area and where the moving body 1 does not exist (hereinafter referred to as the adjacent area) for processing.

[0060] For example, if more than the prescribed phenomena among the following phenomena are included, it may be determined that the moving body 1 is moving in the lane. For example, there are no static obstacles (signboards, mailboxes, electric poles, trash cans, etc.) other than the vehicle 203 in the area 200, the vehicle 203 is moving in the area 200, there are road markings 205 on the road surface of the area 200, there is a crosswalk 206 in the area 200, and the area 200 is on the lower side relative to the steps 202. Figure 3is a diagram showing an example of a scene in which a moving object 1 is moving in a lane. Figure 3 In (and subsequent figures), for the sake of explanation, the bird's-eye view is shown as viewed from above, but each processing can also be performed on the image plane.

[0061] The recognition unit 130 recognizes objects existing around the mobile body 1 based on the output of the external detection device 10. The so-called object includes a part or all of the mobile bodies such as vehicles, bicycles, pedestrians, road boundaries such as road dividing lines, steps, guardrails, shoulders, central median strips, road signs, billboards and other structures set on the road, and obstacles such as fallen objects existing (falling) on ​​the road. The recognition unit 130 obtains information such as the existence, position, and category of other mobile bodies by, for example, inputting an image captured by an external camera to a learned model. The learned model is learned by outputting information such as the existence, position, and category of an object when an image captured by an external camera of the external detection device 10 is input. The category of other mobile bodies can also be estimated based on the size in the image, the intensity of the reflected wave received by the radar device of the external detection device 10, etc. The recognition unit 130 obtains the speed of other mobile bodies detected by the radar device using Doppler shift, etc.

[0062] The recognition unit 130 recognizes traffic lights including vehicle traffic lights and pedestrian traffic lights. The recognition unit 130 recognizes traffic information of the traffic lights. Traffic information of the traffic lights refers to information indicating that travel (movement) in a specified direction is allowed, information indicating a stop instruction, etc. The traffic light recognition unit 150 recognizes the traffic information by recognizing the color of the lights of the traffic lights and the direction of the arrows.

[0063] The sound recognition unit 140 obtains the sound received by the microphone 20 and recognizes the obtained sound. The sound recognition unit 140, for example, recognizes whether the received sound is the same as or similar to a predetermined sound stored in the storage device 70. The predetermined sound refers to, for example, the sound output by a pedestrian signal. The sound output by a pedestrian signal refers to, for example, the sound output when the pedestrian signal indicates to go forward. The sound is output in a manner that enables a visually impaired person to recognize the traffic information of the pedestrian signal. The sound recognition unit 140 may also use audio-visual localization technology to recognize the direction and position of the sound being output.

[0064] The signal recognition unit 150 uses the auxiliary information described later to recognize the traffic information of the pedestrian signal. The signal recognition unit 150 uses the auxiliary information to recognize (or infer) the traffic information of the pedestrian signal when the recognition unit 130 cannot recognize the traffic information of the pedestrian signal or when the confidence of the recognition result of the traffic information of the pedestrian signal is less than a threshold value. The confidence less than the threshold value refers to, for example, a state in which the traffic information of the pedestrian signal cannot be recognized or a state in which the traffic information of the pedestrian signal cannot be fully recognized. For example, when the camera of the external detection device 10 cannot clearly capture the pedestrian signal or cannot obtain the traffic information of the signal from the image (or when the accuracy of the image processing is low), the signal recognition unit 150 cannot recognize the traffic information of the pedestrian signal based on the image. In this case, it is determined that the confidence is less than the threshold value.

[0065] The control unit 160 controls the drive device 40 according to the set driving mode, for example. The mobile body 1 may execute only a part of the driving modes described below, but the control unit 160 makes the speed limit value different when the mobile body 1 moves in a lane and when the mobile body 1 moves on a sidewalk. In this case, the mode switching switch 22 can be omitted.

[0066] In mode A-1, the control unit 160 refers to the information on the driving path and the object obtained based on the output of the recognition unit 130, and controls the motor MT of the drive device 40 in the following manner when the moving body 1 is moving in the lane, that is, the distance to the object existing in front of the moving body 1 is maintained at a certain level or above, and the moving body 1 is moved at a first speed V1 (for example, a speed of more than ten [km / h] and less than several tens [km / h]) when the distance to the object existing in front of the moving body 1 is sufficiently long.

[0067] When the moving body 1 moves on the sidewalk, the control unit 160 controls the motor MT of the driving device 40 in the following manner, that is, the distance from the object in front of the moving body 1 is maintained at a certain distance or more, and the moving body 1 is moved at a second speed V2 (for example, a speed less than ten [km / h]) when the distance from the object in front of the moving body 1 is sufficiently long. The function involved is the same as the ACC (Adaptive Cruise Control) function of the vehicle that sets the first speed V1 or the second speed V2 to a set speed, and the technology used in ACC can be used. In mode A-1, the control unit 160 controls the steering device SD in a manner that changes the steering angle of the steering wheel based on the operation amount of the operating member 14 such as the steering wheel. The function involved is the same as the function of the power steering device, and the technology used in the power steering device can be used. It should be noted that it is also possible that the steering is not electronically controlled, and the moving body 1 has a steering device obtained by mechanically connecting the operating member 14 to the steering mechanism.

[0068] In mode A-2, the control unit 160 refers to the information of the travel path and the object obtained based on the output of the recognition unit 130, generates a target trajectory that can move while avoiding the object in the travel path, and controls the steering device SD of the drive device 40 so that the moving body 1 moves along the target trajectory. Regarding acceleration and deceleration, the control unit 160 controls the motor MT of the drive device 40 based on the speed of the moving body 1 and the operation amount of the accelerator pedal or the brake pedal. The control unit 160 controls the motor MT of the drive device 40 by setting the first speed V1 as the upper limit speed when the moving body 1 is moving in the lane (that is, in the case of mode A-2, the moving body 1 is not accelerated even if there is a further acceleration instruction when the upper limit speed is reached), and controls the drive device 40 by setting the second speed V2 as the upper limit speed when the moving body 1 is moving on the sidewalk.

[0069] In mode B, the control unit 160 controls the motor MT of the drive device 40 based on the speed of the moving body 1 and the operation amount of the accelerator pedal or the brake pedal. When the moving body 1 is moving in a lane, the control unit 160 sets the first speed V1 as the upper limit speed and controls the motor MT of the drive device 40 (that is, in the case of mode B, when the upper limit speed is reached, even if there is a further acceleration instruction, the moving body 1 is not accelerated). When the moving body 1 is moving on a sidewalk, the second speed V2 is set as the upper limit speed and controls the motor MT of the drive device 40. Regarding steering, it is the same as mode A-1. Hereinafter, the mode when traveling on a lane is sometimes referred to as mode B1 (second moving mode), and the mode when traveling on a sidewalk is sometimes referred to as mode B2 (first moving mode).

[0070] In mode C, the control unit 160 refers to the information of the travel path and the object obtained based on the output of the recognition unit 130, generates a target trajectory that can move while avoiding the object in the travel path, and controls the drive device 40 so that the moving body 1 moves along the target trajectory. In mode C as well, the control unit 160 controls the drive device 40 by setting the first speed V1 as the upper limit speed when the moving body 1 is moving in the lane, and controls the drive device 40 by setting the second speed V2 as the upper limit speed when the moving body 1 is moving on the sidewalk.

[0071] [How to deal with crossing the road]

[0072] Next, the processing when the moving body 1 crosses the road in the mode C will be described. Figure 4 : is a diagram showing an example of a scene in which the traffic information of the pedestrian signal cannot be recognized. For example, it is assumed that the moving body 1 stops on the sidewalk in order to cross the road. In this case, it is assumed that there are multiple traffic participants around the moving body 1, and the moving body 1 cannot recognize the traffic information of the pedestrian signal SP, or even if the traffic information of the pedestrian signal SP can be temporarily recognized, the traffic information of the pedestrian signal SP cannot be reliably recognized. In such a case, the moving body 1 determines that the certainty of the traffic information of the recognized pedestrian signal SP is less than a threshold. For example, in a case where the vehicle height of the moving body 1 is about the height of a human shoulder or lower than a human shoulder, sometimes even if the traffic information of the vehicle signal SV installed at a high position can be recognized, the traffic information of the pedestrian signal SP cannot be recognized. Then, the moving body 1 performs the following processing. For example, the following processing is performed when moving on the sidewalk through mode C.

[0073] The signal recognition unit 150 uses auxiliary information to recognize the traffic information of the pedestrian signal based on the output of the detection device that detects the external condition of the moving body 1. The auxiliary information refers to information that is different from the traffic information indicated by the signal of the pedestrian signal that the moving body 1 is focusing on, and is used to recognize the traffic information indicated by the signal of the pedestrian signal.

[0074] The auxiliary information includes, for example, one or more of (1) and (3).

[0075] (1) This is a warning sound for the visually impaired that is output based on traffic information from pedestrian traffic lights.

[0076] (2) Traffic information from another signal that permits a traffic participant to travel in a direction different from the direction permitted by the pedestrian signal that the moving body 1 is focusing on.

[0077] (3) is information indicating the behavior of people around the pedestrian signal or the moving object 1 being observed.

[0078] (1) is obtained by the voice recognition unit 140. (2) and (3) are obtained by the recognition unit 130. The signal recognition unit 150 recognizes the traffic information of the pedestrian signal of interest based on one or more of the auxiliary information among the above information. For example, the signal recognition unit 150 recognizes that the pedestrian signal of interest permits travel, or recognizes that the pedestrian signal of interest is giving a stop instruction.

[0079] When the moving body 1 is located in a specified area, the control unit 160 controls the movement of the moving body 1 based on the traffic information of the pedestrian signal recognized by the signal recognition unit 150. The specified area refers to, for example, a sidewalk. Controlling the moving body 1 refers to starting the movement of the moving body, or maintaining a state in which the movement of the moving body is stopped in advance. Starting the movement refers to, for example, moving the moving body 1 on the sidewalk. For example, making the moving body 1 move forward and moving the moving body 1 to the front of the road. Controlling the moving body 1 may also refer to, for example, performing control for quickly starting the movement of the moving body 1. Control for quickly starting the movement of the moving body 1 refers to setting the moving body 1 to a preparation state for starting the movement. The preparation state refers to, for example, supplying power to the motor MT, increasing the output of the motor MT, reducing the degree of braking of the braking device, releasing the parking brake, etc.

[0080] Figure 5 This is a diagram for explaining the processing of the signal recognition unit 150 using the auxiliary information. The signal recognition unit 150 determines whether the auxiliary information (1) to (3) above satisfies the conditions corresponding to each auxiliary information, and recognizes the traffic information of the pedestrian signal based on the determination result. The following is a specific description.

[0081] Figure 6 1 is a diagram for explaining processing using a report sound for the visually impaired or other signal. When the signal recognition unit 150 cannot recognize the traffic information of the pedestrian signal SP and recognizes the sound output by the pedestrian signal SP, it recognizes that the traffic information of the pedestrian signal SP has changed from a state indicating a stop to a state indicating a go. For example, the signal recognition unit 150 may also refer to the position and direction of the sound recognized by the sound recognition unit 140 to determine whether the pedestrian signal SP is outputting a sound. The signal recognition unit 150 may also recognize that the pedestrian signal indicates a go when the sound output by the pedestrian signal SP when the pedestrian signal SP indicates a go is output and is pre-stored in the storage device 70.

[0082] The signal recognition unit 150 may recognize the traffic information of the pedestrian signal SP based on the state of the vehicle signal SV (a signal that allows traffic participants to travel in a direction different from the direction in which the pedestrian signal that the moving body 1 is focusing on allows traffic participants to travel). When the recognition unit 130, for example, cannot recognize the traffic information of the pedestrian signal SP but recognizes the traffic information of the vehicle signal SV, the signal recognition unit 150 recognizes the traffic information of the pedestrian signal SP based on the traffic information of the vehicle signal SV. For example, the signal recognition unit 150 may recognize that there is a high possibility that the pedestrian signal SP indicates to go forward when the vehicle signal SV indicates to stop.

[0083] Figure 7 : is a diagram for explaining the processing using the action of a person. When the traffic information of the pedestrian signal SP cannot be recognized, the signal recognition unit 150 recognizes that the pedestrian signal SP has changed from a state indicating a stop to a state indicating a move forward based on the actions of traffic participants in the vicinity of the moving body 1 or the actions of traffic participants estimated to move based on the information shown by the pedestrian signal SP. The traffic participants in the vicinity of the moving body 1 refer to traffic participants H (for example, people) who are within a specified distance from the moving body 1 and exist in front of the moving body 1. The traffic participants estimated to move based on the information shown by the pedestrian signal SP refer to, for example, traffic participants located in the vicinity of the pedestrian signal SP. The traffic participants located in the vicinity of the pedestrian signal SP refer to, for example, traffic participants H located on the sidewalk opposite to the sidewalk on which the pedestrian signal SP is installed across the road (traffic participants located on the sidewalk where the moving body 1 is located). For example, when the traffic participant H starts to move in the direction of the pedestrian signal SP, the signal recognition unit 150 recognizes that the pedestrian signal SP changes from a state indicating a stop to a state indicating a go. The traffic participants located around the pedestrian signal SP may be, for example, traffic participants located on the sidewalk where the pedestrian signal SP is installed (traffic participants not shown in the figure located on the sidewalk opposite to the sidewalk where the moving body 1 is located across the road) instead of (or in addition to) the above.

[0084] When a predetermined number of conditions among (condition 1) to (condition 3) as described above are satisfied, the signal recognition unit 150 recognizes that the pedestrian signal SP indicates a state of going forward.

[0085] (Condition 1) Report tone is output

[0086] (Condition 2) Other traffic lights have changed to the stop state

[0087] (Condition 3) The surrounding traffic participants start moving in accordance with the traffic information displayed by the pedestrian using the traffic signal.

[0088] The signal recognition unit 150 may also obtain a score as described below, and when the obtained score is above a threshold value, recognize that the pedestrian signal SP indicates to go ahead. For example, when the above-mentioned conditions are met, the signal recognition unit 150 assigns a score corresponding to the satisfied condition and weights the assigned score. The weight is a weight set for each condition. Alternatively, the above-mentioned score is added to the above-mentioned confidence, and when the confidence including the obtained score is above a threshold value, recognize that the pedestrian signal indicates to go ahead.

[0089] As described above, when the signal recognition unit 150 recognizes that the pedestrian signal SP indicates the forward movement, the control unit 160 starts preparing to cross the road according to the pedestrian signal SP or starts the action for crossing. For example, the control unit 160 makes the moving body 1 approach the crosswalk or moves the moving body 1 to the front of the crosswalk. For example, after starting the action, when the recognition unit 130 can actually confirm that the pedestrian signal SP indicates the forward movement, the control unit 160 makes the moving body 1 cross the road. The moving body 1 moves on the sidewalk and the crosswalk with the second speed V2 as the upper limit speed.

[0090] In this way, the control unit 160 controls the moving body 1 using the recognition result of the signal recognition unit 150 , thereby enabling control of the moving body 1 according to the surrounding situation even when the traffic information of the pedestrian signal SP cannot be recognized.

[0091] For example, when there are traffic participants in front of the moving body 1 or when the pedestrian signal SP cannot be recognized to indicate the forward movement, even if the traffic participants in front start to move, the moving body 1 cannot move from the spot, and the behavior of the traffic participants behind may be interfered. In contrast, in the present embodiment, the moving body 1 starts control according to the recognition result of the signal recognition unit 150 as described above, so that the moving body 1 moves quickly or starts to move to a position where the pedestrian signal SP is easy to recognize while ensuring safety. As a result, the moving body 1 can expect to recognize the traffic information of the pedestrian signal SP while cooperating with the surrounding behavior, and can cross the road while cooperating with the surrounding conditions.

[0092] [flow chart]

[0093] Figure 81 is a flowchart showing an example of the process flow executed by the control device 100. First, the recognition unit 130 determines whether the traffic information of the pedestrian signal of interest can be recognized (step S100). If the traffic information of the pedestrian signal of interest cannot be recognized, the signal recognition unit 150 recognizes the sound output by the pedestrian signal, the traffic information of other signals, or the actions of traffic participants (steps S102, S104, S106).

[0094] Next, the signal recognition unit 150 derives a score based on the recognition results of steps S102 to S106 and determines whether the score is equal to or greater than a threshold value (step S108 ). If the score is less than the threshold value, the processing of one routine of this flowchart ends.

[0095] When the score is above the threshold, the signal recognition unit 150 recognizes that the pedestrian signal of interest indicates to go ahead (step S110). Next, the control unit 160 controls the moving body 1 based on the recognition result of the signal recognition unit 150 (step S112). For example, the control device 100 makes the moving body 1 approach the crosswalk while recognizing the traffic information of the pedestrian signal, and makes the moving body 1 cross the crosswalk when the confidence of the traffic information of the pedestrian signal is above the threshold. Thus, the processing of routine 1 of this flowchart ends.

[0096] As described above, the control device 100 recognizes the traffic information of the pedestrian signal using the auxiliary information and controls the moving body 1 according to the recognition result, thereby realizing the control of the moving body according to the surrounding situation.

[0097] [Handling at intersections]

[0098] In the following description, an example in which the moving object 1 crosses a crosswalk at an intersection will be described. Fig. 9 This is a diagram showing an example of a scene of crossing a crosswalk at an intersection. For example, assume that four crosswalks C1-C4 are provided at the intersection, and pedestrian signals are provided for each crosswalk. SP1A and SP1B are provided for crosswalk C1, and SP4A and SP4B are provided for crosswalk C4. At the intersection, vehicle signals SV1 and SV3 are provided to restrict traffic in a first direction, and vehicle signals SV2 and SV4 are provided to restrict traffic in a second direction intersecting the first direction. For example, assume that a moving body 1 stops near pedestrian signal SP4A in order to cross crosswalk C4. Moving body 1 crosses crosswalk C4 according to traffic information from pedestrian signals SP4A and SP4B (pedestrian signals corresponding to crosswalk C4), but traffic information from pedestrian signals SP4A and SP4B cannot be recognized due to other traffic participants.

[0099] In such a case, the signal recognition unit 150 recognizes the traffic information of the pedestrian signal SPB of interest based on the sounds output by the pedestrian signals SP4A and SP4B (or one of the sounds output by the pedestrian signal SP4A and the sounds output by SP4B), the traffic information of the vehicle signals SV1-SV4, or the status of traffic participants who are about to cross the crosswalk C4.

[0100] As described above, the control device 100 also recognizes the traffic information of the pedestrian signal using the auxiliary information at the intersection where a plurality of pedestrian signals are present, and controls the moving body 1 according to the recognition result, thereby realizing control of the moving body according to the surrounding conditions.

[0101] [Timing of changes in traffic information at a signal and recognition of changed signals]

[0102] The signal recognition unit 150 recognizes that a pedestrian signal in the vicinity of the moving body 1 has switched from a first state indicating a stop to a second state permitting travel based on one or more of the information in (1), (2), and (3) above, and recognizes whether the pedestrian signal in question has switched to the second state or a pedestrian signal different from the pedestrian signal in question has switched to the second state based on information in (1), (2), and (3) above that is different from the information used when recognizing the switch to the second state. The signal recognition unit 150, for example Fig.10 As shown, the timing when the traffic information of the pedestrian signal is switched is recognized by the reporting sound, traffic information of other signals, or human behavior, and the switched pedestrian signal is recognized using an element different from the element used for judging the timing. Different elements mean that some elements are different, and elements may be repeated.

[0103] For example, in the aforementioned Fig. 9 In the case where the signal recognition unit 150 recognizes the sound, it recognizes that the traffic information of any signal at the intersection has changed, and when the traffic participant starts to move, it recognizes that the traffic information of the pedestrian signal corresponding to the direction in which the traffic participant moves has changed. Fig. 9In the case where the traffic participant moves in accordance with the traffic information of the pedestrian signal SP4B, the signal recognition unit 150 recognizes that the traffic information of the pedestrian signal SP4B has changed. When other traffic participants move in accordance with the traffic information of the pedestrian signals SP3A and SP3B, the signal recognition unit 150 recognizes that the traffic information of the pedestrian signal SP4B has not changed. For example, when the vehicle signal SV3 changes to a state indicating a forward movement, or the vehicle signal SV4 changes to a state indicating a stop, the signal recognition unit 150 recognizes that the traffic information of the pedestrian signal SP4B has changed.

[0104] For example, when the signal recognition unit 150 recognizes that the traffic information of the pedestrian signal has changed, the moving body 1 performs a first action. The first action refers to preparation for the moving body 1 to move. Preparation refers to a mode in which the moving body 1 is shifted from a mode in a stopped state to a mode in a prepared state. For example, it is a mode for operating the motor MT, a mode for reducing the degree of braking of the brake device, and a mode for releasing the parking brake (or preparing for releasing).

[0105] For example, when the signal recognition unit 150 recognizes that the traffic information of the pedestrian signal (the pedestrian signal corresponding to the pedestrian crossing to be crossed) has changed, the moving body 1 performs the second action. The second action refers to, for example, an action to move the moving body 1 or an action to transfer from a mode of a ready state to a mode of a stopped state. For example, when the signal whose traffic information has changed is a pedestrian signal SP4B, the control unit 160 moves the moving body 1 in a manner to cross the pedestrian crossing C4. For example, the control device 100 makes the moving body 1 approach the pedestrian crossing while following the movement of traffic participants.

[0106] For example, when the signal whose traffic information has changed is the pedestrian signal SP3A, the control unit 160 changes the mode from the preparation state mode to the stop state mode and moves the moving body 1 to a position that does not hinder the traffic of traffic participants crossing the crosswalk C3.

[0107] [flow chart]

[0108] Fig.11 1 is a flowchart showing an example of the process flow executed by the control device 100. First, the recognition unit 130 determines whether the traffic information of the pedestrian signal of interest can be recognized (step S200). If the traffic information of the pedestrian signal of interest cannot be recognized, the signal recognition unit 150 determines whether the condition related to the timing of the change of the traffic information of the pedestrian signal is satisfied (step S202).

[0109] When the condition related to the timing of the change of the traffic information of the pedestrian signal is met, the control unit 160 recognizes that the traffic information of the pedestrian signal has changed and performs the first action (step S204). Next, the signal recognition unit 150 determines whether the condition for determining the pedestrian signal is met (step S206). When the condition for determining the pedestrian signal is met, the control unit 160 starts the second action based on the recognition result of the signal recognition unit 150 (step S208). Thus, the processing of routine 1 of this flowchart ends.

[0110] Through the above-described processing, the control device 100 can smoothly transition to an operation for moving the moving body 1 through the first operation, and can realize control of the moving body according to the surrounding situation through the second operation.

[0111] <Modification>

[0112] The signal recognition unit 150 may also recognize the traffic information of the signal based on the signal information 76 stored in the storage device 70. The signal information 76 is information that establishes a correspondence between information related to the pedestrian signal at the intersection and information about the sound output by the pedestrian signal. The information related to the pedestrian signal refers to, for example, the position of the pedestrian signal, information about the movement direction restricted by the pedestrian signal, and the movement direction allowed when the sound is output. The sound output by the pedestrian signal refers to, for example, the sound output by the pedestrian signal when indicating to go ahead.

[0113] Fig.12 1 is a diagram showing an example of the signal information 76. For example, the pedestrian signal 1 at the intersection 1 outputs sound 1, and the pedestrian signal 2 does not output sound 2. For example, the pedestrian signal SW3 at the intersection 3 outputs sound 5, and the pedestrian signal SW4 outputs sound 6. The signal recognition unit 150 refers to the signal information 76 to determine the pedestrian signal corresponding to the sound output by the pedestrian signal recognized by the sound recognition unit 140, and the movement direction allowed by the pedestrian signal.

[0114] Fig.13 1 is a diagram showing an example of a scene in which the signal recognition unit 150 controls the moving object 1 using the signal information 76. Fig. 9Same description. At the intersection, if the pedestrian signal SW4B indicates to go ahead, the moving body 1 is scheduled to cross the pedestrian crossing C4 corresponding to the pedestrian signal SW4. At this time, it is assumed that the pedestrian signal SW3A changes to a state indicating to go ahead, and the pedestrian signal SW3A outputs a sound 5. The signal recognition unit 150 refers to the signal information 76 to determine the position of the pedestrian signal corresponding to the sound 5 recognized by the sound recognition unit 140, and the movement direction allowed by the pedestrian signal. The signal recognition unit 150 recognizes that the pedestrian signal SW3A changes to a state where the pedestrian crossing C3 can be crossed. The signal recognition unit 150 recognizes that the traffic information of the pedestrian signal SW4B corresponding to the pedestrian crossing C4 that the moving body 1 is scheduled to cross has not changed.

[0115] For example, after the sound 5 stops, when the signal recognition unit 150 recognizes the sound 6 corresponding to the pedestrian signal SW4A, it recognizes that the pedestrian signal SW4B has changed to a state where the pedestrian can cross the crosswalk C4. It should be noted that instead of recognizing, the certainty of recognizing that the pedestrian signal SW4B has changed to a state where the pedestrian can cross the crosswalk C4 can be increased, and the above-mentioned score can also be increased.

[0116] As described above, the signal recognition unit 150 can recognize the traffic information of the pedestrian signal with higher accuracy by using the signal information 76 .

[0117] In the above example, the case where the moving body 1 is controlled based on the recognition result of the traffic information of the pedestrian signal in the case of mode C is described, but the process of controlling the moving body 1 based on the recognition result of the traffic information of the pedestrian signal may be applied in other modes instead of (or in addition to) this. For example, the change of the traffic information of the pedestrian signal may be notified to the passengers, and the operation or acceleration and deceleration may be controlled based on the traffic information of the pedestrian signal.

[0118] The control device 100 may also identify the traffic information of the signal based on experience information or provided information in addition to (or instead of) the auxiliary information. Experience information refers to information generated by the mobile body 1 or other mobile body 1 based on information related to the signal obtained when passing through an area where a signal is installed. Provided information is information related to the signal provided by a specified organization or server device. Information related to the signal refers to, for example, information on the timing when a signal among multiple signals indicates going forward or stopping, information on the sound output by a signal among multiple signals, etc. For example, the experience information or provided information includes information including a schedule for switching the traffic information of the signal such as the second signal indicating going forward after a specified time after the first signal stops.

[0119] As described above, the control device 100 controls the moving object 1 using the auxiliary information, thereby enabling control of the moving object in accordance with the surrounding situation.

[0120] The above-described embodiment can be expressed as follows.

[0121] A control device for a moving object, wherein:

[0122] The control device of the mobile body comprises:

[0123] a storage medium storing computer-readable instructions; and

[0124] a processor connected to the storage medium,

[0125] The processor executes the computer-readable instructions to perform the following processing:

[0126] Controlling a moving object capable of moving both in a lane and in a predetermined area different from the lane;

[0127] Based on the output of the detection device that detects the external condition of the moving body, identifying whether the moving body is moving in the lane or in the specified area;

[0128] Acquiring auxiliary information, the auxiliary information being information different from traffic information indicating permission to proceed and stop instructions indicated by the signal of the pedestrian signal that the moving body is focusing on, and used to identify permission to proceed or stop instructions indicated by the signal of the pedestrian signal;

[0129] using the acquired auxiliary information to identify traffic information indicating permission to proceed or an instruction to stop indicated by the signal of the pedestrian signal; and

[0130] When the moving object is located in the predetermined area, the moving object is controlled based on the recognized traffic information of the pedestrian signal.

[0131] Although specific embodiments of the present invention have been described above using the embodiments, the present invention is not limited to such embodiments at all, and various modifications and substitutions can be made without departing from the gist of the present invention.

[0132] Description of reference numerals:

[0133] 10 External detection device

[0134] 12 Moving body sensor

[0135] 14 Operating parts

[0136] 16 Internal Camera

[0137] 18 Positioning device

[0138] 20 Microphone

[0139] 22 Mode switch

[0140] 30 Mobile mechanism

[0141] 40 Drive unit

[0142] 50 External reporting device

[0143] 70 Storage Device

[0144] 100 Control device

[0145] 120 Road type identification unit

[0146] 130 Identification Department

[0147] 140 Voice Recognition Unit

[0148] 150 Signal Identification Department

[0149] 160 Control Department.

Claims

1. A control device for a mobile object, the mobile object being capable of moving in both a lane and a predetermined area different from the lane, wherein: The control device of the mobile body comprises: a road type recognition unit for recognizing whether the moving body is moving in a lane or in the prescribed area based on an output of a detection device for detecting an external condition of the moving body; an acquisition unit that acquires auxiliary information, the auxiliary information being information different from traffic information indicating permission to proceed and stop instructions indicated by the signal of the pedestrian signal that the moving body is focusing on, and used to identify permission to proceed or stop instructions indicated by the signal of the pedestrian signal; a signal recognition unit that uses the auxiliary information acquired by the acquisition unit to recognize traffic information as a permission to proceed or a stop instruction indicated by the signal of the pedestrian signal; as well as A control unit controls the moving body based on the traffic information of the pedestrian signal recognized by the signal recognition unit when the moving body is located in the predetermined area.

2. The control device for a moving object according to claim 1, wherein: The auxiliary information is a notification sound for the visually impaired output based on the traffic information of the pedestrian signal.

3. The control device for a moving object according to claim 1, wherein: The supplementary information is the traffic information of a signal that permits a traffic participant to travel in a direction different from a direction in which the targeted pedestrian signal permits the traffic participant to travel.

4. The control device for a moving object according to claim 1, wherein: The auxiliary information is information indicating the behavior of a person estimated to be moving based on the traffic information indicated by the pedestrian signal of interest or information indicating the behavior of a person around the moving object.

5. The control device for a moving object according to claim 1, wherein: The auxiliary information is a report sound for the visually impaired outputted based on the traffic information displayed by the pedestrian signal, and information indicating the behavior of a person estimated to be moving based on the traffic information displayed by the pedestrian signal or information indicating the behavior of people around the moving body, The signal recognition unit recognizes whether the pedestrian signal of interest permits travel by using the notification sound for the visually impaired and information indicating the person's behavior. The control unit controls the moving object based on a result of recognition by the signal recognition unit.

6. The control device for a moving object according to claim 1, wherein: The auxiliary information is a notification sound for the visually impaired outputted based on the traffic information of the pedestrian signal, and the traffic information of the signal that permits travel in a direction different from the direction permitted by the pedestrian signal of interest, The signal recognition unit recognizes whether the pedestrian signal of interest permits travel by using the notification sound for the visually impaired and the traffic information of the signal permitting travel in the different directions, The control unit controls the moving object based on a result of recognition by the signal recognition unit.

7. The control device for a moving object according to claim 1, wherein: The auxiliary information is information indicating the movement of a person estimated to be moving based on the traffic information indicated by the pedestrian signal of interest or information indicating the movement of a person around the moving body, and the traffic information of a signal that permits movement in a direction different from the direction permitted by the pedestrian signal of interest, The signal recognition unit recognizes whether the pedestrian signal of interest permits travel by using the information indicating the person's behavior and the traffic information of the signal permitting travel in the different directions. The control unit controls the moving object based on a result of recognition by the signal recognition unit.

8. The control device for a moving object according to claim 1, wherein: The auxiliary information is (1) a notification sound for the visually impaired outputted based on the traffic information of the pedestrian signal, (2) information indicating the movement of a person estimated to be moving based on the traffic information indicated by the pedestrian signal or information indicating the movement of a person around the moving body, or (3) the traffic information of a signal that permits movement in a direction different from the direction permitted by the pedestrian signal. The signal recognition unit recognizes that a pedestrian signal in the vicinity of the moving body has switched from a first state indicating a stop to a second state permitting travel based on one or more information among (1), (2) and (3), and recognizes whether it is the pedestrian signal of interest that has changed to the second state or a pedestrian signal different from the pedestrian signal of interest that has changed to the second state based on information among (1), (2) and (3) that is different from the information used when recognizing the switching to the second state.

9. The control device for a moving object according to claim 1, wherein: The auxiliary information is a notification sound for the visually impaired outputted based on the traffic information of the pedestrian signal. The signal recognition unit refers to information stored in a storage device that associates the type of the report sound with the direction in which the pedestrian signal allows traffic participants to travel, and recognizes the traffic information of the pedestrian signal based on the report sound.

10. The control device for a moving object according to claim 1, wherein: The control unit limits the speed of the moving object to a first speed when the moving object moves in the lane, and limits the speed of the moving object to a second speed lower than the first speed when the moving object moves in the predetermined area. When the vehicle is scheduled to move according to the traffic information of the pedestrian signal of interest and the signal recognition unit recognizes that the pedestrian signal of interest permits travel, the control unit starts an operation for moving the moving body in a direction permitted by the pedestrian signal.

11. The control device for a moving object according to claim 1, wherein: The signal recognition unit recognizes the state of the first pedestrian signal based on one or more of the following three pieces of information as the auxiliary information, when the moving body moves according to the traffic information of the first pedestrian signal at an intersection including a first pedestrian signal that restricts traffic in a first direction and a second pedestrian signal that restricts traffic in a second direction different from the first direction, the three pieces of information being: a report sound for visually impaired persons output based on the traffic information of the pedestrian signal; the traffic information of a signal different from the first pedestrian signal; and information indicating the action of a person presumed to be moving based on the traffic information indicated by the first pedestrian signal or information indicating the action of people around the moving body, The control unit moves the moving body according to the state of the first pedestrian signal when the signal recognition unit recognizes that the first pedestrian signal has changed to a state in which travel is permitted.

12. The control device for a moving object according to claim 1, wherein: The acquisition unit acquires the traffic information of the pedestrian signal of interest based on the output of the detection device, The signal recognition unit acquires the traffic information of the pedestrian signal using the auxiliary information when the acquisition unit cannot acquire the traffic information.

13. The control device for a moving object according to claim 1, wherein: The control unit controls the moving body based on the recognition result of the signal recognition unit, and then, when the acquisition unit acquires the traffic information of the pedestrian signal of interest and recognizes that the pedestrian signal of interest permits travel, the control unit controls the moving body from the sidewalk where the moving body is located to the road or crosswalk where the traffic information permits travel and crosses the road or the crosswalk.

14. A method for controlling a moving object, the moving object being capable of moving both in a lane and in a predetermined area different from the lane, wherein: The control device of the mobile body performs the following processing: Based on the output of the detection device that detects the external condition of the moving body, identifying whether the moving body is moving in the lane or in the specified area; Acquiring auxiliary information, the auxiliary information being information different from traffic information indicating permission to proceed and stop instructions indicated by the signal of the pedestrian signal that the moving body is focusing on, and used to identify permission to proceed or stop instructions indicated by the signal of the pedestrian signal; Using the acquired auxiliary information, recognizing permission to proceed or a stop instruction indicated by the signal of the pedestrian signal; and When the moving object is located in the predetermined area, the moving object is controlled based on the recognized state of the pedestrian signal.

15. A storage medium storing a program, wherein: The moving object can move in both the lane and the predetermined area different from the lane. The program causes the control device of the mobile body to perform the following processing: Based on the output of the detection device that detects the external condition of the moving body, identifying whether the moving body is moving in the lane or in the specified area; Acquiring auxiliary information, the auxiliary information being information different from traffic information indicating permission to proceed and stop instructions indicated by the signal of the pedestrian signal that the moving body is focusing on, and used to identify permission to proceed or stop instructions indicated by the signal of the pedestrian signal; Using the acquired auxiliary information, recognizing permission to proceed or a stop instruction indicated by the signal of the pedestrian signal; and When the moving object is located in the predetermined area, the moving object is controlled based on the recognized state of the pedestrian signal.

Citation Information

Patent Citations

  • Position calculation system

    JP2020197387A