MOBILE BODY CONTROL DEVICE, MOBILE BODY CONTROL METHOD, AND STORAGE MEDIUM

Abstract

This apparatus, which is for controlling a mobile body capable of moving both in a roadway and in a prescribed region different from the roadway, comprises: a road type recognition unit which, on the basis of the output from an external detection device for detecting the external condition of the mobile body, recognizes whether the mobile body is moving in a roadway or in the prescribed region; and a control unit which limits the speed of the mobile body, in a case of moving in a roadway, to a first speed, and which limits the speed of the mobile body, in a case of moving in the prescribed region, to a second speed slower than the first speed. The road type recognition unit repeatedly recognizes whether the mobile body is moving in a roadway or in the prescribed region, on the basis of recognition results obtained during a first period in the past and the output from the external detection device, and then stores the recognition results in a storage apparatus. When at least some of the recognition results obtained during the past first period are lost from the storage apparatus, recognition results indicating that the mobile body is moving on a pavement are outputted to the control unit during a second period.

Description

[Technical Field]

[0001] The present invention relates to a control device for a mobile object, a control method for a mobile object, and a storage medium. [Background technology]

[0002] Conventionally, practical application of mobile objects that can move on both sidewalks and roadways has been promoted, and for such mobile objects, it is necessary to set different upper speed limits for the sidewalk and roadway. In relation to this, an invention has been disclosed that includes a device that includes left and right imaging units that capture images of the road surface, a block detection unit that detects the direction in which an electric vehicle can move from image data captured by the left and right imaging units, a distance calculation unit that generates instruction information to control the traveling of the electric vehicle based on the direction detected by the block detection unit, and a vehicle control that controls the traveling of the electric vehicle based on the instruction information generated by the distance calculation unit (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-197328 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional technologies, it has sometimes been impossible to properly recognize, taking robust stability into account, whether a moving object is moving on a roadway or in a predetermined area that is different from the roadway.

[0005] The present invention has been made in consideration of these circumstances, and one of its objects is to provide a control device for a mobile body, a control method for a mobile body, and a storage medium that are capable of appropriately recognizing whether a mobile body is moving on a roadway or in a specified area different from the roadway, taking into account robust stability. [Means for solving the problem]

[0006] The control device for a moving body, the control method for a moving body, and the storage medium according to the present invention employ the following configurations. (1): A control device for a mobile body according to one embodiment of the present invention is a control device for a mobile body that can carry one or more passengers and move both on a roadway and in a predetermined area different from the roadway, and includes: a road type recognition unit that recognizes whether the mobile body is moving on a roadway or in the predetermined area based on the output of an external environment detection device that detects the external conditions of the mobile body; and a control unit that limits the speed of the mobile body when moving on the roadway to a first speed and limits the speed of the mobile body when moving in the predetermined area to a second speed lower than the first speed. The road type recognition unit repeatedly recognizes whether the mobile body is moving on a roadway or in the predetermined area based on the recognition results of a past first period and the output of the external environment detection device, and stores the recognition results in a memory device. If at least a portion of the recognition results of the past first period in the memory device are missing, the road type recognition unit outputs a recognition result to the control unit that the mobile body is moving on a sidewalk for a second period.

[0007] (2): In the above aspect (1), when at least a portion of the recognition results for the past first period in the storage device is missing, the road type recognition unit causes the control unit to perform a predetermined operation to cause the external environment detection device to detect the characteristics of either the roadway or the specified area.

[0008] (3): In the above aspect (2), the external world sensing device is capable of changing its sensing direction, and the predetermined operation is an operation of changing the sensing direction of the external world sensing device and causing the external world sensing device to search for the feature.

[0009] (4): In the above aspect (2), the control unit controls the drive device to autonomously move the moving body, and the predetermined operation is an operation of causing the external environment detection device to search for the feature while changing the orientation of the moving body.

[0010] (5): In the above aspect (2), the control unit controls the drive device to autonomously move the moving body, and the predetermined operation is an operation of bringing the moving body close to the feature and causing the external environment detection device to accurately recognize the feature.

[0011] (6): In the above aspect (1), when the road type recognition unit starts up, if the recognition result before shutdown and the output of the external environment detection device are stored in a storage device, and if the current output of the external environment detection device is similar to the output of the external environment detection device before shutdown, the road type recognition unit treats the recognition result before shutdown as the recognition result of the first past period.

[0012] (7): In the above aspect (6), the external environment detection device is an external camera that captures images of the outside of the moving body, and the road type recognition unit performs a process of extracting feature points from the image captured by the external camera, and if the positions of the feature points match to a predetermined degree or more, determines that the current output of the external environment detection device is similar to the output of the external environment detection device before the shutdown.

[0013] (8): In the above aspect (6), the external environment detection device is an external camera that captures images of the outside of the moving body, and the road type recognition unit determines that the captured images are similar when the total value of the differences in pixel values ​​for each pixel of the captured images of the external camera is less than or equal to a predetermined value.

[0014] (9): Another aspect of the present invention provides a method for controlling a moving body, the method comprising: a computer controlling a moving body having one or more occupants on board and capable of moving both on a roadway and in a predetermined area different from the roadway, recognizing whether the moving body is moving on a roadway or in the predetermined area based on the output of an external environment detection device that detects the external conditions of the moving body; limiting the speed of the moving body when moving on the roadway to a first speed; and limiting the speed of the moving body when moving in the predetermined area to a second speed lower than the first speed; the recognition process includes repeatedly recognizing whether the moving body is moving on a roadway or in the predetermined area based on the recognition results of a past first period and the output of the external environment detection device, storing the recognition results in a storage device; and, if at least a portion of the recognition results of the past first period in the storage device are missing, outputting a recognition result that the moving body is recognized as moving for a second period.

[0015] (10): Another aspect of the present invention is a storage medium that stores a program that causes a computer that controls a mobile body that has one or more occupants on board and that can move both on a roadway and in a predetermined area different from the roadway to recognize whether the mobile body is moving on the roadway or in the predetermined area based on the output of an external environment detection device that detects the external conditions of the mobile body, and limits the speed of the mobile body when moving on the roadway to a first speed and the speed of the mobile body when moving in the predetermined area to a second speed lower than the first speed, wherein the recognition process includes repeatedly recognizing whether the mobile body is moving on the roadway or in the predetermined area based on the recognition results of a past first period and the output of the external environment detection device, storing the recognition results in a storage device, and outputting a recognition result that the mobile body is recognizing movement for a second period if at least a portion of the recognition results of the past first period in the storage device are missing. [Effects of the Invention]

[0016] According to the above aspects (1) to (10), it is possible to appropriately recognize whether the moving object is moving on a roadway or in a predetermined area different from the roadway, taking robust stability into consideration. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a moving object and a control device according to an embodiment. [Figure 2] FIG. [Figure 3] FIG. 10 is a diagram showing an example of the contents of past recognition results stored in a storage device. [Figure 4] FIG. 10 is a diagram comparing a case in which a recognition result for a past first period is stored in a storage device with a case in which a recognition result for a past first period is stored in a storage device. [Figure 5] FIG. 10 is a diagram illustrating a first example of a predetermined operation. [Figure 6] FIG. 10 is a diagram illustrating a second example of a predetermined operation. [Figure 7] 10 is a flowchart (part 1) illustrating an example of the flow of processing executed by a road type recognition unit of the embodiment. [Figure 8] 10 is a flowchart (part 2) illustrating an example of the flow of processing executed by the road type recognition unit of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, with reference to the drawings, embodiments of a mobile body control device, a mobile body control method, and a program of the present invention will be described. A mobile body moves both on roadways and in a predetermined area different from the roadway. A mobile body is sometimes referred to as micromobility. An electric kick scooter is a type of micromobility. The predetermined area is, for example, a sidewalk. The predetermined area may also be part or all of a sidewalk, a bicycle lane, a public open space, etc., or may include all of a sidewalk, a sidewalk, a bicycle lane, a public open space, etc. In the following description, the predetermined area is assumed to be a sidewalk. In the following description, the term "sidewalk" can be appropriately read as "predetermined area." A mobile body is not limited to a vehicle, but may also include a small mobility vehicle that runs alongside a walking user to carry luggage or lead a person, and may also include other mobile bodies capable of autonomous movement (e.g., a walking robot).

[0019] 1 is a diagram showing an example of the configuration of a moving object 1 and a control device 100 according to an embodiment. The moving object 1 is equipped with, for example, an external environment detection device 10, a moving object sensor 12, an operator 14, an internal camera 16, a positioning device 18, a mode selector switch 22, a movement mechanism 30, a drive unit 40, an external notification device 50, a storage device 70, and a control device 100. Note that some of these components that are not essential for realizing the functions of the present invention may be omitted.

[0020] The external environment detection devices 10 are various devices whose detection range is in the traveling direction of the moving object 1. The external environment detection devices 10 include an external camera, a radar device, a LIDAR (Light Detection and Ranging), a sensor fusion device, etc. The external environment detection devices 10 output information indicating the detection results (images, object positions, etc.) to the control device 100. Each of the external environment detection devices 10 may be provided with mechanical or electronic means capable of changing the detection direction. For example, the external camera may be provided with an externally operable mechanism that can mechanically change the optical axis direction.

[0021] The mobile body sensor 12 includes, for example, a speed sensor, an acceleration sensor, a yaw rate (angular velocity) sensor, a direction sensor, and an operation amount detection sensor attached to the operator 14. The operator 14 includes, for example, an operator for instructing acceleration / deceleration (e.g., an accelerator pedal or a brake pedal), and an operator for instructing steering (e.g., a steering wheel). In this case, the mobile body sensor 12 may include an accelerator opening sensor, a brake depression amount sensor, a steering torque sensor, etc. The mobile body 1 may also be provided with an operator of a type other than those described above as the operator 14 (e.g., a non-annular rotary operator, a joystick, a button, etc.).

[0022] Internal camera 16 captures an image of at least the head of an occupant of vehicle 1 from the front. Internal camera 16 is a digital camera that uses an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). Internal camera 16 outputs the captured image to control device 100.

[0023] The positioning device 18 is a device that measures the position of the mobile object 1. The positioning device 18 is, for example, a Global Navigation Satellite System (GNSS) receiver, and identifies the position of the mobile object 1 based on signals received from GNSS satellites and outputs the position information. Note that the position information of the mobile object 1 may be estimated from the position of a Wi-Fi base station to which a communication device (described later) is connected.

[0024] The mode selector switch 22 is a switch operated by the occupant. The mode selector switch 22 may be a mechanical switch or a GUI (Graphical User Interface) switch set on a touch panel. The mode selector switch 22 accepts an operation to switch the driving mode between, for example, Mode A: an assist mode in which one of steering operation and acceleration / deceleration control is performed by the occupant and the other is performed automatically; Mode A-1: ​​in which steering operation is performed by the occupant and acceleration / deceleration control is performed automatically; Mode A-2: in which acceleration / deceleration operation is performed by the occupant and steering control is performed automatically; Mode B: a manual driving mode in which steering operation and acceleration / deceleration operation are performed by the occupant; and Mode C: an automatic driving mode in which operation control and acceleration / deceleration control are performed automatically.

[0025] The locomotion mechanism 30 is a mechanism for moving the mobile object 1 on a road. The locomotion mechanism 30 is, for example, a group of wheels including steering wheels and drive wheels. The locomotion mechanism 30 may also be legs for multi-legged walking.

[0026] The drive unit 40 outputs force to the movement mechanism 30 to move the moving body 1. For example, the drive unit 40 includes a motor that drives the drive wheels, a battery that stores power to be supplied to the motor, a steering device that adjusts the steering angle of the steering wheels, etc. The drive unit 40 may also include an internal combustion engine or a fuel cell as a driving force output means or a power generation means. The drive unit 40 may also include a brake device that utilizes frictional force or air resistance.

[0027] The external notification device 50 is, for example, a lamp, a display device, a speaker, or the like, provided on an outer panel of the mobile object 1, and configured to notify information to the outside of the mobile object 1. The external notification device 50 operates differently depending on whether the mobile object 1 is traveling on a sidewalk or a roadway. For example, the external notification device 50 is controlled to emit a lamp when the mobile object 1 is traveling on a sidewalk and not emit a lamp when the mobile object 1 is traveling on a roadway. The light color of this lamp is preferably a color specified by law. The external notification device 50 may be controlled to emit a green lamp when the mobile object 1 is traveling on a sidewalk and emit a blue lamp when the mobile object 1 is traveling on a roadway. If the external notification device 50 is a display device, the external notification device 50 displays a message in text or graphics indicating that the mobile object 1 is traveling on a sidewalk when the mobile object 1 is traveling on a sidewalk.

[0028] FIG. 2 is a perspective view of the moving body 1 as seen from above. In the figure, FW denotes steering wheels, RW denotes drive wheels, SD denotes a steering device, MT denotes a motor, and BT denotes a battery. The steering device SD, motor MT, and battery BT are included in a drive device 40. AP denotes an accelerator pedal, BP denotes a brake pedal, WH denotes a steering wheel, SP denotes a speaker, and MC denotes a microphone. The moving body 1 shown in the figure is a single-seater moving body, and an occupant P is seated in the driver's seat DS and wearing a seat belt SB. Arrow D1 indicates the direction of travel (velocity vector) of the moving body 1. The external environment detection device 10 is located near the front end of the moving body 1, the internal camera 16 is located in a position where it can capture an image of the occupant P's head from in front of the occupant P, and the mode selector switch 22 is located in the boss portion of the steering wheel WH. An external notification device 50 serving as a display device is also located near the front end of the moving body 1.

[0029] Returning to FIG. 1, the storage device 70 is a non-transitory storage device such as a hard disk drive (HDD), flash memory, or random access memory (RAM). The storage device 70 stores map information 72, a program 74 executed by the control device 100, and the like. In the figure, the storage device 70 is illustrated outside the frame of the control device 100, but the storage device 70 may be included in the control device 100. The storage device 70 may also be provided on a server (not shown).

[0030] [Control device] The control device 100 includes, for example, a road type recognition unit 120, an object recognition unit 130, and a control unit 140. These components are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software) 74. Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in the storage device 70 in advance, or may be stored in a removable storage medium (non-transitory storage medium) such as a DVD or CD-ROM, and installed in the storage device 70 by inserting the storage medium into a drive device.

[0031] The road type recognition unit 120 recognizes whether the moving object 1 is moving on a roadway or a sidewalk. The road type recognition unit 120 recognizes whether the moving object 1 is moving on a roadway or a sidewalk, for example, by analyzing an image captured by an external camera of the external environment detection device 10. Note that the output of a radar device, a LIDAR, a sensor fusion device, etc. may be used supplementarily.

[0032] The road type recognition unit 120 calculates, for example, a roadway score Sc indicating the likelihood that the moving object 1 is moving on a roadway. For example, the road type recognition unit 120 adds points to the roadway score Sr each time it recognizes a plurality of first events indicating that the moving object 1 is moving on a roadway, and subtracts points from the roadway score Sr each time it recognizes a plurality of second events indicating that the moving object 1 is moving on a sidewalk. Examples of first events include (A) the absence of static obstacles other than vehicles (e.g., signboards) in the own area (the area where the moving object 1 is located), (B) the presence of a vehicle moving in the own area, (C) the presence of road markings on the road surface of the own area, (D) the presence of a pedestrian crossing in the own area, and (E) the own area being on the lower side of a step. The second event may be, for example, (a) the second of the two guardrail surfaces facing the sidewalk is captured by an external camera, (b) there are tactile paving blocks on the road surface in the vehicle's own area, (c) there is a static obstacle other than a vehicle in the vehicle's own area, or (d) the vehicle's own area is on the upper step of a step.

[0033] Furthermore, the road type recognition unit 120 stores the past recognition results of the first period in the storage device 70, and recognizes whether the moving object 1 is moving on a roadway or a sidewalk based on the past recognition results of the first period and the roadway score Sc based on the output of the external environment detection device described above. The content of this processing is expressed, for example, by equation (1). In the equation, the argument in parentheses represents the control time. Control time k represents the control time of the current cycle. The argument n is a natural number. RES(k) is the recognition result of the current cycle based on the roadway score Sr(k) in the current cycle and past recognition results, RES(k-1) represents the recognition result of the previous cycle, RES(k-2) represents the recognition result of two cycles ago, and RES(kn) represents the recognition result of n cycles ago. α and β1 to βn are each coefficients. For example, among β1 to βn, β1 is set to the largest value, β2 is set to the next largest, and βn is set to the smallest value. α·Sc(k)+β1·RES(k-1)+β2·RES(k-2)+…+βn·RES(kn) is called the integrated score.

[0034] RES(k)=f{α·Sc(k)+β1·RES(k-1)+β2·RES(k-2)+…+βn·RES(kn)} …(1)

[0035] 3 is a diagram showing an example of the contents of past recognition results stored in the storage device 70. For example, information indicating past recognition results is stored in the storage device 70. The information indicating past recognition results includes, for example, not only information on relative control times but also information on absolute times (which may be times with the time when the control device 100 starts up set to zero).

[0036] The road type recognition unit 120 determines that the road is a roadway if, for example, α·Sc(k)+β1·RES(k-1)+β2·RES(k-2)+...+βn·RES(kn) is equal to or greater than a threshold value Th (RES(k) = roadway). For example, a recognition result indicating that the moving object 1 is moving on a roadway is represented by 1, and a recognition result indicating that the moving object 1 is moving on a sidewalk is represented by 0, and these are used for subsequent recognition processes. In this case, the threshold value Th is set to, for example, a value of approximately 0.8 to 0.9. The "first period in the past" does not mean "a period up to a point in time going back a certain number of cycles in the past," but rather "a period up to a point in time going back a specified absolute time based on the current time." In the above example, the "first period in the past" refers to the period from control time k-1 to kn, assuming that the recognition process is repeatedly executed without interruption. However, the "first period in the past" is not limited to this, and may refer to a single time in the past. Furthermore, the method for reflecting past recognition results is not limited to this, and any method such as a Kalman filter may be used.

[0037] In this way, the road type recognition unit 120 can perform recognition processing taking into consideration robust stability by reflecting recognition results from a certain period of time in the past and recognizing whether the moving body 1 is moving on a roadway or a sidewalk.

[0038] The road type recognition unit 120 may also compare the position information of the mobile object 1 with the map information 72 to recognize whether the mobile object 1 is moving on a roadway or a sidewalk. In this case, the map information must be accurate enough to distinguish between a sidewalk and a roadway from the position coordinates. Furthermore, if the "predetermined area" is not limited to sidewalks, the road type recognition unit 120 performs similar processing on shoulders, bicycle lanes, public open spaces, etc.

[0039] The object recognition unit 130 recognizes objects present around the mobile object 1 based on the output of the external environment detection device 10. The objects include some or all of the following obstacles: moving objects such as vehicles, bicycles, and pedestrians; road boundary structures such as road markings, steps, guardrails, road shoulders, and median strips; road structures such as road signs and billboards; and objects lying (fallen) on the road. The object recognition unit 130 acquires information such as the presence, position, and type of other moving objects by inputting images captured by an external camera of the external environment detection device 10 into a trained model that is trained to output information such as the presence, position, and type of an object when an image captured by the external camera is input. The type of other moving objects can also be estimated based on the size in the image or the intensity of reflected waves received by the radar device of the external environment detection device 10. The object recognition unit 130 also acquires the speed of other moving objects detected by the radar device using, for example, Doppler shift.

[0040] The control unit 140 controls the drive unit 40 according to the set driving mode, for example. The moving object 1 may execute only some of the driving modes described below, but in all cases, the control unit 140 sets different speed limits for when the moving object 1 moves on a roadway and when it moves on a sidewalk. In this case, the mode selector switch 22 may be omitted.

[0041] In mode A-1, the control unit 140 refers to the information about the travel path and the object based on the output of the object recognition unit 130, and controls the motor MT of the drive unit 40 so that when the moving body 1 moves on a roadway, the distance to an object ahead of the moving body 1 is maintained at a certain level or more, and when the distance to the object ahead of the moving body 1 is sufficiently long, the moving body 1 moves at a first speed V1 (for example, a speed of at least 10 km / h and less than several tens of km / h). When the moving body 1 moves on a sidewalk, the control unit 140 controls the motor MT of the drive unit 40 so that the distance to an object ahead of the moving body 1 is maintained at a certain level or more, and when the distance to the object ahead of the moving body 1 is sufficiently long, the moving body 1 moves at a second speed V2 (for example, a speed of less than 10 km / h). This function is similar to the adaptive cruise control (ACC) function of a vehicle in which the first speed V1 or the second speed V2 is set as a set speed, and the technology used in ACC can be utilized. In mode A-1, the control unit 140 controls the steering device SD to change the steering angle of the steered wheels based on the amount of operation of the operator 14, such as a steering wheel. This function is similar to that of a power steering device, and the technology used in power steering devices can be used. Note that, instead of electronically controlling steering, the mobile object 1 may have a steering device in which the operator 14 and a steering mechanism are mechanically connected.

[0042] In mode A-2, the control unit 140 references information about the traveling path and objects based on the output of the object recognition unit 130, generates a target trajectory that allows the moving object 1 to avoid objects on the traveling path, and controls the steering device SD of the drive unit 40 so that the moving object 1 moves along the target trajectory. Regarding acceleration and deceleration, the control unit 140 controls the motor MT of the drive unit 40 based on the speed of the moving object 1 and the amount of operation of the accelerator pedal or the brake pedal. When the moving object 1 is traveling on a roadway, the control unit 140 controls the motor MT of the drive unit 40 with a first speed V1 as the upper limit speed (in mode A-2, this means that the moving object 1 will not accelerate even if a further acceleration command is given once the upper limit speed has been reached), and controls the drive unit 40 with a second speed V2 as the upper limit speed when the moving object 1 is traveling on a sidewalk.

[0043] In mode B, the control unit 140 controls the motor MT of the drive unit 40 based on the speed of the moving object 1 and the amount of operation of the accelerator pedal or the brake pedal. When the moving object 1 is moving on a roadway, the control unit 140 controls the motor MT of the drive unit 40 with the first speed V1 as the upper limit speed (in mode B, this means that the moving object 1 will not accelerate even if a further acceleration command is given once the upper limit speed has been reached), and when the moving object 1 is moving on a sidewalk, the control unit 140 controls the motor MT of the drive unit 40 with the second speed V2 as the upper limit speed. Steering is the same as in mode A-1.

[0044] In mode C, the control unit 140 references information about the path and objects based on the output of the object recognition unit 130, generates a target trajectory along which the moving body 1 can move while avoiding objects within the path, and controls the drive unit 40 so that the moving body 1 moves along the target trajectory. Even in mode C, the control unit 140 controls the drive unit 40 with a first speed V1 as the upper limit speed when the moving body 1 is moving on a roadway, and controls the drive unit 40 with a second speed V2 as the upper limit speed when the moving body 1 is moving on a sidewalk.

[0045] [Control when at least some of the recognition results from the first period in the past are missing] However, a situation may arise in which at least some of the recognition results from the past first period are not stored (are missing) in the storage device 70, such as when the mobile object 1 is parked and restarted, when the control device 100 is restarted after some kind of system trouble, or when the control device 100 is reset due to a malfunction of the external environment detection device 10. FIG. 4 is a diagram comparing a case in which the recognition results from the past first period are stored in the storage device 70 with a case in which the recognition results from the past first period are not stored in the storage device 70. The upper diagram of FIG. 4 shows a case in which the recognition results from the past first period are stored in the storage device 70, and the lower diagram of FIG. 4 shows a case in which the recognition results from the past first period are not stored in the storage device 70. As shown in the figure, if a period occurs in which the road type recognition unit 120 does not continuously perform recognition processing, a phenomenon may occur in which the recognition results from the past first period are missing, even if the "recognition result from one cycle ago" exists in terms of control time. The road type recognition unit 120 acquires absolute time information by some means (from an internal clock or via communication) and determines whether all recognition results for the past first period are stored in the memory device 70 (i.e., whether any are missing).

[0046] If all of the past recognition results for the first period are not stored in the storage device 70, the road type recognition unit 120 outputs to the control unit 140 a recognition result that the moving object 1 is moving on a sidewalk for the second period. The second period may be a variable period or a fixed period. For example, the second period may be a period whose upper limit is the first period and ends when a sufficiently high value (a value higher than the threshold value Th) is calculated as Sc(k) or when a sufficiently low value is calculated as Sc(k).

[0047] As described above, when the moving object 1 is traveling on a roadway, it is permitted to travel at a higher speed than when traveling on a sidewalk. Therefore, in consideration of safety, it is preferable that the conditions for recognizing that the moving object 1 is traveling on a roadway be set more conservatively than the conditions for recognizing that the moving object 1 is traveling on a sidewalk. In this regard, according to the embodiment, when there are not enough past recognition results, the validity of the recognition result can be ensured by outputting a recognition result that the moving object 1 is traveling on a sidewalk. In other words, it is possible to appropriately recognize whether the moving object 1 is traveling on a sidewalk or a roadway, in consideration of safety.

[0048] When the recognition results for the past first period are not stored in the storage device 70, and when the control device 100 is started up, if the recognition results before shutdown and the output of the external environment sensing device 10 (e.g., images captured by an external camera) are stored in the storage device 70, and if the current output of the external environment sensing device 10 (same as above) is similar to the output of the external environment sensing device 10 before shutdown, the road type recognition unit 120 may treat the recognition results before shutdown (recognition results with the same time length as the first period) as the recognition results for the past first period. Regarding whether the captured images are similar, for example, the road type recognition unit 120 performs a process to extract feature points of the captured images and determines that the captured images are similar if the positions of the feature points match by a predetermined percentage or more. Alternatively, the road type recognition unit 120 may determine that the captured images are similar if the sum of the differences in pixel values ​​for each pixel of the captured images is equal to or less than a predetermined value. Alternatively, the road type recognition unit 120 may perform the above process after excluding dynamic objects (e.g., vehicles, pedestrians, bicycles) captured in the captured images. It is assumed that the moving object 1 was parked between the time the control device 100 was shut down and the time it was started up, and therefore the recognition result before the shutdown is likely to be valid at that time.

[0049] If the recognition result for the past first period is not stored in the storage device 70, the road type recognition unit 120 may output the recognition result that the moving object 1 is moving on a sidewalk to the control unit 140 during the second period, and may cause the control unit 140 to perform a predetermined operation to cause the external environment detection device 10 to detect the characteristics of either the roadway or the sidewalk. The characteristics are the examples of the first event or the second event described above (steps, tactile paving blocks, road markings, etc.). The predetermined operation is an operation for detecting the first event or the second event more accurately and increasing the reliability of the recognition result.

[0050] (1) For example, if the external world sensing device 10 is capable of changing its sensing direction, the predetermined operation may be an operation of causing the external world sensing device 10 to search for a feature while changing the sensing direction of the external world sensing device 10. FIG. 5 is a diagram showing a first example of the predetermined operation. As shown in the figure, the predetermined operation may be an operation of searching for a step 200 (an example of a feature) while changing the sensing direction of the external world sensing device 10 (here, an external camera) left and right, and confirming the orientation of the step 200 (which side is the top and which side is the bottom) based on an image of the step 200 captured at a position closer to the external world sensing device 10.

[0051] (2) When operating in mode C, the predetermined operation may be an operation of changing the orientation of the moving object 1 within the range of the steering performance of the drive unit 40 to cause the external environment detection device 10 to search for a feature, or an operation of bringing the moving object 1 closer to a feature to cause the external environment detection device 10 to accurately recognize the feature, or an operation including both of these. FIG. 6 is a diagram showing a second example of the predetermined operation. As shown in the figure, the predetermined operation may be an operation of turning the moving object 1 so that a road marking 202 (or an object whose presence or absence is unknown at the time) is in front of the moving object 1, and then bringing the moving object 1 closer to the road marking 202.

[0052] 7 and 8 are flowcharts showing an example of the flow of processing executed by the road type recognition unit 120 according to the embodiment. First, the road type recognition unit 120 acquires information for recognition, such as an image from an external camera (step S300). Next, the road type recognition unit 120 recognizes a first event and a second event (step S302), and calculates a roadway score Sc (step S304).

[0053] Next, the road type recognition unit 120 determines whether or not the recognition results for the past first period are stored in the storage device 70 (step S306). If the recognition results for the past first period are stored in the storage device 70, the road type recognition unit 120 calculates an integrated score from the roadway score Sc and the recognition results for the past first period (step S308), and determines whether or not the integrated score is equal to or greater than a threshold value Th (step S310).

[0054] If the combined score is equal to or greater than the threshold value Th, the road type recognition unit 120 recognizes that the moving object 1 is moving on a roadway (step S312) and stores the recognition result in the storage device 70 (step S316). If the combined score is less than the threshold value Th, the road type recognition unit 120 recognizes that the moving object 1 is moving on a sidewalk (step S314) and stores the recognition result in the storage device 70 (step S316).

[0055] If it is determined in step S306 that the past recognition result for the first period is not stored in the storage device 70 (proceeding to FIG. 8), the road type recognition unit 120 determines whether the current time is the startup time of the control device 100 (step S320). If it is determined that the current time is the startup time of the control device 100, the road type recognition unit 120 determines whether the past recognition result is stored in the storage device 70 and whether the captured images are similar to each other (step S322), as described above. If the past recognition result is stored in the storage device 70 and the captured images are similar to each other, the road type recognition unit 120 treats the past recognition result (which does not correspond to the first period) as the recognition result for the past first period (step S324), and proceeds to step S308.

[0056] If a negative determination result is obtained in step S320 or S322, the road type recognition unit 120 continues to recognize that the moving object 1 is moving on a sidewalk during the second period (holds the recognition result) (step S326), and instructs the control unit 140 to perform a predetermined action (step S328). The process then proceeds to step S316, but the recognition result stored in the storage device 70 when proceeding from step S328 to step S316 may be a recognition result different from the held recognition result. For example, the road type recognition unit 120 may store a provisional recognition result based mainly on the roadway score Sc in the storage device 70 as the recognition result.

[0057] According to the embodiment described above, the road type recognition unit 120 basically recognizes whether the moving body 1 is moving on the roadway or in a specified area (sidewalk) based on the output of the external environment detection device 10, which detects the external situation of the moving body 1. Based on the recognition result of the past first period and the output of the external environment detection device 10, the road type recognition unit 120 repeatedly recognizes whether the moving body 1 is moving on the roadway or in a specified area, and stores the recognition result in the storage device 70. If the recognition result of the past first period is not stored in the storage device 70, the recognition result that the moving body 1 is moving on a sidewalk is output to the control unit 140 during the second period, so that it is possible to appropriately recognize whether the moving body 1 is moving on the roadway or in a specified area different from the roadway, taking robust stability into consideration.

[0058] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: Based on the output of an external environment detection device that detects the external situation of a mobile body that can move on both a roadway and a predetermined area different from the roadway, the mobile body recognizes whether the mobile body is moving on a roadway or in the predetermined area; limiting the speed of the moving body when it moves on a roadway to a first speed, and limiting the speed of the moving body when it moves in the predetermined area to a second speed lower than the first speed; The recognizing process includes: repeatedly recognizing whether the moving object is moving on a roadway or in the predetermined area based on a recognition result of a past first period and an output of the external environment detection device, and storing the recognition result in a storage device; and outputting a recognition result indicating that the moving object has recognized movement for a second period when at least a part of the recognition result for the past first period in the storage device is missing. Control device for a moving object.

[0059] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0060] 10. External sensing devices 12 Mobile Sensor 14 Controls 16 Internal Camera 18 Positioning equipment 22 Mode switch 30 Moving mechanism 40 Drive unit 50 External alarm device 70 Storage device 100 control device 120 Road type recognition unit 130 Object recognition section 140 Control Unit

Claims

1. A control device for a mobile body that can move on both a roadway and a predetermined area different from the roadway, a road type recognition unit that recognizes whether the moving object is moving on a roadway or in the predetermined area based on an output of an external environment detection device that detects an external situation of the moving object; a control unit that limits the speed of the moving body when it moves on a roadway to a first speed and limits the speed of the moving body when it moves in the predetermined area to a second speed that is lower than the first speed; Equipped with The road type recognition unit repeatedly recognize whether the moving object is moving on a roadway or in the predetermined area based on a recognition result of a past first period and an output of the external environment detection device, and store the recognition result in a storage device; outputting, to the control unit, a recognition result indicating that the moving object is moving on a sidewalk during a second period when at least a part of the recognition result for the past first period stored in the storage device is missing; Control device for a moving object.

2. the road type recognition unit, when at least a part of the recognition results for the past first period in the storage device is missing, causes the control unit to perform a predetermined operation to cause the external environment detection device to detect characteristics of either the roadway or the predetermined area; The control device for a moving body according to claim 1.

3. the external environment detection device is capable of changing a detection direction, The predetermined operation is an operation of changing a detection direction of the external environment detection device and causing the external environment detection device to search for the feature. The control device for a moving body according to claim 2.

4. the control unit controls a drive device to autonomously move the moving body, The predetermined operation is an operation of causing the external environment detection device to search for the feature while changing the orientation of the moving object. The control device for a moving body according to claim 2.

5. the control unit controls a drive device to autonomously move the moving body, the predetermined action is an action of bringing the moving object close to the feature and causing the external environment detection device to accurately recognize the feature; The control device for a moving body according to claim 2.

6. The road type recognition unit, upon startup of the device, the recognition result before the shutdown and the output of the external environment sensing device are stored in a storage device, and if the current output of the external environment sensing device is similar to the output of the external environment sensing device before the shutdown, the recognition result before the shutdown is treated as the recognition result of the past first period; The control device for a moving body according to claim 1.

7. the external environment detection device is an external camera that captures an image of the outside of the moving object, The road type recognition unit performs a process of extracting feature points of the image captured by the external camera, and when the positions of the feature points match at a predetermined rate or more, determines that the current output of the external environment sensing device and the output of the external environment sensing device before the shutdown are similar. The control device for a moving body according to claim 6.

8. the external environment detection device is an external camera that captures an image of the outside of the moving object, the road type recognition unit determines that the captured images are similar to each other when a total value of differences in pixel values ​​for each pixel of the captured images of the external camera is equal to or less than a predetermined value; The control device for a moving body according to claim 6.

9. A computer that controls a mobile object that can move on both a roadway and a predetermined area different from the roadway, Based on an output from an external environment detection device that detects an external situation of the moving object, the moving object recognizes whether it is moving on a roadway or in the predetermined area; limiting the speed of the moving body when it moves on a roadway to a first speed, and limiting the speed of the moving body when it moves in the predetermined area to a second speed lower than the first speed; The recognizing process includes: repeatedly recognizing whether the moving object is moving on a roadway or in the predetermined area based on a recognition result of a past first period and an output of the external environment detection device, and storing the recognition result in a storage device; and outputting a recognition result indicating that the moving object is moving on a sidewalk during a second time period when at least a part of the recognition result for the past first time period stored in the storage device is missing. A method for controlling a moving object.

10. A computer that controls a mobile object that can move on both a roadway and a predetermined area different from the roadway, Based on an output from an external environment detection device that detects an external situation of the moving object, the moving object is made to recognize whether it is moving on a roadway or in the predetermined area; a program that limits the speed of the moving body when it moves on a roadway to a first speed, and limits the speed of the moving body when it moves in the predetermined area to a second speed that is lower than the first speed, The recognizing process includes: repeatedly recognizing whether the moving object is moving on a roadway or in the predetermined area based on a recognition result of a past first period and an output of the external environment detection device, and storing the recognition result in a storage device; and outputting a recognition result indicating that the moving object is moving on a sidewalk during a second time period when at least a part of the recognition result for the past first time period stored in the storage device is missing. A storage medium that stores a program.

Images