Control device, control method, and storage medium

Abstract

A control device, control method, and storage medium are provided to support more accurate object detection. The control device includes: a surrounding condition acquisition unit that acquires surrounding condition information representing the surrounding condition of the moving body, identified by an object recognition device mounted on the moving body; a reference object detection unit that detects a predetermined reference object present in the direction of travel of the moving body based on the surrounding condition information; a drivable area recognition unit that identifies a drivable area of ​​the moving body based on the surrounding condition information; and a travel control unit that controls the travel of the moving body to travel within the drivable area. When a reference object is detected, the travel control unit moves the moving body in a manner close to the reference object. The drivable area recognition unit detects objects attached to the reference object based on the surrounding condition information identified by the object recognition device after the moving body has moved in a manner close to the reference object, and identifies the drivable area.

Description

Technical Field

[0001] This invention relates to control devices, control methods, and storage media. Background Technology

[0002] Previously, an invention of an electric vehicle that can be ridden by one person and move on a sidewalk was disclosed (Japanese Patent Application Publication No. 2020-189536).

[0003] In previous technologies, the processing related to the detection of objects around the moving object was sometimes not adequately considered. Summary of the Invention

[0004] The present invention was made in consideration of such circumstances, and one of its objectives is to provide a control device, control method, and storage medium that can support the detection of objects with greater accuracy.

[0005] The control device, control method, and storage medium of the present invention adopt the following structure.

[0006] (1): The present invention relates to a control device, wherein the control device comprises: a surrounding condition acquisition unit that acquires surrounding condition information representing the surrounding condition of the mobile body, which is identified by an object recognition device mounted on the mobile body; a reference object detection unit that detects a predetermined reference object present in the direction of travel of the mobile body based on the surrounding condition information; a drivable area identification unit that identifies a drivable area of ​​the mobile body based on the surrounding condition information; and a driving control unit that controls the driving of the mobile body so that the mobile body drives in the drivable area, wherein when the reference object is detected, the driving control unit causes the mobile body to move in a manner close to the reference object, and the drivable area identification unit detects an accompanying object attached to the reference object based on the surrounding condition information identified by the object recognition device after the mobile body moves in a manner close to the reference object, and identifies the drivable area by excluding the reference object and the accompanying object.

[0007] (2): Based on the above (1) scheme, when the reference object is detected, the driving control unit moves the moving body to the vicinity of the reference object. The vicinity of the reference object means that, when the accompanying object is present, the distance between the reference object and the distance at which the object recognition device can recognize the accompanying object is within the range of the reference position.

[0008] (3): Based on the above scheme (1) or (2), the reference object is at least two objects that are set on the ground on which the moving body travels, and whose height difference is within a specified value and whose height is above a threshold.

[0009] (4): Based on the above (3) scheme, when the reference object detection unit detects at least two objects whose height difference is within a specified value and whose height is above a threshold, the driving control unit moves the moving body in a manner close to the reference object, and the drivable area identification unit identifies the drivable area based on the surrounding identification information obtained by identifying the space between the at least two objects at the position where the moving body moves in a manner close to the reference object.

[0010] (5): Based on the above (4) scheme, when the reference object detection unit detects at least two objects whose height difference is within a specified value and whose height is above a threshold, the driving control unit moves the moving body in a manner close to the reference object so as to tilt in a direction with respect to the extension configuration direction of the accompanying object that is assumed to exist between the two or more objects.

[0011] (6): Based on the above scheme (1) or (2), the reference object is a suspension that is set on the ground on which the moving body travels and has a height above the height of the moving body.

[0012] (7): Based on the above (6) scheme, when the reference object detection unit detects a suspension object with a height of more than the height of the moving body as the reference object, the drivable area identification unit identifies the drivable area based on the surrounding identification information obtained by identifying the hollow space of the suspension object at the position where the moving body moves in a manner close to the reference object.

[0013] (8): Based on the above schemes (1) to (7), when the reference object is detected and the map information held by the moving body shows that the detection position of the reference object is consistent with the boundary position of multiple areas represented by the map information, the driving control unit moves the moving body in a manner close to the reference object.

[0014] (9): Another aspect of the present invention relates to a control method, wherein the control method causes a computer to perform the following processing: acquiring surrounding condition information representing the surrounding conditions of the mobile body, identified by an object recognition device mounted on the mobile body; detecting a predetermined reference object present in the direction of travel of the mobile body based on the surrounding condition information; identifying a drivable area of ​​the mobile body based on the surrounding condition information; controlling the movement of the mobile body to drive in the drivable area; moving the mobile body in a manner close to the reference object when the reference object is detected; and detecting an accompanying object attached to the reference object based on the surrounding condition information identified by the object recognition device after the mobile body moves in a manner close to the reference object, and identifying the drivable area by excluding the reference object and the accompanying object.

[0015] (10): Another aspect of the present invention relates to a storage medium storing a program, wherein the program causes a computer to perform the following processes: acquiring surrounding condition information representing the surrounding conditions of the mobile body, identified by an object recognition device mounted on the mobile body; detecting a predetermined reference object present in the direction of travel of the mobile body based on the surrounding condition information; identifying a drivable area of ​​the mobile body based on the surrounding condition information; controlling the movement of the mobile body to drive in the drivable area; moving the mobile body in a manner close to the reference object when the reference object is detected; and detecting an accompanying object attached to the reference object based on the surrounding condition information identified by the object recognition device after the mobile body moves in a manner close to the reference object, and identifying the drivable area by excluding the reference object and the accompanying object.

[0016] According to the schemes (1)-(10), it is possible to provide support for detecting objects with greater accuracy. Attached Figure Description

[0017] Figure 1 This is a diagram illustrating an example of the structure of the moving body and control device in an embodiment.

[0018] Figure 2 This is a perspective view of a moving object viewed from above.

[0019] Figure 3 This is an example of a scenario where object recognition by the object recognition unit fails.

[0020] Figure 4 This diagram shows an example of an object identified by the object recognition unit after a moving object has moved to a nearby area.

[0021] Figure 5 This is another example of a scenario where object recognition by the object recognition unit fails.

[0022] Figure 6 This is a diagram showing another example of an object identified by the object recognition unit after the moving object has moved to the nearby area.

[0023] Figure 7 This diagram illustrates a method for controlling moving objects by utilizing map information during the detection of a reference object.

[0024] Figure 8 This is a flowchart illustrating an example of the processing flow performed by the control device.

[0025] Figure 9 This is a flowchart representing another example of the process executed by the control device. Detailed Implementation

[0026] Hereinafter, embodiments of the control device, control method, and storage medium mounted on the mobile body of the present invention will be described with reference to the accompanying drawings. The mobile body moves between a lane and a designated area different from the lane. The designated area is, for example, a sidewalk. Alternatively, the designated area may be part or all of a roadside area, bicycle lane, open space, etc., or may include all of the sidewalk, roadside area, bicycle lane, open space, etc. In the following description, the designated area is the sidewalk. The part referred to as "sidewalk" in the following description can be appropriately interpreted as "designated area".

[0027] Hereinafter, the forward direction of the moving object is sometimes referred to as the positive X direction, the backward direction of the moving object is referred to as the negative X direction, the right direction (the right direction when facing the positive X direction) among the directions orthogonal to the forward and backward directions is referred to as the positive Y direction, the left direction (the left direction when facing the positive X direction) among the directions orthogonal to the forward and backward directions is referred to as the negative Y direction, the direction orthogonal to both the X and Y directions and vertically upward is referred to as the positive Z direction, and the direction orthogonal to both the X and Y directions and vertically downward is referred to as the negative Z direction.

[0028] <Implementation Method>

[0029] Figure 1 This diagram illustrates an example of the structure of the mobile body 1 and the control device 200 according to an embodiment. The mobile body 1 includes, for example, an external detection device 10, a mobile body sensor 20, an operating element 30, an internal camera 40, a positioning device 50, a dialogue device 60, a movement mechanism 70, a drive device 80, an external reporting device 90, a storage device 100, and a control device 200. It should be noted that some structures that are not essential for achieving the functions of this invention may be omitted.

[0030] The external detection device 10 comprises various devices that define the direction of travel of the moving body 1 as the detection range. The external detection device 10 includes, for example, an external camera 12 and a radar device 14. The external camera 12 is, for example, a digital camera utilizing a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The external camera 12 is installed at least in a position capable of capturing images of the direction of travel of the moving body 1, for example, periodically and repeatedly capturing images of the periphery of the vehicle M. The external camera 12 can also be a stereo camera. The radar device 14 emits millimeter-wave or other radio waves towards the periphery of the moving body 1 and detects the radio waves (reflected waves) reflected by the object to at least detect the object's position (distance and orientation).

[0031] Furthermore, the external detection device 10 may include, for example, a LIDAR (Light Detection and Ranging) system. The external detection device 10 performs sensor fusion processing on the detection results from some or all of the external camera 12, radar device 14, and LIDAR to identify the position, type, speed, etc., of objects existing in the vicinity of the moving body 1. The external detection device 10 outputs the identification results to the control device 200. Alternatively, the external detection device 10 may directly output the detection results from the external camera 12, radar device 14, and LIDAR to the control device 200. The external detection device 10 is an example of an "object identification device," and the identification result is an example of "surrounding situation information."

[0032] The moving body sensor 20 includes, for example, a speed sensor, an acceleration sensor, a yaw rate (angular velocity) sensor, an orientation sensor, and an operation amount detection sensor mounted on the operating member 30. The operating member 30 includes, for example, operating members for indicating acceleration / deceleration (e.g., accelerator pedal, brake pedal) and operating members for indicating steering (e.g., steering wheel). In this case, the moving body sensor 20 may include a throttle opening sensor, a brake pedal application sensor, a steering torque sensor, etc. The moving body 1 may also serve as the operating member 30 and possess an operating member with a form other than those described above (e.g., a non-annular rotary operating member, a joystick, a button, etc.).

[0033] The internal camera 40 captures images of at least the heads of the occupants of the moving body 1 from the front. The internal camera 40 is a digital camera that utilizes imaging elements such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor). The internal camera 40 outputs the captured images to the control device 200.

[0034] The positioning device 50 is a device for determining the position of the mobile body 1. The positioning device 50 is, for example, a GNSS (Global Navigation Satellite System) receiver, which determines the position of the mobile body 1 based on signals received from GNSS satellites and outputs this as position information. It should be noted that the position information of the mobile body 1 can also be estimated based on the location of the Wi-Fi base station to which the communication device is connected (described later).

[0035] The dialogue device 60 includes, for example, a speaker, a microphone, a touch panel, and a communication device 62. The dialogue device 60 processes the occupant's voice received by the microphone appropriately and transmits it to a server device via a network through the communication device 62. Based on the information received from the server device, it provides voice-based information through the speaker. The dialogue device 60 is sometimes also referred to as an intelligent agent device, a concierge device, or an assistant device. The server device has voice recognition, natural language processing, meaning interpretation, and response content determination functions. Alternatively, the dialogue device 60 can also be a device that sends location information to a server device, and the server device responds with information about suitable facilities based on the location information and guidance requests made by the occupant (e.g., "What are some good ramen restaurants nearby?"). In this case, the dialogue device 60 provides voice guidance such as "Turn left ahead." Not limited to this, the dialogue device 60 also has the function of accepting natural speech from the occupant and providing appropriate responses. In addition, the dialogue device 60 may also have functions such as asking questions from the device side and receiving replies, and conducting simple dialogues without going through the server device, and asking questions to the passenger according to the request from the control device 200.

[0036] The moving mechanism 70 is a mechanism for moving the moving body 1 on a road. The moving mechanism 70 is, for example, a wheel assembly including steering wheels and drive wheels. Alternatively, the moving mechanism 70 can also be a leg for multi-legged walking.

[0037] The drive unit 80 outputs force to the moving mechanism 70, causing the moving body 1 to move. For example, the drive unit 80 includes a motor that drives the drive wheels, a battery that stores the electricity supplied to the motor, a steering device that adjusts the steering angle of the steering wheels, and a braking device that is controlled according to information input from the control device 200 or from the operating element 30. The drive unit 80 may also be equipped with an internal combustion engine, fuel cell, etc., as a drive force output mechanism or a power generation mechanism.

[0038] The external reporting device 90 may be, for example, a lamp, display device, speaker, etc., installed on the outer panel of the mobile body 1 for reporting external information to the mobile body 1. The external reporting device 90 performs different actions depending on whether the mobile body 1 is moving on a sidewalk or a driveway. For example, the external reporting device 90 may be controlled to illuminate when the mobile body 1 is moving on a sidewalk and not illuminate when the mobile body 1 is moving on a driveway. The color of the light emitted is preferably a color specified by regulations. The external reporting device 90 may also be controlled to emit green light when the mobile body 1 is moving on a sidewalk and blue light when the mobile body 1 is moving on a driveway. When the external reporting device 90 is a display device, it displays the message "Moving on the sidewalk" using text or graphics when the mobile body 1 is moving on a sidewalk.

[0039] Figure 2 This is a perspective view of the moving body 1 from above. In the figure, FW is the steering wheel, RW is the drive wheel, SD is the steering mechanism, MT is the motor, and BT is the battery. The steering mechanism SD, motor MT, and battery BT are included in the drive unit 80. Additionally, AP is the accelerator pedal, BP is the brake pedal, WH is the steering wheel, SP is the speaker, and MC is the microphone. The moving body 1 shown is a one-person vehicle, with the occupant P seated in the driver's seat DS and wearing a seatbelt SB. Arrow D1 indicates the direction of travel (velocity vector) of the moving body 1. An external detection device 10 is located near the front end of the moving body 1, and an internal camera 40 is positioned to capture images of the occupant P's head from the front. Furthermore, an external reporting device 90, serving as a display device, is located near the front end of the moving body 1.

[0040] return Figure 1 The storage device 100 is a non-temporary storage device such as an HDD (Hard Disk Drive), flash memory, or RAM (Random Access Memory). Map information 110 and programs 120 executed by the control device 200 are stored in the storage device 100. Figure 1 The storage device 100 is described outside the frame of the control device 200, but the storage device 100 may also be included in the control device 200.

[0041] [Control Device]

[0042] The control device 200 includes, for example, a road recognition unit 210, an object recognition unit 220, a reference object detection unit 230, a drivable area recognition unit 240, and a driving control unit 250. The road recognition unit 210, object recognition unit 220, reference object detection unit 230, drivable area recognition unit 240, and driving control unit 250 are implemented, for example, by executing programs (software) using a hardware processor such as a CPU (Central Processing Unit). Some or all of these components can be implemented using hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit), or through a combination of software and hardware. The program can be pre-saved in a storage device (not shown) or stored in a removable storage medium (non-temporary storage medium) such as a DVD or CD-ROM, and installed in the storage device by mounting the storage medium onto a drive device.

[0043] The road identification unit 210 identifies whether the moving body 1 is moving on a lane or a sidewalk. For example, the road identification unit 210 identifies whether the moving body 1 is moving on a lane or a sidewalk by analyzing images captured by the external camera 12 of the external detection device 10. Semantic segmentation can be cited as an example of image analysis. The road identification unit 210 categorizes each pixel within the frame of the image (lane, sidewalk, boundary, object, etc.) and assigns labels. If more pixels in the area corresponding to the front of the moving body 1 are labeled as lanes, the unit identifies that the moving body 1 is moving on a lane; if more pixels in the area corresponding to the front of the moving body 1 are labeled as sidewalks, the unit identifies that the moving body 1 is moving on a sidewalk. Not limited to this, the road recognition unit 210 can also recognize that the moving body 1 is moving on a lane when a vehicle is detected in the area corresponding to the front of the moving body 1 in the image, and recognize that the moving body 1 is moving on a sidewalk when a pedestrian is detected in the area corresponding to the front of the moving body 1 in the image. Furthermore, the road recognition unit 210 can also recognize that the moving body 1 is moving on a lane when the width of the road surface area corresponding to the front of the moving body 1 in the image is large, and recognize that the moving body 1 is moving on a sidewalk when the width of the road surface area corresponding to the front of the moving body 1 in the image is small. Additionally, the road recognition unit 210 can compare the position information of the moving body 1 with the map information 110 to determine whether the moving body 1 is moving on a lane or a sidewalk. In this case, the map information needs to have the accuracy to distinguish between sidewalks and lanes based on position coordinates. When the "designated area" is not limited to sidewalks, the road recognition unit 210 performs the same processing for roadside areas, bicycle lanes, open spaces, etc.

[0044] The object recognition unit 220 identifies objects existing around the moving body 1 based on the output of the external detection device 10. These objects include moving bodies such as vehicles, bicycles, and pedestrians; road boundaries such as road markings, steps, guardrails, shoulders, and median strips; road signs and notice boards; objects that have fallen onto the road; and obstacles such as objects in the direction of travel of the moving body 1. For example, the object recognition unit 220 can input images captured by the external camera 12 into a learned model that outputs information such as the existence, position, and category of objects when images captured by the external camera 12 of the external detection device 10 are input, thereby obtaining information such as the existence, position, and category of other moving bodies. The category of other moving bodies can also be estimated based on their size in the image and the intensity of the reflected waves received by the radar device of the external detection device 10. Furthermore, the object recognition unit 220 can also obtain the speed of other moving bodies detected by the radar device using Doppler frequency shift, for example. Alternatively, the object recognition unit 220 can also identify obstacles based on information input from the LIDAR 12. The object recognition unit 220 can also be included in an external detection device instead of being included in the control device 200. The object recognition unit 220 is an example of a "surrounding situation acquisition unit".

[0045] Details regarding the reference object detection unit 230 will be described later.

[0046] The drivable area recognition unit 240 identifies the drivable area of ​​the moving body 1 based on the surrounding environment information obtained by the object recognition unit 220. More specifically, the drivable area recognition unit 240 identifies the drivable area of ​​the moving body 1 by excluding objects identified by the object recognition unit 220 from the area of ​​the image captured by the external camera 12. At this time, the excluded objects can also be set with any threshold (e.g., 5 cm or more) regarding height and volume.

[0047] The driving control unit 250 generates a track by referring to information about the driving road obtained from the output of the road recognition unit 210 and information about the drivable area obtained from the output of the drivable area recognition unit 240, and controls the drive device 80 to make the mobile body 1 automatically travel on the generated track. The track is the path that the mobile body 1 will automatically (independent of the driver's operation) will follow. The track includes, for example, a speed element. For example, the track is represented by a sequence of locations (track points) that the mobile body 1 should reach. Track points are locations that the mobile body 1 should reach at predetermined travel distances (e.g., a few meters) along the route. In contrast, target speed and target acceleration are generated as part of the target track at predetermined sampling times (e.g., a few tenths of a second). Alternatively, track points can be positions that the mobile body 1 should reach at predetermined sampling times. In this case, the target speed and target acceleration information are represented by the intervals of the track points.

[0048] For example, when the moving body 1 is moving on a lane, the driving control unit 250 controls the motor MT, braking device, and operating device of the drive unit 80 to maintain the distance between the moving body 1 and objects surrounding the moving body 1 at a first distance or more, thereby causing the moving body 1 to move. When the moving body 1 is moving on a sidewalk, the driving control unit 250 controls the motor MT, braking device, and operating device of the drive unit 80 to maintain the distance between the moving body 1 and objects in front of the moving body 1 at a second distance or more, thereby causing the moving body 1 to move. The second distance is, for example, a distance longer than the first distance. When the moving body 1 is moved by the driver's operation, the driving control unit 250 controls the drive unit 80 based on the user's operation of the operating device, thereby causing the moving body 1 to move in a manner corresponding to the operation.

[0049] Thus, the object recognition unit 220 detects objects based on images captured by the external camera 12 and the intensity of reflected waves received by the radar device 14. The drivable area recognition unit 240 identifies drivable areas by excluding the areas in the images captured by the external camera 12 from the objects identified by the object recognition unit 220. The driving control unit 250 controls the moving body 1 to travel within the identified drivable areas. However, depending on the type and shape of the object, the object recognition unit 220 may sometimes fail to recognize the object due to the small number of pixels and low intensity of reflected waves in the images captured by the external camera 12. As a result, the following situation may occur: the drivable area recognition unit 240 identifies the area containing the object as a drivable area, and the moving body 1 collides with the object.

[0050] Figure 3 This is a diagram illustrating an example of a scenario where object recognition performed by the object recognition unit 220 fails. Figure 3 An example is shown where an object R, which is narrow (i.e., thin) and long in the vertical direction (such as a rope or chain), is suspended from two pillars LP and RP in the direction of travel of the moving body 1. In this case, the object recognition unit 220 may fail to recognize the object R because the number of pixels occupied by the object R in the image captured by the external camera 12 is small and the intensity of the reflected wave reflected by the object R is low.

[0051] In the context described above, the reference object detection unit 230 detects pre-defined reference objects existing in the direction of travel of the moving body 1 based on the surrounding environment information identified by the object recognition unit 220. More specifically, for example, the reference object detection unit 230 detects two or more objects whose height difference is within a predetermined value and whose height is above a threshold as reference objects. Alternatively, the reference object detection unit 230 may compare reference images of pillars, road signs, etc., stored in advance in the storage device 100 with images captured by the external camera 12 and perform pattern matching processing to detect reference objects. Pillars and road signs are examples of "reference objects," and ropes and chains are examples of "attached objects."

[0052] When the reference object detection unit 230 detects a reference object, the driving control unit 250 moves the moving body 1 toward the vicinity of the detected reference object (i.e., in a manner close to the detected reference object). For example, in Figure 3 In this case, the driving control unit 250 first moves the moving body 1 to the vicinity of the area NR containing the location NRC, which is the location obtained by drawing an imaginary line VL through the pillar LP and pillar RP as the detected reference objects, and drawing a line segment (e.g., 3m) orthogonal to the imaginary line VL towards the moving body 1, starting from the center CP of the pillar LP (or pillar RP) in the ground.

[0053] When the moving object 1 moves towards the nearby area NR, the object recognition unit 220 re-identifies the object based on the image captured by the external camera 12 after the moving object 1 moves, the intensity of the reflected wave received by the radar device 14, and other factors. Figure 3 In some cases, for example, the object recognition unit 220 uses the external camera 12 to photograph the space between the left pillar LP and the right pillar LP, or uses the radar device 14 to radiate radio waves toward the space between the left pillar LP and the right pillar LP. More precisely, the driving control unit 250 moves the moving body 1 to the nearby area NR so that it tilts in a direction relative to the extended configuration direction of an object imagined to exist between the left pillar LP and the right pillar LP, and in this state, the object recognition unit 220 photographs the space between the left pillar LP and the right pillar LP, or radiates radio waves toward that space.

[0054] Figure 4 This diagram illustrates an example of an object identified by the object recognition unit 220 after the moving body 1 has moved to the nearby region NR. For example... Figure 4 As shown, the moving body 1 moves towards the nearby region NR, thereby... Figure 3 The object R captured in a horizontal shot is in Figure 4 This becomes longitudinal identification. As a result, the number of pixels occupied by object R in the image captured by external camera 12 increases, or the intensity of the reflected wave reflected by object R increases. Thus, it becomes possible to identify object R that could not be identified before moving to the nearby region NR.

[0055] The drivable area identification unit 240 identifies drivable areas by excluding objects R identified after moving to the nearby area NR from the area captured by the external camera 12. The driving control unit 250 generates a track that causes the mobile body 1 to travel within the identified drivable area. As a result, objects that are difficult to detect from the front using the external camera 12 and radar device 14 can be detected, allowing the mobile body 1 to travel appropriately.

[0056] It should be noted that, in the above description, the reference object detection unit 230 detects two or more objects whose height difference is within a predetermined value and whose height is above a threshold as reference objects. However, the present invention is not limited to such a structure. For example, the reference object detection unit 230 can detect two or more objects arranged at the same interval as reference objects, or it can detect two or more objects having a predetermined shape (e.g., cylinder, triangle) as reference objects. Multiple conditions used for detecting reference objects can also be arbitrarily combined.

[0057] Figure 5 This is another example of a scenario where object recognition by the object recognition unit 220 fails. Figure 5 This example illustrates a structure S (suspension) with a mesh-like fence F installed in the direction of travel of the moving body 1. In this case, the object recognition unit 220 may fail to recognize the fence F because the number of pixels occupied by the fence F in the image captured by the external camera 12 is small and the intensity of the reflected wave reflected by the fence F is low.

[0058] Therefore, the reference object detection unit 230 detects, for example, a suspension structure that is set on the ground on which the mobile body 1 travels and has a height greater than or equal to that of the mobile body 1 as a reference object. This is because it is known that the mobile body 1 cannot travel in the hollow space of the suspension structure if the height of the suspension structure is less than the height of the mobile body 1. Alternatively, the reference object detection unit 230 may compare a reference image of a fence or the like stored in the storage device 100 with an image captured by the external camera 12 and perform pattern matching processing to detect a reference object.

[0059] When the reference object detection unit 230 detects a reference object, the driving control unit 250 moves the moving body 1 towards the vicinity of the detected reference object. For example, in Figure 5 In this case, the driving control unit 250 first moves the moving body 1 toward the vicinity of the location NRC, which is the location obtained by drawing an imaginary line VL through the two pillars of the suspension S, which is a detected reference object, and drawing a line segment (e.g., 3m) orthogonal to the imaginary line VL from the center CP of the left or right pillar in the ground toward the moving body 1.

[0060] When the moving object 1 moves towards the nearby area NR, the object recognition unit 220 re-identifies the object based on the image captured by the external camera 12 after the moving object 1 moves, the intensity of the reflected wave received by the radar device 14, and other factors. Figure 5 In some cases, the object recognition unit 220 may use an external camera 12 to photograph the hollow space of the suspension S, or use a radar device 14 to radiate radio waves toward the hollow space of the suspension S. More precisely, the driving control unit 250 moves the moving body 1 to the vicinity NR so that it is tilted in the direction of the extended configuration of an object that is imagined to exist in the hollow space of the suspension S, and in this state, the object recognition unit 220 photographs the hollow space of the suspension S and radiates radio waves toward the hollow space.

[0061] Figure 6 This diagram illustrates an example of an object identified by the object recognition unit 220 after the moving body 1 moves to the nearby region NR. (See diagram for example.) Figure 6 As shown, the moving body 1 moves towards the nearby region NR, thereby... Figure 5 The fence F, captured in a horizontal shot, is in Figure 6 This becomes longitudinal identification. As a result, the number of pixels occupied by object R in the image captured by external camera 12 increases, or the intensity of the reflected wave reflected by fence F increases. Thus, fence F, which could not be identified before moving to the nearby area NR, can be identified.

[0062] The drivable area identification unit 240 identifies drivable areas by excluding the fence F identified after moving to the nearby area NR from the area captured by the external camera 12. The driving control unit 250 generates a track that causes the mobile body 1 to travel within the identified drivable area. As a result, objects that are difficult to detect from the front using the external camera 12 and radar device 14 can be detected, allowing the mobile body 1 to travel appropriately.

[0063] It should be noted that, in the above description, the reference object detection unit 230 detects suspension objects that are disposed on the ground on which the moving body 1 travels and have a height of 1 or higher as reference objects. However, the present invention is not limited to such a structure; for example, the reference object detection unit 230 may also detect suspension objects that generally have a height of 1 or higher as reference objects. The multiple conditions used for detecting reference objects can also be arbitrarily combined.

[0064] Furthermore, in the above description, when the reference object detection unit 230 detects a reference object, the driving control unit 250 moves the moving body 1 towards the vicinity of the detected reference object. However, the present invention is not limited to that structure, and the driving control unit 250 may also refer to the map information 110 and move the moving body 1 towards the vicinity of the detected reference object only in scenarios where the possibility of an accompanying object attached to the reference object is high.

[0065] Figure 7 This diagram illustrates a method for controlling the moving body 1 by utilizing map information 110 during the detection of a reference object. Figure 7 In this context, CL represents the current location of the moving body 1, and is determined by referring to map information 110 based on the location information of the moving body 1 determined by the positioning device 50. Figure 7 As shown in the map information 110, the moving body 1 is located at the boundary between the division representing the park and the division representing the road. Generally, for a reference object detected at the boundary position of multiple divisions in the map information 110, there is a high probability that there are accompanying objects that have not been identified by the object identification unit 220.

[0066] Therefore, when a reference object is detected and the map information 110 shows that the detection position of the reference object coincides with the boundary position of the multiple areas shown by the map information 110, the driving control unit 250 moves the moving body 1 towards the vicinity of the reference object. On the other hand, when a reference object is detected but the map information 110 does not show that the detection position of the reference object coincides with the boundary position of the multiple areas shown by the map information 110, the driving control unit 250 does not move the moving body 1 towards the vicinity of the reference object. This reduces the possibility of the moving body 1 moving towards the vicinity of a reference object even when no accompanying object is present.

[0067] [flow chart]

[0068] Next, refer to Figure 8 This will explain the process flow performed by the control device 200. Figure 8This is a flowchart illustrating an example of the processing flow performed by the control device 200. First, the object recognition unit 220 detects objects present in the vicinity of the moving body 1 based on images captured by the external camera 12 and the intensity of reflected waves received by the radar device 14, and obtains this as surrounding situation information (step S100). Next, the reference object detection unit 230 determines whether a predetermined reference object present in the direction of travel of the moving body 1 has been detected based on the obtained surrounding situation information (step S102).

[0069] If no reference object is detected, the control device 200 returns the process to step S100. On the other hand, if a reference object is detected, the driving control unit 250 moves the moving body 1 towards the vicinity of the detected reference object (step S104). Next, the object recognition unit 220 detects objects present in the vicinity of the moving body 1 based on the image captured by the external camera 12, the intensity of the reflected waves received by the radar device 14, etc., and obtains them again as surrounding situation information (step S106).

[0070] Next, the drivable area recognition unit 240 determines whether an object attached to the detected reference object is detected based on the surrounding situation information obtained again by the object recognition unit 220 (step S108). If an object is detected, the drivable area recognition unit 240 identifies the drivable area by excluding the detected reference object and the object from the area of ​​the image captured by the external camera 12 (step S110). On the other hand, if no object is detected, the drivable area recognition unit 240 identifies the drivable area by excluding the detected reference object from the area of ​​the image captured by the external camera 12 (step S112).

[0071] Next, the driving control unit 250 generates a track that allows the mobile body 1 to travel within the identified drivable area, and the mobile body 1 travels along the generated track (step S114). Thus, the processing of this flowchart ends.

[0072] Figure 9 This is a flowchart illustrating another example of the process executed by the control device 200. The following is about... Figure 8 The same process applies to the flowchart, so the explanation is omitted.

[0073] If a reference object is detected during the processing in step S102, the driving control unit 250 refers to the map information 110 and determines whether the current position of the moving body 1 is consistent with the boundary positions of multiple divisions in the map information 110 (step S103).

[0074] If the current position of the moving body 1 is determined to be inconsistent with the boundary positions of multiple divisions in the map information 110, the control device 200 returns the process to step S100. Conversely, if the current position of the moving body 1 is determined to be consistent with the boundary positions of multiple divisions in the map information 110, the control device 200 advances the process to step S104. Regarding other processes, ... Figure 8 The same applies to the flowchart processing. This reduces the likelihood that the moving body 1 will move towards the vicinity of the reference object even when no accompanying object is present.

[0075] It should be noted that the above embodiments illustrate an example where the reference object is a structure of a post or fence, and the accompanying object is a rope or the mesh portion of the fence. However, the present invention is not limited to such a structure. More generally, by typifying the combination of objects that can be identified even from a distance with objects that cannot be identified unless at close range, the present invention can be applied to a wide variety of reference objects and combinations of reference objects.

[0076] According to the implementation described above, when a reference object is detected around the moving body 1, the moving body 1 is moved to the vicinity of the reference object. At the moved position, the surrounding situation information of the reference object is obtained again. When an accessory object attached to the reference object is detected, the accessory object is removed from the drivable area of ​​the moving body 1. As a result, support for detecting objects with higher accuracy can be provided.

[0077] The implementation methods described above can be performed as follows.

[0078] A control device mounted on a moving body, wherein,

[0079] The control device includes:

[0080] Storage media that hold commands that can be read by a computer; and

[0081] The processor, which is connected to the storage medium,

[0082] The processor performs the following processing by executing commands that can be read by the computer:

[0083] Obtain surrounding condition information representing the surrounding conditions of the moving body as identified by the object recognition device mounted on the moving body;

[0084] Based on the surrounding environment information, a pre-defined reference object is detected in the direction of travel of the moving body;

[0085] Based on the surrounding environment information, the drivable area of ​​the moving body is identified;

[0086] Control the movement of the mobile body so that the mobile body can travel within the drivable area;

[0087] Upon detecting the reference object, the driving control unit causes the moving body to move in a manner approaching the reference object; and

[0088] The drivable area identification unit detects accompanying objects attached to the reference object based on the surrounding situation information identified by the object identification device after the moving body moves in a manner close to the reference object, and identifies the drivable area by excluding the reference object and the accompanying objects.

[0089] The above description illustrates specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A control device mounted on a moving body, wherein, The control device includes: The surrounding situation acquisition unit acquires surrounding situation information representing the surrounding situation of the moving body, which is identified by the object recognition device mounted on the moving body; The reference object detection unit detects a pre-defined reference object present in the direction of travel of the moving body based on the surrounding situation information. The drivable area identification unit identifies the drivable area of ​​the moving body based on the surrounding condition information; as well as A driving control unit controls the movement of the mobile body so that it travels within the drivable area. Upon detecting the reference object, the driving control unit causes the moving body to move in a manner approaching the reference object. The drivable area identification unit detects accompanying objects attached to the reference object based on the surrounding situation information identified by the object recognition device after the moving body moves in a manner close to the reference object, and identifies the drivable area by excluding the reference object and the accompanying objects. The reference objects are at least two objects placed on the ground on which the moving body travels, whose height difference is within a specified value and whose height is above a threshold value. When the reference object detection unit detects at least two objects whose height difference is within a predetermined value and whose height is above a threshold, the driving control unit moves the moving body to approach the reference object. The drivable area identification unit identifies the drivable area based on surrounding identification information obtained by identifying the space between at least two objects at a location where the moving body has moved close to the reference object. When the reference object detection unit detects at least two objects whose height difference is within a predetermined value and whose height is above a threshold, the driving control unit moves the moving body in a manner close to the reference object so as to tilt in a direction with respect to the extension configuration direction of the accompanying object that is assumed to exist between the two or more objects.

2. A control device mounted on a moving body, wherein, The control device includes: The surrounding situation acquisition unit acquires surrounding situation information representing the surrounding situation of the moving body, which is identified by the object recognition device mounted on the moving body; The reference object detection unit detects a pre-defined reference object present in the direction of travel of the moving body based on the surrounding situation information. The drivable area identification unit identifies the drivable area of ​​the moving body based on the surrounding condition information; as well as A driving control unit controls the movement of the mobile body so that it travels within the drivable area. Upon detecting the reference object, the driving control unit causes the moving body to move in a manner approaching the reference object. The drivable area identification unit detects accompanying objects attached to the reference object based on the surrounding situation information identified by the object recognition device after the moving body moves in a manner close to the reference object, and identifies the drivable area by excluding the reference object and the accompanying objects. When the reference object is detected and the map information held by the moving body shows that the detection position of the reference object is consistent with the boundary position of multiple areas represented by the map information, the driving control unit causes the moving body to move in a manner close to the reference object.

3. The control device according to claim 1 or 2, wherein, When the reference object is detected, the driving control unit moves the moving body toward the vicinity of the reference object. The vicinity of the reference object means, in the presence of the accompanying object, the range within which the object recognition device can recognize the accompanying object, with the reference object as the reference position.

4. The control device according to claim 1 or 2, wherein, The reference object is a suspension structure that is set on the ground on which the moving body travels and has a height above that of the moving body.

5. The control device according to claim 4, wherein, When the reference object detection unit detects a suspension object with a height greater than or equal to that of the moving body, the drivable area identification unit identifies the drivable area based on the surrounding identification information obtained by identifying the hollow space of the suspension object at a position where the moving body moves close to the reference object.

6. A control method, wherein, The control method causes the computer of the moving body to perform the following processing: Obtain surrounding condition information representing the surrounding condition of the moving body, as identified by the object recognition device mounted on the moving body; Based on the surrounding environment information, a pre-defined reference object is detected in the direction of travel of the moving body; Based on the surrounding environment information, the drivable area of ​​the moving body is identified; Control the movement of the mobile body so that the mobile body can travel within the drivable area; Upon detecting the reference object, the moving body is moved in a manner that approaches the reference object; as well as The system detects accompanying objects attached to the reference object based on the surrounding situation information identified by the object recognition device after the moving body moves in a manner close to the reference object, and identifies the drivable area by excluding the reference object and the accompanying objects. The reference objects are at least two objects placed on the ground on which the moving body travels, whose height difference is within a specified value and whose height is above a threshold value. If, when at least two objects are detected that have a height difference within a specified value and a height greater than a threshold, serving as the reference object, the moving body is moved to approach the reference object. The drivable area is identified based on surrounding identification information obtained by identifying the space between at least two objects at a location where the moving body has moved close to the reference object. If at least two or more objects are detected as the reference object, with a height difference within a specified value and a height above a threshold, the moving body is moved in a manner close to the reference object so as to tilt in a direction relative to the extended configuration direction of the accompanying object conceived to exist between the two or more objects.

7. A control method, wherein, The control method causes the computer of the moving body to perform the following processing: Obtain surrounding condition information representing the surrounding condition of the moving body, as identified by the object recognition device mounted on the moving body; Based on the surrounding environment information, a pre-defined reference object is detected in the direction of travel of the moving body; Based on the surrounding environment information, the drivable area of ​​the moving body is identified; Control the movement of the mobile body so that the mobile body can travel within the drivable area; Upon detecting the reference object, the moving body is moved in a manner that approaches the reference object; as well as The system detects accompanying objects attached to the reference object based on the surrounding situation information identified by the object recognition device after the moving body moves in a manner close to the reference object, and identifies the drivable area by excluding the reference object and the accompanying objects. When the reference object is detected and the map information held by the moving body shows that the detection position of the reference object is consistent with the boundary position of multiple areas represented by the map information, the moving body moves in a manner that approaches the reference object.

8. A storage medium storing a program, wherein, The program causes the mobile computer to perform the following processing: Obtain surrounding condition information representing the surrounding condition of the moving body, as identified by the object recognition device mounted on the moving body; Based on the surrounding environment information, a pre-defined reference object is detected in the direction of travel of the moving body; Based on the surrounding environment information, the drivable area of ​​the moving body is identified; Control the movement of the mobile body so that the mobile body can travel within the drivable area; Upon detecting the reference object, the moving body is moved in a manner that approaches the reference object; as well as The system detects accompanying objects attached to the reference object based on the surrounding situation information identified by the object recognition device after the moving body moves in a manner close to the reference object, and identifies the drivable area by excluding the reference object and the accompanying objects. The reference objects are at least two objects placed on the ground on which the moving body travels, whose height difference is within a specified value and whose height is above a threshold value. If, when at least two objects are detected that have a height difference within a specified value and a height greater than a threshold, serving as the reference object, the moving body is moved to approach the reference object. The drivable area is identified based on surrounding identification information obtained by identifying the space between at least two objects at a location where the moving body has moved close to the reference object. If at least two or more objects are detected as the reference object, with a height difference within a specified value and a height above a threshold, the moving body is moved in a manner close to the reference object so as to tilt in a direction relative to the extended configuration direction of the accompanying object conceived to exist between the two or more objects.

9. A storage medium storing a program, wherein, The program causes the mobile computer to perform the following processing: Obtain surrounding condition information representing the surrounding condition of the moving body, as identified by the object recognition device mounted on the moving body; Based on the surrounding environment information, a pre-defined reference object is detected in the direction of travel of the moving body; Based on the surrounding environment information, the drivable area of ​​the moving body is identified; Control the movement of the mobile body so that the mobile body can travel within the drivable area; Upon detecting the reference object, the moving body is moved in a manner that approaches the reference object; as well as The system detects accompanying objects attached to the reference object based on the surrounding situation information identified by the object recognition device after the moving body moves in a manner close to the reference object, and identifies the drivable area by excluding the reference object and the accompanying objects. When the reference object is detected and the map information held by the moving body shows that the detection position of the reference object is consistent with the boundary position of multiple areas represented by the map information, the moving body moves in a manner that approaches the reference object.

Images