Control system, control method, and storage medium

Abstract

This invention provides a control system, control method, and storage medium capable of providing more accurate and effective support for object detection. The control system is a control system for a moving body, comprising: an identification unit that identifies the surrounding conditions of the moving body based on the detection result of a first sensor; a second sensor, different from the first sensor, mounted on the moving body and detecting an object by the moving body contacting or approaching it; and a control unit that controls the movement of the moving body based on the surrounding conditions. If the identification unit identifies a predetermined first object, the control unit assumes that a second object, which is unidentifiable or difficult to identify, exists at a reference position set based on the position of the first object, and moves the moving body to a position where the second sensor can detect the second object.

Description

[0001] This application claims priority based on Japanese Patent Application No. 2022-007777, filed on January 21, 2022, the contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to control systems, control methods, and storage media. Background Technology

[0003] Previously, an invention of an electric vehicle that can be ridden by one person and move on a sidewalk was disclosed (Japanese Patent Application Publication No. 2020-189536). Summary of the Invention

[0004] In previous technologies, the processing related to the detection of objects around the moving object was sometimes not adequately considered.

[0005] This invention was made with the consideration of such circumstances in mind, and one of its objectives is to provide a control system, control method, and storage medium capable of supporting more accurate detection of objects.

[0006] Solution for solving the problem

[0007] The control system, control method, and storage medium of the present invention adopt the following structure.

[0008] (1): A control system according to one aspect of the present invention includes: a first sensor; an identification unit that identifies the surrounding conditions of the moving body based on the detection result of the first sensor; a second sensor different from the first sensor, which is mounted on the moving body and detects the object by the moving body contacting or approaching the object; and a control unit that controls the movement of the moving body based on the surrounding conditions, wherein when the identification unit identifies a predetermined first object, the control unit assumes that there is a second object that the identification unit cannot identify or is difficult to identify at a reference position set based on the position of the first object, and moves the moving body to a position where the second sensor can detect the second object.

[0009] (2): In the above (1) scheme, when the identification unit identifies a third object that is different from the first object, the control unit controls the movement of the moving body in a manner that does not assume the existence of the second object.

[0010] (3): Another aspect of the control system of the present invention includes: a first sensor; an identification unit that identifies the surrounding conditions of the moving body based on the detection result of the first sensor; a position identification unit that identifies the position of the moving body; a second sensor different from the first sensor, the second sensor being mounted on the moving body and detecting the object by the moving body contacting or approaching the object; and a control unit that controls the movement of the moving body based on the surrounding conditions, wherein when the position identification unit identifies that the moving body has reached a predetermined position, the control unit assumes that there is a second object that the identification unit cannot identify or is difficult to identify at a reference position set based on the position of the first object, and moves the moving body to a position where the second sensor can detect the second object, wherein the first object exists within a predetermined range from the predetermined position.

[0011] (4): In the above (3) scheme, the control unit controls the movement of the moving body in a manner that does not assume the existence of the second object when the position recognition unit does not recognize that the moving body has reached the predetermined position.

[0012] (5): In the above schemes (1)-(4), the control system further includes a determination unit that determines, based on the detection result of the second sensor, that the moving body has contacted or approached the object. If the determination unit determines, based on the detection result of the second sensor, that the moving body has contacted or approached the object when the moving body has been moved to a position where the second sensor can detect the second object, the control unit causes the moving body to move along a second direction orthogonal to the first direction, the first direction being the direction of movement of the moving body immediately preceding the contact or approach.

[0013] (6): In the above (5) scheme, when the control unit determines that the moving body is contacting or approaching the second object after the moving body has moved for a specified time or a specified distance along the orthogonal direction, the determination unit determines that the moving body is contacting or approaching the second object, and causes the moving body to perform an action to avoid the second object.

[0014] (7): In the above scheme (5) or (6), when the control unit moves the moving body along the orthogonal direction for a specified time or a specified distance, the determination unit determines that the moving body has not contacted or approached the second object, determines that the second object does not exist, and moves the moving body along the first direction.

[0015] (8): In any of the above schemes (1) to (7), the control system further includes a determination unit that determines, based on the detection result of the second sensor, that the moving body has contacted or approached an object. If the control unit performs the process of moving the moving body to a position where the second sensor can detect the second object and the determination unit does not determine, based on the detection result of the second sensor, that the moving body has contacted or approached the object, it determines that the second object does not exist and moves the moving body by passing the reference position.

[0016] (9): In any of the above schemes (1) to (7), the first sensor is one or more of the following sensors: radar device, camera and optical radar device.

[0017] (10): In the above (9) scheme, the second object is an object whose vertical length is less than the reference and which is difficult or impossible for the identification unit to identify based on the detection results of the radar device, the camera and the optical radar device.

[0018] (11): Another aspect of the control method of the present invention enables the computer of the control system of the mobile body to perform the following processing: identification processing to identify the surrounding conditions of the mobile body based on the detection result of the first sensor; control processing to control the movement of the mobile body based on the surrounding conditions; in the case that a predetermined first object is identified by the identification processing, assuming that there is a second object that cannot be identified or is difficult to identify by the identification processing at a reference position set based on the position of the first object, the mobile body is moved to a position where a second sensor different from the first sensor can detect the second object, the second sensor being mounted on the mobile body and detecting the object by the mobile body contacting or approaching the object.

[0019] (12): Another aspect of the present invention is a storage medium storing a program, wherein the program causes a computer of the control system of the mobile body to perform the following processing: identification processing to identify the surrounding conditions of the mobile body based on the detection result of a first sensor; control processing to control the movement of the mobile body based on the surrounding conditions, the program also causes the computer of the control system of the mobile body to perform the following processing: if a predetermined first object is identified by the identification processing, assuming that a second object that cannot be identified or is difficult to identify by the identification processing exists at a reference position set based on the position of the first object, the mobile body is moved to a position where a second sensor different from the first sensor can detect the second object, the second sensor being mounted on the mobile body and detecting the object by the mobile body contacting or approaching the object.

[0020] (13): Another aspect of the control method of the present invention enables the computer of the control system of the moving body to perform the following processing: identification processing to identify the surrounding conditions of the moving body based on the detection result of the first sensor; control processing to control the movement of the moving body based on the surrounding conditions; position identification processing to identify the position of the moving body, wherein if the moving body is identified to have reached a predetermined position through the position identification processing, it is assumed that there is a second object that cannot be identified or is difficult to identify through the identification processing at a reference position set based on the position of the first object, and the moving body is moved to a position where the second sensor, which is different from the first sensor, can detect the second object, the first object is located within a predetermined range from the predetermined position, and the second sensor is mounted on the moving body and detects the object by the moving body contacting or approaching the object.

[0021] (14): Another aspect of the present invention is a storage medium storing a program, wherein the program causes a computer of a control system of a mobile body to perform the following processing: identification processing to identify the surrounding conditions of the mobile body based on the detection result of a first sensor; control processing to control the movement of the mobile body based on the surrounding conditions; and position identification processing to identify the position of the mobile body. The program also causes a computer of a control system of the mobile body to perform the following processing: if the mobile body is identified to have reached a predetermined position through the position identification processing, assuming that a second object that cannot be identified or is difficult to identify exists at a reference position set based on the position of a first object, the mobile body is moved to a position where a second sensor different from the first sensor can detect the second object, the first object exists within a predetermined range from the predetermined position, and the second sensor is mounted on the mobile body and detects the object by the mobile body contacting or approaching the object.

[0022] Invention Effects

[0023] According to the schemes (1)-(14), the control device can provide support for more accurate object detection. For example, as a support for object recognition, the control device can move the moving body to a position where the second sensor can detect objects that the recognition unit cannot or has difficulty recognizing. Moreover, by detecting objects with greater accuracy and controlling the movement of the moving body based on the detection results, the control device can more smoothly guide the moving body to its destination. Attached Figure Description

[0024] Figure 1 This is a diagram illustrating an example of the structure of the moving body and control device in an embodiment.

[0025] Figure 2This is a perspective view of a moving object viewed from above.

[0026] Figure 3 This is a diagram showing the moving object viewed from the horizontal (positive Y direction).

[0027] Figure 4 This is a diagram (Figure 1) used to illustrate an example of the movement pattern of a moving body.

[0028] Figure 5 This is a diagram representing an example of the first object.

[0029] Figure 6 This is a diagram (Figure 2) used to illustrate an example of the movement pattern of a moving body.

[0030] Figure 7 This is a diagram (Figure 2) used to illustrate an example of the movement pattern of a moving body.

[0031] Figure 8 This is a diagram showing the moving object viewed from the side.

[0032] Figure 9 This is a graph illustrating an example of how the output value of a contact detection sensor changes over time.

[0033] Figure 10 This is a diagram (Figure 3) used to illustrate an example of the movement pattern of a moving body.

[0034] Figure 11 This is a graph illustrating an example of how the output value of a contact detection sensor changes over time.

[0035] Figure 12 Figure 4 is an example used to illustrate the movement pattern of a moving body.

[0036] Figure 13 This is a graph illustrating an example of how the output value of a contact detection sensor changes over time.

[0037] Figure 14 This is a flowchart illustrating an example of a process executed by a control device.

[0038] Figure 15 This is a diagram representing an example of the content of the corresponding information. Detailed Implementation

[0039] Hereinafter, with reference to the accompanying drawings, embodiments of the control system, control method, and storage medium mounted on the mobile body of the present invention will be described. The mobile body moves between a lane and a designated area different from the lane. The designated area is, for example, a sidewalk. The designated area may also be part or all of a roadside area, a bicycle lane, an open space, etc., or may include all of the sidewalk, roadside area, bicycle lane, open space, etc. In the following description, the designated area is assumed to be a sidewalk. The part referred to as "sidewalk" in the following description can be appropriately read as "designated area".

[0040] Hereinafter, the forward direction of the moving object is sometimes referred to as the positive X direction, the backward direction of the moving object as the negative X direction, the right direction (the right direction when facing the positive X direction) in a direction orthogonal to the forward and backward directions is referred to as the positive Y direction, the left direction (the left direction when facing the positive X direction) in a direction orthogonal to the forward and backward directions is referred to as the negative Y direction, the direction orthogonal to both the X and Y directions and vertically upward is referred to as the positive Z direction, and the direction orthogonal to both the X and Y directions and vertically downward is referred to as the negative Z direction.

[0041] <Implementation Method>

[0042] Figure 1 This diagram illustrates an example of the structure of the mobile body 1 and control device 200 according to an embodiment. The mobile body 1 includes, for example, an external detection device 10 (an example of a first sensor), a contact detection sensor 12 (an example of a second sensor), a mobile body sensor 20, an operating element 30, an internal camera 40, a positioning device 50, a dialogue device 60, a movement mechanism 70, a drive device 80, an external reporting device 90, a storage device 100, and a control device 200. Structures not essential for achieving the functions of this invention may be omitted.

[0043] The external detection device 10 is a variety of devices whose detection range is the direction of travel of the moving body 1. Furthermore, the external detection device 10 includes, for example, an external camera, a radar device, a LIDAR (Light Detection and Ranging) device, a sensor fusion device, etc. The external detection device 10 outputs information representing the detection result (image, object position, etc.) to the control device 200.

[0044] The contact detection sensor 12 is a sensor that detects when the moving body 1 comes into contact with an object. The contact detection sensor 12 detects changes in output values ​​such as resistance and capacitance when the moving body 1 comes into contact with the object, thereby detecting the contact. In this embodiment, as an example, the contact detection sensor 12 is assumed to be a pressure sensor.

[0045] The moving body sensor 20 may include, for example, a speed sensor, an acceleration sensor, a yaw rate (angular velocity) sensor, an orientation sensor, and an operation amount detection sensor mounted on the operating member 30. The operating member 30 may include, for example, operating members for indicating acceleration / deceleration (e.g., accelerator pedal, brake pedal) and operating members for indicating steering (e.g., steering wheel). In this case, the moving body sensor 20 may also include a throttle opening sensor, a brake pedal input sensor, a steering torque sensor, etc. The moving body 1, as the operating member 30, may also have operating members of other forms than those described above (e.g., non-ring-shaped rotary operating members, joysticks, buttons, etc.).

[0046] The internal camera 40 captures images of at least the heads of the occupants of the moving body 1 from the front. The internal camera 40 is a digital camera that utilizes imaging elements such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor). The internal camera 40 outputs the captured images to the control device 200.

[0047] The positioning device 50 is a device for measuring the position of the mobile body 1. The positioning device 50 is, for example, a GNSS (Global Navigation Satellite System) receiver, and determines the position of the mobile body 1 based on signals received from GNSS satellites, outputting this as position information. The position information of the mobile body 1 can also be estimated based on the location of the Wi-Fi base station to which the communication device is connected (described later).

[0048] The dialogue device 60 includes, for example, a speaker, a microphone, a touch panel, and a communication device 62. The dialogue device 60 appropriately processes the occupant's voice received by the microphone, transmits it to a server device via a network through the communication device 62, and provides voice-based information through the speaker based on the information received from the server device. The dialogue device 60 is also sometimes referred to as an intelligent agent device, a concierge device, or an assistant device. The server device has voice recognition, natural language processing, meaning interpretation, and response content determination functions. The dialogue device 60 sends location information to the server device, and the server device responds with information about suitable facilities based on the location information and the guidance requests made by the occupant (e.g., "What are some good ramen restaurants nearby?"). In this case, the dialogue device 60 provides voice guidance such as "It's located after turning left ahead." Not limited to this, the dialogue device 60 also has the function of accepting natural speech from the occupant and providing appropriate responses. The dialogue device 60 may also have the function of conducting simple dialogues without going through the server device, such as asking questions and receiving replies from the device side, and asking the passenger questions according to the request from the control device 200.

[0049] The moving mechanism 70 is a mechanism for moving the moving body 1 on a road. The moving mechanism 70 can be, for example, a wheel assembly including steering wheels and drive wheels. The moving mechanism 70 can also be legs for multi-legged travel.

[0050] The drive unit 80 outputs force to the moving mechanism 70, causing the moving body 1 to move. For example, the drive unit 80 includes a motor that drives the drive wheels, a battery that stores the electricity supplied to the motor, a steering device that adjusts the steering angle of the steering wheels, and a braking device that is controlled according to information input from the control device 200 or from the operating element 30. The drive unit 80 may also be equipped with an internal combustion engine, fuel cell, etc., as a drive force output mechanism or a power generation mechanism.

[0051] The external reporting device 90 may be, for example, a lamp, display device, or speaker installed on the outer panel of the mobile body 1 for reporting external information to the mobile body 1. The external reporting device 90 operates differently depending on whether the mobile body 1 is moving on a sidewalk or a driveway. For example, the external reporting device 90 may be controlled to illuminate when the mobile body 1 is moving on a sidewalk and not illuminate when the mobile body 1 is moving on a driveway. The color of the light emitted is preferably a color defined by law. The external reporting device 90 may also be controlled to emit green light when the mobile body 1 is moving on a sidewalk and blue light when the mobile body 1 is moving on a driveway. When the external reporting device 90 is a display device, it displays the message "Moving on the sidewalk" in text or graphics when the mobile body 1 is moving on a sidewalk.

[0052] Figure 2 This is a perspective view of the moving body 1 from above. In the figure, FW is the steering wheel, RW is the drive wheel, SD is the steering mechanism, MT is the motor, and BT is the battery. The steering mechanism SD, motor MT, and battery BT are included in the drive unit 80. AP is the accelerator pedal, BP is the brake pedal, WH is the steering wheel, SP is the speaker, and MC is the microphone. The moving body 1 shown is a one-person vehicle, with the occupant P sitting in the driver's seat DS and wearing the seatbelt SB. Arrow D1 indicates the direction of travel (velocity vector) of the moving body 1. An external detection device 10 is located near the front end of the moving body 1, and an internal camera 40 is positioned to capture images of the occupant P's head from in front of P. An external reporting device 90, serving as a display device, is located near the front end of the moving body 1.

[0053] Figure 3This is a view of the moving body 1 viewed from the horizontal (positive Y direction). A bumper section BNP is provided in front of the moving body 1 (X direction side). Contact detection sensors 12 (12-1 to 12-3) are provided between the bumper section BNP and the moving body 1. The contact detection sensors 12 are configured to detect the position of the force applied to the bumper section BNP by contact when it comes into contact with an object. The bumper section BNP is, for example, at a height that does not obstruct the occupant's view when the occupant is sitting on the moving body 1.

[0054] return Figure 1 The storage device 100 is a non-temporary storage device such as an HDD (Hard Disk Drive), flash memory, or RAM (Random Access Memory). The storage device 100 stores map information 110, programs 120 for execution by the control device 200, etc. Figure 1 The storage device 100 is described outside the frame of the control device 200, but the storage device 100 may be included in the control device 200.

[0055] [Control Device]

[0056] The control device 200 includes, for example, a road recognition unit 210, an object recognition unit 220, a position recognition unit 230, a determination unit 240, and a control unit 250. The road recognition unit 210, object recognition unit 220, position recognition unit 230, determination unit 240, and control unit 250 are implemented, for example, by executing a program (software) using a hardware processor such as a CPU (Central Processing Unit). Some or all of these components can also be implemented using hardware (including the circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit), or through the coordinated use of software and hardware. The program can be pre-saved in a storage device (not shown), or it can be saved in a removable storage medium (non-temporary storage medium) such as a DVD or CD-ROM, and installed in the storage device by mounting the storage medium onto a drive device.

[0057] The road recognition unit 210 identifies whether the moving body 1 is moving on a lane or a sidewalk. For example, the road recognition unit 210 identifies whether the moving body 1 is moving on a lane or a sidewalk by analyzing an image captured by an external camera of the external detection device 10. Semantic segmentation can be cited as an example of image analysis. The road recognition unit 210 classifies and labels each pixel within a frame of the image according to a category (lane, sidewalk, boundary, object, etc.). If more pixels in the area corresponding to the front of the moving body 1 are labeled as "lane," the moving body 1 is identified as moving on a lane; if more pixels in the area corresponding to the front of the moving body 1 are labeled as "sidewalk," the moving body 1 is identified as moving on a sidewalk. Not limited to this, the road recognition unit 210 can also identify the moving body 1 as moving on a lane if a vehicle is detected in the area corresponding to the front of the moving body 1 in the image, and identify the moving body 1 as moving on a sidewalk if a pedestrian is detected in the area corresponding to the front of the moving body 1 in the image. The road recognition unit 210 can also identify whether the moving body 1 is moving on a lane when the width of the road surface area corresponding to the front of the moving body 1 in the image is large, and identify whether the moving body 1 is moving on a sidewalk when the width of the road surface area corresponding to the front of the moving body 1 in the image is small. The road recognition unit 210 can also compare the position information of the moving body 1 with the map information 110 to identify whether the moving body 1 is moving on a lane or a sidewalk. In this case, the map information needs to have the accuracy to distinguish between a sidewalk and a lane based on the position coordinates. When the "designated area" is not only a sidewalk, the road recognition unit 210 also performs the same processing for roadside areas, bicycle lanes, open spaces, etc.

[0058] The object recognition unit 220 identifies objects surrounding the moving body 1 based on the output of the external detection device 10. Objects include moving bodies such as vehicles, bicycles, and pedestrians; road boundaries such as road markings, steps, guardrails, shoulders, and median strips; road signs and notice boards; fallen objects on the road; and obstacles such as objects in the direction of travel of the moving body 1. The object recognition unit 220 can, for example, input images captured by the external camera into a learned model that outputs information such as the presence, position, and category of objects when images captured by the external camera of the external detection device 10 are input, thereby obtaining information such as the presence, position, and category of other moving bodies. The category of other moving bodies can also be estimated based on their size in the image and the intensity of reflected waves received by the radar device of the external detection device 10. The object recognition unit 220 can also, for example, obtain the speed of other moving bodies detected by the radar device using Doppler frequency shift, etc. The object recognition unit 220 can also identify obstacles based on information input from the LIDAR, as described later. The object recognition unit 220 can also be included in an external detection device instead of the control device 200.

[0059] The object recognition unit 220 may also identify the category of an object based on its shape, size, etc. The object recognition unit 220 may identify whether the identified object belongs to a predetermined category. An object of a predetermined category (first object) is an object, such as a pillar or support (described later), that has a higher probability of containing linear objects like ropes or chains compared to other objects (third objects). In addition to the above, linear objects may also include fences with large mesh made of thin wire or barbed wire.

[0060] The position identification unit 230 acquires position information measured by the positioning device 50 and determines whether the acquired position is a predetermined position. The predetermined position information is stored in the storage device 100.

[0061] The determination unit 240 determines whether the moving body 1 has come into contact with an object based on the detection result of the contact detection sensor 12. For example, if the output value of the signal from the contact detection sensor 12 is above a threshold, the determination unit 240 determines that the moving body 1 has come into contact with an object.

[0062] The control unit 250 generates a track by referring to information about the driving path obtained from the output of the road recognition unit 210 and information about the object obtained from the output of the object recognition unit 220, and controls the drive device 80 to make the mobile body 1 automatically travel on the generated track. The track is the path that the mobile body 1 will automatically (independent of the driver's operation) will travel in the future. The track includes, for example, a speed element. For example, the track is represented by arranging the locations (track points) that the mobile body 1 should reach in sequence. Track points are the locations that the mobile body 1 should reach at predetermined travel distances (e.g., a few meters) along the route. In contrast, target speed and target acceleration are generated as part of the target track at predetermined sampling times (e.g., a few tenths of a second). Track points can also be the positions that the mobile body 1 should reach at the sampling time at predetermined sampling times. In this case, the information of target speed and target acceleration is represented by the interval of track points.

[0063] For example, when the mobile body 1 is moving on a roadway, the control unit 250 controls the motor MT, braking device, and operating device of the drive unit 80 to maintain the distance between the mobile body 1 and objects surrounding the mobile body 1 at a first distance or more. When the mobile body 1 is moving on a sidewalk, the control unit 250 controls the motor MT, braking device, and operating device of the drive unit 80 to maintain the distance between the mobile body 1 and objects in front of the mobile body 1 at a second distance or more. The second distance is, for example, a distance longer than the first distance. When the mobile body 1 is moved by the driver's operation, the control unit 250 controls the drive unit 80 to move the mobile body 1 in a manner corresponding to the operation based on the user's operation of the operating device. As described above, the mobile body 1 moves by automatic driving or by the user's operation. In the case of automatic driving, it is also possible that no occupant is riding in the mobile body 1.

[0064] [Specific examples of controlling moving objects]

[0065] The control unit 250 controls the moving body 1 based on the category of the object identified by the object recognition unit 220. For example, the control unit 250 switches the movement mode (control mode) of the moving body 1 according to the category of the object. Specific examples will be given below. These controls are performed by the control unit 250.

[0066] [The case where the object's category is not the first object (the third object)]

[0067] When the object recognition unit 220 identifies a third object that is different from the first object, the control unit 250 controls the movement of the moving body 1 in a manner that does not consider the second object (does not assume or presuppose the existence of a second object). The case of "assuming (presuming) the existence of a second object" will be described later.

[0068] Figure 4 Figure 1 illustrates an example of the movement pattern of moving body 1. At time T, moving body 1 detects bicycle B (an example of a third object). In this case, at time T+1, moving body 1 decelerates, and between times T+2 and T+5, it passes bicycle B at a predetermined distance without approaching it.

[0069] [The case where the object's category is the first object]

[0070] When the object recognition unit 220 detects a first object, the control unit 250 moves the moving body 1 towards a position where the second sensor can detect a second object. This second object exists at a reference position surrounding the first object and is either undetectable or difficult to detect by the object recognition unit 220. The control unit 250 assumes (or presupposes) that the second object, which the object recognition unit 220 cannot detect, exists at a reference position surrounding the first object (or a reference position set based on the position of the first object), and moves the moving body 1 closer to or to the reference position in order for the contact detection sensor 12 to detect the second object.

[0071] Figure 5 This diagram illustrates an example of the first object. When the moving body 1 detects the pillar P, which is an example of the first object, it moves in a different manner than when it detects the bicycle B (the third object). For example, between the two pillars P, in... Figure 5 The first object is a second object with a shorter width in the vertical direction and a longer horizontal direction, such as a rope or chain, suspended from two pillars P as shown. The first object is not limited to pillars; it can be any support or a second object with a shorter width in the vertical direction and a longer horizontal direction (the second object with a shorter vertical length than the reference). The information of the first object is stored in the storage device 100.

[0072] The second object is an object that the object recognition unit 220 cannot recognize or cannot recognize based on the detection results of the radar device, the image captured by the camera, and the detection results of the optical radar device.

[0073] The external detection device 10 of the moving body 1 sometimes fails to detect a second object that is short in width and long in width in the vertical direction, as described above. For example, in an image captured by an external camera, the second object occupies few pixels, resulting in limited information for identifying it. Therefore, the object recognition unit 220 sometimes fails to recognize the second object based on the image. For example, when using a radar device, the observed value of the reflected wave from the second object is small, making it difficult for the object recognition unit 220 to distinguish between the reflected wave and noise, and thus difficult to recognize the second object based on the small observed value. For example, when using a LiDAR, the scanning direction of the LiDAR light is parallel to the shape of the second object. When part or all of the second object exists between the scanned light beams, the object recognition unit 220 sometimes fails to recognize the second object and cannot grasp the overall structure of the second object.

[0074] When the object recognition unit 220 recognizes the first object using the external detection device 10 but cannot recognize the second object, it performs the following control.

[0075] Figure 6 Figure 2 illustrates an example of the movement pattern of the moving body 1. At time T, the moving body 1 detects the pillar P. In this case, at time T+1, the moving body 1 decelerates and approaches the pillar P. At time T+2, the moving body 1 decelerates further. At this time, the object recognition unit 220 attempts to identify whether an object (e.g., a rope) exists between the two pillars P, but fails to detect the existence of an object between the two pillars P. In this case, the moving body 1 may also move to a position different from the position where the recognition process was attempted, and attempt the recognition process again at the moved position. The recognition process refers to the process of identifying the object using the external detection device 10.

[0076] At time T+3, the moving body 1 decelerates while moving. At time T+4, the moving body 1 assumes that an object (a second object) exists between the two pillars P and moves accordingly. For example, the moving body 1 controls itself to contact the assumed second object. For example, the moving body 1 approaches or reaches a reference position in order for the contact detection sensor 12 to detect the second object. The reference position (the position where the second object is assumed to exist) is, for example, the position corresponding to the line connecting the reference positions of the two pillars (and...). Figure 6 (The same position as the rope R). The reference position can also replace the above position and be a position preset based on the position of the first object.

[0077] At time T+5, the moving body 1 further decelerates and moves to the reference position at a speed below a specified speed. The specified speed refers to any speed, such as slow speed, 5 km / h, or 1 km / h. For example, the specified speed is the speed at which the moving body 1, based on the detection result of the contact detection sensor 12, can identify contact with a second object when it comes into contact with it, and the impact on the moving body 1 is below a specified level. At time T+6, the moving body 1 contacts the rope R, and the situation of the moving body 1 contacting a second object is identified.

[0078] As described above, the moving body 1 can identify the second object by contacting the second object that the object recognition unit 220 failed to recognize.

[0079] When the control unit 250 determines that the moving body 1 has contacted the second object based on the detection result of the contact detection sensor 12, and the determination unit 240 determines that the moving body 1 has contacted the second object based on the detection result of the contact detection sensor 12, the determination unit 240 moves the moving body 1 in a second direction orthogonal to the first direction, which is the direction in which the moving body 1 moves before the contact is about to occur.

[0080] Figure 7 Figure 2 illustrates an example of the movement pattern of moving body 1. At time T+6, after moving body 1 comes into contact with the second object, at time T+7, moving body 1, for example, while still in contact with the second object, moves in a direction orthogonal (or approximately orthogonal) to the direction of travel (or the direction in which the pillar P exists). At time T+8, moving body 1 moves further in the orthogonal direction.

[0081] Figure 8 This is a diagram showing the movement of object 1 viewed from the side. (Example) Figure 9 As shown, when the moving body 1 comes into contact with the second object, the moving body M moves in the direction (positive Y direction) where the pillar P exists.

[0082] Figure 9 This is a graph illustrating an example of how the output value of a contact detection sensor changes over time. Figure 9 The vertical axis represents the output value of the contact detection sensor. Figure 10 The horizontal axis represents time. The output value of the moving body 1 is zero or close to zero before it comes into contact with the object. When the moving body 1 comes into contact with the object (at time T+6), the output value increases (for example, becomes above a threshold). Thus, the control device 200 can recognize that the moving body 1 has come into contact with the object.

[0083] After the moving body 1 comes into contact with the second object, it moves towards the direction where the pillar P is located while still in contact with the second object, thereby maintaining an output value above a predetermined output value. In this way, while the moving body 1 moves laterally and maintains an output value above a predetermined output value, the control device 200 determines that there is an object between the two pillars P that cannot be detected by the object recognition unit 220.

[0084] When the control unit 250 moves the mobile body 1 along the aforementioned orthogonal direction for a predetermined time or distance, and the determination unit 240 determines that the mobile body 1 has contacted or approached the second object, the control unit 250 confirms the presence of the second object and causes the mobile body 1 to perform an action to avoid the second object. For example, the control unit 250 directs the mobile body 1 to the destination via a different path or by passing the outside of the pillar P.

[0085] If a second object is detected, the moving body 1 can also perform object identification while moving near the object it is in contact with in order to identify the location of a more specific object. For example, the moving body 1 can also change its relative position to the column P or the rope R, or change its posture, so that the external detection device 10 can attempt to detect the object.

[0086] As described above, the control unit 250 can identify the position and presence of the second object by bringing the moving body 1 into contact with the second object.

[0087] [In the case where there is no second object]

[0088] When the control unit 250 moves the moving body 1 close to or to the reference position (performs the process of moving the moving body towards the position where the second sensor can detect the second object), if the determination unit 240 determines that the moving body 1 does not contact or approach the second object based on the detection result of the contact detection sensor 12, it determines that there is no second object, and moves the moving body 1 by passing the reference position.

[0089] Figure 10 Figure 3 illustrates an example of the movement pattern of the moving body 1. At time T+6#, the moving body 1 slowly approaches the space between the two pillars. For example, the moving body 1 assumes the existence of a second object and sets a reference position Rp, and approaches the set reference position Rp. At this time, if the determination unit 240 does not detect the second object, the moving body 1 continues to move. If the state of not detecting the second object continues for a predetermined time (if the state of moving continues for a predetermined distance), the moving body 1 presumes that there is no second object between the two pillars, passes between the two pillars P, and proceeds to its destination.

[0090] Figure 11This is a graph illustrating an example of how the output value of a contact detection sensor changes over time. Figure 11 The vertical axis represents the output value of the contact detection sensor. Figure 11 The horizontal axis represents time. In, for example... Figure 10 As explained, when the moving body 1 does not contact the second object, it continues to move with an output value of zero or close to zero. Therefore, the control device 200 can recognize that there is no second object in the direction of travel of the moving body 1.

[0091] [The case where the second object exists within a portion of the region]

[0092] If the determination unit 240 determines that the moving body 1 has contacted or approached a second object based on the fact that the moving body 1 has approached or reached a reference position, the control unit 250 moves the moving body 1 along a second direction orthogonal to the first direction, which is the immediate moving direction, for a predetermined time or distance. If the determination unit 240 determines that the moving body 1 has not contacted or approached the second object, the control unit 250 determines that there is no second object and moves the moving body along the first direction.

[0093] Figure 12 Figure 4 illustrates an example of the movement pattern of moving body 1. At time T+5#, moving body 1 slowly approaches the reference position between the two pillars P. At time T+6##, moving body 1 comes into contact with a second object OB that exists between the two pillars. At time T+7##, moving body 1 moves laterally while still in contact with the second object OB. Thus, moving body 1 is positioned to avoid object OB. At this point, moving body 1 becomes unaware of contact with the second object OB. At time T+8##, moving body 1 recognizes that there is no second object OB ahead and moves straight. Then, moving body 1 passes between the two pillars P to its destination.

[0094] Figure 13 This is a graph illustrating an example of how the output value of a contact detection sensor changes over time. Figure 13 The vertical axis represents the output value of the contact detection sensor. Figure 13 The horizontal axis represents time. In, for example... Figure 12 As explained, when the moving body 1 comes into contact with the second object, the output value increases; the output value also increases when the moving body 1 moves laterally while in contact with the second object. Subsequently, when the moving body 1 moves further laterally and avoids the second object, the output value becomes zero or near zero. Thus, the control device 200 can recognize that a second object exists in a portion of the area between the two pillars P.

[0095] As described above, even when a second object cannot be identified in the detection results of the external detection device 10, the control device 200 can still identify the second object by bringing the moving body 1 close to or to the reference position and using the detection results of the contact detection sensor 12, as described above. That is, by bringing the moving body 1 close to or to the reference position, the control device 200 can provide support for more accurate detection of the second object or detect the object with more accurate precision. Furthermore, when a second object is detected, the control device 200 can avoid the second object and move the moving body 1 towards the destination. Therefore, even when a second object cannot be identified by the object recognition unit 220, the moving body 1 can reach its destination more smoothly.

[0096] [flow chart]

[0097] Figure 14 This is a flowchart illustrating an example of the processing flow performed by the control device 200. First, the control device 200 determines whether a predetermined condition is met (step S100). If the predetermined condition is met, the control device 200 decelerates the moving body 1, bringing it closer to a reference position where a second object is assumed to exist (step S102). The predetermined condition is, for example, the detection of a first object.

[0098] Next, the control device 200 determines whether the output value of the contact detection sensor 12 meets the first condition (step S104). The first condition is a preset output value indicating that the moving body 1 has contacted the second object.

[0099] If the output value does not meet the first condition, the control device 200 determines whether the moving body 1 has moved a predetermined distance (step S106). If the moving body 1 has not moved the predetermined distance, the process returns to step S102. If the moving body 1 has moved the predetermined distance, the control device 200 directs the moving body to its destination while monitoring its surroundings (step S112). For example, if the moving body 1 does not come into contact with the second object even after moving the predetermined distance, the control device 200 recognizes that there is no second object around the first object and directs the moving body 1 past the first object.

[0100] If the output value meets the first condition in step S104 (the case of contact with the second object), the control device 200 moves the moving body 1 laterally (step S108). Next, the control device 200 determines whether the output value of the contact detection sensor 12 meets the second condition after the moving body 1 has moved laterally a predetermined distance (step S110). The second condition is the output value indicating contact with the second object (continuing contact). Specifically, the second condition is that the output value above a threshold is maintained for a predetermined time while the moving body 1 is moving laterally, or that the output value above the threshold occurs more than a predetermined number of times while the moving body 1 is moving laterally.

[0101] If the output value does not meet the second condition, the control device 200 moves the mobile body 1 to its destination while monitoring its surroundings (step S112). If the second condition is not met, the mobile body 1 is positioned where there is no second object in front of it (a position that avoids the second object), so the mobile body 1 can pass the first object without contacting the second object even if it moves straight.

[0102] If the output value satisfies the second condition, the control device 200 presumes the existence of a second object and generates a path to avoid the second object, and causes the moving body 1 to go to the destination based on the generated path (step S114). Thus, the processing of one routine of this flowchart ends.

[0103] Through the above processing, the control device 200 can identify the presence or absence of a second object that the object recognition unit 220 cannot recognize, as well as the position of the second object, and can make the moving body 1 smoothly go to its destination based on the recognition results.

[0104] In the above description and flowchart, it was explained that when the control device 200 detects the first object, it performs a process to bring the moving body 1 closer to or to a reference position in order to detect the second object. However, it is also possible to move the moving body towards a position where the second sensor can detect the second object in order to detect the second object when the moving body 1 reaches or approaches a predetermined position. For example, the control device 200 may also assume that there is a second object that cannot be detected or is difficult to detect by the external detection device 10 at a reference position set based on the position of the first object, and move the moving body towards a position where the second sensor can detect the second object, where the first object exists within a predetermined range from the predetermined position.

[0105] The aforementioned "predefined location" is, for example, map information stored in a storage device. The predefined location is, for example, a location where the second object is highly likely to exist, or a location set based on the location where the second object exists. The predefined location is either a location where the second object is highly likely to exist (an example of a reference location), or a location a few meters or tens of meters closer to the location where the second object exists (an example of a reference location).

[0106] When the moving body 1 reaches a predetermined position, the control device 200 assumes that a second object is at a reference position surrounding the predetermined position, and moves the moving body 1 closer to or to the reference position (moving the moving body towards a position where the second sensor can detect the second object, which is an object existing at a reference position surrounding the first object and that the recognition unit cannot or has difficulty recognizing). When the moving body 1 does not reach the predetermined position, the control device 200 controls the movement of the moving body 1 without assuming the existence of a second object (ignoring the second object).

[0107] The control device 200 can also identify objects and control the moving body 1 based on the corresponding information stored in the storage device of the control device 200. Figure 15 This is a diagram illustrating an example of the content of the correspondence information 300. The correspondence information 300 establishes a correspondence between the action of the moving body 1 used to identify the object, information indicating the shift in the output value of the contact detection sensor 12 generated by the action, the position of the assumed object corresponding to the shift in the output value, and the action taken by the moving body 1 in relation to the position of the assumed object.

[0108] (1) If the following action a is performed and condition a is satisfied, the moving body 1 performs action 1.

[0109] (Action a) After the moving body 1 comes into contact with the second object, the moving body 1 moves laterally.

[0110] (Condition a) The output value of the contact detection sensor 12 is consistent with the movement indicating the presence of a second object.

[0111] If the conditions (Action a) and (Condition a) above are met, the moving body 1 is presumed to have a second object and performs Action 1 to avoid the second object.

[0112] (2) If the following action b is performed and condition b is met, the moving body 1 performs action 2.

[0113] (Action b) The state in which the moving body 1 does not contact the second object and does not contact the second object when it moves further continues for a specified time.

[0114] (Condition b) The output value of the contact detection sensor 12 is consistent with the movement indicating that there is no second object.

[0115] If the conditions of (Action b) and (Condition b) above are met, the moving body 1 is presumed to be without a second object, and the moving body 1 moves.

[0116] (3) If the following action c is performed and condition c is met, the moving body 1 performs action 3.

[0117] (Action c) The moving body 1 comes into contact with the second object and moves laterally a specified distance.

[0118] (Condition c) The output value of the contact detection sensor 12 is consistent with the movement indicating the presence of a second object with a specified width.

[0119] If the conditions (action c) and (condition c) above are met, the moving body 1 is presumed to have a second object with a specified width, and performs action 3 to avoid the second object.

[0120] As described above, the control device 200 can also identify the second object based on the corresponding information 300, and control the moving body 1 based on the identification result. In addition to the information described above, the corresponding information 300 may also include information representing the relationship between the shift of the output value of the contact detection sensor 12 and the position of the object, as well as information on the actions taken by the moving body 1, in other modes.

[0121] [other]

[0122] The above description illustrates an example of detecting a second object by having the moving body 1 contact it. However, it is also possible to detect the second object by having the moving body 1 approach it instead. The contact detection sensor 12 can be a sensor that detects contact with the second object or a sensor that detects approach to the second object. A sensor whose output value changes upon contact or approach to the second object can be used.

[0123] Sensors whose output values ​​change upon contact or proximity to a second object include, but are not limited to, pressure sensors, microphones, sound sensors, accelerometers, capacitance sensors, conductivity sensors, temperature sensors, light sensors, and magnetic sensors. The determination unit 240 determines whether the moving body 1 has approached or contacted the second object based on changes in the sensor output values ​​(changes in sound, frequency, resistance, capacitance, conductivity, temperature, light, and magnetism) when the moving body 1 approaches the second object. As described above, the control device 200 is not limited to pressure sensors and can use any sensor to identify proximity or contact with a second object.

[0124] According to the embodiments described above, when a second object, which is difficult to identify or cannot be identified based on the detection results of a radar device, camera device, or optical radar device, exists in the direction of travel of the moving body 1, the control device 200 moves the moving body 1 by making it contact or approach the second object, and avoids the second object based on the detection results of a sensor corresponding to the movement of the moving body 1. Therefore, the control device 200 can identify the second object with greater accuracy, and further, by identifying the second object, the moving body 1 can be smoothly moved towards its destination.

[0125] The implementation methods described above can be performed as follows.

[0126] A control device for a moving body, which is capable of moving in both a lane and a designated area different from the lane, is configured to include:

[0127] Storage device, which stores a program; and

[0128] Hardware processor,

[0129] The hardware processor executes the program stored in the storage device to perform the following processing:

[0130] Identification processing to identify the surrounding conditions of the moving object based on the detection results of the first sensor;

[0131] Based on the surrounding conditions, control processing is used to control the movement of the moving body;

[0132] If a pre-defined first object is identified in the identification process, it is assumed that a second object exists at a reference position set based on the position of the first object, which cannot be identified or is difficult to identify by the identification process, and the moving body moves toward a position where the second sensor can detect the second object.

[0133] The above description illustrates specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A control system for a moving body, wherein, The control system includes: First sensor; A second sensor, different from the first sensor, is mounted on the moving body and detects the object by the moving body contacting the object; The identification unit identifies the surrounding conditions of the moving object based on the detection results of the first sensor; as well as The control unit controls the movement of the moving body based on the surrounding conditions. When the identification unit detects a pre-defined first object, the control unit assumes that a second object, which the identification unit cannot or is difficult to identify, exists at a reference position set based on the position of the first object. The control unit then moves the moving body to a position where the second sensor can detect the second object. The first object is an object of a category that has the potential to contain linear objects. The second object is a linear object.

2. The control system according to claim 1, wherein, If the identification unit detects a third object that is different from the first object, the control unit controls the movement of the moving body in a manner that does not assume the existence of the second object.

3. A control system, which is a control system for a moving body, wherein, The control system includes: First sensor; A second sensor, different from the first sensor, is mounted on the moving body and detects the object by the moving body contacting the object; The identification unit identifies the surrounding conditions of the moving object based on the detection results of the first sensor; The position recognition unit identifies the position of the moving body; as well as The control unit controls the movement of the moving body based on the surrounding conditions. When the position recognition unit detects that the moving body has reached a predetermined position, the control unit assumes that a second object, which cannot be recognized or is difficult to recognize, exists at a reference position set based on the position of the first object. The control unit then moves the moving body to a position where the second sensor can detect the second object, and the first object exists within a predetermined range from the predetermined position. The first object is an object of a category that has the potential to contain linear objects. The second object is a linear object.

4. The control system according to claim 3, wherein, If the position recognition unit does not recognize that the moving body has reached a predetermined position, the control unit controls the movement of the moving body in a manner that does not assume the existence of the second object.

5. The control system according to any one of claims 1 to 4, wherein, The control system further includes a determination unit that determines, based on the detection result of the second sensor, whether the moving body has come into contact with the object. If the determination unit determines that the moving body has contacted the object based on the detection result of the second sensor when the moving body has moved to a position where the second sensor can detect the second object, the control unit moves the moving body along a second direction orthogonal to the first direction, the first direction being the direction of movement of the moving body immediately preceding the contact.

6. The control system according to claim 5, wherein, When the control unit determines that the moving body is in contact with the second object after the determination unit has moved the moving body along the orthogonal direction for a predetermined time or distance, the control unit determines that the second object exists and causes the moving body to perform an action to avoid the second object.

7. The control system according to claim 5, wherein, When the control unit moves the moving body along the orthogonal direction for a specified time or a specified distance, and the determination unit determines that the moving body has not contacted the second object, it determines that the second object does not exist and moves the moving body along the first direction.

8. The control system according to any one of claims 1 to 4, wherein, The control system further includes a determination unit that determines, based on the detection result of the second sensor, whether the moving body has come into contact with the object. If the control unit performs the process of moving the moving body to a position where the second sensor can detect the second object, and the determination unit does not determine that the moving body has contacted the object based on the detection result of the second sensor, it determines that the second object does not exist, and moves the moving body by passing the reference position.

9. The control system according to any one of claims 1 to 4, wherein, The first sensor is one or more sensors from a radar device, a camera, and an optical radar device.

10. The control system according to claim 9, wherein, The second object is an object whose vertical length is less than the reference and which is difficult or impossible for the identification unit to identify based on the detection results of the radar device, the camera, and the optical radar device.

11. A control method, wherein, The control method causes the computer of the control system of the moving body to perform the following processes: Identification processing to identify the surrounding conditions of the moving object based on the detection results of the first sensor; The control process that controls the movement of the moving body based on the surrounding conditions. If a pre-defined first object is identified through the identification process, and assuming that a second object exists at a reference position based on the position of the first object that cannot be identified or is difficult to identify through the identification process, the moving body is moved to a position where a second sensor, different from the first sensor, can detect the second object. The second sensor is mounted on the moving body and detects the object by the moving body contacting it. The first object is an object of a category that has the potential to contain linear objects. The second object is a linear object.

12. A storage medium storing a program, wherein, The program causes the computer of the control system for the moving body to perform the following processing: Identification processing to identify the surrounding conditions of the moving object based on the detection results of the first sensor; The control process that controls the movement of the moving body based on the surrounding conditions. The program also causes the computer of the control system for the moving body to perform the following processing: If a pre-defined first object is identified through the identification process, and assuming that a second object exists at a reference position based on the position of the first object that cannot be identified or is difficult to identify through the identification process, the moving body is moved to a position where a second sensor, different from the first sensor, can detect the second object. The second sensor is mounted on the moving body and detects the object by the moving body contacting it. The first object is an object of a category that has the potential to contain linear objects. The second object is a linear object.

13. A control method, wherein, The control method causes the computer of the control system of the moving body to perform the following processes: Identification processing to identify the surrounding conditions of the moving object based on the detection results of the first sensor; Based on the surrounding conditions, control processing is used to control the movement of the moving body; Location recognition processing to identify the position of the moving object. If the moving body is identified as having reached a predetermined position through the position recognition process, assuming that a second object exists at a reference position set based on the position of the first object, which cannot be identified or is difficult to identify through the recognition process, the moving body is moved to a position where a second sensor, different from the first sensor, can detect the second object. The first object exists within a predetermined range from the predetermined position, and the second sensor is mounted on the moving body and detects the object by the moving body contacting it. The first object is an object of a category that has the potential to contain linear objects. The second object is a linear object.

14. A storage medium storing a program, wherein, The program causes the computer of the control system for the moving body to perform the following processing: Identification processing to identify the surrounding conditions of the moving object based on the detection results of the first sensor; Based on the surrounding conditions, control processing is used to control the movement of the moving body; Location recognition processing to identify the position of the moving object. The program also causes the computer of the control system for the moving body to perform the following processing: If the moving body is identified as having reached a predetermined position through the position recognition process, assuming that a second object exists at a reference position set based on the position of the first object, which cannot be identified or is difficult to identify through the recognition process, the moving body is moved to a position where a second sensor, different from the first sensor, can detect the second object. The first object exists within a predetermined range from the predetermined position, and the second sensor is mounted on the moving body and detects the object by the moving body contacting it. The first object is an object of a category that has the potential to contain linear objects. The second object is a linear object.

Images