White line recognition device, mobile object control system, white line recognition method, and program

Abstract

To provide a white line recognition device, a mobile object control system, a white line recognition method, and a program, which can suppress erroneous recognition of white lines.SOLUTION: A white line recognition device includes: a white line recognition section that recognizes a linear feature portion located in an image captured by a monocular camera as a contour of a white line drawn on a road surface; and a linear object detection section that detects a linear object located at a distance from the road surface on the basis of an output of a distance measurement sensor provided to have at least a detection range in the same direction as the monocular camera. The white line recognition section prevents the linear feature portion corresponding to a position of the linear object from being recognized as the contour of the white line.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a white line recognition device, a control system for a moving object, a white line recognition method, and a program. [Background technology]

[0002] An invention has been disclosed in the past that extracts linear objects and steps from camera images (Patent Document 1). Also, an invention has been disclosed that distinguishes roadways, center lines, white lines, curbs, and sidewalks from three-dimensional point cloud data (Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-135596 [Patent Document 2] Japanese Patent Application Publication No. 2019-190975 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, progress has been made in practical application of mobile objects that can move not only on roadways but also on sidewalks, but conventional technology has not fully considered the possibility of misrecognizing white lines, especially in spaces involving sidewalks.

[0005] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a white line recognition device, a control system for a moving object, a white line recognition method, and a program that can suppress erroneous recognition of white lines. [Means for solving the problem]

[0006] The white line recognition device, the control system for a moving object, the white line recognition method, and the program according to the present invention employ the following configurations. (1): A white line recognition device according to one embodiment of the present invention includes a white line recognition unit that recognizes linear features present in an image captured by a monocular camera as the outline of a white line painted on the road surface, and a linear object detection unit that detects linear objects that are spaced apart from the road surface based on the output of a ranging sensor that is arranged to have a detection range at least in the same direction as the monocular camera, and the white line recognition unit is configured not to recognize the linear features that correspond to the position of the linear object as the outline of the white line.

[0007] (2): In the above aspect (1), the white line recognition unit projects the position of the linear object onto the image plane of the image to derive a linear area, and does not recognize the linear feature that matches the linear area as the outline of the white line.

[0008] (3): In the above-mentioned aspect (1) or (2), the white line recognition unit searches for the linear object in the spatial region corresponding to the recognized linear feature, and if the linear object is present in the spatial region, the linear feature is not recognized as the outline of the white line.

[0009] (4): In any of the above aspects (1) to (3), the output of the distance measurement sensor is information that allows color recognition, and the white line recognition unit prevents the linear feature corresponding to the position of the white linear object from being recognized as the outline of the white line.

[0010] (5): A control system for a moving body comprising a white line recognition device according to any one of the aspects (1) to (4) above, and a control device that controls the movement of the moving body based on the white line recognized by the white line recognition unit.

[0011] (6): Another aspect of the white line recognition method of the present invention is a white line recognition method executed using a white line recognition device, and includes the steps of: recognizing linear features present in an image captured by a monocular camera as the outline of a white line drawn on the road surface; and detecting linear objects that are spaced apart from the road surface based on the output of a ranging sensor that is provided so as to have a detection range at least in the same direction as the monocular camera, wherein the recognition step includes preventing the linear features from being recognized as the outline of the white line.

[0012] (7): Another aspect of the present invention is a program that causes a computer to recognize linear features present in an image captured by a monocular camera as the outline of a white line painted on the road surface, and detect linear objects that are spaced apart from the road surface based on the output of a ranging sensor that is configured to have a detection range at least in the same direction as the monocular camera, wherein the recognition includes preventing the linear features from being recognized as the outline of the white line. [Effects of the Invention]

[0013] According to the aspects (1) to (7), it is possible to suppress erroneous recognition of white lines. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a diagram illustrating an example of the configuration of a moving object 1 and a control device 100 according to an embodiment. [Figure 2] FIG. 2 is a perspective view of the moving body 1 seen from above. [Figure 3] FIG. 10 is a diagram illustrating a scene in which a white line drawn on a roadway is recognized through the boundary between the sidewalk and the roadway. [Figure 4] 4 is a diagram showing an example of a captured image IM1 captured by a monocular camera 10A of a moving object 1 in the scene shown in FIG. [Figure 5] FIG. 10 is a diagram illustrating Method 1. [Figure 6] FIG. 10 is a diagram illustrating Method 2. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, with reference to the drawings, embodiments of a white line recognition device, a control system for a mobile body, a white line recognition method, and a program according to the present invention will be described. The white line recognition device is mounted on, for example, a mobile body. A mobile body is, for example, one that moves both on a roadway and a predetermined area different from the roadway. A mobile body is sometimes referred to as micromobility. An electric kick scooter is a type of micromobility. A mobile body may also be a vehicle that can accommodate a passenger, or may be an autonomous mobile body that can travel autonomously without a driver. The latter autonomous mobile body is used, for example, for transporting luggage, etc. Alternatively, the mobile body may be a vehicle or a motorcycle that travels exclusively on a roadway. The white line recognition device is particularly suitable for use on a mobile body that travels on a sidewalk.

[0016] The specified area is, for example, a sidewalk. The specified area may also be part or all of a roadside strip, bicycle lane, public open space, etc., or may include all of the sidewalk, roadside strip, bicycle lane, public open space, etc. In the following description, the specified area is assumed to be a sidewalk. In the following description, the term "sidewalk" can be read as "specified area" as appropriate.

[0017] FIG. 1 is a diagram illustrating an example of the configuration of a mobile object 1 and a control device 100 according to an embodiment. The mobile object 1 is equipped with, for example, an external environment detection device 10, a mobile object sensor 12, an operator 14, an internal camera 16, a positioning device 18, a mode selector switch 22, a movement mechanism 30, a drive unit 40, an external notification device 50, a storage device 70, and a control device 100. Note that some of these components that are not essential for realizing the functions of the present invention may be omitted. The mobile object is not limited to a vehicle, and may include a small mobility device that runs alongside a walking user to carry luggage or lead a person, as well as other mobile objects capable of autonomous movement (e.g., a walking robot).

[0018] The external environment sensing device 10 is a device of various types whose sensing range is in the traveling direction of the moving object 1. The external environment sensing device 10 includes, for example, a monocular camera 10A and a distance measuring sensor 10B. The external environment sensing device 10 may further include a radar device, a LIDAR (Light Detection and Ranging), a sensor fusion device, etc. The external environment sensing device 10 outputs information indicating the detection result (images, object positions, etc.) to the control device 100.

[0019] Monocular camera 10A captures an image of the space in the traveling direction of moving object 1 and outputs an RGB image or a black and white image. The image output by monocular camera 10A is a projection of reflecting objects in the space onto a two-dimensional image space.

[0020] Distance measurement sensor 10B has a detection range of the space in the same direction as monocular camera 10A, and outputs spatial information. For example, distance measurement sensor 10B is a stereo camera, and the spatial information is a distance image in which the distance from distance measurement sensor 10B is added for each pixel (reflection point). The distance image is in the form of an image in data format, but represents information of three-dimensional space. Monocular camera 10A may also be used as one of the cameras constituting the stereo camera. Distance measurement sensor 10B may also be a LIDAR, and the spatial information may be three-dimensional point cloud data. In either case, color or brightness information may or may not be added to the points (pixels) constituting the spatial information. In the following explanation, it is assumed that distance measurement sensor 10B is a stereo camera.

[0021] The mobile body sensor 12 includes, for example, a speed sensor, an acceleration sensor, a yaw rate (angular velocity) sensor, a direction sensor, and an operation amount detection sensor attached to the operator 14. The operator 14 includes, for example, an operator for instructing acceleration / deceleration (e.g., an accelerator pedal or a brake pedal), and an operator for instructing steering (e.g., a steering wheel). In this case, the mobile body sensor 12 may include an accelerator opening sensor, a brake depression amount sensor, a steering torque sensor, etc. The mobile body 1 may also be provided with an operator of a type other than those described above as the operator 14 (e.g., a non-annular rotary operator, a joystick, a button, etc.).

[0022] Internal camera 16 captures an image of at least the head of an occupant of vehicle 1 from the front. Internal camera 16 is a digital camera that uses an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). Internal camera 16 outputs the captured image to control device 100.

[0023] The positioning device 18 is a device that measures the position of the mobile object 1. The positioning device 18 is, for example, a Global Navigation Satellite System (GNSS) receiver, and identifies the position of the mobile object 1 based on signals received from GNSS satellites and outputs the position information. Note that the position information of the mobile object 1 may be estimated from the position of a Wi-Fi base station to which a communication device (described later) is connected.

[0024] The mode selector switch 22 is a switch operated by the occupant. The mode selector switch 22 may be a mechanical switch or a GUI (Graphical User Interface) switch set on a touch panel. The mode selector switch 22 accepts an operation to switch the driving mode between, for example, Mode A: an assist mode in which one of steering operation and acceleration / deceleration control is performed by the occupant and the other is performed automatically; Mode A-1: ​​in which steering operation is performed by the occupant and acceleration / deceleration control is performed automatically; Mode A-2: in which acceleration / deceleration operation is performed by the occupant and steering control is performed automatically; Mode B: a manual driving mode in which steering operation and acceleration / deceleration operation are performed by the occupant; and Mode C: an automatic driving mode in which operation control and acceleration / deceleration control are performed automatically.

[0025] The locomotion mechanism 30 is a mechanism for moving the mobile object 1 on a road. The locomotion mechanism 30 is, for example, a group of wheels including steering wheels and drive wheels. The locomotion mechanism 30 may also be legs for multi-legged walking.

[0026] The drive unit 40 outputs force to the movement mechanism 30 to move the moving body 1. For example, the drive unit 40 includes a motor that drives the drive wheels, a battery that stores power to be supplied to the motor, a steering device that adjusts the steering angle of the steering wheels, etc. The drive unit 40 may also include an internal combustion engine or a fuel cell as a driving force output means or a power generation means. The drive unit 40 may also include a brake device that utilizes frictional force or air resistance.

[0027] The external notification device 50 is, for example, a lamp, a display device, a speaker, or the like, provided on an outer panel of the mobile object 1, and configured to notify information to the outside of the mobile object 1. The external notification device 50 operates differently depending on whether the mobile object 1 is traveling on a sidewalk or a roadway. For example, the external notification device 50 is controlled to emit a lamp when the mobile object 1 is traveling on a sidewalk and not emit a lamp when the mobile object 1 is traveling on a roadway. The light color of this lamp is preferably a color specified by law. The external notification device 50 may be controlled to emit a green lamp when the mobile object 1 is traveling on a sidewalk and emit a blue lamp when the mobile object 1 is traveling on a roadway. If the external notification device 50 is a display device, the external notification device 50 displays a message in text or graphics indicating that the mobile object 1 is traveling on a sidewalk when the mobile object 1 is traveling on a sidewalk.

[0028] FIG. 2 is a perspective view of the moving body 1 as seen from above. In the figure, FW denotes steering wheels, RW denotes drive wheels, SD denotes a steering device, MT denotes a motor, and BT denotes a battery. The steering device SD, motor MT, and battery BT are included in a drive device 40. AP denotes an accelerator pedal, BP denotes a brake pedal, WH denotes a steering wheel, SP denotes a speaker, and MC denotes a microphone. The moving body 1 shown in the figure is a single-seater moving body, and an occupant P is seated in the driver's seat DS and wearing a seat belt SB. Arrow D1 indicates the direction of travel (velocity vector) of the moving body 1. The external environment detection device 10 is located near the front end of the moving body 1, the internal camera 16 is located in a position where it can capture an image of the occupant P's head from in front of the occupant P, and the mode selector switch 22 is located in the boss portion of the steering wheel WH. An external notification device 50 serving as a display device is also located near the front end of the moving body 1.

[0029] Returning to FIG. 1 , the storage device 70 is a non-transitory storage device such as a hard disk drive (HDD), flash memory, or random access memory (RAM). The storage device 70 stores map information 72, a program 74 executed by the control device 100, correspondence information 76 (described later), and the like. In the figure, the storage device 70 is shown outside the frame of the control device 100, but the storage device 70 may be included in the control device 100. The storage device 70 may also be provided on a server (not shown).

[0030] [Control device] The control device 100 includes, for example, a road type recognition unit 120, an object recognition unit 130, and a control unit 140. The object recognition unit 130 includes a white line recognition unit 132 and a linear object detection unit 134. These components are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software) 74. Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in the storage device 70 in advance, or may be stored in a removable storage medium (non-transitory storage medium) such as a DVD or CD-ROM, and installed in the storage device 70 by inserting the storage medium into a drive device. The combination of the white line recognition unit 132 and the linear object detection unit 134 is an example of a white line recognition device. In the embodiment, the white line recognition device is an internal function of the control device 100, but the white line recognition device may be an independent device.

[0031] The road type recognition unit 120 recognizes whether the moving object 1 is moving on a roadway or a sidewalk. The road type recognition unit 120 recognizes whether the moving object 1 is moving on a roadway or a sidewalk, for example, by analyzing an image captured by an external camera of the external environment detection device 10. Note that the output of a radar device, a LIDAR, a sensor fusion device, etc. may be used supplementarily.

[0032] The road type recognition unit 120 adds points to the roadway score Sr each time it recognizes a plurality of first events indicating that the moving object 1 is moving on a roadway in an image captured by the external camera. If the roadway score Sr is equal to or greater than a first threshold, it recognizes that the moving object 1 is moving on a roadway. Examples of first events include: (A) the absence of static obstacles other than a vehicle in the own area (i.e., the area inside the outer edge of the area where the moving object 1 is located); (B) the presence of a vehicle moving in the own area; (C) the presence of road markings on the road surface of the own area; (D) the presence of a pedestrian crossing in the own area; and (E) the own area being on the lower side of a step. The road type recognition unit 120 weights points according to the degree of certainty when recognizing each of the plurality of first events and adds the weighted points to the roadway score Sr. Points corresponding to the first events are designated P1 to Pn (n is a natural number equal to the number of events assumed as first events). The points P1 to Pn may have the same value, or may have different values ​​depending on the type of the first event. In addition, weights according to the confidence level of the recognition process are set to α1 to αn. The confidence level of the recognition process is output incidentally during the process of recognizing the first event (including, for example, a discrimination process using a trained model by machine learning). The roadway score Sr is expressed by equation (1).

[0033] Sr=α1×P1+α1+P2+…+αn×Pn…(1) (If the corresponding first event k is not recognized, Pk = 0 (k = 1 to n))

[0034] However, when the road type recognition unit 120 recognizes any one of a plurality of second events indicating that the moving object 1 is moving on a sidewalk in the image captured by the external camera, the road type recognition unit 120 recognizes that the moving object 1 is moving on a sidewalk regardless of the roadway score Sr. The second events are, for example, (a) the external camera capturing an image of the second of the two faces of a guardrail that faces the sidewalk, (b) the presence of tactile paving blocks on the road surface in the own area, (c) the presence of a static obstacle other than a vehicle in the own area, (d) the own area being on the upper side of a step, etc.

[0035] The road type recognition unit 120 may compare the position information of the mobile object 1 with the map information 72 to recognize whether the mobile object 1 is moving on a roadway or a sidewalk. In this case, the map information must be accurate enough to distinguish between a sidewalk and a roadway from the position coordinates. Furthermore, if the "predetermined area" is not limited to sidewalks, the road type recognition unit 120 performs similar processing for shoulders, bicycle lanes, public open spaces, etc.

[0036] The object recognition unit 130 recognizes objects present around the moving object 1 based on the output of the external environment detection device 10. Objects include some or all of the following obstacles: moving objects such as vehicles, bicycles, and pedestrians; road boundary structures such as road markings, steps, guardrails, road shoulders, and median strips; road structures such as road signs and billboards; and objects lying (fallen) on the road. The object recognition unit 130 acquires information about the presence, position, and type of other moving objects by inputting images captured by an external camera of the external environment detection device 10 into a trained model that is trained to output information about the presence, position, and type of an object when an image captured by the external camera is input. The type of other moving objects can also be estimated based on the size in the image or the intensity of reflected waves received by the radar device of the external environment detection device 10. The object recognition unit 130 also acquires the speed of other moving objects detected by the radar device using, for example, Doppler shift. The functions of the white line recognition unit 132 and the linear object detection unit 134 will be described later.

[0037] The control unit 140 controls the drive unit 40 according to the set driving mode, for example. The moving object 1 may execute only some of the driving modes described below, but in all cases, the control unit 140 sets different speed limits for when the moving object 1 moves on a roadway and when it moves on a sidewalk. In this case, the mode selector switch 22 may be omitted.

[0038] In mode A-1, the control unit 140 refers to the information about the travel path and the object based on the output of the object recognition unit 130, and controls the motor MT of the drive unit 40 so that when the moving body 1 moves on a roadway, the distance to an object ahead of the moving body 1 is maintained at a certain level or more, and when the distance to the object ahead of the moving body 1 is sufficiently long, the moving body 1 moves at a first speed V1 (for example, a speed of at least 10 km / h and less than several tens of km / h). When the moving body 1 moves on a sidewalk, the control unit 140 controls the motor MT of the drive unit 40 so that the distance to an object ahead of the moving body 1 is maintained at a certain level or more, and when the distance to the object ahead of the moving body 1 is sufficiently long, the moving body 1 moves at a second speed V2 (for example, a speed of less than 10 km / h). This function is similar to the adaptive cruise control (ACC) function of a vehicle in which the first speed V1 or the second speed V2 is set as a set speed, and the technology used in ACC can be utilized. In mode A-1, the control unit 140 controls the steering device SD to change the steering angle of the steered wheels based on the amount of operation of the operator 14, such as a steering wheel. This function is similar to that of a power steering device, and the technology used in power steering devices can be used. Note that, instead of electronically controlling steering, the mobile object 1 may have a steering device in which the operator 14 and a steering mechanism are mechanically connected.

[0039] In mode A-2, the control unit 140 references information about the traveling path and objects based on the output of the object recognition unit 130, generates a target trajectory that allows the moving object 1 to avoid objects on the traveling path, and controls the steering device SD of the drive unit 40 so that the moving object 1 moves along the target trajectory. Regarding acceleration and deceleration, the control unit 140 controls the motor MT of the drive unit 40 based on the speed of the moving object 1 and the amount of operation of the accelerator pedal or the brake pedal. When the moving object 1 is traveling on a roadway, the control unit 140 controls the motor MT of the drive unit 40 with a first speed V1 as the upper limit speed (in mode A-2, this means that the moving object 1 will not accelerate even if a further acceleration command is given once the upper limit speed has been reached), and controls the drive unit 40 with a second speed V2 as the upper limit speed when the moving object 1 is traveling on a sidewalk.

[0040] In mode B, the control unit 140 controls the motor MT of the drive unit 40 based on the speed of the moving object 1 and the amount of operation of the accelerator pedal or the brake pedal. When the moving object 1 is moving on a roadway, the control unit 140 controls the motor MT of the drive unit 40 with the first speed V1 as the upper limit speed (in mode B, this means that the moving object 1 will not accelerate even if a further acceleration command is given once the upper limit speed has been reached), and when the moving object 1 is moving on a sidewalk, the control unit 140 controls the motor MT of the drive unit 40 with the second speed V2 as the upper limit speed. Steering is the same as in mode A-1.

[0041] In mode C, the control unit 140 references information about the path and objects based on the output of the object recognition unit 130, generates a target trajectory along which the moving body 1 can move while avoiding objects within the path, and controls the drive unit 40 so that the moving body 1 moves along the target trajectory. Even in mode C, the control unit 140 controls the drive unit 40 with a first speed V1 as the upper limit speed when the moving body 1 is moving on a roadway, and controls the drive unit 40 with a second speed V2 as the upper limit speed when the moving body 1 is moving on a sidewalk.

[0042] [White line recognition] The functions of the white line recognition unit 132 and the linear object detection unit 134 will be described below. The white line recognition unit 132 extracts horizontal edges whose luminance difference in the left-right direction is equal to or greater than a threshold in a region of interest in an image captured by the monocular camera 10A, and detects linear or curved horizontal edges that resemble straight or curved lines and linear characteristic locations extending toward the top dead center of the image as candidate contours of white lines. The white line recognition unit 132 then recognizes, as white lines, regions that are surrounded by the candidate contours and whose width in the short direction is within the range assumed for a white line, and outputs information projecting the positions of the white lines from the image plane of the captured image onto a virtual plane viewed from above the moving object 1 to the control unit 140.

[0043] Here, the mobile object 1 is capable of moving on a sidewalk, and a situation is assumed in which the mobile object 1 is about to move from the sidewalk to a roadway. Therefore, the mobile object 1 may recognize a white line drawn on the roadway through the boundary between the sidewalk and the roadway. FIG. 3 is a diagram illustrating such a situation. In this situation, the mobile object 1 is moving on a sidewalk 200 and is about to move onto the roadway 202 through a connecting section 206 where a step 204 between the sidewalk 200 and the roadway 202 is reduced. A guardrail 208 is provided on the sidewalk side of the step 204. The guardrail 208 is a thin fence provided at the boundary between the sidewalk and the roadway. In addition to such a situation, a situation is also assumed in which the mobile object 1 moves from a roadway to the sidewalk and then moves onto the opposite roadway. In this way, the mobile object 1 may recognize a white line in a space involving a sidewalk.

[0044] Fig. 4 is a diagram showing an example of a captured image IM1 captured by the monocular camera 10A of the moving object 1 in the scene shown in Fig. 3. The guard pipe 208 is composed of, for example, multiple support posts 208A and one or more main pipe sections 208B (three in the figure: 208B1, 208B2, and 208B3) suspended horizontally between the support posts 208A. Three white lines 210-1, 210-2, and 210-3 are visible in the captured image IM1 (white line 210-2 is shown as a dashed line). Each of the main pipe sections 208B has characteristics that make them easily confused with white lines in computer image processing for reasons such as (1) they extend along the road, (2) their width as seen from the sidewalk is similar to that of a white line, and (3) many of them are white. For this reason, after moving from the sidewalk 200 onto the roadway 202, it may be impossible to correctly recognize the positions of the left and right white lines 210-1, 210-2 that should be used as reference for controlling the movement of the moving object 1 when moving within the roadway 202.

[0045] Therefore, in this embodiment, the linear object detection unit 134 detects a linear object that exists apart from the road surface (i.e., floating in the air), and the white line recognition unit 132 prevents the linear feature corresponding to the position of the linear object from being recognized as the outline of a white line. This prevents erroneous recognition of white lines caused by objects that are easily confused with white lines, such as the main pipe 208B. "Preventing recognition as the outline of a white line" generally means not recognizing the object as the outline of a white line. For example, if only the middle portion of a linear feature corresponds to the position of a linear object and the ends do not, it is likely that the position of the white line as seen by the monocular camera 10A coincides with the linear object. In such a case, the white line recognition unit 132 does not immediately determine that the object is "not the outline of a white line." Instead, if the probability that the object is the outline of a white line is sufficiently high based on its connection with the surrounding area, the white line recognition unit 132 may recognize the object as the outline of a white line. Hereinafter, "preventing recognition as the outline of a white line" is also referred to as "excluding from the recognition result."

[0046] The linear object detection unit 134 refers to the correspondence information 76 stored in the storage device 70 and detects a linear object based on the spatial information output by the distance measurement sensor 10B.

[0047] When the spatial information is in the form of a distance image (hereinafter referred to as a spatial information image), the corresponding information corresponds to each coordinate of the spatial information image and is information that indicates the distance when it is assumed that the road surface has been detected as a light reflection point (hereinafter referred to as reference distance 1). In this case, the linear object detection unit 134 extracts coordinates whose distance is shorter than reference distance 1 from each coordinate of the spatial information image, and detects, as a linear object, a target formed by a group of coordinates extending in a straight line or curved line with a predetermined width.

[0048] When the spatial information is in the form of a three-dimensional point cloud, the correspondence information is information that indicates the direction (azimuth angle, depression angle, or elevation angle) as seen from the distance measuring sensor 10B and the distance when it is assumed that the road surface is detected as a light reflection point (hereinafter, reference distance 2). In this case, the linear object detection unit 134 extracts points whose distance is shorter than reference distance 2 from each point that constitutes the three-dimensional point cloud, and detects a target that is constituted by a group of points extending in a straight line or curved line with a predetermined width as a linear object.

[0049] When the spatial information includes color information, if the linear object detection unit 134 detects a non-white linear object, the white line recognition unit 132 does not need to perform any special processing to exclude the non-white linear object from the recognition result, because the probability that a non-white linear object will be confused with a white line is low.

[0050] There are two possible methods for excluding linear feature points corresponding to the positions of linear objects from the recognition results. (Method 1) For example, the white line recognition unit 132 projects the positions of the linear objects onto the image plane of the captured image IM1 to derive a linear region, and then excludes linear feature points that match the linear region from the recognition results. FIG. 5 is a diagram for explaining Method 1. Parameters required for the projection include the mounting position, optical axis direction, and distortion degree of the monocular camera 10A, and the mounting position, optical axis direction, and distortion degree of the distance measurement sensor 10B. The white line recognition unit 132 performs the projection using a projection function calculated from the physical relationship between these parameters. The position of the linear object may be represented by the coordinate axes of a spatial information image or by the coordinate axes of a three-dimensional space. In either case, projecting the linear object onto the image plane of the captured image IM1 makes it possible to obtain information on "where the linear object would be if it were on the captured image IM1."

[0051] (Method 2) The white line recognition unit 132 searches for a linear object in a spatial region (a region on the coordinate axes constituting the spatial information; obtained by performing the inverse process of the above-mentioned projection) corresponding to the white line on the recognized captured image IM1, and if a linear object is present in the spatial region, the linear feature location is excluded from the recognition result. FIG. 6 is a diagram for explaining method 2. As shown in the figure, the white line recognition unit 132 searches for a linear object in a search region obtained by inversely projecting the position of the linear feature location in the captured image IM1 onto the spatial information, and if a linear object is present, the linear feature location corresponding to the hit search region is excluded from the recognition result.

[0052] The recognition result of the white lines, in which erroneous recognition is reduced by excluding linear objects in this way, is used by the control unit 140 to control the drive device 40. For example, in mode A-2 or mode C, the control unit 140 generates a target trajectory so that the vehicle stays within the lane defined by the recognized white lines.

[0053] According to the embodiment described above, there is provided a white line recognition unit 132 that recognizes white lines drawn on the road surface in an image IM1 captured by the monocular camera 10A, and a linear object detection unit 134 that detects linear objects that are spaced apart from the road surface based on the output of a distance measurement sensor 10B that is arranged to have a detection range at least in the same direction as the monocular camera 10A. The white line recognition unit 132 excludes white lines that correspond to the positions of linear objects from the recognition results, thereby suppressing erroneous recognition of white lines.

[0054] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: Recognizing linear feature points present in the image captured by the monocular camera as the outlines of white lines painted on the road surface; and detecting a linear object that is present at a distance from the road surface based on an output of a distance measuring sensor that is provided so as to have a detection range in at least the same direction as the monocular camera; The recognizing includes preventing the linear characteristic portion from being recognized as the outline of the white line. White line recognition device.

[0055] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0056] 10. External sensing devices 10A Monocular Camera 10B Distance Sensor 12 Mobile Sensor 14 Controls 16 Internal Camera 18 Positioning equipment 22 Mode switch 30 Moving mechanism 40 Drive unit 50 External alarm device 70 Storage device 76 Compatibility Information 100 control device 120 Road type recognition unit 130 Object recognition section 132 White line recognition section 134 Linear object detection unit 140 Control Unit

Claims

1. a white line recognition unit that recognizes linear feature points present in the image captured by the monocular camera as the outlines of white lines painted on the road surface; a linear object detection unit that detects a linear object that exists at a distance from the road surface based on an output of a distance measuring sensor that is provided so as to have a detection range in at least the same direction as the monocular camera, the white line recognition unit derives a linear area by projecting the position of the linear object onto an image plane of the image, and prevents the linear characteristic portion that matches the linear area from being recognized as the outline of the white line. White line recognition device.

2. A white line recognition unit that recognizes linear feature points present in an image captured by a monocular camera as the outline of a white line painted on the road surface; a linear object detection unit that detects a linear object that exists at a distance from the road surface based on an output of a distance measuring sensor that is provided so as to have a detection range in at least the same direction as the monocular camera, the white line recognition unit searches for the linear object in a spatial region corresponding to the recognized linear characteristic portion, and when the linear object is present in the spatial region, the linear characteristic portion is not recognized as the contour of the white line. White line recognition device.

3. A white line recognition unit that recognizes linear feature points present in an image captured by a monocular camera as the outline of a white line painted on the road surface; a linear object detection unit that detects a linear object that exists at a distance from the road surface based on an output of a distance measuring sensor that is provided so as to have a detection range in at least the same direction as the monocular camera, The output of the distance measuring sensor is information that allows color recognition, the white line recognition unit prevents the linear characteristic portion corresponding to the position of the white linear object from being recognized as the outline of the white line. White line recognition device.

4. A white line recognition device according to any one of claims 1 to 3; a control device that controls the movement of a moving object based on the white line recognized by the white line recognition unit; A control system for a moving object comprising:

5. A white line recognition method executed using a white line recognition device, Recognizing linear feature points present in the image captured by the monocular camera as the outlines of white lines painted on the road surface; detecting a linear object that is present at a distance from the road surface based on an output of a distance measuring sensor that is provided so as to have a detection range in at least the same direction as the monocular camera; The recognizing step includes deriving a linear region by projecting the position of the linear object onto an image plane of the image, and preventing the linear feature portion that matches the linear region from being recognized as the outline of the white line. White line recognition method.

6. On the computer, Recognizing linear feature points present in the image captured by the monocular camera as the outlines of white lines painted on the road surface; and detecting a linear object that is spaced apart from the road surface based on an output of a distance measuring sensor that is provided so as to have a detection range in at least the same direction as the monocular camera, The recognizing step includes deriving a linear region by projecting the position of the linear object onto an image plane of the image, and preventing the linear feature portion that matches the linear region from being recognized as the outline of the white line. program.

Images