Mobile control device, mobile control method, and program

Abstract

To provide a mobile body control device, a mobile body control method and a program capable of recognizing a compartment line compartmentalizing a travel lane with improved accuracy.SOLUTION: A mobile body control device comprises: a candidate compartment line recognition section which recognizes candidate compartment lines compartmentalizing a travel lane where a mobile body travels thereon from a first image, including surroundings of the mobile body, taken through an imaging section; and a compartment line search section determining the candidate compartment line, among the candidate compartment lines recognized with the candidate compartment line recognition section, which is the closest to a past compartment line compartmentalizing the travel lane where the mobile body traveled thereon predetermined time ago as the compartment line compartmentalizing the current travel lane of the mobile body.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a movement control device, a movement control method, and a program.

Background Art

[0002] In recent years, research on automatic driving for automatically controlling the running of vehicles has been underway. In this regard, based on the lanes around the vehicle detected by a sensor that detects the situation around the vehicle and the lanes around the vehicle acquired from map data based on the position and attitude of the vehicle, a technique for adjusting the lane arrangement is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, due to the instability of the recognition by a sensor that detects the situation around a moving object such as a vehicle, when there is a disappearance or break in the lane dividing lines on an actual road, more dividing lines may be recognized than the actual number of dividing lines. Therefore, there are cases where the dividing lines that demarcate the lane in which the host vehicle is traveling cannot be accurately recognized.

[0005] One of the objects of the embodiments of the present invention is to provide a movement control device, a movement control method, and a program that can more accurately recognize the dividing lines that demarcate the driving lane in consideration of such circumstances.

Means for Solving the Problems

[0006] The movement control device, movement control method, and program according to this invention adopt the following configuration. (1): The mobile body control device according to one aspect of the present invention includes a lane line candidate recognition unit that recognizes candidates for lane lines that demarcate the driving lane on which the mobile body travels from a first image including the periphery of the mobile body imaged by an imaging unit, and among the candidates for lane lines recognized by the lane line candidate recognition unit, Partition the travel lane of the moving body a lane line search unit that sets the lane line candidate closest to the past lane line as the lane line that demarcates the current driving lane of the mobile body, and a driving control unit that executes driving control by controlling one or both of the speed or steering of the mobile body so that the mobile body travels along the lane line that demarcates the current driving lane of the mobile body. The lane line search unit detects the proximity between the candidate lane line recognized by the lane line candidate recognition unit and the past lane line in an interval partially cut out based on a reference position for the candidate lane line and the past lane line, The past lane line is a lane line that has already been searched as the lane line that demarcates the driving lane of the mobile body by the lane line search unit at the time when the lane line candidate is recognized by the lane line candidate recognition unit. The lane line search unit sets the past lane line as the lane line of the current driving lane when the proximity of the lane line candidate closest to the past lane line among the candidates for lane lines recognized by the lane line candidate recognition unit is equal to or greater than a threshold value. The driving control unit continues the driving control using the past lane line even when the proximity of the lane line candidate closest to the past lane line is equal to or greater than the threshold value.

[0007] (2): In the aspect of (1) above, the past lane line is Before the first image is captured extracted from a second image including the periphery of the mobile body previously imaged by the imaging unit.

[0009] ( 3 ): In the aspect of (1) above, Or (2) when the lane line search unit compares each of the lane line candidates recognized by the lane line candidate recognition unit with the past lane line, it determines the lane line closest to the past lane line based on the distances from a predetermined number of points on the past lane line from a predetermined search start point to the lane line candidate.

[0010] ( 4 ): In the above ( 3In the aspect of (), the lane line search unit converts the first image and a second image including the periphery of the moving body imaged by the imaging unit before the first image is imaged, into an image of the moving body viewed from above, and based on the mean square value of the horizontal position difference between each candidate of the lane line included in the converted image and the past lane line, determines the lane line closest to the past lane line.

[0011] ( 5 ): In the aspect of the above ( 3 ), the search start point is set based on the parameter information of the imaging unit.

[0012] ( 6 ): In any one of the aspects of the above (1) to ( 5 ), the lane line search unit corrects the position of the past lane line based on one or both of the movement amount and movement direction of the moving body or the road shape.

[0013] ( 7 ): In the aspect of the above (1), the interface control unit further includes an interface control unit that displays driving support information including the driving lane of the moving body searched by lane line search using the past lane line and the candidate of the lane line on an interface provided in the moving body.

[0015] ( 8 ): A moving body control method according to an aspect of the present invention includes a computer of a moving body control device recognizing candidates for lane lines that divide a driving lane on which the moving body travels from a first image including the periphery of the moving body imaged by an imaging unit, and among the recognized candidates for the lane lines, Partition the travel lane of the moving body selecting the lane line candidate closest to the past lane line as the lane line that divides the current driving lane of the moving body, and performing driving control by controlling one or both of the speed or steering of the moving body so that the moving body travels along the lane line that divides the current driving lane of the moving body. Detect the proximity between the recognized candidate lane line and the past lane line in an interval partially cut out based on a reference position for the candidate lane line and the past lane line,The past lane line is a lane line that has already been searched as a lane line that demarcates the traveling lane of the moving body at the time when the candidate for the lane line is recognized. When the proximity of the lane line candidate closest to the past lane line among the recognized lane line candidates is equal to or greater than a threshold value, the past lane line is set as the lane line of the current traveling lane. Even when the proximity of the lane line candidate closest to the past lane line is equal to or greater than the threshold value, the driving control is continued using the past lane line. This is a moving body control method.

[0016] ( 9 ): A program according to one aspect of the present invention causes a computer of a moving body control device to recognize a candidate for a lane line that demarcates the traveling lane of the moving body from a first image including the periphery of the moving body captured by an imaging unit. Among the recognized lane line candidates, Partition the travel lane of the moving body the lane line candidate closest to the past lane line is set as the lane line that demarcates the current traveling lane of the moving body, and driving control is executed by controlling one or both of the speed or steering of the moving body so that the moving body travels along the lane line that demarcates the current traveling lane of the moving body. Cause the proximity between the recognized candidate lane line and the past lane line to be detected in an interval partially cut out based on a reference position for the candidate lane line and the past lane line, The past lane line is a lane line that has already been searched as a lane line that demarcates the traveling lane of the moving body at the time when the candidate for the lane line is recognized. When the proximity of the lane line candidate closest to the past lane line among the recognized lane line candidates is equal to or greater than a threshold value, the past lane line is set as the lane line of the current traveling lane. Even when the proximity of the lane line candidate closest to the past lane line is equal to or greater than the threshold value, the driving control is continued using the past lane line. This is a program.

Advantages of the Invention

[0017] According to the above aspects (1) to ( 9 ), it is possible to more accurately recognize the lane line that demarcates the host vehicle lane.

Brief Description of the Drawings

[0018]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0019] Hereinafter, with reference to the drawings, embodiments of the movement control device, movement control method, and program of the present invention will be described. The movement control device is a device that controls the movement of a moving body. The moving body includes vehicles such as three-wheeled or four-wheeled vehicles, two-wheeled vehicles, micromobility, etc., and may include any moving body that can move on a road surface where, for example, a person (occupant) rides and there is a driving lane such as a driving lane. In the following description, the moving body is assumed to be a four-wheeled vehicle and is referred to as "own vehicle M". Further, hereinafter, the own vehicle M will be mainly described as an autonomous driving vehicle. Autonomous driving is, for example, to execute a driving control that automatically controls one or both of the steering and speed of the own vehicle M. The driving control of the own vehicle M includes, for example, LKAS (Lane Keeping Assistance System), which is a driving assistance control for the own vehicle M to travel without deviating from the own lane (driving lane) in which the own vehicle M travels. Further, the driving control may include various driving assistance controls such as ALC (Auto Lane Changing) and ACC (Adaptive Cruise Control). Also, the autonomous driving vehicle may be controlled by manual driving of an occupant (driver). Alternatively, the drive source of the own vehicle M is, for example, an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using the generated electric power from a generator connected to the internal combustion engine, or the discharged electric power from a secondary battery or a fuel cell.

[0020] [Overall Configuration] FIG. 1 is a configuration diagram of a vehicle system 1 using a movement control device according to an embodiment. The vehicle system 1 includes, for example, a camera 10, a radar device 12, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driving operator 80, an automatic driving control device 100, a traveling driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. Note that the configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted, or another configuration may be added. The camera 10 is an example of an "imaging unit". The automatic driving control device 100 is an example of a "movement control device".

[0021] The camera 10 is, for example, a digital camera using a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary position of the host vehicle M on which the vehicle system 1 is mounted. When imaging the front, the camera 10 is attached to the upper part of the front windshield, the back surface of the rearview mirror, or the like. The camera 10, for example, periodically and repeatedly images the periphery of the host vehicle M. The camera 10 may be a stereo camera.

[0022] The radar device 12 radiates radio waves such as millimeter waves around the host vehicle M and detects radio waves (reflected waves) reflected by an object to detect at least the position (distance and azimuth) of the object. The radar device 12 is attached to an arbitrary position of the host vehicle M. The radar device 12 may detect the position and speed of an object by an FM-CW (Frequency Modulated Continuous Wave) method.

[0023] The object recognition device 16 analyzes an image representing the front situation of the host vehicle M captured by the camera 10 and extracts necessary information. Then, the object recognition device 16 performs sensor fusion processing on the detection results of the camera 10 and the radar device 12 to recognize the position, type, speed, etc. of the object, and outputs the recognition result to the automatic driving control device 100. In the present invention, the radar device 12 may be omitted. In this case, the object recognition device 16 may have only the function of analyzing the image. Further, without performing sensor fusion processing, the detection result of the radar device 12 may be directly output to the automatic driving control device 100. Further, the function of the object recognition device 16 may be included in the automatic driving control device 100 (more specifically, the recognition unit 130 described later). In this case, the object recognition device 16 may be omitted from the vehicle system 1.

[0024] The communication device 20 communicates with other vehicles existing around the host vehicle M by using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., or communicates with various server devices via a wireless base station.

[0025] The HMI 30 outputs various information to the passengers of the host vehicle M under the control of the HMI control unit 170. Further, the HMI 30 may function as a reception unit that receives input operations by the passengers. The HMI 30 includes, for example, a display device, a speaker, a microphone, a buzzer, a key, an indicator lamp, etc. The display device is, for example, an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display device, etc.

[0026] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around the vertical axis, a direction sensor that detects the direction of the host vehicle M, and the like. Further, the vehicle sensor 40 may include a position sensor that acquires the position of the host vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. Further, the position sensor may be a sensor that acquires position information using the GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50.

[0027] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 holds first map information 54 in a storage device such as an HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 identifies the position of the host vehicle M based on signals received from GNSS satellites. The position of the host vehicle M may be identified or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The navigation HMI 52 may share some or all of the components with the aforementioned HMI 30. The route determination unit 53 determines, for example, a route (hereinafter, the on-map route) from the position of the host vehicle M identified by the GNSS receiver 51 (or an arbitrary input position) to the destination input by the occupant using the navigation HMI 52 with reference to the first map information 54. The first map information 54 is information in which the road shape is represented by, for example, links indicating roads and nodes connected by the links. The first map information 54 may include information such as the curvature of the road and POI (Point Of Interest) information. The on-map route is output to the MPU 60. The navigation device 50 may perform route guidance using the navigation HMI 52 based on the on-map route. The navigation device 50 may be realized by, for example, the functions of a terminal device such as a smartphone or a tablet terminal held by the occupant. The navigation device 50 may transmit the current position and the destination to the navigation server via the communication device 20 and acquire a route equivalent to the on-map route from the navigation server.

[0028] The MPU60 includes, for example, a recommended lane determination unit 61 and holds second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route on the map provided from the navigation device 50 into a plurality of blocks (for example, divides every 100 [m] in the vehicle traveling direction) and determines the recommended lane for each block with reference to the second map information 62. For example, when the currently traveling lane or the road to be traveled in the near future has a plurality of lanes, the recommended lane determination unit 61 makes a determination as to which lane from the left to travel in. Further, when there is a branch point in the route on the map, the recommended lane determination unit 61 determines the recommended lane so that the host vehicle M can travel on a reasonable route for proceeding to the branch destination.

[0029] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of the lane or information on the boundary of the lane (for example, road markings). Further, the second map information 62 may include road information (road type), the number of lanes of the road, presence or absence of branch and merge, legal speed (speed limit, maximum speed, minimum speed), traffic regulation information, address information (address · postal code), facility information, telephone number information, and the like. The second map information 62 may be updated at any time when the communication device 20 communicates with other devices.

[0030] The driving operator 80 includes, for example, a steering wheel, an accelerator pedal, a brake pedal, a shift lever, and other operators. A sensor for detecting the operation amount or the presence or absence of an operation is attached to the driving operator 80, and the detection result is output to a part or all of the automatic driving control device 100, the traveling driving force output device 200, the brake device 210, and the steering device 220.

[0031] Next, prior to the description of the automatic driving control device 100, the traveling driving force output device 200, the brake device 210, and the steering device 220 will be described. The traveling driving force output device 200 outputs the traveling driving force (torque) for the host vehicle M to travel to the drive wheels. The traveling driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above configuration according to information input from the automatic driving control device 100 (specifically, the second control unit 160 described later) or information input from the operation operator 80.

[0032] The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the operation operator 80, so that brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include, as a backup, a mechanism that transmits the hydraulic pressure generated by the operation of the brake pedal included in the operation operator 80 to the cylinder via the master cylinder. Note that the brake device 210 is not limited to the configuration described above, and may be an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 and transmits the hydraulic pressure of the master cylinder to the cylinder.

[0033] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies a force to a rack and pinion mechanism to change the direction of the steered wheels. The steering ECU drives the electric motor according to information input from the second control unit 160 or information input from the steering wheel 82 of the operation operator 80, and changes the direction of the steered wheels.

[0034] Next, the automatic driving control device 100 will be described. The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 170, and a storage unit 180. The first control unit 120, the second control unit 160, and the HMI control unit 170 are each realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Also, some or all of these components may be realized by hardware (including a circuit unit; circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device including a non-transitory storage medium) such as an HDD or a flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD or a CD-ROM, and may be installed in the HDD or flash memory of the automatic driving control device 100 when the storage medium (non-transitory storage medium) is mounted on a drive device. The combination of the action plan generation unit 140 and the second control unit 160 is an example of a "driving control unit".

[0035] The storage unit 180 may be implemented by the above-described various storage devices, or an SSD (Solid State Drive), EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory), etc. The storage unit 180 stores, for example, image information 182, lane line information 184, information necessary for executing various processes in the present embodiment, programs, and other various information. The image information 182 is information on a peripheral image of the host vehicle M (an image including at least the traveling direction of the host vehicle M) captured by the camera 10. Hereinafter, the image captured by the camera 10 is referred to as a "camera image". Further, in the image information 182, the imaging time by the camera 10 and the position and direction of the host vehicle M at the time of shooting may be associated with the camera image. The image information 182 stores a camera image before a predetermined time (before a predetermined image frame). The predetermined time in this case may be a fixed time or a variable time according to the speed of the host vehicle M or the road shape. Hereinafter, the camera image before the predetermined time is referred to as a "past camera image". The past camera image is an example of the "second image". The lane line information 184 is information on road lane lines (hereinafter simply referred to as "lane lines") that divide lanes on the road recognized by the recognition unit 130 a predetermined time ago. The lane line information 184 may include, for example, position information of a lane (hereinafter referred to as the "own lane") in which the host vehicle M travels obtained from the analysis result of a past camera image. In this case, the storage unit 180 holds lane lines that divide the own lane in the past camera image. Further, map information (first map information 54, second map information 62) may be stored in the storage unit 180.

[0036] Figure 2 is a functional block diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120 realizes, for example, functions by AI (Artificial Intelligence) and functions by a pre-provided model in parallel. For example, the function of "recognizing an intersection" may be realized by executing the recognition of an intersection by deep learning or the like and the recognition based on pre-provided conditions (such as signals capable of pattern matching, road markings, etc.) in parallel, scoring both, and comprehensively evaluating them. Thereby, the reliability of the automatic driving is ensured.

[0037] Based on the information input from the camera 10 and the radar device 12 via the object recognition device 16, the recognition unit 130 recognizes the position (relative position), speed (relative speed), acceleration, and other states of the objects (such as other vehicles and other obstacles) around the host vehicle M. The position of the object is recognized, for example, as the position on the absolute coordinates (host vehicle center coordinate system) with the representative point (such as the center of gravity or the center of the drive shaft) of the host vehicle M as the origin, and is used for control. The position of the object may be represented by the representative point such as the center of gravity or the corner of the object, or may be represented by a region. The "state" of the object may include, when the object is a moving body such as another vehicle, the acceleration, jerk, or "behavior state" (such as whether it is changing lanes or about to change lanes) of the other vehicle.

[0038] Also, the recognition unit 130 recognizes the host vehicle lane based on, for example, at least the information input from the camera 10. Specifically, the recognition unit 130 includes, for example, a point cloud acquisition unit 132, a lane line candidate recognition unit 134, and a lane line search unit 136, and recognizes the lane lines that divide the host vehicle lane and the host vehicle lane itself by each function. The details of each function will be described later.

[0039] In addition to (or instead of) recognizing the host vehicle lane by the functions of the point cloud acquisition unit 132, the lane line candidate recognition unit 134, and the lane line search unit 136, the recognition unit 130 may recognize the lane line that divides the host vehicle lane and the host vehicle lane itself by comparing the pattern of the lane line (e.g., the arrangement of solid lines and broken lines) obtained from the second map information 62 with the pattern of the lane line around the host vehicle M recognized from the image captured by the camera 10. Further, the recognition unit 130 may recognize not only lane lines but also driving boundaries (road boundaries) including road shoulders, curbs, median strips, guardrails, etc. In this recognition, the position of the host vehicle M acquired from the navigation device 50 and the processing result by the INS may be taken into account. Further, the recognition unit 130 recognizes a stop line, an obstacle, a red signal, a toll gate, a road sign, and other road events. Further, the recognition unit 130 may recognize an adjacent lane adjacent to the host vehicle lane or an oncoming lane facing the host vehicle lane. The adjacent lane is, for example, a lane that can proceed in the same direction as the host vehicle lane driving lane.

[0040] Further, when the recognition unit 130 recognizes the host vehicle lane, it may recognize the position and posture of the host vehicle M with respect to the host vehicle lane. For example, the recognition unit 130 may recognize the deviation of the reference point of the host vehicle M from the center of the lane and the angle formed with respect to the line connecting the centers of the lanes in the traveling direction of the host vehicle M as the relative position and posture of the host vehicle M with respect to the host vehicle lane. Instead of this, the recognition unit 130 may recognize the position of the reference point of the host vehicle M with respect to any side end (e.g., a lane line or a road boundary) of the host vehicle lane as the relative position of the host vehicle M with respect to the traveling lane. Here, the reference point of the host vehicle M may be the center of the host vehicle M or the center of gravity. Further, the reference point may be an end (front end, rear end) of the host vehicle M or a position where one of a plurality of wheels provided in the host vehicle M exists.

[0041] The action plan generation unit 140 basically travels in the recommended lane determined by the recommended lane determination unit 61. Furthermore, in order to be able to respond to the surrounding situation of the host vehicle M, the host vehicle M automatically (regardless of the driver's operation) generates a target trajectory for future travel based on the recognition result by the recognition unit 130 or the like. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points that the host vehicle M should reach (trajectory points). The trajectory points are points that the host vehicle M should reach at regular driving distances (for example, about several [m]) in the road-aligned distance. Separately from that, the target speed and target acceleration at predetermined sampling times (for example, about 0 comma several [sec]) are generated as part of the target trajectory. Also, the trajectory points may be the positions that the host vehicle M should reach at the sampling times at predetermined sampling times. In this case, the information on the target speed and target acceleration is expressed by the interval between the trajectory points. Also, when a preset speed of the host vehicle M is determined, the action plan generation unit 140 may generate a target trajectory such that the speed of the host vehicle M becomes the preset speed within the feasible range.

[0042] When generating the target trajectory, the action plan generation unit 140 may set an automatic driving event (function). Examples of automatic driving events include a constant speed driving event, a low-speed following driving event, a lane change event, a branching event, a merging event, a takeover event, and the like. The action plan generation unit 140 generates a target trajectory according to the activated event. Also, when executing the driving control of the host vehicle M or a predetermined event or the like, the action plan generation unit 140 recommends (recommends) the execution of the driving control or the event to the occupant according to the driving mode of the host vehicle M described later, and generates a corresponding target trajectory when the recommendation is approved.

[0043] The second control unit 160 controls the driving force output device 200, the brake device 210, and the steering device 220 so that the host vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

[0044] The second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the travel drive force output device 200 or the brake device 210 based on the speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the degree of curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road in front of the host vehicle M and feedback control based on the deviation from the target trajectory.

[0045] The HMI control unit 170 notifies a predetermined information to the occupant (driver) of the host vehicle M through the HMI 30. The predetermined information includes, for example, driving assistance information. For example, the HMI control unit 170 may generate an image including the above-described predetermined information and display the generated image on the display device of the HMI 30, or may generate a voice indicating the predetermined information and output the generated voice from the speaker of the HMI 30. Further, the HMI control unit 170 may output the information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, etc., for example.

[0046] [Point cloud acquisition unit, lane line candidate recognition unit, lane line search unit] Hereinafter, the details of the functions of the point cloud acquisition unit 132, the lane line candidate recognition unit 134, and the lane line search unit 136 will be described.

[0047] The point cloud acquisition unit 132 acquires the current camera image (an image representing the front situation of the host vehicle M) from the camera 10, and acquires the point cloud of the object included in the acquired camera image. The current camera image is an example of the "first image". For example, the point cloud acquisition unit 132 extracts edge points from the camera image (the first image) by an existing image analysis process, and groups the points within a predetermined interval among the extracted edge points as the point cloud of the object. The point cloud acquired at this time is, for example, the point cloud defined in the image obtained by converting the camera image into an image in a two-dimensional coordinate system (bird's-eye view coordinate system) looking down on the host vehicle M from above. The reference position (origin) of the two-dimensional coordinates in this case is, for example, the representative point of the host vehicle M. Note that the point cloud acquisition unit 132 may store the camera image acquired from the camera 10 as the image information 182 in the storage unit 180.

[0048] The lane line candidate recognition unit 134 recognizes candidates for lane lines that demarcate the host lane in which the host vehicle M travels from the point cloud acquired by the point cloud acquisition unit 132. FIG. 3 is a diagram for explaining the function of the lane line candidate recognition unit 134. In the example of FIG. 3, the host vehicle M traveling at a speed VM in the arrow direction and lane line candidates recognized by the method described later are shown.

[0049] For example, the lane line candidate recognition unit 134 forms a point sequence from the point cloud within a predetermined distance at intervals in the same direction (including the allowable angle range) among the point cloud acquired by the point cloud acquisition unit 132, and compares the formed point sequence with a predetermined point sequence pattern to recognize candidates for lane lines. For example, the lane line candidate recognition unit 134 recognizes the point sequence as a lane line candidate when the point sequence is linear and extends for a predetermined distance or more, or when it is non-linear but extends for a predetermined distance or more with a predetermined curvature.

[0050] In addition, the lane line candidate recognition unit 134 inputs the point cloud data acquired by the point cloud acquisition unit 132 into a learned model such as a DNN (deep neural network) that has been trained to take, for example, point cloud data as input and output candidates for lane lines corresponding to the point cloud data, thereby obtaining candidates for lane lines at the current time. Further, the lane line candidate recognition unit 134 may input the camera image into a learned model such as a DNN that has been trained to take the camera image as input and output candidates for lane lines corresponding to the image, thereby obtaining candidates for lane lines at the current time. In this case, the point cloud acquisition unit 132 may be omitted from the recognition unit 130. The above-mentioned learned model may be stored in the storage unit 180 or may be acquired from an external device through communication via the communication device 20. Hereinafter, it will be described on the assumption that the lane line candidate recognition unit 134 has recognized lane line candidates LC1 to L7 as shown in FIG. 3.

[0051] The lane line search unit 136 searches (tracks) for the lane lines that demarcate the host vehicle lane among the lane line candidates LC1 to LC7 recognized by the lane line candidate recognition unit 134. For example, the lane line search unit 136 obtains the lane lines of the current driving lane of the host vehicle M based on the lane line candidates LC1 to LC7 and the information indicating the lane lines of the past driving lanes included in the past camera images (second images) included in the lane line information 184 of the storage unit 180. Note that the lane line search unit 136 may obtain the information indicating the lane lines of the past driving lanes by analyzing the past camera images stored in the storage unit 180 instead of obtaining the information from the lane line information 184.

[0052] FIG. 4 is a diagram showing an example of past lane lines of the host vehicle lane. In the example of FIG. 4, the left and right lane lines PL1 and PL2 of the host vehicle lane acquired by the lane line search unit 136 in the past and stored in the lane line information 184 of the storage unit 180 are shown. Hereinafter, the lane lines PL1 and PL2 are referred to as "past lane lines PL1 and PL2". For example, based on the position information of the past lane lines PL1 and PL2, the lane line search unit 136 replaces each of the past lane lines PL1 and PL2 with a coordinate system (host vehicle center coordinate system) based on the position of the host vehicle M in the two-dimensional coordinate system of the camera image at the current time as shown in FIG. 3.

[0053] Here, the position and orientation of the host vehicle M at the time of shooting the past camera image may be different from the position and orientation of the current host vehicle M. Therefore, the lane line search unit 136 corrects the positions of the past lane lines PL1 and PL2 based on one or both of the movement amount and movement direction of the host vehicle M from the shooting time of the past camera image to the present (more specifically, the shooting time of the camera image in which the lane line candidates LC1 to LC7 are recognized), or the road shape. For example, when the host vehicle M has been turning since the time of shooting the past camera image, the lane line search unit 136 positions the host vehicle so that the orientation (movement direction) of the host vehicle is the same in the current camera image (first image) and the past camera image (second image), and corrects the positions of the past lane lines PL1 and PL2 corresponding to that orientation. Also, when the inclination angle or curvature of the road has changed since the time of shooting the past camera image of the host vehicle M, the positions of the past lane lines PL1 and PL2 are corrected according to the inclination angle or curvature. Thereby, the positional relationship between the past lane lines PL1 and PL2 and the lane line candidates LC1 to LC7 can be acquired more accurately.

[0054] Next, the lane line search unit 136 searches for the lane line candidates LC1 to LC7 that are closest (nearest neighbor) to each of the past lane lines PL1 and PL2 among the lane line candidates LC1 to LC7 based on the positional relationship between the past lane lines PL1 and PL2 and the lane line candidates LC1 to LC7. "Closest" means, for example, the shortest distance. In addition to (or instead of) the distance, at least one of the following may be included: the slope of the line when the traveling direction of the host vehicle is used as a reference, the shape of the line in a predetermined section, the thickness of the line, the color of the line, the line type, etc. being the closest.

[0055] FIG. 5 is a diagram for explaining the comparison of the positions of the past lane line PL1 and the lane line candidates LC1 to L7. For example, as shown in FIG. 5, the lane line search unit 136 searches for the lane line candidate that exists at the closest distance from the reference position RP1 set on the past lane line PL1 among the lane line candidates LC1 to LC7. The reference position RP1 is set based on, for example, the position of the host vehicle M. In the example of FIG. 5, the point where the line extended in the horizontal direction (Y-axis direction) on the image from the tip of the host vehicle M contacts the past lane line PL1 is used as the reference position RP1, but a line extended in the lateral direction of the road from the tip may be used, or other points may be used as the reference. Similarly, the lane line search unit 136 searches for the lane line candidate that is closest to the reference position set on the past lane line PL2 among the lane line candidates LC1 to LC7 for the past lane line PL2 as well.

[0056] FIG. 6 is a diagram for explaining the method of deriving the distance between the past lane line and the lane line candidate. In the following explanation, the method of deriving the distance between the past lane line PL1 and the lane line candidate LC2 will be described, but the same processing is performed for deriving the distances between the past lane line PL1 and the other lane line candidates LC1, LC3 to LC7, and between the past lane line PL2 and the lane line candidates LC1 to LC7. In the example of FIG. 6, the sections of the past lane line PL1 and the lane line candidate LC2 that are partially cut out are shown.

[0057] The section line search unit 136, for example, sets a search start point SP on the section line candidate LC2, and partially cuts out each section of the past section line PL1 from the search start point SP to the reference position RP1 and the section line candidate LC2. The search start point SP is, for example, a position determined based on camera information (parameter information of the camera 10). The camera information includes, for example, the field of view angle, the shootable distance (limit distance), and the like. For example, by setting a search start point within a range that does not exceed at least the shootable distance based on the camera information, recognition errors can be suppressed and more accurate line comparison can be performed.

[0058] The section line search unit 136 sets a predetermined number of points on the section line candidate LC2 at predetermined intervals ΔI in the direction in which the reference position RP1 exists from the search start point SP. From each of the set points, the section line search unit 136 derives the respective distances to the points where the lines extended in the horizontal direction (Y-axis direction) of the image (or the lines extended vertically with respect to the extension direction of the section line candidate LC2) contact the past section line PL1. The predetermined interval ΔI may be, for example, a fixed interval determined in advance, or a variable interval based on the speed of the host vehicle M detected by the vehicle sensor 40, the road shape, and the like. Also, the predetermined number may be, for example, a fixed number determined in advance, or a variable number based on the speed of the host vehicle M, the road shape, and the like. In the example of FIG. 6, four search points PO1 to PO4 are set. The section line search unit 136 derives the distance at each point based on the difference between the search start point SP and each of the search points PO1 to PO4 and the past section line PL1 in the horizontal position direction of the image. In the example of FIG. 6, since there is no past section line PL1 in the horizontal direction of the search start point SP, the distance from this point is not derived, and the distances D11 to D14 corresponding to the search points PO1 to PO4 are derived. Then, the section line search unit 136 derives the distance between the past section line PL1 and the section line candidate LC2 based on each of the distances D11 to D14. The distance is derived, for example, by obtaining the root mean square value of each of the distances D11 to D14. Also, the distance may be derived by adding or averaging the distances D11 to D14, or by substituting the values of the distances D11 to D14 into a predetermined function. Also, the distance may be the distance on the image (host vehicle center coordinate system image) converted from the view looking down on the host vehicle M from above, or the actual distance adjusted based on the shooting information of the camera image.

[0059] The lane line search unit 136 similarly derives the distances for the other lane line candidates LC1, LC3 to LC7, extracts the lane line candidate closest to the past lane line PL1, and determines the extracted lane line candidate as the dividing line (more specifically, the left dividing line that divides the own vehicle lane) that divides the current own vehicle lane. Similarly, the lane line search unit 136 derives the distances between each of the past lane lines PL2 and the lane line candidates LC1 to LC7, and among the lane line candidates LC1 to LC7, determines the lane line candidate with the closest distance to the past lane line PL2 as the dividing line (more specifically, the right dividing line that divides the own vehicle lane) that divides the current own vehicle lane. Note that if either one of the past lane lines PL1 and PL2 cannot be searched for among the lane line candidates LC1 to LC7, the lane line search unit 136 may output only the position information of the other searched lane line. Thereby, the host vehicle M can be made to travel based on the information of one lane line obtained as the search result.

[0060] Note that in the above example, the lane line search unit 136 sets each point for deriving the distance at a predetermined interval from the search start point SP toward the reference position RP1 side (in other words, the front side of the host vehicle M). However, a predetermined number of points may be set at a predetermined interval △I on the past lane line toward the search start point SP with the reference position of the past lane line as the start point. Further, the lane line search unit 136 may set the predetermined interval △I so that a predetermined number of points can be extracted from the search start point SP.

[0061] Further, when the distances between two lane line candidates that continuously extend in the traveling direction are close, the lane line search unit 136 may interpolate and connect a line between the two lane line candidates. Further, when there is no past lane line in the lateral direction of the lane line candidate, or when the length of the past lane line is short, the lane line candidate or the past lane line may be extended in the extending direction so that the distance based on a predetermined number of search points can be derived. By setting a predetermined number of search points, it is possible to more accurately derive the proximity between lines including the shape in a predetermined section, rather than just the distance of a single point.

[0062] Further, the lane line search unit 136 may, for example, exclude from the distance derivation target a lane line candidate whose distance from a past lane line is farther than that of other lane line candidates based on the positional relationship of each lane line candidate. For example, in the case of FIG. 5, since the lane line candidates LC5 to LC7 are located at positions (distances separated by a predetermined distance or more) away from the past lane line RP1 compared to LC1 to LC4, they are excluded from the target of distance derivation from the past lane line RP1. Thereby, the lane line candidates to be searched can be reduced, and the processing efficiency can be improved.

[0063] Further, when the distance (closeness) between the past lane line and the closest lane line candidate is equal to or greater than a threshold value (when it is not less than the threshold value), the lane line search unit 136 may determine that there is no lane line corresponding to the past lane line among the lane line candidates LC1 to LC7. In this case, the lane line search unit 136 continues to use the lane line of the past own vehicle lane searched in the past as the lane line that divides the current own vehicle lane. Thereby, even when the lane line candidates cannot be temporarily recognized from the camera image depending on the surrounding environment of the own vehicle M and the state of the lane lines drawn on the road surface, based on the position of the lane line of the driving lane which is the search result in the past (for example, immediately before), driving support such as LKAS can be continued.

[0064] The lane line search unit 136 stores information on the lane line of the own vehicle lane obtained by the current search (for example, position information of the lane line based on the position of the own vehicle M) and information on the own vehicle lane itself divided by the lane line in the lane line information 184 of the storage unit 180, and uses it when searching for the lane line that divides the next own vehicle lane.

[0065] The recognition unit 130 outputs a recognition result including the lane line information of the own vehicle lane determined by the lane line search unit 136 to the action plan generation unit 140. Based on the recognition result, the action plan generation unit 140 generates a target trajectory of the own vehicle M so that the own vehicle M travels toward the destination set by the navigation device 50 and executes driving control so that the own vehicle M travels along the generated target trajectory.

[0066] [Processing Flow] FIG. 7 is a flowchart showing an example of the flow of processing executed by the automatic driving control device 100 of the embodiment. Hereinafter, among the processes executed by the automatic driving control device 100, the process of mainly searching for a lane dividing line that divides the driving lane of the host vehicle M and executing the driving control of the host vehicle M based on the information such as the lane dividing line obtained as the search result will be mainly described. The process of FIG. 7 may be repeatedly executed at a predetermined interval while the automatic driving control for the host vehicle M is being executed.

[0067] In the example of FIG. 7, the point cloud acquisition unit 132 acquires a camera image representing the front situation of the host vehicle M captured by the camera 10. Next, the point cloud acquisition unit 132 converts the acquired image into an image coordinate system of an image of the host vehicle M viewed from above (step S102), and acquires a point cloud of an object included in the image-converted camera image (step S104). Next, the lane dividing line candidate recognition unit 134 recognizes a lane dividing line candidate from the point cloud of the object (step S106).

[0068] Next, the lane line search unit 136 acquires information indicating a lane line that demarcates the past own vehicle lane of the host vehicle M from the lane line information 184 stored in the storage unit 180 (step S108). In the process of step S108, the lane line search unit 136 may perform correction to convert the position of the acquired past lane line into the host vehicle center coordinate system based on the position of the current host vehicle M. Next, the lane line search unit 136 extracts the lane line with the shortest distance from the lane line candidates to the past lane line (step S110). Next, the lane line search unit 136 determines whether the distance between the extracted closest lane line candidate and the past lane line is less than a threshold value (step S112). If it is determined that the distance is less than the threshold value, the lane line search unit 136 determines that lane line candidate as the lane line that demarcates the current own vehicle lane of the host vehicle M (step S114). Also, in the process of step S112, if it is determined that the distance is not less than the threshold value (is greater than or equal to the threshold value), the lane line search unit 136 determines the past lane line (for example, the previously searched lane line) stored in the storage unit 180 as the lane line that demarcates the current own vehicle lane (step S116). After the processes of steps 116 and S118, the information indicating the determined lane line of the own vehicle lane is stored in the storage unit 180 as the lane line information 184 (step S118).

[0069] Next, the action plan generation unit 140 generates a target trajectory for the host vehicle M to travel along the center of the own vehicle lane demarcated by the determined lane line (step S120). Next, the second control unit 160 executes driving control to control one or both of the steering and speed of the host vehicle M so that the host vehicle M travels along the generated target trajectory (step S122). Thereby, the processing of this flowchart ends.

[0070] <Modification Example> The above-described vehicle system 1 may be provided with LIDAR (Light Detection and Ranging). LIDAR irradiates light (or an electromagnetic wave with a wavelength close to light) around the host vehicle M and measures the scattered light. LIDAR detects the distance to an object based on the time from light emission to light reception. The light to be irradiated is, for example, pulsed laser light. LIDAR can be attached to any part of the host vehicle M. For example, in addition to the camera 10 and the radar device 12, the object recognition device 16 may receive an input from the LIDER and recognize surrounding objects (including lane lines) from the result of sensor fusion processing based on the input information from each of them.

[0071] Further, in the above-described embodiment, when the destination by the navigation device 50 is not set and the host vehicle M is traveling by the operation of the driver, driving control for assisting a part of the steering and speed of the host vehicle M may be performed using the information on the lane lines of the searched host lane by the above-described lane line search unit 136.

[0072] Further, when the lane line search unit 136 searches for the lane line candidate closest to the past lane line from the lane line candidates, for example, the closest (for example, the highest degree of matching) lane line candidate may be derived using an existing matching algorithm.

[0073] Further, when the lane line search unit 136 searches for the line closest to the past lane line from the lane line candidates, for example, the line closest to (for example, the highest degree of matching) the past lane line may be derived using an existing matching algorithm. Also, information on the past lane line (the lane line searched last time) used during the search in the lane line search unit 136 may be obtained from the image stored in the image information 182 instead of the lane line information 184.

[0074] According to the above-described embodiment, in an automatic driving control device (an example of a movement control device), a lane line candidate recognition unit 134 that recognizes candidates for lane lines that demarcate a travel lane on which a host vehicle (an example of a moving body) travels from a first image including the periphery of the moving body captured by a camera (an example of an imaging unit) 10, and among the candidates for lane lines recognized by the lane line candidate recognition unit 134, a lane line search unit 136 that sets, as a lane line that demarcates the current travel lane of the host vehicle M, the lane line candidate that is closest to a past lane line that demarcates the travel lane of the host vehicle M at a predetermined time ago. By including these components, it is possible to more accurately recognize the lane line that demarcates the host lane.

[0075] Specifically, according to the above-described embodiment, by sequentially searching for the closest lane line candidate to the host lane recognized in the past among the lane line candidates recognized from the camera image, it is possible to travel while constantly grasping the host lane. Further, according to the embodiment, since the current lane line is searched considering not only the past lane line information but also the movement amount, movement direction, road shape, etc. of the host vehicle M, it is possible to suppress losing sight of the lane line of the host lane, and even when the host vehicle M deviates from the travel lane, it is possible to track the host lane. Also, according to the embodiment, even when the lane line cannot be recognized temporarily, it is possible to continue traveling using past information.

[0076] The above-described embodiment can be expressed as follows. A storage device storing a program, A hardware processor, and is configured such that by the hardware processor executing the program stored in the storage device, candidates for lane lines that demarcate a travel lane on which the moving body travels are recognized from a first image including the periphery of the moving body captured by the imaging unit, and among the recognized candidates for lane lines, the lane line candidate that is closest to a past lane line that demarcates the travel lane of the moving body at a predetermined time ago is set as the lane line that demarcates the current travel lane of the moving body. A movement control device configured as described above.

[0077] As described above, the embodiments for carrying out the present invention have been described using the embodiments. However, the present invention is not limited to such embodiments, and various modifications and substitutions can be made without departing from the gist of the present invention.

Explanation of Reference Numerals

[0078] 1…Vehicle system, 10…Camera, 12…Radar device, 16…Object recognition device, 20…Communication device, 30…HMI, 40…Vehicle sensor, 50…Navigation device, 60…MPU, 80…Driving operator, 100…Autonomous driving control device, 120…First control unit, 130…Recognition unit, 132…Point cloud acquisition unit, 134…Lane line candidate recognition unit, 136…Lane line search unit, 140…Action plan generation unit, 160…Second control unit, 162…Acquisition unit, 164…Speed control unit, 166…Steering control unit, 170…HMI control unit, 180…Memory unit, 200…Travel driving force output device, 210…Brake device, 220…Steering device

Claims

1. A lane line candidate recognition unit that recognizes candidates for lane lines that demarcate a driving lane on which the moving object travels from a first image including the periphery of the moving object imaged by an imaging unit; A lane line search unit that sets, as a lane line that demarcates the current driving lane of the moving object, a lane line candidate that is closest to a past lane line that demarcates the driving lane of the moving object among the lane line candidates recognized by the lane line candidate recognition unit; A driving control unit that executes driving control by controlling one or both of the speed or steering of the moving object so that the moving object travels along a lane line that demarcates the current driving lane of the moving object, comprising: The lane line search unit detects the proximity between the lane line candidate and the past lane line in an interval partially cut out based on a reference position for the lane line candidate recognized by the lane line candidate recognition unit and the past lane line; The past lane line is a lane line that has already been searched as a lane line that demarcates the driving lane of the moving object by the lane line search unit at the time when the lane line candidate is recognized by the lane line candidate recognition unit; The lane line search unit sets the past lane line as the lane line of the current driving lane when the proximity of the lane line candidate closest to the past lane line among the lane line candidates recognized by the lane line candidate recognition unit is equal to or greater than a threshold value; The driving control unit continues the driving control using the past lane line even when the proximity of the lane line candidate closest to the past lane line is equal to or greater than a threshold value; A moving body control device.

2. The past lane line is extracted from a second image including the periphery of the moving object imaged by the imaging unit before the first image is imaged; The moving body control device according to claim 1.

3. When the lane line search unit compares each of the lane line candidates recognized by the lane line candidate recognition unit with the past lane line, the lane line closest to the past lane line is determined based on the distances from points on a predetermined number of the past lane lines from a predetermined search start point to the lane line candidate; The moving body control device according to claim 1 or 2.

4. The lane line search unit converts the first image and a second image including the periphery of the moving body captured by the imaging unit before the first image is captured into an image of the moving body viewed from above, and based on the mean square value of the horizontal position difference between each candidate for the lane line included in the converted image and the past lane line, determines the lane line closest to the past lane line. The moving body control device according to claim 3.

5. The search start point is set based on the parameter information of the imaging unit. The moving body control device according to claim 3.

6. The lane line search unit corrects the position of the past lane line based on one or both of the movement amount and movement direction of the moving body or the road shape. The moving body control device according to any one of claims 1 to 5.

7. The moving body control device further includes an interface control unit that causes an interface provided in the moving body to display driving support information including the driving lane of the moving body searched by lane line search using the past lane line and the candidate for the lane line. The moving body control device according to claim 1.

8. The computer of the moving body control device recognizes candidates for lane lines that demarcate the driving lane on which the moving body travels from a first image including the periphery of the moving body captured by the imaging unit. Among the recognized candidates for the lane lines, the candidate for the lane line closest to the past lane line that demarcates the driving lane of the moving body is set as the lane line that demarcates the current driving lane of the moving body. Performs driving control by controlling one or both of the speed or steering of the moving body so that the moving body travels along the lane line that demarcates the current driving lane of the moving body. Detects the proximity between the candidate for the lane line and the past lane line in a section partially cut out based on a reference position for the recognized candidate for the lane line and the past lane line. The past lane line is a lane line that has already been searched as a lane line that demarcates the driving lane of the moving body at the time when the candidate for the lane line is recognized. When the proximity of the candidate for the lane line closest to the past lane line among the recognized candidates for the lane line is equal to or greater than a threshold value, the past lane line is set as the lane line of the current driving lane. Even when the proximity of the candidate for the lane line closest to the past lane line is equal to or greater than the threshold value, the driving control is continued using the past lane line. Moving body control method.

9. In the computer of the moving body control device Cause a candidate for a dividing line that demarcates a driving lane on which the moving object travels to be recognized from a first image including the periphery of the moving object imaged by an imaging unit. Of the recognized candidates for the dividing line, cause the dividing line candidate that is closest to a past dividing line that demarcates the driving lane of the moving object to be the dividing line that demarcates the current driving lane of the moving object. Execute driving control by controlling one or both of the speed or steering of the moving object so that the moving object travels along the dividing line that demarcates the current driving lane of the moving object. Cause the proximity between the candidate for the dividing line and the past dividing line to be detected in a section that is partially cut out based on a reference position from among the recognized candidates for the dividing line and the past dividing line. The past dividing line is a dividing line that has already been searched for as a dividing line that demarcates the driving lane of the moving object at the time when the candidate for the dividing line is recognized. When the proximity of the dividing line candidate that is closest to the past dividing line among the recognized candidates for the dividing line is equal to or greater than a threshold value, cause the past dividing line to be the dividing line of the current driving lane. Even when the proximity of the dividing line candidate that is closest to the past dividing line is equal to or greater than a threshold value, continue the driving control using the past dividing line. Program.

Images