MOBILE BODY CONTROL DEVICE, MOBILE BODY CONTROL METHOD, AND PROGRAM

Abstract

To properly determine timing of lane change at the time of travel in a merging lane.SOLUTION: A movable body control device comprises: an image acquisition unit which acquires an image obtained by imaging an external space of a movable body; a merging lane determination unit which determines whether or not the movable body is traveling in a merging lane on the basis of the image; a detection unit which detects a first reference object and a second reference object with the image processing when it is determined that the movable body is traveling in the merging lane; a setting unit which virtually sets a plurality of third reference objects extending in parallel with a first prescribed width to the first reference object on an assumption plane virtually viewed from above; and a derivation unit which derives information about a place where the merging lane disappears on the basis of the first intersection point of at least the second reference object and the plurality of third reference objects.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a mobile object control device, a mobile object control method, and a program. [Background technology]

[0002] There is known a technique for determining whether or not the lane on which a vehicle is traveling is a merging lane. For example, Patent Document 1 discloses a technique for determining whether or not the lane on which a vehicle is traveling is a merging lane based on map information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-27316 A Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 is for determining whether a lane is a merging lane or a merging lane. However, this technology does not detect the start point at which the lane width of the merging lane starts to decrease, and the end point at which the lane width ends to decrease. As a result, for example, in the automatic driving control of a vehicle, there are cases where the timing of lane change cannot be appropriately determined when traveling in a merging lane.

[0005] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a mobile body control device, a mobile body control method, and a program that can appropriately determine the timing of lane changes when traveling in a merging lane. [Means for solving the problem]

[0006] A mobile object control device, a mobile object control method, and a program according to the present invention employ the following configuration. (1): A mobile body control device according to one embodiment of the present invention includes an image acquisition unit that acquires an image of an external space of a mobile body; a merging lane determination unit that determines based on the image whether the mobile body is traveling in a merging lane that disappears in the direction of travel of the mobile body; a detection unit that, when the merging lane determination unit determines that the mobile body is traveling in a merging lane, detects through image processing a first reference object that is present on the main line side of the mobile body in the image and extends in the direction of travel of the mobile body, and a second reference object that is present on the opposite side of the main line of the mobile body in the image and extends in the direction of travel of the mobile body; a setting unit that virtually sets a number of third reference objects that extend parallel to the first reference object with a first predetermined width on an imaginary plane virtually viewed from above; and a derivation unit that derives information regarding the location where the merging lane disappears based on at least a first intersection of the second reference object and the number of third reference objects.

[0007] (2): In the above aspect (1), the detection unit detects an end point in the first reference object where the lane width of the merging lane ends decreasing, and the derivation unit derives a start point where the lane width of the merging lane begins to decrease, which is information regarding the point where the merging lane disappears, based on the first intersection and the end point.

[0008] (3): In the above aspect (1), the setting unit virtually sets a plurality of fourth reference objects extending parallel to the first reference object with a second predetermined width different from the first predetermined width on an imaginary plane viewed from above, and the derivation unit derives information regarding the location where the merging lane disappears, such as a start point where the lane width of the merging lane begins to decrease and an end point where the lane width of the merging lane ends decreasing, based on the first reference objects, the first intersection, and the second intersection of the second reference object and the plurality of fourth reference objects.

[0009] (4): In any of the above aspects (1) to (3), the setting unit extracts a first object from the multiple third reference objects that intersects with the second reference object and an object immediately preceding the first object, sets multiple objects parallel to the first reference object in a direction from the immediately preceding object to the first object at intervals shorter than the intervals between the multiple third reference objects, and sets the first object obtained by repeating the process of extracting a first object from the multiple objects set at the short intervals that intersects with the second reference object and an object immediately preceding the first object a predetermined number of times as the first intersection.

[0010] (5): In the above aspect (3) or (4), the setting unit extracts a first object from the multiple fourth reference objects that intersects with the second reference object and an object immediately preceding the first object, sets multiple objects parallel to the first reference object in a direction from the immediately preceding object to the first object at intervals shorter than the intervals between the multiple fourth reference objects, and sets the first object obtained by repeating the process of extracting a first object from the multiple objects set at the shorter intervals that intersects with the second reference object and an object immediately preceding the first object a predetermined number of times as the second intersection.

[0011] (6): In any of the above aspects (3) to (5), the derivation unit extends a line segment connecting the first intersection and the second intersection, thereby detecting a third intersection of the extended line segment with the first reference object, and a fourth intersection on the second reference object where the length of a perpendicular line drawn from the extended line segment to the first reference object is equal to the lane width, and derives the third intersection as a starting point where the lane width of the merging lane begins to decrease, and derives the fourth intersection as an end point where the lane width of the merging lane ends to decrease.

[0012] (7): In any one of the above aspects (1) to (6), the first predetermined width is the width of the moving body.

[0013] (8): In any of the above aspects (1) to (7), the mobile body control device further includes a driving control unit that generates a target trajectory for the mobile body such that the mobile body completes the lane change before the mobile body reaches the end point of the merging lane derived by the derivation unit, and controls the steering and acceleration / deceleration of the mobile body so that the mobile body travels along the generated target trajectory without relying on the operation of the driver of the mobile body.

[0014] (9): In one embodiment of the mobile body control method of the present invention, a computer mounted on a mobile body acquires an image of the external space of the mobile body, and determines based on the image whether the mobile body is traveling in a merging lane that disappears in the direction of travel of the mobile body. If the merging lane determination unit determines that the mobile body is traveling in a merging lane, a first reference object that is present on the main line side of the mobile body in the image and extends in the direction of travel of the mobile body, and a second reference object that is present on the opposite side of the main line of the mobile body in the image and extends in the direction of travel of the mobile body, are detected by image processing, and a plurality of third reference objects are virtually set on an imaginary plane viewed from above, extending parallel to the first reference object with a first predetermined width, and information regarding the location where the merging lane disappears is derived based on at least a first intersection of the second reference object and the plurality of third reference objects.

[0015] (10): A program according to one embodiment of the present invention causes a computer mounted on a moving body to acquire an image of the external space of the moving body, and determine based on the image whether the moving body is traveling in a merging lane that disappears in the moving body's direction of travel. If the merging lane determination unit determines that the moving body is traveling in a merging lane, the program causes a computer mounted on a moving body to detect through image processing a first reference object that is located on the main line side of the moving body in the image and extends in the moving body's direction of travel, and a second reference object that is located on the opposite side of the main line of the moving body in the image and extends in the moving body's direction of travel, and virtually sets a plurality of third reference objects that extend parallel to the first reference object with a first predetermined width on an imaginary plane viewed virtually from above, and derives information regarding the location where the merging lane disappears based on at least a first intersection of the second reference object and the plurality of third reference objects. Effect of the Invention

[0016] According to aspects (1) to (10), the timing of changing lanes can be appropriately determined when traveling in a merging lane. [Brief description of the drawings]

[0017] [Figure 1] 1 is a configuration diagram of a vehicle system 1 that uses a mobile object control device according to the present embodiment. [Diagram 2] 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. FIG. [Diagram 3] FIG. 2 is a diagram showing an example of a scene in which processing of a mobile object control device according to the present embodiment is executed; [Figure 4] FIG. 13 is a diagram showing an example of a plurality of third reference objects RO3 set by a setting unit 130C. [Diagram 5] FIG. 13 is a diagram showing an example of a plurality of third reference objects RO3 additionally set by a setting unit 130C. [Figure 6] FIG. 13 is a diagram showing an example of a plurality of fourth reference objects RO4 set by a setting unit 130C. [Figure 7] 11 is a diagram showing an example of a start point SP and an end point EP of a merging lane L1 derived by a derivation unit 130D. FIG. [Figure 8] 13 is a diagram showing an example of a target trajectory generated by a behavior plan generating unit 140. FIG. [Figure 9] 5 is a flowchart showing an example of a process flow of a mobile object control device according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of a mobile object control device, a mobile object control method, and a program of the present invention will be described with reference to the drawings. The mobile object in the present invention is a four-wheeled vehicle, a two-wheeled vehicle, micromobility, a robot, etc. In the following description, the mobile object is a four-wheeled vehicle.

[0019] [Overall configuration] 1 is a configuration diagram of a vehicle system 1 that uses a mobile object control device according to this embodiment. The vehicle on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a secondary battery or a fuel cell.

[0020] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driving operator 80, an automatic driving control device 100, a driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by multiple communication lines such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the configuration may be omitted, or other configuration may be added.

[0021] The camera 10 is, for example, a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary location of a vehicle (hereinafter, the vehicle M) in which the vehicle system 1 is mounted. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, or the like. The camera 10, for example, periodically and repeatedly captures images of the surroundings of the vehicle M. The camera 10 may be a stereo camera.

[0022] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by an object (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 12 is attached to an arbitrary location of the vehicle M. The radar device 12 may detect the position and speed of an object by an FM-CW (Frequency Modulated Continuous Wave) method.

[0023] The LIDAR 14 irradiates light (or electromagnetic waves with a wavelength close to that of light) around the vehicle M and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between light emission and light reception. The irradiated light is, for example, a pulsed laser light. The LIDAR 14 is attached to an arbitrary location of the vehicle M.

[0024] The object recognition device 16 performs sensor fusion processing on some or all of the detection results from the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, etc. of an object. The object recognition device 16 outputs the recognition result to the automatic driving control device 100. The object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the automatic driving control device 100. In this embodiment, the object recognition device 16 includes an image acquisition unit 16A, which acquires an image of the external space of the vehicle captured by the camera 10, and outputs the image to the automatic driving control device 100 described later.

[0025] The communication device 20 communicates with other vehicles in the vicinity of the vehicle M, for example, using a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., or communicates with various server devices via a wireless base station.

[0026] The HMI 30 presents various information to an occupant of the vehicle M and accepts input operations by the occupant. The HMI 30 includes various display devices, a speaker, a buzzer, a touch panel, switches, keys, and the like.

[0027] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, a direction sensor that detects the direction of the host vehicle M, and the like.

[0028] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 holds first map information 54 in a storage device such as a HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on a signal received from a GNSS satellite. The position of the vehicle M may be identified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or entirely shared with the above-mentioned HMI 30. The route determination unit 53, for example, determines a route (hereinafter, a route on a map) from the position of the vehicle M identified by the GNSS receiver 51 (or an arbitrary position input) to a destination input by a driver using the navigation HMI 52, with reference to the first map information 54. The first map information 54 is, for example, information in which road shapes are expressed by links indicating roads and nodes connected by the links. The first map information 54 may include road curvature and POI (Point Of Interest) information. The route on the map is output to the MPU 60. The navigation device 50 may perform route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized by, for example, a function of a terminal device such as a smartphone or tablet terminal owned by a passenger. The navigation device 50 may transmit a current position and a destination to a navigation server via the communication device 20, and obtain a route equivalent to the route on the map from the navigation server.

[0029] The MPU 60 includes, for example, a recommended lane determination unit 61, and holds second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, by dividing it into 100 m intervals in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines which lane from the left the vehicle should travel in. When a branch point is present on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the host vehicle M can travel along a reasonable route to proceed to the branch point.

[0030] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of lanes or information on boundaries of lanes. The second map information 62 may also include road information, traffic regulation information, address information (address and zip code), facility information, telephone number information, and the like. The second map information 62 may be updated as needed by the communication device 20 communicating with another device.

[0031] The driving operators 80 include, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, an irregular steering wheel, a joystick, and other operators. The driving operators 80 are fitted with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to the automatic driving control device 100, or some or all of the driving force output device 200, the brake device 210, and the steering device 220.

[0032] The automatic driving control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). In addition, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (storage device having a non-transient storage medium) such as an HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the automatic driving control device 100 by mounting the storage medium (non-transient storage medium) in a drive device. The combination of the object recognition device 16 and the automatic driving control device 100 is an example of a "mobile body control device", and the combination of the action plan generation unit 140 and the second control unit 160 is an example of a "driving control unit".

[0033] FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120 realizes, for example, a function based on AI (Artificial Intelligence) and a function based on a pre-given model in parallel. For example, the function of "recognizing an intersection" may be realized by executing in parallel recognition of an intersection by deep learning or the like and recognition based on pre-given conditions (such as traffic lights and road markings that can be pattern matched), and by scoring both and evaluating them comprehensively. This ensures the reliability of the autonomous driving.

[0034] The recognition unit 130 recognizes the position, speed, acceleration, and other states of objects around the vehicle M based on information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. The position of the object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of the object may include the acceleration or jerk of the object, or the "behavior state" (for example, whether or not the object is changing lanes or is about to change lanes).

[0035] The recognition unit 130 also recognizes, for example, the lane in which the vehicle M is traveling (driving lane). For example, the recognition unit 130 recognizes the driving lane by comparing a pattern of road dividing lines (for example, an arrangement of solid lines and dashed lines) obtained from the second map information 62 with a pattern of road dividing lines around the vehicle M recognized from an image captured by the camera 10. The recognition unit 130 may recognize the driving lane by recognizing road boundaries (road boundaries) including not only road dividing lines but also road dividing lines, shoulders, curbs, median strips, guard rails, etc. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results by the INS may be taken into consideration. The recognition unit 130 also recognizes stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0036] When recognizing the driving lane, the recognition unit 130 recognizes the position and attitude of the host vehicle M with respect to the driving lane. For example, the recognition unit 130 may recognize the deviation of the reference point of the host vehicle M from the center of the lane and the angle with respect to a line connecting the centers of the lanes in the traveling direction of the host vehicle M as the relative position and attitude of the host vehicle M with respect to the driving lane. Alternatively, the recognition unit 130 may recognize the position of the reference point of the host vehicle M with respect to either side end of the driving lane (a road dividing line or a road boundary) as the relative position of the host vehicle M with respect to the driving lane.

[0037] In this embodiment, the recognition unit 130 particularly includes a merging lane determination unit 130A, a detection unit 130B, a setting unit 130C, and a derivation unit 130D, the functions of which will be described in detail later.

[0038] The action plan generating unit 140 generates a target trajectory along which the host vehicle M will travel in the future automatically (without relying on the driver's operation) so that the host vehicle M will travel in the recommended lane determined by the recommended lane determining unit 61 in principle and can further respond to the surrounding conditions of the host vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) to be reached by the host vehicle M. The trajectory points are points to be reached by the host vehicle M at each predetermined travel distance (for example, about several [m]) along the road, and separately, the target speed and target acceleration for each predetermined sampling time (for example, about a few tenths [sec]) are generated as part of the target trajectory. The trajectory points may also be positions to be reached by the host vehicle M at each sampling time for each predetermined sampling time. In this case, the information on the target speed and target acceleration is expressed as the interval between the trajectory points.

[0039] The behavior plan generating unit 140 may set an event of autonomous driving when generating the target trajectory. The autonomous driving events include a constant speed driving event, a low speed following driving event, a lane change event, a branching event, a merging event, a takeover event, etc. The behavior plan generating unit 140 generates a target trajectory according to the activated event.

[0040] The second control unit 160 controls the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the host vehicle M passes the target trajectory generated by the action plan generating unit 140 at the scheduled time.

[0041] Returning to FIG. 2, the second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140, and stores it in a memory (not shown). The speed control unit 164 controls the traveling driving force output device 200 or the brake device 210 based on a speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the host vehicle M and feedback control based on the deviation from the target trajectory.

[0042] The driving force output device 200 outputs a driving force (torque) for the vehicle to travel to the driving wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components according to information input from the second control unit 160 or information input from the driving operation device 80.

[0043] The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the driving operation element 80, so that a brake torque corresponding to a braking operation is output to each wheel. The brake device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of a brake pedal included in the driving operation element 80 to the cylinder via a master cylinder. Note that the brake device 210 is not limited to the configuration described above, and may be an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure in the master cylinder to the cylinder.

[0044] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor changes the direction of the steered wheels by applying a force to, for example, a rack and pinion mechanism. The steering ECU drives the electric motor in accordance with information input from the second control unit 160 or information input from the driving operator 80 to change the direction of the steered wheels.

[0045] [Operation] Next, the processing of the mobile object control device according to this embodiment will be described with reference to Figs. 3 to 8. Fig. 3 is a diagram showing an example of a scene in which the processing of the mobile object control device according to this embodiment is executed. In Fig. 3, the vehicle M is traveling in the merging lane L1 by automatic driving, executes a lane change, and enters the merging lane L2. Generally, the lane width of the merging lane starts to decrease from a certain point, and the lane width becomes zero at another point ahead and ends the decrease. Therefore, in order to determine the timing of executing the lane change, the mobile object control device needs to know in advance the start point at which the lane width of the merging lane starts to decrease and the end point at which the lane width of the merging lane ends the decrease. The present invention derives such start point and end point and utilizes them in determining the timing of executing the lane change.

[0046] When the image acquisition unit 16A acquires an image of the external space of the host vehicle M captured by the camera 10, the merging lane determination unit 130A determines, based on the image, whether the host vehicle M is traveling in a merging lane that disappears on the traveling direction side of the host vehicle M. Specifically, for example, when the second map information 62 indicates traveling in a merging lane, when the image of the external space of the host vehicle M indicates that the road dividing lines (or guard rails) on the left and right of the traveling lane intersect, or when it shows a dashed line shape specific to the merging lane, the merging lane determination unit 130A determines that the host vehicle M is traveling in a merging lane.

[0047] When the merging lane determination unit 130A determines that the host vehicle M is traveling on the merging lane, the detection unit 130B detects a first reference object RO1 that exists on the main line side of the host vehicle M and extends in the traveling direction of the host vehicle M in the image acquired from the image acquisition unit 16A, and a second reference object RO2 that exists on the opposite side of the main line of the host vehicle M and extends in the traveling direction of the host vehicle M in the image by image processing. Specifically, for example, the detection unit 130B extracts edge points that have a large luminance difference with adjacent pixels in the image acquired from the image acquisition unit 16A, and detects two reference objects, the first reference object RO1 and the second reference object RO2 on the left and right sides in the image based on the extension length and shape of the outline of the edge points. Here, the first reference object RO1 and the second reference object RO2 are, for example, a road dividing line or a guardrail. In FIG. 3, the detection unit 130B detects a road dividing line as the first reference object RO1 and a guardrail as the second reference object RO2.

[0048] The setting unit 130C virtually projects the image acquired from the image acquisition unit 16A onto an imaginary plane viewed from the sky, and virtually sets a plurality of third reference objects RO3 extending parallel to the first reference object detected by the detection unit 130B with a first predetermined width on the imaginary plane. The setting unit 130C may perform such processing on the image plane. The same applies to other processing described below.

[0049] Fig. 4 is a diagram showing an example of a plurality of third reference objects RO3 set by the setting unit 130C. In Fig. 4, the setting unit 130C sets a plurality of point-like third reference objects RO3 at a predetermined interval FI1 with a first predetermined width from the first reference object RO1. Here, the first predetermined width is, for example, the vehicle width MW of the host vehicle M.

[0050] The setting unit 130C further extracts, from among the multiple third reference objects RO3, a first object FP1 that intersects with the second reference object RO2 and an object PP1 immediately before the first object FP1. If an object that intersects with the second reference object RO2 cannot be extracted, the setting unit 130C resets the predetermined interval to a shorter value and sets the multiple third reference objects RO3 again.

[0051] Next, the setting unit 130C additionally sets a plurality of third reference objects RO3 parallel to the first reference object RO1 in the direction from the extracted previous object PP1 to the first object FP1 at intervals shorter than the intervals of the plurality of third reference objects RO3. FIG. 5 is a diagram showing an example of a plurality of third reference objects RO3 additionally set by the setting unit 130C. In FIG. 5, the setting unit 130C sets a plurality of third reference objects RO3 in the direction from the previous object PP1 to the first object FP1 at a first predetermined width from the first reference object RO1 at a predetermined interval FI2. The predetermined interval FI2 at this time is set shorter than the predetermined interval FI1. The setting unit 130C extracts the first object FP2 intersecting with the second reference object RO2 and the object PP2 immediately before the first object FP2 from the plurality of third reference objects RO3 generated at short intervals. As shown in FIG. 5, the first object FP2 extracted at this time is set closer to the host vehicle M compared to the previous first object FP1. The setting unit 130C sets the first object FP(N) obtained by repeating the above process a predetermined number of times (N times) as the first intersection I1 between the second reference object RO2 and the third reference object RO3. Alternatively, the setting unit 130C may set the first object FP(N) as the first intersection I1 when the distance between the previous first object FP(N-1) and the current first object FP(N) becomes less than a threshold value (for example, 1 m) instead of repeating the above process a predetermined number of times.

[0052] Next, the setting unit 130C virtually sets a plurality of fourth reference objects RO4 extending in parallel with the first reference object RO1 at a second predetermined width different from the first predetermined width, in addition to the plurality of third reference objects RO3 set. FIG. 6 is a diagram showing an example of a plurality of fourth reference objects RO4 set by the setting unit 130C. In FIG. 6, the setting unit 130C sets a plurality of point-like fourth reference objects RO4 at a predetermined interval SI1 with a second predetermined width from the first reference object RO1. Here, the second predetermined width is, for example, half the length of the vehicle width MW of the host vehicle M, but it is sufficient that a value different from the first predetermined width is set. The predetermined interval SI1 may be the same value as the predetermined interval FI1 or may be a different value.

[0053] The setting unit 130C further extracts, from among the multiple third reference objects RO4, a first object FP1 that intersects with the second reference object RO2 and an object PP1 immediately before the first object FP1. If an object that intersects with the second reference object RO2 cannot be extracted, the setting unit 130C resets the predetermined interval to a shorter value and sets multiple fourth reference objects RO4 again.

[0054] Next, the setting unit 130C additionally sets a plurality of third reference objects RO4 parallel to the first reference object RO1 in the direction from the extracted previous object PP1 to the first object FP1 at intervals shorter than the intervals of the plurality of third reference objects RO4. The setting unit 130C extracts a first object FP2 intersecting with the second reference object RO2 and an object PP2 immediately preceding the first object FP2 from among the plurality of fourth reference objects RO4 generated at short intervals in the same manner as in the case of FIG. 5. The setting unit 130C sets the first object FP(N) obtained by repeating the above process a predetermined number of times (N times) as the second intersection I2 between the second reference object RO2 and the fourth reference object RO4. Alternatively, the setting unit 130C may set the first object FP(N) as the second intersection I2 when the distance between the previous first object FP(N-1) and the current first object FP(N) becomes less than a threshold value (for example, 1 m) instead of repeating the above process a predetermined number of times. Through the above process, a first intersection I1 between the second reference object RO2 and the third reference object RO3, and a second intersection I2 between the second reference object RO2 and the fourth reference object RO4 are set.

[0055] The derivation unit 130D derives a start point S1 where the lane width of the merging lane L1 starts to decrease and an end point EP where the lane width of the merging lane L1 ends to decrease, which is information about the disappearance point of the merging lane L1, based on the first reference object RO1, the first intersection I1, and the second intersection I2. FIG. 7 is a diagram showing an example of the start point SP and end point EP of the merging lane L1 derived by the derivation unit 130D. As shown in FIG. 7, the derivation unit 130D extends a line segment RL connecting the first intersection I1 and the second intersection I2, and derives a point on the second reference object RO2 where the length of a perpendicular line drawn from the extended line segment RL to the first reference object RO1 is equal to the width LW of the lane L1 as the start point S1 of the merging lane L1. The derivation unit 130D further derives the intersection point of the extended line segment RL and the first reference object as the end point EP of the merging lane L1.

[0056] In the above description, the derivation unit 130D derives the start point S1 and the end point EP of the merging lane L1 using the first intersection I1 set using the third reference object RO3 and the second intersection I2 set using the third reference object RO3. However, the present invention is not limited to this configuration. For example, when the detection unit 130B detects the end point EP using any image processing method, the derivation unit 130D may derive the start point S1 based on the first intersection I1 and the end point EP. Specifically, the derivation unit 130D extends a line segment connecting the first intersection I1 and the end point EP, and derives, as the start point S1, a point on the second reference object RO2 where the length of a perpendicular line drawn from the extended line segment to the first reference object RO1 is equal to the width LW of the lane L1.

[0057] Thereafter, the behavior plan generating unit 140 acquires information on the start point S1 and the end point EP of the merging lane L1 from the recognition unit 130, and generates a target trajectory based on the information. FIG. 8 is a diagram showing an example of a target trajectory generated by the behavior plan generating unit 140. As shown in FIG. 8, the behavior plan generating unit 140 generates a target trajectory TT of the host vehicle M such that the host vehicle M completes a lane change before the host vehicle M reaches the end point EP of the merging lane L1. The second control unit 160 controls the steering and acceleration / deceleration of the host vehicle M without relying on the operation of the driver of the host vehicle M so that the host vehicle M travels along the generated target trajectory TT.

[0058] Next, the flow of processing of the mobile body control device according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the flow of processing of the mobile body control device according to this embodiment. The processing of this flowchart is executed by the mobile body control device at every predetermined control cycle (for example, 10 milliseconds).

[0059] First, the mobile object control device uses the image acquisition unit 16A to acquire an image of the external space of the vehicle M captured by the camera 10 (step S100). Next, the mobile object control device uses the merging lane determination unit 130A to determine whether the vehicle M is traveling in a merging lane based on the image acquired by the image acquisition unit 16A (step S101). If the merging lane determination unit 130A determines that the vehicle M is not traveling in a merging lane, the mobile object control device returns the process to step S100.

[0060] On the other hand, when it is determined that the host vehicle M is traveling in the merging lane, the mobile object control device detects, by image processing using the detection unit 130B, a first reference object RO1 that exists on the main line side of the host vehicle M in the image and extends in the traveling direction of the host vehicle M, and a second reference object RO2 that exists on the opposite side of the main line of the host vehicle M in the image and extends in the traveling direction of the host vehicle M (step S102). Next, the mobile object control device uses the setting unit 130C to virtually set a plurality of third reference objects that extend parallel to the first reference object RO1 with a first predetermined width on an imaginary plane virtually seen from above, and detects a first intersection with the second reference object RO2 (step S103). In addition, the mobile object control device uses the setting unit 130C to virtually set a plurality of fourth reference objects that extend parallel to the first reference object RO1 with a second predetermined width on the imaginary plane, and detects a second intersection with the second reference object RO2 (step S104).

[0061] Next, the mobile object control device derives the start point S1 and end point EP of the merging lane L1 based on the first reference object RO1, the first intersection point I1, and the second intersection point I2 using the derivation unit 130D (step S105). Next, the mobile object control device uses the action plan generation unit 140 to generate a target trajectory TT for the host vehicle M such that the host vehicle M completes the lane change before the host vehicle M reaches the end point EP of the merging lane L1. This ends the processing of this flowchart.

[0062] In the above embodiment, an example in which the mobile body control device of the present invention is applied to automatic driving has been described. However, the mobile body control device of the present invention is not limited to this configuration and can also be applied to manual driving. In that case, the mobile body control device of the present invention may further include a driving instruction unit, instead of the driving control unit, that generates a target trajectory based on the derived start point and end point of the merging lane and gives at least one of a steering instruction and an acceleration / deceleration instruction so that the occupant of the host vehicle M drives along the generated target trajectory. The driving instruction unit can be realized, for example, as a part of the function of the navigation device 50.

[0063] According to the present embodiment described above, the mobile object control device derives the start and end points of the merging lane based on the image captured by the camera 10, and determines the timing of lane change in consideration of the start and end points. This makes it possible to appropriately determine the timing of lane change when traveling in a merging lane.

[0064] The above-described embodiment can be expressed as follows. A storage device storing a program; a hardware processor; The hardware processor executes the program stored in the storage device, Acquire an image of the external space of the moving object; determining whether the moving object is traveling in a merging lane that disappears in a traveling direction of the moving object based on the image; When the merging lane determination unit determines that the moving body is traveling in a merging lane, a first reference object that is present on the main line side of the moving body in the image and extends in the traveling direction of the moving body, and a second reference object that is present on the opposite side of the main line of the moving body in the image and extends in the traveling direction of the moving body are detected by image processing; Virtually setting a plurality of third reference objects extending parallel to the first reference object with a first predetermined width on an imaginary plane viewed from above; deriving information about a disappearance point of the merging lane based on at least a first intersection point of the second reference object and the plurality of third reference objects; Mobile control device.

[0065] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0066] 10. Camera 16 Object recognition device 16A Image acquisition unit 100 Automatic driving control device 120 First Control Section 130 Recognition part 130A Merging lane judgement section 130B Detector 130C setting section 130D Lead-out section 140 Action Plan Generation Unit 160 Second Control Section

Claims

1. an image acquisition unit that acquires an image of an external space of the moving object; a merging lane determination unit that determines whether the moving object is traveling in a merging lane that disappears in a traveling direction of the moving object based on the image; a detection unit that detects, when the merging lane determination unit determines that the moving body is traveling in a merging lane, a road dividing line that is on the main line side of the moving body in the image and extends in the moving direction of the moving body, and a guard rail that is on the opposite side of the main line of the moving body in the image and extends in the moving direction of the moving body, by image processing; a setting unit that virtually sets a plurality of third reference objects extending parallel to the road dividing line with a first predetermined width on an imaginary plane virtually seen from above; a derivation unit that derives information regarding a disappearance point of the merging lane based on at least a first intersection point between the guardrail and the plurality of third reference objects, The detection unit detects an end point on the road dividing line at which a lane width of the merging lane ends decreasing, The derivation unit derives a start point at which a lane width of the merging lane starts to decrease, the start point being information regarding a location where the merging lane disappears, based on the first intersection point and the end point. Mobile control device.

2. an image acquisition unit that acquires an image of an external space of the moving object; a merging lane determination unit that determines whether the moving object is traveling in a merging lane that disappears in a traveling direction of the moving object based on the image; a detection unit that detects, when the merging lane determination unit determines that the moving body is traveling in a merging lane, a road dividing line that is on the main line side of the moving body in the image and extends in the moving direction of the moving body, and a guard rail that is on the opposite side of the main line of the moving body in the image and extends in the moving direction of the moving body, by image processing; a setting unit that virtually sets a plurality of third reference objects extending parallel to the road dividing line with a first predetermined width on an imaginary plane virtually seen from above; a derivation unit that derives information regarding a disappearance point of the merging lane based on at least a first intersection point between the guardrail and the plurality of third reference objects, the setting unit virtually sets a plurality of fourth reference objects extending parallel to the road dividing line with a second predetermined width different from the first predetermined width on an imaginary plane virtually seen from above; the derivation unit derives, based on the road dividing line, the first intersection, and the second intersections of the guardrail and the plurality of fourth reference objects, information about a location where the merging lane disappears, the information being a start point at which the lane width of the merging lane starts to decrease and an end point at which the lane width of the merging lane ends to decrease; Mobile control device.

3. the setting unit extracts, from among the plurality of third reference objects, a first object that intersects with the guardrail and an object immediately preceding the first object, sets a plurality of objects parallel to the road dividing line in a direction from the immediately preceding object to the first object at intervals shorter than the intervals between the plurality of third reference objects, and sets, as the first intersection, the first object obtained by repeating a process of extracting, from among the plurality of objects set at the short intervals, a first object that intersects with the guardrail and an object immediately preceding the first object, a predetermined number of times. The mobile object control device according to claim 1 or 2.

4. the setting unit extracts, from among the plurality of fourth reference objects, a first object that intersects with the guardrail and an object immediately preceding the first object, sets a plurality of objects parallel to the road dividing line in a direction from the immediately preceding object to the first object at intervals shorter than the intervals between the plurality of fourth reference objects, and sets, as the second intersection, the first object obtained by repeating a process of extracting, from among the plurality of objects set at the short intervals, a first object that intersects with the guardrail and an object immediately preceding the first object, a predetermined number of times. The mobile object control device according to claim 2 .

5. the derivation unit detects a third intersection point between the extended line segment and the road dividing line and a fourth intersection point on the guardrail where a length of a perpendicular line drawn from the extended line segment to the road dividing line is equal to the lane width by extending the line segment connecting the first intersection point and the second intersection point, and derives the fourth intersection point as a start point where the lane width of the merging lane starts to decrease and derives the third intersection point as an end point where the lane width of the merging lane ends to decrease. The mobile object control device according to claim 2 .

6. The first predetermined width is a width of the moving body. The mobile object control device according to any one of claims 1 to 5.

7. a driving control unit that generates a target trajectory of the moving body such that the moving body completes a lane change before the moving body reaches the end point of the merging lane derived by the derivation unit, and controls steering and acceleration / deceleration of the moving body so that the moving body travels along the generated target trajectory without depending on an operation of a driver of the moving body; The mobile object control device according to any one of claims 1 to 6.

8. A computer installed in a vehicle Acquire an image of the external space of the moving object; determining whether the moving object is traveling in a merging lane that disappears in a traveling direction of the moving object based on the image; When it is determined that the moving object is traveling in a merging lane, a road dividing line that is on the main line side of the moving object in the image and extends in the moving direction of the moving object, and a guard rail that is on the opposite side of the main line of the moving object in the image and extends in the moving direction of the moving object are detected by image processing; Virtually setting a plurality of third reference objects extending parallel to the road dividing line with a first predetermined width on an imaginary plane viewed from above; deriving information about a disappearance point of the merging lane based on at least a first intersection point between the guardrail and the plurality of third reference objects; detecting an end point on the road dividing line where the lane width of the merging lane ends decreasing; deriving a start point at which a lane width of the merging lane starts to decrease, which is information about a location where the merging lane disappears, based on the first intersection point and the end point; A method for controlling a moving object.

9. A computer installed in a moving vehicle Acquire an image of the external space of the moving object; determining whether or not the moving object is traveling in a merging lane that disappears in a traveling direction of the moving object based on the image; when it is determined that the moving body is traveling in a merging lane, detecting by image processing a road dividing line that is on the main line side of the moving body in the image and extends in the traveling direction of the moving body, and a guard rail that is on the opposite side of the main line of the moving body in the image and extends in the traveling direction of the moving body; a plurality of third reference objects are virtually set on an imaginary plane viewed from above, the third reference objects extending parallel to the road dividing line with a first predetermined width; deriving information regarding a disappearance point of the merging lane based on at least a first intersection point between the guardrail and the plurality of third reference objects; detecting an end point on the road dividing line where a lane width of the merging lane ends decreasing; deriving a start point at which a lane width of the merging lane starts to decrease, which is information about a location where the merging lane disappears, based on the first intersection point and the end point; program.

10. A computer installed in a vehicle Acquire an image of the external space of the moving object; determining whether the moving object is traveling in a merging lane that disappears in a traveling direction of the moving object based on the image; When it is determined that the moving object is traveling in a merging lane, a road dividing line that is on the main line side of the moving object in the image and extends in the moving direction of the moving object, and a guard rail that is on the opposite side of the main line of the moving object in the image and extends in the moving direction of the moving object are detected by image processing; Virtually setting a plurality of third reference objects extending parallel to the road dividing line with a first predetermined width on an imaginary plane viewed from above; deriving information about a disappearance point of the merging lane based on at least a first intersection point between the guardrail and the plurality of third reference objects; a plurality of fourth reference objects are virtually set so as to extend in parallel to the road dividing line with a second predetermined width different from the first predetermined width on an imaginary plane virtually seen from above; deriving a start point at which the lane width of the merging lane starts to decrease and an end point at which the lane width of the merging lane ends to decrease, which is information about a location where the merging lane disappears, based on the road dividing line, the first intersection, and second intersections of the guardrail and the plurality of fourth reference objects; A method for controlling a moving object.

11. A computer installed in a moving vehicle Acquire an image of the external space of the moving object; determining whether or not the moving object is traveling in a merging lane that disappears in a traveling direction of the moving object based on the image; when it is determined that the moving body is traveling in a merging lane, detecting by image processing a road dividing line that is on the main line side of the moving body in the image and extends in the traveling direction of the moving body, and a guard rail that is on the opposite side of the main line of the moving body in the image and extends in the traveling direction of the moving body; a plurality of third reference objects are virtually set on an imaginary plane viewed from above, the third reference objects extending parallel to the road dividing line with a first predetermined width; deriving information regarding a disappearance point of the merging lane based on at least a first intersection point between the guardrail and the plurality of third reference objects; a plurality of fourth reference objects are virtually set on an imaginary plane viewed from above, the fourth reference objects extending parallel to the road dividing line and having a second predetermined width different from the first predetermined width; deriving a start point at which the lane width of the merging lane starts to decrease and an end point at which the lane width of the merging lane ends to decrease, which is information about a location where the merging lane disappears, based on the road dividing line, the first intersection, and second intersections of the guardrail and the plurality of fourth reference objects; program.

Images