Mobile object control device, mobile object control method, and storage medium

Abstract

Provided are a mobile control device, mobile control method, and storage medium capable of appropriately determining the timing of a lane change when traveling in a merging lane. The mobile control device includes: an image acquisition unit that acquires an image captured of a space outside the mobile device; a merging lane determination unit that determines, based on the image, whether the mobile device is traveling in the merging lane; a detection unit that, when determining that the mobile device is traveling in the merging lane, detects a first reference object and a second reference object through image processing; a setting unit that hypothetically sets, in an imaginary plane viewed hypothetically from above, a plurality of third reference objects arranged to extend parallel to and have a first predetermined width relative to the first reference object; and a derivation unit that derives information related to a location where the merging lane disappears based on at least a first intersection point between the second reference object and the plurality of third reference objects.

Description

Technical Field

[0001] The present invention relates to a mobile body control device, a mobile body control method and a storage medium. Background Art

[0002] There is a known technology for determining whether the lane a vehicle is traveling in is a merging lane. For example, Japanese Patent Application Laid-Open No. 2020-27316 discloses a technology for determining whether the lane a vehicle is traveling in is a merging lane based on map information. Summary of the Invention

[0003] The technology described in Japanese Patent Application Laid-Open No. 2020-27316 determines whether a lane is a merging lane or a merging lane. However, this technology does not detect the start and end points of the lane width reduction in the merging lane. As a result, for example, during autonomous driving control of a vehicle, it is sometimes impossible to appropriately determine the timing of a lane change when traveling in a merging lane.

[0004] The present invention has been made in consideration of such circumstances, and one object thereof is to provide a mobile object control device, a mobile object control method, and a storage medium that can appropriately determine the timing of lane change when traveling in a merging lane.

[0005] The mobile object control device, mobile object control method, and storage medium of the present invention employ the following configurations.

[0006] (1): A mobile body control device according to one embodiment of the present invention comprises: an image acquisition unit for acquiring an image obtained by photographing an external space of the mobile body; a merging lane determination unit for determining, based on the image, whether the mobile body is traveling in a merging lane that disappears on the side of the moving direction of the mobile body; a detection unit for detecting, by image processing, a first reference object that exists on the main road side of the mobile body in the image and extends toward the moving direction of the mobile body, and a second reference object that exists on the side of the mobile body opposite to the main road in the image and extends toward the moving direction of the mobile body, when the merging lane determination unit determines that the mobile body is traveling in the merging lane; a setting unit for imaginarily setting, in an imaginary plane observed from above, a plurality of third reference objects that extend parallel to and have a first specified width relative to the first reference object; and a derivation unit for deriving information related to the location where the merging lane disappears based on at least a first intersection point between the second reference object and the plurality of third reference objects.

[0007] (2): In the scheme of (1) above, the detection unit detects the end point where the lane width of the merging lane ends decreasing, which is located on the first reference object, and the derivation unit derives the start point where the lane width of the merging lane begins to decrease as information related to the location where the merging lane disappears, based on the first intersection and the end point.

[0008] (3): In the scheme of (1) above, the setting unit hypothetically sets a plurality of fourth reference objects extending parallel to the first reference object and having a second prescribed width different from the first prescribed width in an imaginary plane hypothetically observed from above, and the deriving unit derives the starting point at which the lane width of the merging lane begins to decrease and the ending point at which the lane width of the merging lane ends to decrease as information related to the location where the merging lane disappears based on the first reference object, the first intersection, and the second intersection of the second reference object and the plurality of fourth reference objects.

[0009] (4): In any of the above schemes (1) to (3), the setting unit extracts the initial object that intersects with the second reference object and the object immediately before the initial object from the multiple third reference objects, sets multiple objects parallel to the first reference object at intervals shorter than the intervals between the multiple third reference objects in the direction of the initial object, and sets the initial object obtained by repeatedly extracting the initial object that intersects with the second reference object and the object immediately before the initial object from the multiple objects set at short intervals for a specified number of times as the first intersection point.

[0010] (5): In the above-mentioned scheme (3) or (4), the setting unit extracts the initial object intersecting with the second reference object and the object immediately preceding the initial object from the multiple fourth reference objects, sets multiple objects parallel to the first reference object at intervals shorter than the intervals between the multiple fourth reference objects in the direction of the initial object, and repeatedly extracts the initial object intersecting with the second reference object and the object immediately preceding the initial object from the multiple objects set at short intervals for a specified number of times, and sets the initial object obtained as the second intersection point.

[0011] (6): In any of the above schemes (3) to (5), the derivation unit extends the line segment connecting the first intersection and the second intersection, detects a third intersection of the extended line segment and the first reference object, and a fourth intersection on the second reference object whose length of a perpendicular line from the extended line segment to the first reference object is equal to the lane width, derives the third intersection as the starting point at which the lane width of the merging lane begins to decrease, and derives the fourth intersection as the ending point at which the lane width of the merging lane ends to decrease.

[0012] (7): In any one of the above (1) to (6), the first predetermined width is the width of the moving body.

[0013] (8): In any of the above schemes (1) to (7), the mobile body control device further includes a driving control unit that generates a target trajectory of the mobile body in which the mobile body completes the lane change before the mobile body reaches the end point of the merging lane derived by the derivation unit, and controls the steering and acceleration and deceleration of the mobile body independently of the operation of the driver of the mobile body so that the mobile body travels along the generated target trajectory.

[0014] (9): A mobile body control method according to one embodiment of the present invention causes a computer mounted on the mobile body to perform the following processing: obtaining an image obtained by photographing the external space of the mobile body, and based on the image, determining whether the mobile body is traveling on a merging lane that disappears on the side of the moving direction of the mobile body; if it is determined that the mobile body is traveling on the merging lane, detecting by image processing a first reference object that exists on the main road side of the mobile body in the image and extends toward the side of the moving direction of the mobile body, and a second reference object that exists on the side of the mobile body opposite to the main road in the image and extends toward the side of the moving direction of the mobile body; imaginarily setting a plurality of third reference objects that extend parallel to the first reference object with a first specified width in an imaginary plane that is observed from above; and deriving information related to the location where the merging lane disappears based on at least a first intersection point between the second reference object and the plurality of third reference objects.

[0015] (10): A storage medium according to one embodiment of the present invention stores a program that enables a computer mounted on a mobile body to obtain an image obtained by photographing an external space of the mobile body, and based on the image, to determine whether the mobile body is traveling on a merging lane that disappears on the side of the moving direction of the mobile body. When it is determined that the mobile body is traveling on the merging lane, a first reference object that exists in the image on the main road side of the mobile body and extends toward the moving direction of the mobile body, and a second reference object that exists in the image on the side of the mobile body opposite to the main road and extends toward the moving direction of the mobile body are detected by image processing. In an imaginary plane that is hypothetically observed from above, a plurality of third reference objects that extend parallel to the first reference object and have a first specified width are hypothetically set, and information related to the location where the merging lane disappears is derived based on at least a first intersection point between the second reference object and the plurality of third reference objects.

[0016] According to the solutions (1) to (10), it is possible to appropriately determine the timing of changing lanes when traveling in a merging lane. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 1 is a configuration diagram of a vehicle system 1 using the mobile object control device according to the present embodiment.

[0018] Figure 2 This is a functional structure diagram of the first control unit and the second control unit.

[0019] Figure 3 This is a diagram showing an example of a scene in which processing by the mobile object control device according to this embodiment is executed.

[0020] Figure 4 This is a diagram showing an example of a plurality of third reference objects set by the setting unit.

[0021] Figure 5 This is a diagram showing an example of a plurality of third reference objects additionally set by the setting unit.

[0022] Figure 6 This is a diagram showing an example of a plurality of fourth reference objects set by the setting unit.

[0023] Figure 7 This is a diagram showing an example of the start point and end point of the merging lane derived by the deriving unit.

[0024] Figure 8 This is a diagram showing an example of a target trajectory generated by the action plan generation unit.

[0025] Figure 9 This is a flowchart showing an example of the flow of processing of the mobile object control device according to this embodiment. DETAILED DESCRIPTION

[0026] Embodiments of the mobile object control device, mobile object control method, and storage medium of the present invention are described below with reference to the accompanying drawings. The mobile object in the present invention refers to a four-wheeled vehicle, a two-wheeled vehicle, a micro-mobile object, a robot, and the like. In the following description, the mobile object is assumed to be a four-wheeled vehicle.

[0027] [Overall structure]

[0028] Figure 1 This is a structural diagram of a vehicle system 1 that utilizes the mobile vehicle control device of this embodiment. The vehicle equipped with vehicle system 1 is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its driving source is an internal combustion engine such as a diesel engine or gasoline engine, an electric motor, or a combination thereof. The electric motor operates using power generated by a generator connected to the internal combustion engine, or power discharged from a secondary battery or fuel cell.

[0029] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) device 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driving operating element 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are interconnected via multiple communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, and wireless communication networks. It should be noted that Figure 1 The structure shown is only an example, and part of the structure may be omitted or another structure may be added.

[0030] The camera 10 is, for example, a digital camera utilizing a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is mounted anywhere on the vehicle (hereinafter, the vehicle M) equipped with the vehicle system 1. To image the front, the camera 10 is mounted on the top of the windshield, behind the rearview mirror, or elsewhere. For example, the camera 10 periodically and repeatedly captures images of the surroundings of the vehicle M. The camera 10 may also be a stereo camera.

[0031] The radar device 12 radiates radio waves, such as millimeter waves, around the vehicle M and detects the radio waves (reflected waves) reflected by objects to detect at least the object's position (range and direction). The radar device 12 is mounted anywhere on the vehicle M. The radar device 12 can also detect the object's position and velocity using the FM-CW (Frequency Modulated Continuous Wave) method.

[0032] LIDAR 14 irradiates light (or electromagnetic waves with a wavelength close to that of light) around the vehicle M and measures the scattered light. LIDAR 14 detects the distance to an object based on the time between light emission and light reception. The irradiated light is, for example, a pulsed laser. LIDAR 14 is mounted anywhere on the vehicle M.

[0033] The object recognition device 16 performs sensor fusion processing on some or all of the detection results obtained by the camera 10, the radar device 12, and the LIDAR 14 to identify the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the automatic driving control device 100. The object recognition device 16 can directly transmit the detection results of the camera 10, the radar device 12, and the LIDAR 14 to the automatic driving control device 100. In this embodiment, the object recognition device 16 includes an image acquisition unit 16A. The image acquisition unit 16A acquires an image of the space outside the vehicle captured by the camera 10 and outputs it to the automatic driving control device 100, which will be described later.

[0034] The communication device 20 communicates with other vehicles around the host vehicle M using, for example, a cellular network, Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or communicates with various server devices via a wireless base station.

[0035] The HMI 30 presents various information to the occupants of the vehicle M and receives input operations from the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys, and the like.

[0036] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects angular velocity about a vertical axis, an azimuth sensor that detects the orientation of the host vehicle M, and the like.

[0037] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M can also be determined or supplemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, buttons, etc. The navigation HMI 52 can also be partially or entirely shared with the aforementioned HMI 30. The route determination unit 53, for example, refers to the first map information 54 to determine a route (hereinafter referred to as a route on the map) from the position of the vehicle M determined by the GNSS receiver 51 (or an arbitrary position input) to the destination input by the occupant using the navigation HMI 52. The first map information 54 is information that represents the shape of the road using, for example, road links representing the road and nodes connected by the road links. The first map information 54 may also include road curvature, POI (Point of Interest) information, and the like. The route on the map is output to the MPU 60. The navigation device 50 may also provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may also be implemented as a function of a terminal device such as a smartphone or tablet computer held by the passenger. The navigation device 50 may also transmit the current location and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

[0038] The MPU 60, for example, includes a recommended lane determination unit 61, which stores second map information 62 in a storage device such as a HDD or flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into multiple blocks (e.g., every 100 meters in the vehicle's direction of travel) and, referring to the second map information 62, determines a recommended lane for each block. The recommended lane determination unit 61 determines the lane from the left that the vehicle M should travel on. If the route on the map branches, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable route to the branch destination.

[0039] The second map information 62 is higher-precision map information than the first map information 54. The second map information 62 includes, for example, information about lane centers or lane boundaries. Furthermore, the second map information 62 may include road information, traffic restriction information, address information (address and postal code), facility information, and telephone number information. The second map information 62 can be updated at any time by communicating with other devices via the communication device 20.

[0040] Driving control elements 80 include, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a special-shaped steering wheel, a joystick, and other operating elements. Sensors are mounted on driving control elements 80 to detect the amount of operation or the presence or absence of an operation. These detection results are output to the automatic driving control device 100, or to some or all of the driving force output device 200, the braking device 210, and the steering device 220.

[0041] The automatic driving control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are each implemented by executing a program (software) on a hardware processor such as a CPU (Central Processing Unit). In addition, some or all of these components can be implemented by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or by a combination of software and hardware. The program can be pre-stored in a storage device (a storage device having a non-transitory storage medium) such as an HDD or flash memory of the automatic driving control device 100, or can be stored in a removable storage medium such as a DVD or CD-ROM and installed in the HDD or flash memory of the automatic driving control device 100 by assembling the storage medium (non-transitory storage medium) in a drive device. The object recognition device 16 and the automatic driving control device 100 are collectively an example of a “mobile body control device”, and the action plan generation unit 140 and the second control unit 160 are collectively an example of a “driving control unit”.

[0042] Figure 2The following is a functional structure diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, an identification unit 130 and an action plan generation unit 140. The first control unit 120, for example, concurrently implements functions based on AI (Artificial Intelligence) and functions based on pre-assigned models. For example, the "intersection recognition" function can be implemented by concurrently executing intersection recognition based on deep learning and other methods and recognition based on pre-assigned conditions (such as the presence of pattern-matchable signals and road signs), scoring both and comprehensively evaluating them. This ensures the reliability of autonomous driving.

[0043] The recognition unit 130 recognizes the position, velocity, acceleration, and other states of objects in the vicinity of the vehicle M based on information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. The position of the object is recognized, for example, as a position on an absolute coordinate with a representative point (center of gravity, drive shaft center, etc.) of the vehicle M as the origin, and is used for control. The position of the object can be represented by a representative point such as the center of gravity or a corner of the object, or by a represented area. The so-called "state" of the object may also include the acceleration, jerk, or "action state" of the object (for example, whether a lane change is in progress or about to be made).

[0044] In addition, the recognition unit 130, for example, recognizes the lane in which the vehicle M is traveling (driving lane). For example, the recognition unit 130 recognizes the driving lane by comparing the pattern of road dividing lines (e.g., the arrangement of solid lines and dashed lines) obtained based on the second map information 62 with the pattern of road dividing lines around the vehicle M recognized based on the image captured by the camera 10. It should be noted that the recognition unit 130 is not limited to road dividing lines, and can also recognize the driving lane by recognizing road dividing lines, driving road boundaries (road boundaries) including shoulders, curbs, central medians, guardrails, etc. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results based on the INS can also be taken into consideration. In addition, the recognition unit 130 recognizes temporary stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0045] When identifying a driving lane, the recognition unit 130 identifies the position and posture of the host vehicle M relative to the driving lane. For example, the recognition unit 130 may identify the deviation of the host vehicle M's reference point from the lane center and the angle formed by the vehicle M's travel direction with respect to a line connecting the lane centers as the relative position and posture of the host vehicle M relative to the driving lane. Alternatively, the recognition unit 130 may identify the position of the host vehicle M's reference point relative to either side of the driving lane (a road dividing line or a road boundary) as the relative position of the host vehicle M relative to the driving lane.

[0046] In the present embodiment, the recognition unit 130 particularly includes a merging lane determination unit 130A, a detection unit 130B, a setting unit 130C, and a derivation unit 130D, but details of their functions will be described later.

[0047] The action plan generation unit 140 generates a target trajectory for the vehicle M to automatically (independent of the driver's operation) travel in the future, in a manner that, in principle, travels on the recommended lane determined by the recommended lane determination unit 61 and can cope with the surrounding conditions of the vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of locations (track points) that the vehicle M should arrive at. A track point is a location that the vehicle M should arrive at for every specified driving distance (for example, a few meters) along the way. Different from this, a target speed and a target acceleration for every specified sampling time (for example, a few tenths of a second) are generated as part of the target trajectory. In addition, a track point can also be a location that the vehicle M should arrive at at the sampling moment for every specified sampling time. In this case, the information on the target speed and target acceleration is expressed at intervals between track points.

[0048] When generating a target trajectory, the action plan generation unit 140 can set an autonomous driving event. These events include constant speed driving, low-speed following driving, lane change, diverging, merging, and takeover. The action plan generation unit 140 generates a target trajectory corresponding to the activated event.

[0049] The second control unit 160 controls the driving force output device 200 , the braking device 210 , and the steering device 220 so that the host vehicle M passes through the target trajectory generated by the action plan generation unit 140 at a predetermined time.

[0050] return Figure 2 The second control unit 160, for example, includes an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires the information of the target track (track point) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the braking device 210 based on the speed element attached to the target track stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target track stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is implemented, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 performs a combination of feedforward control corresponding to the curvature of the road in front of the vehicle M and feedback control based on the deviation from the target track.

[0051] The driving force output device 200 outputs the driving force (torque) used to propel the vehicle to the drive wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls them. The ECU controls the aforementioned components based on information input from the second control unit 160 or from the driving control element 80.

[0052] Braking device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor based on information input from the second control unit 160 or from the driver's operating element 80, outputting braking torque to each wheel in response to the braking operation. Braking device 210 may include a mechanism that transmits hydraulic pressure generated by operation of the brake pedal included in the driver's operating element 80 to the cylinder via a master cylinder as a backup. It should be noted that braking device 210 is not limited to the structure described above and may also be an electronically controlled hydraulic braking device that controls an actuator based on information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0053] The steering system 220 includes, for example, a steering ECU and an electric motor. The electric motor applies force to, for example, a rack-and-pinion mechanism to change the direction of the steering wheel. The steering ECU drives the electric motor to change the direction of the steering wheel based on information input from the second control unit 160 or information input from the driving operating element 80.

[0054] [action]

[0055] Next, refer to Figures 3 to 8 , the processing of the mobile body control device of this embodiment is explained. Figure 3 : is a diagram showing an example of a scene in which the processing of the mobile object control device of this embodiment is executed. Figure 3 In the example, the vehicle M is traveling in the merging lane L1 through automated driving, executing a lane change and entering the merging lane L2. Typically, the lane width of a merging lane begins decreasing at a certain point and ends at a different point ahead, reaching zero. Therefore, the vehicle control device must determine the timing for executing a lane change by first knowing the starting point and the ending point of the lane width reduction in the merging lane. The present invention derives these starting and ending points and utilizes them to determine the timing for executing a lane change.

[0056] The merging lane determination unit 130A, upon acquiring an image of the space outside the host vehicle M captured by the camera 10 from the image acquisition unit 16A, determines based on the image whether the host vehicle M is traveling in a merging lane that disappears in the direction of travel of the host vehicle M. Specifically, for example, the merging lane determination unit 130A determines that the host vehicle M is traveling in a merging lane when the second map information 62 indicates travel in a merging lane, when the image of the space outside the host vehicle M indicates that the left and right road dividing lines (or guardrails) of the travel lane intersect, or when the image of the space outside the host vehicle M indicates a dotted line shape unique to a merging road.

[0057] When the merging lane determination unit 130A determines that the vehicle M is traveling in the merging lane, the detection unit 130B detects, through image processing, a first reference object RO1 that exists on the main road side of the vehicle M and extends toward the direction of travel of the vehicle M in the image obtained from the image acquisition unit 16A, and a second reference object RO2 that exists on the opposite side of the main road from the vehicle M and extends toward the direction of travel of the vehicle M in the image. Specifically, for example, the detection unit 130B extracts edge points with a large brightness difference from adjacent pixels in the image obtained from the image acquisition unit 16A, and detects the left and right first reference objects RO1 and second reference objects RO2 in the image based on the length and shape of the extended contour connecting the edge points. Here, the first reference object RO1 and the second reference object RO2 are, for example, road dividing lines and guardrails. Figure 3 In the example, the detection unit 130B detects the road dividing line as the first reference object RO1 and detects the guardrail as the second reference object RO2.

[0058] The setting unit 130C projects the image acquired by the image acquisition unit 16A onto an imaginary plane viewed virtually from above. In this imaginary plane, a plurality of third reference objects RO3 are imaginarily set, extending parallel to the first reference object detected by the detection unit 130B at a first predetermined width. The setting unit 130C may also perform this processing on the image plane. The same applies to the other processing described below.

[0059] Figure 4 130C is a diagram showing an example of a plurality of third reference objects RO3 set by the setting unit 130C. Figure 4 In the example, the setting unit 130C sets a plurality of dot-shaped third reference objects RO3 at predetermined intervals FI1 with a first predetermined width spaced apart from the first reference object RO1. Here, the first predetermined width is, for example, the vehicle width MW of the host vehicle M.

[0060] The setting unit 130C further extracts the first object FP1 intersecting with the second reference object RO2 and the object PP1 immediately preceding the first object FP1 from the plurality of third reference objects RO3. If no object intersecting with the second reference object RO2 is extracted, the setting unit 130C resets the predetermined interval to a shorter value and sets a plurality of third reference objects RO3 again.

[0061] Next, the setting unit 130C additionally sets a plurality of third reference objects RO3 parallel to the first reference object RO1 at intervals shorter than those of the plurality of third reference objects RO3 , starting from the extracted immediately preceding object PP1 toward the initial object FP1 . Figure 5 130C is a diagram showing an example of a plurality of third reference objects RO3 additionally set by the setting unit 130C. Figure 5 In the embodiment, the setting unit 130C sets a plurality of third reference objects RO3 at a predetermined interval FI2, starting from the immediately preceding object PP1 and moving toward the initial object FP1, with a first predetermined width spaced apart from the first reference object RO1. The predetermined interval FI2 is set shorter than the predetermined interval FI1. The setting unit 130C extracts the initial object FP2 intersecting with the second reference object RO2 and the immediately preceding object PP2 of the initial object FP2 from the plurality of third reference objects RO3 generated at a short interval. Figure 5 As shown, the first object FP2 extracted at this time is set closer to the host vehicle M than the previous first object FP1. Setting unit 130C sets the first object FP(N), obtained by repeating the above process a predetermined number of times (N), as the first intersection I1 between the second reference object RO2 and the third reference object RO3. Alternatively, setting unit 130C may set the first object FP(N) as the first intersection I1 when the distance between the previous first object FP(N-1) and the current first object FP(N) becomes less than a threshold value (e.g., 1 meter) without repeating the above process a predetermined number of times.

[0062] Next, the setting unit 130C virtually sets, in addition to the plurality of third reference objects RO3 , a plurality of fourth reference objects RO4 arranged to extend parallel to the first reference object RO1 and have a second predetermined width different from the first predetermined width. Figure 6 is a diagram showing an example of a plurality of fourth reference objects RO4 set by the setting unit 130C. Figure 6In the example, the setting unit 130C sets a plurality of dot-shaped fourth reference objects RO4 at predetermined intervals SI1, spaced apart from the first reference object RO1 by a second predetermined width. Here, the second predetermined width is, for example, half the width MW of the host vehicle M, but may be a different value from the first predetermined width. The predetermined interval SI1 may be the same as or different from the predetermined interval FI1.

[0063] The setting unit 130C further extracts the first object FP1 intersecting with the second reference object RO2 and the object PP1 immediately preceding the first object FP1 from the plurality of third reference objects RO4. If no object intersecting with the second reference object RO2 is extracted, the setting unit 130C resets the predetermined interval to a shorter value and sets a plurality of fourth reference objects RO4.

[0064] Next, the setting unit 130C sets a plurality of third reference objects RO4 parallel to the first reference object RO1 at intervals shorter than those of the plurality of third reference objects RO4, starting from the extracted immediately preceding object PP1 in the direction of the initial object FP1. Figure 5 Similarly, among the multiple fourth reference objects RO4 generated at short intervals, the initial object FP2 that intersects with the second reference object RO2 and the object PP2 immediately preceding the initial object FP2 are extracted. Setting unit 130C sets the initial object FP(N), obtained by repeating the above process a predetermined number of times (N), as the second intersection point I2 between the second reference object RO2 and the fourth reference object RO4. Alternatively, setting unit 130C may set the initial object FP(N) as the second intersection point I2 when the distance between the previous initial object FP(N-1) and the current initial object FP(N) becomes less than a threshold value (e.g., 1 meter) without performing the above process a predetermined number of times. The above process sets the first intersection point I1 between the second reference object RO2 and the third reference object RO3, and the second intersection point I2 between the second reference object RO2 and the fourth reference object RO4.

[0065] The deriving unit 130D derives a start point SP where the lane width of the merging lane L1 starts to decrease and an end point EP where the lane width of the merging lane L1 ends to decrease, as information on the location where the merging lane L1 disappears, based on the first reference object RO1, the first intersection I1, and the second intersection I2. Figure 7 1 is a diagram showing an example of the start point SP and the end point EP of the merging lane L1 derived by the deriving unit 130D. Figure 7As shown, the derivation unit 130D extends the line segment RL connecting the first intersection point I1 and the second intersection point I2, and derives a point on the second reference object RO2 where the length of the perpendicular from the extended line segment RL to the first reference object RO1 is equal to the width LW of the lane L1, as the starting point SP of the merging lane L1. The derivation unit 130D also derives the intersection point of the extended line segment RL with the first reference object as the ending point EP of the merging lane L1.

[0066] It should be noted that in the above description, the derivation unit 130D uses the first intersection I1 and the second intersection I2 set using the third reference object RO3 to derive the starting point SP and the end point EP of the merging lane L1. However, the present invention is not limited to this configuration. For example, if the detection unit 130B detects the end point EP using any image processing method, the derivation unit 130D may derive the starting point SP based on the first intersection I1 and the end point EP. Specifically, the derivation unit 130D extends the line segment connecting the first intersection I1 and the end point EP and derives a point on the second reference object RO2 where the length of the perpendicular line from the extended line segment to the first reference object RO1 is equal to the width LW of the lane L1 as the starting point SP.

[0067] Thereafter, the action plan generating unit 140 acquires information on the start point SP and the end point EP of the merging lane L1 from the identifying unit 130 , and generates a target trajectory based on the information. Figure 8 : is a diagram showing an example of a target trajectory generated by the action plan generation unit 140. Figure 8 As shown, the action plan generation unit 140 generates a target trajectory TT of the vehicle M such that the vehicle M completes the lane change before reaching the end point EP of the merging lane L1. The second control unit 160 controls the steering and acceleration / deceleration of the vehicle M independently of the driver's operation so that the vehicle M travels along the generated target trajectory TT.

[0068] Next, refer to Figure 9 , the processing flow of the mobile body control device of this embodiment is explained. Figure 9 This is a flowchart showing an example of the flow of processing by the mobile object control device of this embodiment. The processing in this flowchart is executed by the mobile object control device every predetermined control cycle (for example, 10 milliseconds).

[0069] First, the mobile object control device uses the image acquisition unit 16A to acquire an image of the space outside the host vehicle M captured by the camera 10 (step S100). Next, the mobile object control device uses the merging lane determination unit 130A to determine whether the host vehicle M is traveling in a merging lane based on the image acquired by the image acquisition unit 16A (step S101). If the merging lane determination unit 130A determines that the host vehicle M is not traveling in a merging lane, the mobile object control device returns the process to step S100.

[0070] On the other hand, if it is determined that the host vehicle M is traveling in a merging lane, the mobile object control device uses the detection unit 130B to detect, through image processing, a first reference object RO1 located on the main road side of the host vehicle M and extending in the direction of travel of the host vehicle M, and a second reference object RO2 located on the opposite side of the main road from the host vehicle M and extending in the direction of travel of the host vehicle M (step S102). Next, the mobile object control device uses the setting unit 130C to hypothetically set a plurality of third reference objects extending parallel to the first reference object RO1 at a first predetermined width in a hypothetical plane viewed hypothetically from above, and detects a first intersection point with the second reference object RO2 (step S103). Furthermore, the mobile object control device uses the setting unit 130C to hypothetically set a plurality of fourth reference objects extending parallel to the first reference object RO1 at a second predetermined width in the hypothetical plane, and detects a second intersection point with the second reference object RO2 (step S104).

[0071] Next, the mobile control device uses the derivation unit 130D to derive the start point SP and end point EP of the merging lane L1 based on the first reference object RO1, the first intersection I1, and the second intersection I2 (step S105). Next, the mobile control device uses the action plan generation unit 140 to generate a target trajectory TT for the vehicle M, ensuring that the vehicle M completes the lane change before reaching the end point EP of the merging lane L1. This concludes the processing in this flowchart.

[0072] It should be noted that the above embodiments illustrate an example in which the mobile control device of the present invention is applied to automated driving. However, the mobile control device of the present invention is not limited to this configuration and can also be applied to manual driving. In this case, the mobile control device of the present invention may further include a driving instruction unit, in place of the driving control unit. This driving instruction unit generates a target trajectory based on the derived starting and ending points of the merging lane and provides at least one of a steering instruction and an acceleration / deceleration instruction to guide the occupants of the host vehicle M to drive along the generated target trajectory. The driving instruction unit may be implemented, for example, as part of the functions of the navigation device 50.

[0073] According to the present embodiment described above, the vehicle control device derives the start and end points of the merging lane based on images captured by the camera 10, and determines the timing of a lane change based on these start and end points. This allows for appropriate lane change timing when traveling in the merging lane.

[0074] The above-described embodiment can be expressed as follows.

[0075] A mobile object control device comprising:

[0076] a storage device storing a program; and

[0077] Hardware processor,

[0078] Executing the program stored in the storage device by the hardware processor,

[0079] Acquire an image of the external space of the moving object,

[0080] determining, based on the image, whether the moving object is traveling in a merging lane that disappears on the moving direction side of the moving object,

[0081] When it is determined that the moving body is traveling in a merging lane, a first reference object that exists on the main road side of the moving body in the image and extends toward the direction of travel of the moving body, and a second reference object that exists on the opposite side of the main road from the moving body in the image and extends toward the direction of travel of the moving body are detected through image processing.

[0082] A plurality of third reference objects are virtually set in an imaginary plane virtually viewed from above and arranged to extend parallel to the first reference object with a first predetermined width.

[0083] Information related to a location where the merging lane disappears is derived based on at least a first intersection point between the second reference object and the plurality of third reference objects.

[0084] While specific embodiments of the present invention have been described above, the present invention is not limited to these embodiments at all, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A mobile body control device, wherein: The mobile body control device comprises: an image acquisition unit that acquires an image obtained by photographing a space outside the moving object; a merging lane determination unit that determines, based on the image, whether the mobile body is traveling in a merging lane that disappears on the side of the moving direction of the mobile body; a detection unit configured to detect, when the merging lane determination unit determines that the mobile body is traveling in a merging lane, by image processing a first reference object that is present on the main road side of the mobile body in the image and is arranged extending toward the direction of travel of the mobile body, and a second reference object that is present on the side of the mobile body opposite to the main road in the image and is arranged extending toward the direction of travel of the mobile body; a setting unit configured to virtually set a plurality of third reference objects arranged to extend parallel to and have a first predetermined width relative to the first reference object in an imaginary plane virtually viewed from above; and A deriving unit derives information related to a location where the merging lane disappears based on at least a first intersection point between the second reference object and the plurality of third reference objects.

2. The mobile body control device according to claim 1, wherein: The detection unit detects an end point on the first reference object where the lane width of the merging lane ends decreasing. The deriving unit derives a starting point at which the lane width of the merging lane starts to decrease as information on a location where the merging lane disappears, based on the first intersection point and the end point.

3. The mobile body control device according to claim 1, wherein: The setting unit virtually sets a plurality of fourth reference objects arranged to extend parallel to the first reference object and have a second predetermined width different from the first predetermined width in an imaginary plane virtually viewed from above. The derivation unit derives, as information related to a location where the merging lane disappears, a start point at which the lane width of the merging lane begins to decrease and an end point at which the lane width of the merging lane ends to decrease, based on the first reference object, the first intersection, and second intersections of the second reference object and the plurality of fourth reference objects.

4. The mobile object control device according to any one of claims 1 to 3, wherein: The setting unit extracts the initial object that intersects with the second reference object and the object immediately before the initial object from the multiple third reference objects, sets multiple objects parallel to the first reference object at intervals shorter than the intervals between the multiple third reference objects in the direction of the initial object, and sets the initial object obtained by repeatedly extracting the initial object that intersects with the second reference object and the object immediately before the initial object from the multiple objects set at short intervals for a specified number of times as the first intersection point.

5. The mobile body control device according to claim 3, wherein: The setting unit extracts the initial object that intersects with the second reference object and the object immediately before the initial object from the multiple fourth reference objects, sets multiple objects parallel to the first reference object at intervals shorter than the intervals between the multiple fourth reference objects in the direction of the initial object, and sets the initial object obtained by repeatedly extracting the initial object that intersects with the second reference object and the object immediately before the initial object from the multiple objects set at short intervals for a specified number of times as the second intersection point.

6. The mobile body control device according to claim 3 or 5, wherein: The derivation unit extends a line segment connecting the first intersection point and the second intersection point, detects a third intersection point between the extended line segment and the first reference object, and a fourth intersection point on the second reference object whose length of a perpendicular line drawn from the extended line segment to the first reference object is equal to the lane width, derives the third intersection point as a starting point at which the lane width of the merging lane begins to decrease, and derives the fourth intersection point as an ending point at which the lane width of the merging lane ends to decrease.

7. The mobile object control device according to any one of claims 1 to 3, wherein: The first predetermined width is the width of the moving object.

8. The mobile object control device according to any one of claims 1 to 3, wherein: The invention further includes a driving control unit that generates a target trajectory of the moving body in which the moving body completes the lane change before the moving body reaches the end point of the merging lane derived by the derivation unit, and controls the steering and acceleration and deceleration of the moving body independently of the operation of the driver of the moving body so that the moving body travels along the generated target trajectory.

9. A method for controlling a moving object, wherein: The mobile object control method causes a computer mounted on the mobile object to perform the following processing: Acquire an image of the external space of the moving object, determining, based on the image, whether the moving object is traveling in a merging lane that disappears on the moving direction side of the moving object, When it is determined that the moving body is traveling in a merging lane, a first reference object that exists on the main road side of the moving body in the image and extends toward the direction of travel of the moving body, and a second reference object that exists on the side of the moving body opposite to the main road in the image and extends toward the direction of travel of the moving body are detected through image processing. A plurality of third reference objects are virtually set in an imaginary plane virtually viewed from above and arranged to extend parallel to the first reference object with a first predetermined width. Information related to a location where the merging lane disappears is derived based on at least a first intersection point between the second reference object and the plurality of third reference objects.

10. A storage medium storing a program, wherein: The program causes the computer mounted on the mobile object to perform the following processing: Acquire an image of the external space of the moving object, determining, based on the image, whether the moving object is traveling in a merging lane that disappears on the moving direction side of the moving object, When it is determined that the moving body is traveling in a merging lane, a first reference object that exists on the main road side of the moving body in the image and extends toward the direction of travel of the moving body, and a second reference object that exists on the side of the moving body opposite to the main road in the image and extends toward the direction of travel of the moving body are detected through image processing. A plurality of third reference objects are virtually set in an imaginary plane virtually viewed from above and arranged to extend parallel to the first reference object with a first predetermined width. Information related to a location where the merging lane disappears is derived based on at least a first intersection point between the second reference object and the plurality of third reference objects.

Images