Vehicle control system

Abstract

This improves the accuracy of recognizing lane markings in sections with significant curves or road gradients. [Solution] The vehicle control system comprises a camera that photographs the area around the vehicle, an electromagnetic wave sensor that irradiates electromagnetic waves toward the area around the vehicle and receives the reflected waves, an area recognition unit that recognizes lane markings in front of the vehicle in the direction of travel based on the detection result of at least one of the camera and the electromagnetic wave sensor, and a driving control unit that performs driving control of the vehicle based on the recognition result of the area recognition unit. The vehicle's track includes a straight section where both the degree of curvature and the road surface gradient are below a threshold, and a specific section where at least one of the degree of curvature or the road surface gradient is above a threshold. The area recognition unit performs a first recognition process as a lane marking recognition process, and in the first recognition process, it prioritizes the detection result of the camera over the detection result of the electromagnetic wave sensor to recognize the lane markings in the straight section, and prioritizes the detection result of the electromagnetic wave sensor over the detection result of the camera to recognize the lane markings in the specific section.

Description

Technical Field

[0006] , ,

[0005] , ,

[0001] The present invention relates to a vehicle control system.

Background Art

[0002] In recent years, efforts have been actively made to provide a sustainable transportation system that takes into account people in vulnerable positions among traffic participants. In order to further improve traffic safety and convenience towards this realization, research and development on driving support technologies have been carried out. [[ID=​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​[Means for solving the problem]

[0007] To solve the above problems, one aspect of the present invention provides a vehicle control system comprising: a camera for photographing the area around a vehicle; an electromagnetic wave sensor for irradiating electromagnetic waves toward the area around the vehicle and receiving the reflected waves; an area recognition unit for recognizing lane markings in front of the vehicle in the direction of travel based on the detection result of at least one of the camera and the electromagnetic wave sensor; and a driving control unit for performing driving control of the vehicle based on the recognition result of the area recognition unit, wherein the vehicle's track includes a straight section where both the degree of curvature and the road surface gradient are below a threshold, and a specific section where at least one of the degree of curvature or the road surface gradient is equal to or greater than the threshold, and the area recognition unit performs a first recognition process as a lane marking recognition process, and in the first recognition process, the detection result of the camera is given priority over the detection result of the electromagnetic wave sensor to recognize the lane markings in the straight section, and the detection result of the electromagnetic wave sensor is given priority over the detection result of the camera to recognize the lane markings in the specific section. [Effects of the Invention]

[0008] According to the above embodiment, it is possible to provide a vehicle control system that can improve the accuracy of recognizing lane markings in sections with a large degree of curvature or road surface gradient. [Brief explanation of the drawing]

[0009] [Figure 1] Block diagram showing a vehicle to which the vehicle control system according to the embodiment is applied. [Figure 2] Table showing the recognition process of lane lines by the surrounding recognition unit according to the embodiment. [Figure 3] Plan view showing the vehicle according to the embodiment traveling in a straight section. [Figure 4] Plan view showing the vehicle according to the embodiment traveling through a curved section. [Figure 5] Plan view showing the state in which the vehicle according to the embodiment enters a straight section from a curved section. [Figure 6]Flowchart showing an example of vehicle control execution according to the embodiment (Example 1) [Figure 7] Flowchart showing an example of vehicle control execution according to the embodiment (Example 2) [Figure 8] Flowchart showing an example of vehicle control execution 3 according to the embodiment. [Modes for carrying out the invention]

[0010] The following describes an embodiment of the vehicle control system 1 with reference to the drawings.

[0011] <Vehicle 3> Figure 1 is a block diagram showing a vehicle 3 to which the vehicle control system 1 is applied. For example, vehicle 3 is an automobile. In other embodiments, vehicle 3 may be a vehicle other than an automobile (for example, a two-wheeled vehicle).

[0012] Vehicle 3 includes a drive unit 5, a brake unit 6, a steering unit 7, a navigation unit 8, a driver control unit 9, an external sensor 11, a vehicle sensor 12, an HMI 13 (Human Machine Interface), and a control device 15. In this embodiment, the external sensor 11, the vehicle sensor 12, the HMI 13, and the control device 15 constitute the vehicle control system 1.

[0013] The drive unit 5 is a device that provides driving force to the vehicle 3. The drive unit 5 includes a drive source that generates driving force to move the vehicle 3. For example, the drive source is composed of an internal combustion engine and / or an electric motor.

[0014] The braking device 6 is a device that applies braking force to the vehicle 3. For example, the braking device 6 includes a brake caliper that presses pads against a brake rotor and an electric cylinder that supplies hydraulic pressure to the brake caliper.

[0015] The steering device 7 is a device that changes the steering angle of the wheels by steering the wheels. For example, the steering device 7 includes a rack and pinion mechanism connected to the wheels and an electric motor that drives the rack and pinion mechanism.

[0016] The navigation device 8 is a device that provides route guidance to the destination of the vehicle 3 and the like. The navigation device 8 stores map information. The navigation device 8 identifies the current position of the vehicle 3 based on GNSS signals (for example, GPS signals) received from artificial satellites.

[0017] The driving operator 9 is a device that receives driving operations by the occupants of the vehicle 3 (hereinafter simply referred to as "occupants"). The driving operator 9 includes a steering operator 22 (for example, a steering wheel) that receives a steering operation of the vehicle 3 by the occupant, an acceleration operator 23 (for example, an accelerator pedal) that receives an acceleration operation of the vehicle 3 by the occupant, and a deceleration operator 24 (for example, a brake pedal) that receives a deceleration operation of the vehicle 3 by the occupant.

[0018] The external sensor 11 is a device that acquires information about the surroundings of the vehicle 3. The external sensor 11 includes a plurality of cameras 26, a plurality of radars 27, and a plurality of lidars 28 (LiDAR). Each camera 26 captures an image of an object (other vehicles, pedestrians, structures on the road, lane lines, etc.) existing around the vehicle 3. Each radar 27 emits radio waves such as millimeter waves toward the surroundings of the vehicle 3 and detects the position of an object existing around the vehicle 3 by receiving the reflected wave. Each lidar 28 is an example of an electromagnetic wave sensor. Each lidar 28 irradiates light (an example of electromagnetic waves) such as infrared rays toward the surroundings of the vehicle 3 and detects the position of an object existing around the vehicle 3 by receiving the reflected wave.

[0019] The vehicle sensor 12 is a sensor that detects the driving state of the vehicle 3. The vehicle sensor 12 includes a vehicle speed sensor 30 that detects the vehicle speed of the vehicle 3 and an acceleration sensor 31 that detects the acceleration and deceleration of the vehicle 3.

[0020] The HMI13 is a device that provides notifications to the crew and accepts operations from the crew. The HMI13 includes a touch panel 32 and an audio output device 33. The touch panel 32 displays various screens to the crew and accepts input operations from the crew on these screens. The audio output device 33 outputs voice guidance, warning sounds, etc.

[0021] <Control device 15> The control device 15 is a computer having a processor 41 and a memory 42 that is communicatively connected to the processor 41. The processor 41 may include, for example, at least one of a CPU, GPU, or MPU as a core. The memory 42 stores programs executed by the processor 41 and various data. The memory 42 may include at least one of volatile memory and non-volatile memory. The volatile memory may be, for example, DRAM or SRAM. The non-volatile memory may be an SSD, flash memory, magnetic disk storage device, or optical disk storage device. At least a part of the control device 15 may be implemented by hardware such as an LSI, ASIC, or FPGA, or by a combination of software and hardware. The control device 15 may be composed of a single piece of hardware, or it may be composed of multiple pieces of hardware that can communicate with each other.

[0022] The control device 15 comprises a map storage unit 43, a surrounding area recognition unit 44, and a driving control unit 45 as functional units. The processor 41 functions as the map storage unit 43, the surrounding area recognition unit 44, and the driving control unit 45 by executing a program stored in the memory 42.

[0023] The map storage unit 43 stores high-precision map information 47. The high-precision map information 47 is three-dimensional map information with higher precision than the map information stored by the navigation device 8. For example, the high-precision map information 47 includes information about lane markings X on the road T of the vehicle 3 (e.g., the position and type of lane markings X). For example, lane markings X on the high-precision map are represented by nodes placed at predetermined intervals and links connecting the nodes. In other embodiments, the map storage unit 43 stores learned map information generated from the detection results of on-board sensors of the vehicle 3 (i.e., its own vehicle) and / or other vehicles, and this learned map information may also include information about lane markings X.

[0024] The surrounding area recognition unit 44 recognizes the conditions around the vehicle 3 based on the detection results of the external sensor 11. For example, the surrounding area recognition unit 44 recognizes the lane markings X in front of the vehicle 3 in the direction of travel based on the detection results of the camera 26 (i.e., the image captured by the camera 26) and the detection results of the lidar 28 (i.e., the position of the target detected by the lidar 28). Hereinafter, the lane markings X recognized by the surrounding area recognition unit 44 based on the detection results of the camera 26 will be referred to as "camera lane markings XC", and the lane markings X recognized by the surrounding area recognition unit 44 based on the detection results of the lidar 28 will be referred to as "lidar lane markings XL". Based on the detection results of the lidar 28, the surrounding area recognition unit 44 recognizes the degree of curvature and the road surface gradient of the vehicle 3's road T.

[0025] The surrounding area recognition unit 44 recognizes the conditions around the vehicle 3 based on the map information stored in the map storage unit 43 (for example, high-precision map information 47 or learned map information) and the current position of the vehicle 3. Hereinafter, the lane markings X recognized by the surrounding area recognition unit 44 based on the map information (for example, high-precision map information 47 or learned map information) and the current position of the vehicle 3 will be referred to as "map lane markings XM".

[0026] The driving control unit 45 performs driving control of the vehicle 3 based on the recognition results of the surrounding recognition unit 44. For example, the driving control unit 45 performs lane keeping support control as driving control of the vehicle 3. In lane keeping support control, the driving control unit 45 recognizes the lane based on the lane markings X and controls the steering control knob 22 and the steering device 7 so that the vehicle 3 maintains its position within the lane.

[0027] The driving control of the vehicle 3 performed by the driving control unit 45 includes a first level of driving control and a second level of driving control. The second level of driving control provides a higher degree of driving assistance than the first level of driving control, or reduces the workload for the occupants of the vehicle 3. For example, the driving control unit 45 performs lane keeping assistance control in hands-on mode (a mode that requires the driver to hold the steering controls 22) as the first level of driving control. For example, the driving control unit 45 performs lane keeping assistance control in hands-off mode (a mode that allows the driver to release their hands from the steering controls 22) as the second level of driving control.

[0028] <Road line recognition processing by surrounding area recognition unit 44> Next, the lane marking recognition process by the surrounding recognition unit 44 will be explained with reference to Figures 2 to 5. In Figure 2, the "straight section" indicates a section of the vehicle 3's road T where both the degree of curvature and the road surface gradient are below a threshold. In Figure 2, the "specific section" indicates a section of the vehicle 3's road T where at least one of the degree of curvature or the road surface gradient is above a threshold. In Figure 2, the "curved section" indicates a section within the specific section where the degree of curvature is above a threshold. In Figure 2, the "gradient section" indicates a section within the specific section where the road surface gradient is above a threshold. Hereinafter, lane markings X present in the straight section, specific section, and curved section will be referred to as lane markings X of the straight section, lane markings X of the specific section, and lane markings X of the curved section, respectively. In Figures 3 to 5, arrow A indicates the direction of travel of the vehicle 3.

[0029] Referring to Figures 2 and 3, when vehicle 3 is traveling in a straight section, both camera 26 and lidar 28 can accurately detect the lane markings X of the straight section. However, the detection range of camera 26 is longer than that of lidar 28. Also, the computational processing load of camera 26 is less than that of lidar 28. Furthermore, camera 26 can detect the color of lane markings X, but lidar 28 cannot. Therefore, when vehicle 3 is traveling in a straight section, the surrounding recognition unit 44 recognizes the lane markings X of the straight section based on the camera lane markings XC.

[0030] When vehicle 3 is traveling in a straight section, camera 26 cannot accurately detect the lane markings X in a specific section. On the other hand, lidar 28 can accurately detect the lane markings X in a specific section (because lidar 28 detects the position of an object based on the reflected laser light). Therefore, when vehicle 3 is traveling in a straight section, the surrounding recognition unit 44 recognizes the lane markings X in a specific section based on the lidar lane markings XL.

[0031] As described above, when vehicle 3 is traveling in a straight section, the surrounding recognition unit 44 prioritizes the detection result of camera 26 over the detection result of lidar 28 to recognize the lane markings X in the straight section, and prioritizes the detection result of lidar 28 over the detection result of camera 26 to recognize the lane markings X in a specific section. Hereinafter, this lane marking X recognition process will be referred to as the "first recognition process".

[0032] Referring to Figures 2 and 4, when vehicle 3 is traveling through a curved section, the vehicle 3 tilts due to its roll, making it difficult for camera 26 to accurately detect the lane markings X in the straight section and specific sections. Furthermore, processing to correct the tilt of camera 26 is also necessary. On the other hand, lidar 28 can accurately detect the lane markings X in the straight section and specific sections (because lidar 28 detects the position of the target based on the reflected laser light). Therefore, when vehicle 3 is traveling through a curved section, the surrounding recognition unit 44 recognizes the lane markings X in the straight section and specific sections based on the lidar lane markings XL.

[0033] As described above, when vehicle 3 is traveling through a curved section, the surrounding recognition unit 44 prioritizes the detection result of the lidar 28 over the detection result of the camera 26 to recognize the lane markings X in the straight section and the specific section. Hereinafter, this process of recognizing lane markings X will be referred to as the "second recognition process".

[0034] Referring to Figure 5, there are cases where vehicle 3 passes through a curved section and enters a straight section. In such cases, the surrounding recognition unit 44 changes the recognition process of the lane marking X from the second recognition process to the first recognition process after vehicle 3 has passed through the curved section and entered the straight section. However, in other embodiments, the surrounding recognition unit 44 may change the recognition process of the lane marking X from the second recognition process to the first recognition process before vehicle 3 passes through the curved section and enters the straight section.

[0035] Although not shown in the diagram, there are cases where vehicle 3 passes through a straight section and enters a curved section. In such cases, the surrounding recognition unit 44 changes the recognition process of the lane marking X from the first recognition process to the second recognition process before vehicle 3 passes through the straight section and enters the curved section. However, in other embodiments, the surrounding recognition unit 44 may change the recognition process of the lane marking X from the first recognition process to the second recognition process after vehicle 3 passes through the straight section and enters the curved section.

[0036] <Parking line correction processing> Next, with reference to Figure 3, the lane marking correction process by the surrounding recognition unit 44 will be explained. The lane marking correction process corrects the lidar lane marking XL using the camera lane marking XC for straight sections. In other words, the lane marking correction process corrects the detection result of the lidar 28 based on the detection result of the camera 26 for the lane marking X of the straight section.

[0037] Vehicle 3 is traveling in a straight section, and there may be a specific section (a curved section in Figure 3) ahead of the direction of travel of vehicle 3. In such a case, the surrounding recognition unit 44 performs lane marking correction processing before vehicle 3 enters the specific section from the straight section (i.e., before changing the recognition processing of lane marking X from the first recognition processing to the second recognition processing).

[0038] In the lane marking correction process, the surrounding recognition unit 44 corrects the lidar lane marking line XL using the camera lane marking line XC for straight sections. For example, if there is a discrepancy between the angle of the camera lane marking line XC and the angle of the lidar lane marking line XL, the surrounding recognition unit 44 corrects the lidar lane marking line XL by rotating it toward the camera lane marking line XC. Also, if there is a discrepancy between the lateral position of the camera lane marking line XC and the lateral position of the lidar lane marking line XL, the surrounding recognition unit 44 corrects the lidar lane marking line XL by translating it toward the camera lane marking line XC. In other embodiments, in the lane marking correction process, the surrounding recognition unit 44 may correct the camera lane marking line XC using the lidar lane marking line XL for straight sections.

[0039] <Loss of road markings> Next, the process for detecting lost road markings by the surrounding recognition unit 44 will be explained with reference to Figures 3 and 4. The road marking loss process is performed when road marking X cannot be recognized based on the camera road marking XC or the lidar road marking XL (i.e., when road marking X based on the camera road marking XC or the lidar road marking XL is lost).

[0040] Referring to Figure 3, when vehicle 3 is traveling in a straight section, if the lane marking X of the straight section cannot be recognized based on the camera lane marking XC, the surrounding recognition unit 44 recognizes the lane marking X of the straight section based on the lidar lane marking XL. In other words, when vehicle 3 is traveling in a straight section, if the lane marking X of the straight section cannot be recognized based on the detection result of camera 26, the surrounding recognition unit 44 recognizes the lane marking X of the straight section based on the detection result of lidar 28.

[0041] When vehicle 3 is traveling in a straight section, if the lane marking X of a specific section cannot be recognized based on the lidar lane marking XL, the surrounding recognition unit 44 recognizes the lane marking X of the specific section based on the map lane marking XM. In other words, when vehicle 3 is traveling in a straight section, if the lane marking X of a specific section cannot be recognized based on the detection result of the lidar 28, the surrounding recognition unit 44 recognizes the lane marking X of the specific section based on map information (for example, high-precision map information 47 or learned map information).

[0042] Referring to Figure 4, when vehicle 3 is traveling through a curved section, if the lidar lane marking XL does not allow recognition of at least one lane marking X of the straight section and the specific section, the surrounding recognition unit 44 recognizes the at least one lane marking X of the straight section and the specific section based on the map lane marking XM. In other words, when vehicle 3 is traveling through a curved section, if the detection result of the lidar 28 does not allow recognition of at least one lane marking X of the straight section and the specific section, the surrounding recognition unit 44 recognizes the at least one lane marking X of the straight section and the specific section based on map information (for example, high-precision map information 47 or learned map information).

[0043] <Level change processing by the operation control unit 45> Next, the level change process by the operation control unit 45 will be explained with reference to Figures 3 and 4. The level change process is a process for changing the level of the operation control.

[0044] Referring to Figure 3, when vehicle 3 is traveling in a straight section with second-level driving control, the surrounding recognition unit 44 determines whether the discrepancy between at least one of the camera lane markings XC or the lighter lane markings XL and the map lane markings XM is less than or equal to a predetermined value for the straight section. If the surrounding recognition unit 44 determines that the discrepancy between at least one of the camera lane markings XC or the lighter lane markings XL and the map lane markings XM is less than or equal to a predetermined value, the driving control unit 45 continues with second-level driving control. On the other hand, if the surrounding recognition unit 44 determines that the discrepancy between both the camera lane markings XC and the lighter lane markings XL and the map lane markings XM exceeds a predetermined value, the driving control unit 45 stops second-level driving control and starts first-level driving control. In other words, the driving control unit 45 lowers the level of driving control from second level to first level.

[0045] When vehicle 3 is traveling in a straight section with second-level driving control, the surrounding recognition unit 44 determines whether the discrepancy between the rider lane line XL and the map lane line XM in a specific section is less than or equal to a predetermined value. If the surrounding recognition unit 44 determines that the discrepancy between the rider lane line XL and the map lane line XM is less than or equal to a predetermined value, the driving control unit 45 continues with second-level driving control. On the other hand, if the surrounding recognition unit 44 determines that the discrepancy between the rider lane line XL and the map lane line XM exceeds a predetermined value, the driving control unit 45 stops second-level driving control and starts first-level driving control. In other words, the driving control unit 45 lowers the level of driving control from second level to first level.

[0046] Referring to Figure 4, when vehicle 3 is traveling through a curved section with second-level driving control, the surrounding recognition unit 44 determines whether the discrepancy between the rider lane markings XL and the map lane markings XM is less than or equal to a predetermined value for the straight section and specific sections. If the surrounding recognition unit 44 determines that the discrepancy between the rider lane markings XL and the map lane markings XM is less than or equal to a predetermined value, the driving control unit 45 continues with second-level driving control. On the other hand, if the surrounding recognition unit 44 determines that the discrepancy between the rider lane markings XL and the map lane markings XM exceeds a predetermined value, the driving control unit 45 stops second-level driving control and starts first-level driving control. In other words, the driving control unit 45 lowers the level of driving control from second level to first level.

[0047] <Example of vehicle control execution 1> Next, with reference to Figure 6, we will explain Example 1 of vehicle control, including driving control. Example 1 of vehicle control is an example of vehicle control when vehicle 3 is traveling in a straight section.

[0048] When the main power of vehicle 3 is turned ON, the surrounding recognition unit 44 recognizes the camera lane markings XC, the rider lane markings XL, and the map lane markings XM (step ST1). Next, the driving control unit 45 starts lane keeping assist control in response to the driver's operation on the HMI 13 (step ST2).

[0049] When lane keeping assist control is initiated, the surrounding recognition unit 44 recognizes the lane markings X of the straight section based on the camera lane markings XC (step ST3). The driving control unit 45 generates a trajectory for the vehicle 3 for lane keeping assist control based on the lane markings X of the straight section recognized by the surrounding recognition unit 44, and drives the vehicle 3 along the generated trajectory.

[0050] While vehicle 3 is traveling in the straight section, the surrounding recognition unit 44 determines whether or not a specific section exists in front of vehicle 3 in the direction of travel (step ST4). If no specific section exists in front of vehicle 3 in the direction of travel (step ST4: No), the surrounding recognition unit 44 recognizes the lane marking X of the straight section based on the camera lane marking XC (step ST3) and repeats the determination in step ST4.

[0051] If a specific section exists in the direction of travel ahead of vehicle 3 (step ST4: Yes), the surrounding recognition unit 44 performs the lane marking correction process described above (step ST5). In other words, the surrounding recognition unit 44 corrects the rider lane marking XL using the camera lane marking XC for straight sections.

[0052] Next, the surrounding area recognition unit 44 recognizes the lane markings X of the straight section based on the camera lane markings XC and recognizes the lane markings X of the specific section based on the rider lane markings XL (step ST6). The driving control unit 45 generates a trajectory for the vehicle 3 for lane keeping support control based on the lane markings X of the straight section and the specific section recognized by the surrounding area recognition unit 44, and drives the vehicle 3 along the generated trajectory.

[0053] <Example of vehicle control execution 2> Next, with reference to Figure 7, we will explain the execution example 2 of vehicle control, including driving control. Execution example 2 of vehicle control is an example of vehicle control when vehicle 3 enters a curved section from a straight section and then returns to a straight section from the curved section. Note that steps ST11-ST13 of execution example 2 of vehicle control are the same as steps ST1-ST3 of execution example 1 of vehicle control, so their explanation will be omitted.

[0054] When step ST13 is completed, while vehicle 3 is traveling in the straight section, the surrounding recognition unit 44 determines whether or not there is a curved section ahead in the direction of travel of vehicle 3 (step ST14). If there is no curved section ahead in the direction of travel of vehicle 3 (step ST14: No), the surrounding recognition unit 44 recognizes the lane marking X of the straight section based on the camera lane marking XC (step ST13) and repeats the determination in step ST14.

[0055] If a curved section exists ahead of the vehicle 3 in the direction of travel (step ST14: Yes), the surrounding recognition unit 44 performs the lane marking correction process described above (step ST15). In other words, the surrounding recognition unit 44 corrects the rider lane marking XL using the camera lane marking XC for straight sections.

[0056] Next, the surrounding recognition unit 44 recognizes the lane markings X of the straight section based on the camera lane markings XC and recognizes the lane markings X of the curved section based on the rider lane markings XL (step ST16). The driving control unit 45 generates a trajectory for the vehicle 3 for lane keeping support control based on the lane markings X of the straight section and curved section recognized by the surrounding recognition unit 44, and drives the vehicle 3 along the generated trajectory.

[0057] Next, the surrounding recognition unit 44 determines whether or not vehicle 3 has entered a curved section (step ST17). If vehicle 3 has not entered a curved section (step ST17: No), the surrounding recognition unit 44 recognizes the lane markings X of the straight section based on the camera lane markings XC and the lane markings X of the curved section based on the rider lane markings XL (step ST16), and repeats the determination in step ST17.

[0058] When vehicle 3 enters a curved section (step ST17: Yes), the surrounding recognition unit 44 recognizes the lane markings X of the straight section and specific section (for example, the curved section or the gradient section) based on the rider lane markings XL (step ST18). The driving control unit 45 generates a trajectory for vehicle 3 for lane keeping support control based on the lane markings X of the straight section and specific section recognized by the surrounding recognition unit 44, and drives vehicle 3 along the generated trajectory.

[0059] Next, the surrounding area recognition unit 44 determines whether or not vehicle 3 has returned to the straight section (step ST19). If vehicle 3 has not returned to the straight section (step ST19: No), the surrounding area recognition unit 44 recognizes the straight section and the lane markings X of the specific section based on the rider lane markings XL (step ST18) and repeats the determination in step ST19.

[0060] When vehicle 3 returns to the straight section (step ST19: Yes), the surrounding recognition unit 44 recognizes the lane marking X of the straight section based on the camera lane marking XC (step ST20). The driving control unit 45 generates a trajectory for vehicle 3 for lane keeping support control based on the lane marking X of the straight section recognized by the surrounding recognition unit 44, and drives vehicle 3 along the generated trajectory.

[0061] <Example of vehicle control execution 3> Next, with reference to Figure 8, we will explain the execution example 3 of vehicle control, including driving control. Execution example 3 of vehicle control is an example of vehicle control when vehicle 3 enters a curved section from a straight section and then returns to a straight section from the curved section. Note that step ST31 of execution example 3 of vehicle control is the same as step ST1 of execution example 1 of vehicle control, so its explanation will be omitted. Hereinafter, lane keeping support control in hands-on mode will be abbreviated as "hands-on control," and lane keeping support control in hands-off mode will be abbreviated as "hands-off control."

[0062] When step ST31 is completed, the surrounding recognition unit 44 determines whether the hands-off condition has been met (step ST32). The hands-off condition is a condition for determining whether hands-off control is possible. For example, the surrounding recognition unit 44 determines that the hands-off condition has been met when the camera lane line XC or the rider lane line XL matches the map lane line XM. On the other hand, the surrounding recognition unit 44 determines that the hands-off condition has not been met when the camera lane line XC and the rider lane line XL do not match the map lane line XM. In other embodiments, the hands-off condition may include, in addition to, such conditions relating to the matching of lane lines, or instead of conditions relating to the matching of lane lines, conditions relating to the driving state of the vehicle 3 or the driving state of the driver. If the hands-off condition has not been met (step ST32: No), the surrounding recognition unit 44 repeats the determination in step ST32 until the hands-off condition is met.

[0063] If the hands-off condition is met (step ST32: Yes), the operation control unit 45 starts hands-off control (step ST33). In other words, the operation control unit 45 starts second-level operation control.

[0064] When the driving control unit 45 starts hands-off control, the surrounding recognition unit 44 determines whether the discrepancy between the camera lane marking XC or the rider lane marking XL and the map lane marking XM in the straight section is less than or equal to a predetermined value (step ST34). If the discrepancy between the camera lane marking XC and the rider lane marking XL and the map lane marking XM in the straight section exceeds the predetermined value (step ST34: No), the driving control unit 45 switches the driving control from hands-off control to hands-on control (step ST35). In other words, the driving control unit 45 lowers the level of driving control from the second level to the first level.

[0065] If the discrepancy between the camera lane marking XC or rider lane marking XL and the map lane marking XM in a straight section is less than or equal to a predetermined value (step ST34: Yes), the driving control unit 45 continues hands-off control. In this case, the surrounding recognition unit 44 determines whether or not there is a curved section ahead in the direction of travel of the vehicle 3 (step ST36). If there is no curved section ahead in the direction of travel of the vehicle 3 (step ST36: No), the surrounding recognition unit 44 repeats the determination in step ST36.

[0066] If a curved section exists in the direction of travel ahead of vehicle 3 (step ST36: Yes), the surrounding recognition unit 44 determines whether the discrepancy between the rider lane line XL and the map lane line XM in the curved section is less than or equal to a predetermined value (step ST37). If the discrepancy between the rider lane line XL and the map lane line XM in the curved section exceeds a predetermined value (step ST37: No), the driving control unit 45 switches the driving control from hands-off control to hands-on control (step ST35). In other words, the driving control unit 45 lowers the level of driving control from the second level to the first level.

[0067] If the discrepancy between the rider lane line XL and the map lane line XM in the curved section is less than or equal to a predetermined value (step ST37: Yes), the driving control unit 45 continues hands-off control. In this case, the surrounding recognition unit 44 determines whether or not the vehicle 3 has entered the curved section (step ST38). If the vehicle 3 has not entered the curved section (step ST38: No), the surrounding recognition unit 44 repeats the determination in step ST38.

[0068] If vehicle 3 enters a curved section (step ST38: Yes), the surrounding recognition unit 44 determines whether the discrepancy between the rider lane line XL and the map lane line XM for straight sections and specific sections (for example, curved sections or gradient sections) is less than or equal to a predetermined value (step ST39). If the discrepancy between the rider lane line XL and the map lane line XM for straight sections or specific sections exceeds a predetermined value (step ST39: No), the driving control unit 45 switches the driving control from hands-off control to hands-on control. In other words, the driving control unit 45 lowers the level of driving control from the second level to the first level. If the discrepancy between the rider lane line XL and the map lane line XM for straight sections and specific sections is less than or equal to a predetermined value (step ST39: Yes), the driving control unit 45 continues hands-off control.

[0069] <Effects> When vehicle 3 is traveling in a straight section and lane markings X are present in a specific section (a curved section or a gradient section), the detection accuracy of camera lane markings XC decreases. Therefore, if the system attempts to recognize lane markings X based solely on camera lane markings XC, the recognition accuracy of lane markings X may decrease. If driving control (for example, lane keeping assist control) is performed with reduced recognition accuracy of lane markings X in this state, it may cause discomfort to the driver. Therefore, in the first recognition process (recognition process when vehicle 3 is traveling in a straight section), the surrounding recognition unit 44 recognizes lane markings X in a specific section based on the rider lane markings XL. This improves the recognition accuracy of lane markings X in a specific section, thereby suppressing discomfort for the driver when driving control is performed.

[0070] On the other hand, when vehicle 3 is traveling in a straight section and lane marking X is present in the straight section, the detection accuracy of camera lane marking XC and lidar lane marking XL is high. Also, as mentioned above, camera 26 is superior to lidar 28 in terms of detection distance, computational processing amount, color detection, etc. Therefore, in the first recognition process, the surrounding recognition unit 44 recognizes lane marking X in the straight section based on camera lane marking XC. This improves the recognition accuracy of lane marking X in the straight section, and further reduces the driver's discomfort when driving control is executed.

[0071] When vehicle 3 is traveling through a curved section, the camera lane marking XC shifts due to the roll of vehicle 3. Therefore, if the lane marking X is to be recognized based solely on the camera lane marking XC, the accuracy of lane marking X recognition may decrease. To address this, the surrounding recognition unit 44 recognizes lane marking X in the straight section and specific sections based on the rider lane marking XL during the second recognition process (recognition process when vehicle 3 is traveling through a curved section). This improves the recognition accuracy of lane marking X in the straight section and specific sections, thereby further reducing driver discomfort during the execution of driving control.

[0072] <Other variations> In the above embodiment, in the first recognition process, the peripheral recognition unit 44 recognizes the lane markings X of the straight section based on the camera lane markings XC. In other modified examples, in the first recognition process, the peripheral recognition unit 44 may make the proportion of camera lane markings XC (e.g., 90%) in the recognition of lane markings X of the straight section higher than the proportion of lidar lane markings XL (e.g., 10%). In other words, "prioritizing the detection result of camera 26 over the detection result of lidar 28 to recognize lane markings X of the straight section" in the first recognition process includes "making the proportion of camera lane markings XC in the recognition of lane markings X of the straight section higher than the proportion of lidar lane markings XL". Similarly, "prioritizing the detection result of lidar 28 over the detection result of camera 26 to recognize lane markings X of the straight section (or specific section)" in the first or second recognition process includes "making the proportion of lidar lane markings XL in the recognition of lane markings X of the straight section (or specific section) higher than the proportion of camera lane markings XC".

[0073] Furthermore, in other variations, in the first recognition process, if the peripheral recognition unit 44 cannot recognize the lane marking X of the straight section based on the camera lane marking XC, it may recognize the lane marking X of the straight section based on the lidar lane marking XL. In other words, "prioritizing the detection result of the camera 26 over the detection result of the lidar 28 to recognize the lane marking X of the straight section" in the first recognition process includes "using the lidar lane marking XL instead of the camera lane marking XC when the lane marking X of the straight section is lost." Similarly, "prioritizing the detection result of the lidar 28 over the detection result of the camera 26 to recognize the lane marking X of the straight section (or specific section)" in the first or second recognition process includes "using the camera lane marking XC instead of the lidar lane marking XL when the lane marking X of the straight section (or specific section) is lost."

[0074] Furthermore, in other variations, in the first recognition process, the surrounding recognition unit 44 may compare the map boundary line XM with the camera boundary line XC over the lidar boundary line XL for the straight section boundary line X. In other words, "recognizing the straight section boundary line X by prioritizing the detection result of the camera 26 over the detection result of the lidar 28" in the first recognition process includes "comparing the map boundary line XM with the camera boundary line XC over the lidar boundary line XL for the straight section boundary line X." Similarly, "recognizing the straight section boundary line X (or specific section) by prioritizing the detection result of the lidar 28 over the detection result of the camera 26" in the first or second recognition process includes "comparing the map boundary line XM with the lidar boundary line XL over the camera boundary line XC for the straight section (or specific section) boundary line X."

[0075] In the above embodiment, the lidar 28 is an example of an electromagnetic wave sensor. In other embodiments, a sensor other than the lidar 28 (for example, radar 27) may be an example of an electromagnetic wave sensor.

[0076] <Summary of Embodiments> The vehicle control system 1 includes a camera 26 that photographs the area around the vehicle 3, an electromagnetic wave sensor 28 that irradiates electromagnetic waves toward the area around the vehicle 3 and receives the reflected waves, an area recognition unit 44 that recognizes the lane markings X in front of the vehicle 3 in the direction of travel based on the detection results of at least one of the camera 26 and the electromagnetic wave sensor 28, and a driving control unit 45 that performs driving control of the vehicle 3 based on the recognition results of the area recognition unit 44. The vehicle 3's track T includes a straight section where both the degree of curvature and the road surface gradient are below a threshold, and a specific section where at least one of the degree of curvature or the road surface gradient is equal to or greater than the threshold. The area recognition unit 44 performs a first recognition process as a recognition process for the lane markings X, and in the first recognition process, it recognizes the lane markings X in the straight section by prioritizing the detection results of the camera 26 over the detection results of the electromagnetic wave sensor 28, and recognizes the lane markings X in the specific section by prioritizing the detection results of the electromagnetic wave sensor 28 over the detection results of the camera 26.

[0077] In this embodiment, the surrounding recognition unit 44 prioritizes the detection result of the electromagnetic wave sensor 28 over the detection result of the camera 26 when recognizing the lane marking X in a specific section. This improves the recognition accuracy of the lane marking X in a specific section, thereby suppressing any discomfort the driver may experience during the execution of driving control. Furthermore, it suppresses situations in which driving control cannot be continued due to a decrease in the recognition accuracy of the lane marking X in a specific section, thereby improving the continuity of driving control.

[0078] The surrounding area recognition unit 44 may perform the first recognition process when the vehicle 3 is traveling in a straight section.

[0079] In this configuration, when vehicle 3 is traveling in a straight section, the surrounding recognition unit 44 prioritizes the detection result of the camera 26 over the detection result of the electromagnetic wave sensor 28 to recognize the lane markings X of the straight section. This makes it possible to improve the recognition accuracy of the lane markings X of the straight section by taking advantage of the camera 26, and thus further reduces the driver's discomfort when driving control is performed.

[0080] The specified section includes a curved section whose curvature is greater than or equal to the threshold, and the surrounding recognition unit 44 may perform a second recognition process as the recognition process for the lane marking X when the vehicle 3 is traveling through the curved section, and in the second recognition process, the detection result of the electromagnetic wave sensor 28 may be given priority over the detection result of the camera 26 to recognize the lane marking X of the straight section and the specified section.

[0081] In this embodiment, when vehicle 3 is traveling through a curved section, the surrounding recognition unit 44 prioritizes the detection result of the electromagnetic wave sensor 28 over the detection result of the camera 26 to recognize the lane markings X of the straight section and the specific section. This improves the accuracy of recognizing the lane markings X of the straight section and the specific section, thereby further suppressing any discomfort the driver may experience when driving control is performed.

[0082] The surrounding recognition unit 44 may change the recognition process of the lane marking X from the second recognition process to the first recognition process after the vehicle 3 has passed through the curved section and entered the straight section.

[0083] According to this embodiment, when vehicle 3 passes through a curved section, the recognition process of the lane marking X is returned from the second recognition process to the first recognition process at an appropriate timing, thereby further improving the recognition accuracy of the lane marking X and the continuity of driving control.

[0084] The surrounding area recognition unit 44 recognizes the lane markings X based on the map information 47, and the driving control unit 45 performs a first level of driving control and a second level of driving control which provides a higher degree of driving assistance than the first level of driving control, or which requires less work from the occupants of the vehicle 3 than the first level of driving control. When the vehicle 3 is traveling through the straight section with the second level of driving control, the second level of driving control may be continued if the discrepancy between the lane markings X recognized based on the detection results of the camera 26 or the electromagnetic wave sensor 28 and the lane markings X recognized based on the map information 47 for the straight section is less than or equal to a predetermined value.

[0085] According to this embodiment, when vehicle 3 is traveling in a straight section, it is possible to appropriately determine whether or not to continue the second level of driving control.

[0086] The surrounding area recognition unit 44 recognizes the lane markings X based on the map information 47, and the driving control unit 45 performs a first level of driving control and a second level of driving control which provides a higher degree of driving assistance than the first level of driving control, or which requires less work from the occupants of the vehicle 3 than the first level of driving control. When the vehicle 3 is traveling through the straight section with the second level of driving control, the second level of driving control may be continued if the discrepancy between the lane markings X recognized based on the detection results of the electromagnetic wave sensor 28 and the lane markings X recognized based on the map information 47 for the specific section is less than or equal to a predetermined value.

[0087] According to this embodiment, when vehicle 3 is traveling in a straight section, it is possible to appropriately determine whether or not to continue the second level of driving control.

[0088] The specified section includes a curved section whose curvature is equal to or greater than the threshold, the surrounding recognition unit 44 recognizes the lane marking X based on map information 47, and the driving control unit 45 performs a first level of driving control and a second level of driving control which provides a higher degree of driving assistance than the first level of driving control or which requires less work from the occupants of the vehicle 3 than the first level of driving control. When the vehicle 3 is traveling through the curved section with the second level of driving control, the second level of driving control may be continued if the discrepancy between the lane marking X recognized based on the detection result of the electromagnetic wave sensor 28 and the lane marking X recognized based on the map information 47 for the straight section and the specified section is less than or equal to a predetermined value.

[0089] According to this embodiment, it is possible to appropriately determine whether or not to continue the second level of driving control when vehicle 3 is traveling through a curved section.

[0090] When the vehicle 3 is traveling in the straight section, the surrounding recognition unit 44 may perform lane marking correction processing to correct the detection result of the electromagnetic wave sensor 28 based on the detection result of the camera 26 for the lane marking X of the straight section.

[0091] When vehicle 3 is traveling in a straight section, both camera 26 and electromagnetic wave sensor 28 can accurately detect the lane markings X of the straight section. According to the above configuration, by aligning the accurately detected lane markings X, the influence of measurement errors between camera 26 and electromagnetic wave sensor 28 can be reduced.

[0092] The surrounding recognition unit 44 may perform the lane marking correction process if the vehicle 3 is traveling in the straight section and the specific section exists in front of the vehicle 3 in the direction of travel.

[0093] According to this embodiment, before the vehicle 3 enters a specific section from a straight section and the recognition process for the lane marking X switches, the influence of measurement errors between the camera 26 and the electromagnetic wave sensor 28 can be reduced by lane marking correction processing. This allows for a smooth switch in the recognition process for lane marking X.

[0094] When the vehicle 3 is traveling in the straight section, if the camera 26 cannot recognize the lane markings X in the straight section based on its detection results, the surrounding area recognition unit 44 may recognize the lane markings X in the straight section based on the detection results of the electromagnetic wave sensor 28.

[0095] According to this embodiment, even if the lane markings X of the straight section cannot be recognized based on the detection results of the camera 26 when the vehicle 3 is traveling in the straight section, the lane markings X of the straight section can still be recognized.

[0096] The surrounding area recognition unit 44 recognizes the lane markings X based on the map information 47, and when the vehicle 3 is traveling in the straight section, if it is not possible to recognize the lane markings X in the specific section based on the detection results of the electromagnetic wave sensor 28, it may recognize the lane markings X in the specific section based on the map information 47.

[0097] According to this embodiment, even if the vehicle 3 is traveling in a straight section and cannot recognize the lane markings X of a specific section based on the detection results of the electromagnetic wave sensor 28, the lane markings X of the specific section can still be recognized.

[0098] The specified section includes a curved section whose curvature is greater than or equal to the threshold, and the surrounding recognition unit 44 recognizes the lane markings X based on the map information 47. When the vehicle 3 is traveling through the curved section, if the detection result of the electromagnetic wave sensor 28 does not allow the recognition of at least one of the lane markings X in the straight section and the specified section, the unit may recognize at least one of the lane markings X in the straight section and the specified section based on the map information 47.

[0099] According to this embodiment, even if the vehicle 3 is traveling through a specific section and cannot recognize the straight section or the lane markings X of the specific section based on the detection results of the electromagnetic wave sensor 28, it is still possible to recognize the straight section or the lane markings X of the specific section.

[0100] The electromagnetic wave sensor 28 is a lidar 28 that emits laser light toward the vicinity of the vehicle 3 and receives the reflected wave. The surrounding recognition unit 44 may recognize the degree of curvature and the road surface gradient of the vehicle 3's track T based on the detection results of the lidar 28.

[0101] According to this embodiment, the lidar 28 can be used to appropriately detect situations in which the detection accuracy of the camera 26 is likely to decrease. [Explanation of Symbols]

[0102] 1: Vehicle control system 3: Vehicles 26: Camera 28: LIDA (an example of an electromagnetic wave sensor) 44: Peripheral Recognition Unit 45: Operation Control Unit 47: High-precision map information T: Track X: Plot line

Claims

1. A vehicle control system, A camera that takes pictures around the vehicle, An electromagnetic wave sensor that irradiates electromagnetic waves toward the vicinity of the vehicle and receives the reflected waves, A surrounding area recognition unit recognizes lane markings in front of the vehicle in the direction of travel based on the detection results of at least one of the camera and the electromagnetic wave sensor, The vehicle includes a driving control unit that performs driving control of the vehicle based on the recognition results of the surrounding recognition unit, The vehicle's route is A straight section where both the degree of curvature and the road surface gradient are below the threshold, A specific section in which at least one of the degree of curvature or the road surface gradient is equal to or greater than the threshold, The aforementioned peripheral recognition unit, As the recognition process for the aforementioned road markings, a first recognition process is executed. A vehicle control system that, in the first recognition process, prioritizes the detection result of the camera over the detection result of the electromagnetic wave sensor to recognize the lane markings in the straight section, and prioritizes the detection result of the electromagnetic wave sensor over the detection result of the camera to recognize the lane markings in the specific section.

2. The vehicle control system according to claim 1, wherein the surrounding area recognition unit performs the first recognition process when the vehicle is traveling in the straight section.

3. The aforementioned specific section includes a curved section in which the degree of curvature is equal to or greater than the aforementioned threshold. The aforementioned peripheral recognition unit, When the vehicle is traveling through the curved section, the second recognition process is performed as the recognition process for the lane markings. The vehicle control system according to claim 1 or 2, wherein in the second recognition process, the detection result of the electromagnetic wave sensor is given priority over the detection result of the camera when recognizing the lane markings of the straight section and the specific section.

4. The vehicle control system according to claim 3, wherein the surrounding area recognition unit changes the recognition process of the lane markings from the second recognition process to the first recognition process after the vehicle has passed through the curved section and entered the straight section.

5. The surrounding area recognition unit recognizes the lane lines based on the map information, The aforementioned operation control unit, The system performs a first level of driving control and a second level of driving control that provides a higher degree of driving assistance than the first level of driving control, or that requires fewer tasks for the vehicle occupants than the first level of driving control. The vehicle control system according to claim 1 or 2, wherein when the vehicle is traveling through the straight section with the second level of driving control, the second level of driving control is continued if the discrepancy between the lane markings recognized based on the detection results of the camera or the electromagnetic wave sensor and the lane markings recognized based on the map information is less than or equal to a predetermined value.

6. The surrounding area recognition unit recognizes the lane lines based on the map information, The aforementioned operation control unit, The system performs a first level of driving control and a second level of driving control that provides a higher degree of driving assistance than the first level of driving control, or that requires fewer tasks for the vehicle occupants than the first level of driving control. The vehicle control system according to claim 1 or 2, wherein when the vehicle is traveling in the straight section with the second level of driving control, the second level of driving control is continued if the discrepancy between the lane markings recognized based on the detection results of the electromagnetic wave sensor and the lane markings recognized based on the map information for the specific section is less than or equal to a predetermined value.

7. The aforementioned specific section includes a curved section in which the degree of curvature is equal to or greater than the aforementioned threshold. The surrounding area recognition unit recognizes the lane lines based on the map information, The aforementioned operation control unit, The system performs a first level of driving control and a second level of driving control that provides a higher degree of driving assistance than the first level of driving control, or that requires fewer tasks for the vehicle occupants than the first level of driving control. The vehicle control system according to claim 1 or 2, wherein when the vehicle is traveling through the curved section with the second level of driving control, the second level of driving control is continued if the discrepancy between the lane markings recognized based on the detection results of the electromagnetic wave sensor and the lane markings recognized based on the map information is less than or equal to a predetermined value for the straight section and the specific section.

8. The vehicle control system according to claim 1 or 2, wherein when the vehicle is traveling in the straight section, the surrounding recognition unit performs a lane marking correction process to correct the detection result of the electromagnetic wave sensor for the lane markings in the straight section based on the detection result of the camera.

9. The vehicle control system according to claim 8, wherein the surrounding recognition unit performs the lane marking correction process when the vehicle is traveling in the straight section and the specific section exists in front of the vehicle in the direction of travel.

10. The vehicle control system according to claim 1 or 2, wherein when the vehicle is traveling in the straight section, the surrounding area recognition unit recognizes the lane markings of the straight section based on the detection results of the electromagnetic wave sensor if the lane markings of the straight section cannot be recognized based on the detection results of the camera.

11. The aforementioned peripheral recognition unit, Based on map information, the lane lines are recognized, A vehicle control system according to claim 1 or 2, which, when the vehicle is traveling in the straight section, recognizes the lane markings in the specific section based on the map information if the lane markings in the specific section cannot be recognized based on the detection results of the electromagnetic wave sensor.

12. The aforementioned specific section includes a curved section in which the degree of curvature is equal to or greater than the aforementioned threshold. The aforementioned peripheral recognition unit, Based on map information, the lane lines are recognized, A vehicle control system according to claim 1 or 2, in which, when the vehicle is traveling through the curved section, if the lane markings of at least one of the straight section and the specific section cannot be recognized based on the detection results of the electromagnetic wave sensor, the system recognizes the lane markings of at least one of the straight section and the specific section based on the map information.

13. The electromagnetic wave sensor is a lidar that irradiates laser light toward the vicinity of the vehicle and receives the reflected wave. The vehicle control system according to claim 1 or 2, wherein the surrounding recognition unit recognizes the degree of curvature and road surface gradient of the vehicle's route based on the detection results of the rider.

Images