Vehicle control system and computer program

Abstract

This invention provides a vehicle control device and computer program that improve the accuracy of determining roads where entry is prohibited. [Solution] The vehicle control device 10 includes a control unit 11 that performs driving assistance control of the vehicle 1, and the control unit analyzes image data captured from the environment around the vehicle, recognizes roads in the direction the vehicle is traveling in the environment, recognizes predetermined signs indicating no entry on the road, recognizes no-entry roads on the road where the predetermined signs are located, and if it is determined that the vehicle is entering a no-entry road, it outputs a predetermined alarm to the output unit, and if the road where the predetermined signs are located branches into a first road and a second road, it determines that either the first road or the second road is a no-entry road based on the positional relationship between the predetermined signs and the first road and the second road.

Description

Technical Field

[0001] The present invention relates to a vehicle control device and a computer program for performing driving assistance.

Background Art

[0002] For example, Patent Document 1 describes an entry determination device capable of determining whether a vehicle has entered a road where entry is prohibited. According to the technique described in Patent Document 1, based on imaging data obtained by imaging the environment around the vehicle, a predetermined sign indicating entry prohibition is recognized on both sides of the road in the traveling direction, and when the vehicle passes between the predetermined signs, it is configured to determine that the vehicle has entered a road where entry is prohibited.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] According to the technique described in Patent Document 1, there was a possibility that a vehicle could not be determined to have entered a road where entry is prohibited on a road where a predetermined sign indicating entry prohibition exists only on one side of the road.

[0005] An object of the present invention is to provide a vehicle control device and a computer program that improve the accuracy of determining a road where entry is prohibited.

Means for Solving the Problems

[0006] One aspect of the present invention is a vehicle control device comprising a control unit that performs driving assistance control of a vehicle, the control unit analyzes imaging data obtained by imaging the environment around the vehicle, recognizes a road in the direction the vehicle is traveling in the environment, recognizes a predetermined sign indicating no entry on the road, recognizes a no-entry road on the road where the predetermined sign is located, and, if it is determined that the vehicle is entering the no-entry road, outputs a predetermined alarm to the output unit, and, if the road where the predetermined sign is located branches into a first road and a second road, determines that either the first road or the second road is the no-entry road based on the positional relationship between the predetermined sign and the first road and the second road. [Effects of the Invention]

[0007] According to the present invention, the accuracy of determining roads where entry is prohibited can be improved. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the configuration of vehicle 1 according to an embodiment of the present invention. [Figure 2] This figure shows an example of imaging data obtained by capturing images of the environment around a vehicle. [Figure 3] This figure shows another example of image data. [Figure 4] This flowchart shows the processing flow of the vehicle control method executed in the vehicle control device. [Modes for carrying out the invention]

[0009] As shown in Figure 1, Vehicle 1 is configured to perform driver assistance. Vehicle 1 is configured with a detection unit 2 and a vehicle control device 10 to perform driver assistance such as ADAS (Advanced Driver-Assistance Systems). The detection unit 2 detects the environment around Vehicle 1. The detection unit 2 is composed of multiple devices depending on the application. The detected values ​​detected by the detection unit 2 are used for driver assistance and navigation devices, etc.

[0010] The detection unit 2 is equipped with a camera 2A that captures images of the environment around the vehicle 1. The camera 2A captures images of the area around the vehicle 1 and outputs the captured data. In this embodiment, the camera 2A captures a predetermined range in front of the vehicle 1. The captured data from the camera 2A is used, for example, for driver assistance or as a drive recorder for the vehicle 1. The imaging range and imaging direction of the camera 2A may differ depending on the vehicle 1.

[0011] The detection unit 2 includes a lidar device 2B that detects three-dimensional data around the vehicle 1. The lidar device 2B periodically irradiates laser light in front of or around the vehicle 1 within the detection area and measures the reflected light from the target object. The lidar device 2B is configured to allow adjustment of the detection area. The lidar device 2B scans the laser light within the detection area and acquires measurement data. The lidar device 2B is configured to generate three-dimensional point cloud data around the vehicle 1 based on the measurement data.

[0012] The measurements from the LiDAR device 2B are used to detect other vehicles, pedestrians, bicycles, motorcycles, and other traffic participants around vehicle 1, as well as objects around vehicle 1. The LiDAR device 2B also detects road structures present in the road environment.

[0013] The detection unit 2 includes, for example, a radar device 2C that scans radar waves to detect objects present around the vehicle 1. The radar device 2C is configured to complement the detection of objects with the lidar device 2B. The radar device 2C detects the relative distance to an object by irradiating millimeter-wave radar waves in the detection area and receiving the reflected waves reflected from the object. The radar device 2C is configured to allow adjustment of the detection area.

[0014] The measurements from radar device 2C are used to detect other vehicles, pedestrians, bicycles, motorcycles, and other traffic participants around vehicle 1, as well as objects around vehicle 1. Radar device 2C also detects road structures present in the road environment. The object recognition unit is composed of lidar device 2B and / or radar device 2C.

[0015] The detection unit 2 is equipped with a position sensor 2D that detects the current position of the vehicle 1. The position sensor 2D is, for example, a GPS (Global Positioning System) sensor or a GNSS (Global Navigation Satellite System) sensor. The position sensor 2D may be complemented by autonomous sensors (not shown) used for autonomous navigation, such as a gyro sensor and an accelerometer, to determine the position of the vehicle 1.

[0016] Vehicle 1 is equipped with an input / output unit 3 that accepts user operations and can display information. The input / output unit 3 is composed of, for example, a touch panel that accepts touch operations and can display images. The input / output unit 3 may also be composed of an input unit that accepts input operations such as physical switches and a display unit that can display information such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display.

[0017] The input / output unit 3 may include a speaker for outputting audio information, an indicator for outputting light information, a vibrator for outputting vibration information, etc. The input / output unit 3 also functions as an output unit that outputs information in at least one form, such as text, images, audio, light, or vibration.

[0018] Vehicle 1 includes a communication unit 4 that can be communicatively connected to a network W. The communication unit 4 is a communication interface configured for wireless communication. The communication unit 4 communicatively connects, for example, with a wireless base station existing around the vehicle 1 and mutually communicates with various communication targets via the network W. The vehicle 1 communicates, for example, with a server device 20 communicatively connected to the network W and acquires map data including the current position of the vehicle 1.

[0019] Vehicle 1 includes a drive unit 5 that generates power for traveling. The drive unit 5 is configured, for example, by an internal combustion engine using fuel. The drive unit 5 is configured by an electric motor when the vehicle 1 is an electric vehicle. The drive unit 5 may be configured by combining an internal combustion engine and an electric motor when the vehicle 1 is a hybrid vehicle. The drive unit 5 is controlled by the vehicle control device 10 during execution of driving assistance, and the speed is adjusted.

[0020] Vehicle 1 includes a braking unit 6 for decelerating the vehicle speed and controlling it to a stopped state. The braking unit 6 is configured, for example, by a brake device that generates braking force. The braking unit 6 may be integrated with the drive unit 5 when the vehicle 1 is an electric vehicle. The braking unit 6 is controlled by the vehicle control device 10 during execution of driving assistance.

[0021] Vehicle 1 includes a steering unit 7 for operating the traveling direction. The steering unit 7 is configured by a power steering device or the like that gives a steering angle to the steering wheel according to a steering operation. The steering unit 7 may be integrated with the drive unit 5 that variably controls the driving force on the left and right of the drive wheels when the vehicle 1 is an electric vehicle. The steering unit 7 is controlled by the vehicle control device 10 during execution of driving assistance, and the steering angle is adjusted.

[0022] The vehicle control device 10 includes a control unit 11 that executes control related to the running of the vehicle 1. The control unit 11 integrally executes control such as the running, driving support, navigation, and communication via the network W of the vehicle 1 based on the detection value detected by the detection unit 2. The control unit 11 is constituted by a hardware processor such as at least one CPU (Central Processing Unit). The control unit 11 may be realized by hardware (including a circuit part; circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by the cooperation of software and hardware.

[0023] The vehicle control device 10 includes a storage unit 12 that stores data and programs. The storage unit 12 is constituted by a non-temporary storage medium such as a hard disk drive (HDD) or a solid state drive (SSD). In the storage unit 12, a computer program and data necessary for the control of the vehicle 1 are stored. The program may be stored in the storage unit 12 in advance, or may be stored in an externally connectable storage medium such as a DVD or a CD-ROM, and may be installed in the storage unit 12 when the storage medium is mounted on a drive device. The control unit 11 controls the drive unit 5, the braking unit 6, and the steering unit 7 based on the detection value detected by the detection unit 2, and executes driving support control such as obstacle anticipation assist (OAA).

[0024] As shown in Figure 2, Vehicle 1 is configured to perform driving assistance control based on the recognition results of image data M1 captured by the environment surrounding Vehicle 1. The control unit 11 acquires the image data M1 captured by camera 2A. The control unit 11 is configured to recognize the environment surrounding Vehicle 1 and the contents of informational objects present in the environment by pre-running machine learning such as deep learning using image data of the road environment being driven on as training data.

[0025] The control unit 11 analyzes the image data M1 and recognizes structures such as roads R, road structures, road signs H, road markings P, and buildings B, as well as traffic participants such as other vehicles and pedestrians, from the environment surrounding the vehicle 1. If the control unit 11 extracts informational materials such as road signs H and road markings P from the image data M1, it recognizes the content of the information provided. For example, the control unit 11 recognizes road R located in the direction G in which the vehicle 1 is traveling within the environment included in the image data M1.

[0026] The control unit 11 recognizes road R by setting the road boundary line Ln (n: natural number) as the road boundary line if, for example, a road boundary line Ln exists on road R. The control unit 11 recognizes the road surface between a pair of adjacent road boundary lines Ln as road R. If the control unit 11 cannot recognize a boundary line on road R, it sets a virtual road boundary line based on the arrangement of structures adjacent to the road. If no road boundary line Ln exists on road R, the control unit 11 extracts characteristic features of structures present in the road environment, such as the color of the pavement surface, the presence or absence of sidewalks, the presence or absence of road structures such as guardrails, and the positional relationship with building B, and sets a virtual road boundary line LMm (m: natural number) on road R.

[0027] The control unit 11 performs processing to treat the set virtual road boundary line LMm as a road boundary. The control unit 11 recognizes the road surface between a pair of adjacent virtual road boundary lines LMm as road R. The control unit 11 may also recognize the road surface between an adjacent road boundary line Ln and the virtual road boundary line LMm as road R. The control unit 11 may also recognize road R by comparing map data acquired from the server device 20 with position data from the position sensor 2D.

[0028] The control unit 11 recognizes a road sign H located near the road R. The control unit 11 recognizes the sign board H1 of the road sign H and analyzes the content of the information displayed on the sign board H1. The control unit 11 recognizes the shapes and characters displayed on the sign board H1 and recognizes the content of the information displayed on the sign board H1 based on the shape information and character information. If the content of the information displayed on the sign board H1 is a predetermined sign that restricts the passage of vehicle 1, such as "no entry for vehicles," "road closed," or "no entry except in the specified direction," the control unit 11 determines whether vehicle 1 is allowed to pass on road R.

[0029] If the information displayed on the sign board H1 includes traffic restrictions based on vehicle classification, vehicle weight, vehicle width or height, or cargo, the control unit 11 compares the vehicle's specifications with the traffic restriction data. If the vehicle's specifications fall under a traffic restriction, the control unit 11 determines that vehicle 1 cannot travel on road R. If the vehicle's specifications do not fall under a traffic restriction, the control unit 11 determines that vehicle 1 can travel on road R.

[0030] When the control unit 11 recognizes an auxiliary sign H2 present on road sign H, it analyzes the content of the information displayed on the auxiliary sign H2. If the information displayed on sign board H1 includes traffic restrictions based on vehicle classification, traffic restrictions based on day / time, or traffic restrictions such as the end point, start point, or within a section or area, the control unit 11 compares the vehicle 1's specifications data, current date / time data, and current location data with the content of the traffic restrictions. If the vehicle 1's specifications, date / time, and location fall under a traffic restriction, the control unit 11 determines that vehicle 1 cannot travel on road R. If the vehicle 1's specifications, date / time, and location do not fall under a traffic restriction, the control unit 11 determines that vehicle 1 can travel on road R.

[0031] When the control unit 11 recognizes a predetermined sign indicating no entry on road R, it recognizes road R where the predetermined sign is located as a no-entry road. Based on, for example, position data acquired from position sensor 2D, imaging data acquired from camera 2A, map data acquired from server device 20, etc., if the control unit 11 determines that vehicle 1 is entering a no-entry road, it executes driving support control to output a predetermined warning to input / output unit 3. The control unit 11 may also execute driving support control to output a predetermined warning to input / output unit 3 a predetermined distance or time before vehicle 1 enters a no-entry road if it predicts that vehicle 1 is entering a no-entry road.

[0032] The control unit 11 may, when it predicts that vehicle 1 will enter a restricted road, perform driving support control to suppress vehicle 1 from entering the restricted road by controlling the drive unit 5, braking unit 6, and steering unit 7 a predetermined distance or time before vehicle 1 enters the restricted road. The control unit 11 generates at least one piece of information, such as text information, image information, sound information, light information, or vibration information, which serves as a warning against entering a restricted road, and outputs it from the input / output unit 3.

[0033] As shown in Figure 3, road R may branch into a first road R1 and a second road R2. The illustrated example shows a road environment in which road R branches into a first road R1 and a second road R2, and where one predetermined sign is present at the branching point of road R. When road R, where the predetermined sign is present, branches into a first road R1 and a second road R2, the control unit 11 determines that either the first road R1 or the second road R2 is a no-entry road based on the positional relationship between the predetermined sign and the first road R1 and the second road R2.

[0034] The control unit 11 recognizes the first road R1 and the second road R2 based on the imaging data M2. The control unit 11 recognizes the first road boundary and the second road boundary included in the road boundary or virtual road boundary on a road R that branches into the first road R1 and the second road R2. The control unit 11 recognizes the first road R1 and the second road R2 based, for example, on the road boundary line Ln. The control unit 11 recognizes the road surface between a pair of adjacent road boundary lines Ln as either the first road R1 or the second road R2.

[0035] If no road boundary line Ln exists, the control unit 11 sets a virtual road boundary line LMm and recognizes the road surface between an adjacent pair of virtual road boundary lines LMm as either the first road R1 or the second road R2. The control unit 11 recognizes the road surface between an adjacent road boundary line Ln and a virtual road boundary line LMm as road R. Based on the road boundary line Ln and / or the virtual road boundary line LMm, the control unit 11 recognizes the first road boundary of the first road R1. Based on the road boundary line Ln and / or the virtual road boundary line LMm, the control unit 11 recognizes the second road boundary of the second road R2.

[0036] The control unit 11 determines whether a road sign H is a designated sign if it exists on road R where the first road R1 and the second road R2 diverge. The control unit 11 analyzes the content of the information indicated by the road sign H. If the control unit 11 recognizes that the road sign H is a designated sign, it determines whether the designated sign belongs to either the first road R1 or the second road R2.

[0037] The control unit 11 recognizes, for example, the installation position of a predetermined sign on the road R. The control unit 11 recognizes, for example, the support post H3 that supports the sign plate H1 of the road sign H based on the imaging data M2. The control unit 11 recognizes the installation point H4 of the support post H3 on the road surface. The installation point H4 is the base end of the support post H3 on the road surface. The control unit 11 may also recognize the installation point H4 of the support post H3 on the ground using the detected values ​​of the LiDAR device 2B and / or the radar device 2C. The control unit 11 recognizes the installation point H4 of the support post H3 of the predetermined sign on the ground as the installation position of the road sign H. If the sign plate H1 is installed in the air, the control unit 11 may set the vertical projection position of the sign plate H1 onto the road surface as a virtual installation point.

[0038] The control unit 11 recognizes the first road boundary of the first road R1 and measures the first distance between the installation position of the predetermined marker and the first road boundary. The control unit 11 sets the first distance to, for example, the length of the first perpendicular D1 to the first road boundary adjacent to the installation position. The control unit 11 recognizes the second road boundary of the second road R2 and measures the second distance between the installation position of the predetermined marker and the second road boundary. The control unit 11 sets the second distance to, for example, the length of the second perpendicular D2 to the second road boundary adjacent to the installation position.

[0039] The control unit 11 compares a first distance between the installation location of the designated sign and the first road boundary with a second distance between the installation location of the designated sign and the second road boundary. Based on the comparison result of the first and second distances, the control unit 11 determines that the road to which the closer road belongs is a no-entry road. In the illustrated example, the control unit 11 determines that the second distance of the second road boundary is shorter than the first distance of the first road boundary, and recognizes that the designated sign belongs to the second road R2. The control unit 11 recognizes that the second road R2 to which the designated sign belongs is a no-entry road.

[0040] The control unit 11 controls, for example, the input / output unit 3, the drive unit 5, the braking unit 6, and the steering unit 7 to perform driving support control that prevents the vehicle 1 from entering the second road R2, and also performs driving support control that encourages the vehicle 1 to travel on the first road R1.

[0041] Figure 4 shows the processing flow of the vehicle control method executed in the vehicle control device 10. The vehicle control method is executed based on a computer program installed in the computer mounted on the vehicle control device 10, which performs driver assistance control of the vehicle. The computer program causes the control unit 11 of the vehicle control device 10 to perform the following processes.

[0042] The control unit 11 acquires imaging data from the camera 2A capturing the environment around the vehicle 1 (S100). The control unit 11 analyzes the imaging data (S102). The control unit 11 recognizes roads in the environment that exist in the direction the vehicle is traveling (S104). The control unit 11 determines whether or not there is a predetermined sign indicating no entry on road R (S106). If the control unit 11 determines that there is a predetermined sign, it determines whether or not road R branches off (S108).

[0043] If the road R on which the predetermined sign is located branches into a first road R1 and a second road R2 (S108: Yes), the control unit 11 determines that either the first road R1 or the second road R2 is a no-entry road based on the positional relationship between the predetermined sign and the first road R1 and the second road R2 (S110). If the road R on which the predetermined sign is located does not branch (S108: No), the control unit 11 recognizes that the road R on which the predetermined sign is located is a no-entry road and proceeds to S112. If the control unit 11 determines that vehicle 1 has entered a no-entry road, it performs driving support control such as outputting a predetermined alarm to the input / output unit 3 (S112).

[0044] As described above, the vehicle control device 10 can improve the accuracy of determining which roads are off-limits in a road environment where a predetermined sign indicating "no entry" exists on a branching road. The vehicle control device 10 can determine whether the predetermined sign belongs to one of the branching roads in a branching road environment, and can identify the roads that are off-limits. The vehicle control device 10 uses the positional relationship between the predetermined sign and the first road R1 and the second road R2 in the determination process, so even if there is only one predetermined sign at the branching point of road R, it can determine that either the first road R1 or the second road R2 is a road that is off-limits.

[0045] In the embodiments described above, the computer programs executed in each configuration of the vehicle control device 10 may be provided in the form of being recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. The computer programs may also be provided as a program product. [Explanation of Symbols]

[0046] 1 Vehicle, 2 Detection unit, 2A Camera, 2B LiDAR device, 2C Radar device, 2D Position sensor, 3 Input / Output unit, 4 Communication unit, 5 Drive unit, 6 Braking unit, 7 Steering unit, 10 Vehicle control device, 11 Control unit, 12 Storage unit, 20 Server device, B Building, D1 First perpendicular, D2 Second perpendicular, H Road sign, H1 Sign board, H2 Auxiliary sign, H3 Pillar, H4 Installation point, LMm Virtual road boundary line, Ln Road boundary line, M1 Image data, M2 Image data, P Road surface marking, R Road, R1 First road, R2 Second road, W Network

Claims

1. It includes a control unit that performs driver assistance control for the vehicle, The control unit, By analyzing the image data captured from the environment surrounding the vehicle, In the aforementioned environment, the road present in the direction of travel of the vehicle is recognized. Recognizing a designated sign indicating no entry on the aforementioned road, Recognizing roads where entry is prohibited on the road where the aforementioned designated sign exists, If it is determined that the vehicle is entering the aforementioned no-entry road, a predetermined warning is output to the output unit. If the road on which the prescribed sign exists branches into a first road and a second road, then, based on the positional relationship between the prescribed sign, the first road, and the second road, it is determined that either the first road or the second road is the road where entry is prohibited. Vehicle control device.

2. The control unit, The installation location of the predetermined sign on the road is recognized, Recognizing the first road boundary of the first road, Recognizing the second road boundary of the second road, Based on the comparison result between the first distance between the installation location and the first road boundary and the second distance between the installation location and the second road boundary, the road closer to the installation location is determined to be the no-entry road. The vehicle control device according to claim 1.

3. The control unit, The installation point of the predetermined sign on the road surface is recognized as the installation position. The length of the first perpendicular line to the first road boundary adjacent to the installation location is set as the first distance. The length of the second perpendicular line to the second road boundary adjacent to the installation location is set as the second distance. The vehicle control device according to claim 2.

4. The control unit, When a boundary line is recognized on the aforementioned road, the boundary line is set as the road boundary. If the boundary line cannot be recognized on the road, a virtual road boundary is set based on the arrangement of structures adjacent to the road. The road boundary is recognized in the same way as the first road boundary and the second road boundary. The vehicle control device according to claim 2.

5. A computer program installed on a computer mounted on a vehicle control system that performs driver assistance control for a vehicle, By analyzing the image data captured from the environment surrounding the vehicle, In the aforementioned environment, the road present in the direction of travel of the vehicle is recognized. Recognizing a designated sign indicating no entry on the aforementioned road, Recognizing roads where entry is prohibited on the road where the aforementioned designated sign exists, If it is determined that the vehicle is entering the aforementioned no-entry road, a predetermined warning is output to the output unit. If the road on which the predetermined sign exists branches into a first road and a second road, the computer is instructed to perform a process to determine, based on the positional relationship between the predetermined sign, the first road, and the second road, that either the first road or the second road is the no-entry road. Computer program.

Images