Vehicle control device and computer program

Abstract

A vehicle control device (10) is provided with a control unit (11) for executing driving assistance control of a vehicle (1). The control unit performs: analysis of image data obtained by capturing an image of the surroundings of the vehicle; identifying a road present in the environment in the direction of travel of the vehicle; identifying a specified sign indicating entry prohibition on the road; identifying an entry prohibited road in the roads with the specified marks; when it is determined that the vehicle will enter the entry prohibition road, the control unit causes the output unit to output a predetermined alarm. And determining either the first road or the second road as an entry prohibited road on the basis of the positional relationship between the predetermined sign and the first road or the second road when the road having the predetermined sign is branched into the first road and the second road.

Description

TECHNICAL FIELD

[0001] The present application relates to a vehicle control device and a computer program for performing driving assistance. BACKGROUND

[0002] For example, an entry determination device capable of determining whether a vehicle has entered a road on which entry is prohibited is described in Japanese Patent Application Publication No. 2016-224714. The entry determination device described in Japanese Patent Application Publication No. 2016-224714 recognizes a prescribed sign indicating entry prohibition shown on both sides of a road in a traveling direction from image data obtained by imaging the environment around the vehicle. Further, the entry determination device is configured to determine that the vehicle has entered the road on which entry is prohibited when the vehicle passes between the prescribed signs. SUMMARY

[0003] According to the technology described in Japanese Patent Application Publication No. 2016-224714, in a road in which a prescribed sign indicating entry prohibition exists on only one side of the road, there is a possibility that the vehicle cannot determine the road on which entry is prohibited.

[0004] An object of the present application is to provide a vehicle control device and a computer program that improve the accuracy of determining a road on which entry is prohibited.

[0005] One embodiment of the present application is a vehicle control device including a control unit configured to perform driving assistance control of a vehicle,

[0006] The control unit performs the following processing.

[0007] The control unit analyzes image data obtained by imaging the environment around the vehicle.

[0008] The control unit recognizes a road in the environment that exists in a traveling direction of the vehicle.

[0009] The control unit recognizes a prescribed sign indicating entry prohibition on the road.

[0010] The control unit recognizes a road on which entry is prohibited in the road in which the prescribed sign exists.

[0011] The control unit causes an output unit to output a prescribed warning when it is determined that the vehicle will enter the road on which entry is prohibited, and

[0012] When the road in which the prescribed sign exists branches into a first road and a second road, the control unit determines that one of the first road and the second road is the road on which entry is prohibited, based on a positional relationship between the prescribed sign and the first road and the second road.

[0013] According to the present application, it is possible to improve the accuracy of determining a road on which entry is prohibited. BRIEF DESCRIPTION OF DRAWINGS

[0014] Features, advantages, and technical and industrial significance of exemplary embodiments of the present application will be described below with reference to the accompanying drawings, wherein like numerals designate like elements, and wherein:

[0015] Figure 1 is a block diagram showing the structure of a vehicle 1 to which an embodiment of the present application is applied.

[0016] Figure 2 is a diagram showing an example of camera data obtained by imaging the surroundings of the vehicle.

[0017] Figure 3 is a diagram showing another example of camera data.

[0018] Figure 4 is a flowchart showing the processing flow of a vehicle control method executed in a vehicle control device. DETAILED DESCRIPTION

[0019] As shown in Figure 1 , the vehicle 1 is configured to be able to execute driving assistance. The vehicle 1 is composed of a detection section 2 and a vehicle control device 10 to be able to execute driving assistance such as an Advanced Driver-Assistance System (ADAS). The detection section 2 detects the environment around the vehicle 1. The detection section 2 is composed of a plurality of devices according to the purpose. The detection values detected by the detection section 2 are used for driving assistance or a navigation device, etc.

[0020] The detection section 2 is provided with a camera 2A that images the surroundings of the vehicle 1. The camera 2A images the surroundings of the vehicle 1 and outputs camera data. In the present embodiment, the camera 2A images a prescribed range in front of the vehicle 1. The camera data of the camera 2A is used for driving assistance of the vehicle 1 or a dashcam, for example. The imaging range and the imaging direction of the camera 2A can differ according to the vehicle 1.

[0021] The detection section 2 is provided with a laser radar device 2B that detects three-dimensional data around the vehicle 1. The laser radar device 2B irradiates a laser beam in a detection region in front of the vehicle 1 or around the vehicle 1 at a constant cycle and measures the reflected light from an object. The laser radar device 2B is configured to be able to adjust the detection region. The laser radar device 2B scans the laser beam in the detection region and acquires measurement data. The laser radar device 2B is configured to generate point cloud data of three dimensions around the vehicle 1 from the measurement data.

[0022] The measurement values of the laser radar device 2B are used to detect other vehicles, pedestrians, bicycles, motorcycles, etc. that are traffic participants present around the vehicle 1 or objects present around the vehicle 1. The laser radar device 2B detects road structures present in the road environment.

[0023] The detection section 2 includes, for example, a radar device 2C that scans a radar wave to detect an object present in the surroundings of the vehicle 1. The radar device 2C is configured to complement the detection of the object with the lidar device 2B. The radar device 2C detects the relative distance to the object by irradiating a millimeter wave radar wave in a detection region and receiving a reflected wave reflected by the object. The radar device 2C is configured to be able to adjust the detection region.

[0024] The measurement value of the radar device 2C is used to detect other vehicles, pedestrians, bicycles, motorcycles, and the like that are traffic participants present in the surroundings of the vehicle 1 or objects present in the surroundings of the vehicle 1. The radar device 2C detects road structures present in the road environment. The object recognition section is configured by the lidar device 2B and / or the radar device 2C.

[0025] The detection section 2 includes a position sensor 2D that detects the current position of the vehicle 1. The position sensor 2D is, for example, a Global Positioning System (GPS) sensor or a Global Navigation Satellite System (GNSS) sensor. The position of the vehicle 1 can be complemented by autonomous sensors (not shown) such as a gyro sensor, an acceleration sensor, and the like that are used for autonomous navigation.

[0026] The vehicle 1 includes an input / output section 3 that receives an operation by a user and is able to display information. The input / output section 3 is configured by, for example, a touch panel that receives a touch operation and is able to display a display image. The input / output section 3 can be configured by an input section and a display section. The input section receives an input operation of a physical switch or the like, and the display section is able to display information of a display device such as a Liquid Crystal Display (LCD) or an organic Electro Luminescence (EL) display or the like.

[0027] The input / output section 3 can include a speaker that outputs sound information, an indicator that outputs light emission information, a vibrator that outputs vibration information, and the like. The input / output section 3 has a function as an output section that outputs information in at least one of a character, an image, a sound, a light emission, a vibration, and the like.

[0028] The vehicle 1 includes a communication section 4 that is able to communicate with a network W. The communication section 4 is a communication interface configured to be able to perform wireless communication. The communication section 4 is, for example, in communication connection with a wireless base station present in the surroundings of the vehicle 1 and performs mutual communication with various communication objects via the network W. The vehicle 1 is, for example, in communication with a server device 20 that is in communication connection with the network W and acquires map data including the current position of the vehicle 1.

[0029] The vehicle 1 is provided with a drive section 5 that generates a travel power. The drive section 5 is constituted by, for example, an internal combustion engine that uses fuel. In a case where the vehicle 1 is an electrified vehicle, the drive section 5 is constituted by an electric motor. In a case where the vehicle 1 is a hybrid electric vehicle, the drive section 5 can be constituted by a combination of an internal combustion engine and an electric motor. The drive section 5 is controlled by the vehicle control device 10 at the time of execution of the drive assist, and adjusts a speed.

[0030] The vehicle 1 is provided with a brake section 6 that is used to decelerate a vehicle speed and control to a stopped state. The brake section 6 is constituted by, for example, a brake device that generates a braking force. In a case where the vehicle 1 is an electrified vehicle, the brake section 6 can be integrated with the drive section 5. The brake section 6 is controlled by the vehicle control device 10 at the time of execution of the drive assist.

[0031] The vehicle 1 is provided with a steering section 7 that is used to operate a travel direction. The steering section 7 is constituted by a power steering device or the like that imparts a steering angle to a steering wheel in accordance with a steering wheel operation. In a case where the vehicle 1 is an electrified vehicle, the steering section 7 can be integrated with the drive section 5 that variably controls driving forces of left and right drive wheels. The steering section 7 is controlled by the vehicle control device 10 at the time of execution of the drive assist, and adjusts a steering angle.

[0032] The vehicle control device 10 is provided with a control section 11 that executes control related to travel of the vehicle 1. The control section 11 integrates control of execution of travel, drive assist, navigation, communication via a network W, and the like of the vehicle 1 in accordance with a detection value detected by the detection section 2. The control section 11 is constituted by at least one hardware processor such as a Central Processing Unit (CPU). The control section 11 can be realized by a Large Scale Integration (LSI) or an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), a Graphics Processing Unit (GPU), or the like (circuit section; including circuitry). Also, the control section 11 can be realized by cooperation of software and hardware.

[0033] The vehicle control unit 10 includes a storage unit 12 for storing data or programs. The storage unit 12 is composed of a non-transitory storage medium such as a hard disk drive (HDD) or a solid-state drive (SSD). The storage unit 12 stores computer programs and data required for the control of the vehicle 1. The programs can be pre-stored in the storage unit 12, or stored on an externally connectable storage medium such as a DVD or CD-ROM, and installed in the storage unit 12 by installing the storage medium in the drive unit. The control unit 11 controls the drive unit 5, the braking unit 6, and the steering unit 7 based on detection values ​​detected by the detection unit 2, and performs driver assistance controls such as Obstacle Anticipation Assist (OAA).

[0034] like Figure 2 As shown, vehicle 1 is configured to perform driver assistance control based on the recognition results of camera data M1 obtained by capturing images of the environment surrounding vehicle 1. Control unit 11 acquires the camera data M1 captured by camera 2A. Control unit 11 is configured to perform machine learning such as deep learning, which uses the captured camera data of the driving road environment as teacher data, and is capable of recognizing the environment surrounding vehicle 1 or the content of information providers existing within the environment.

[0035] Control unit 11 analyzes camera data M1 and identifies road R, road structures, road signs H, pavement markings P, buildings B, and other traffic participants such as other vehicles and pedestrians from the environment surrounding vehicle 1. When information providers such as road signs H and pavement markings P are extracted from camera data M1, control unit 11 identifies the content of the provided information. For example, control unit 11 identifies road R existing in the direction of travel G of vehicle 1 within the environment contained in camera data M1.

[0036] For example, if a road boundary line Ln (n: a natural number) exists on road R, control unit 11 sets the road boundary line Ln as a road boundary and identifies road R. Control unit 11 identifies the road surface between a pair of adjacent road boundary lines Ln as road R. If no boundary line can be identified on road R, control unit 11 sets a virtual road boundary based on the configuration of structures adjacent to the road. If no road boundary line Ln exists on road R, control unit 11 extracts the characteristic parts of structures existing in the road environment, including: road surface color, presence of sidewalks, presence of road structures such as guardrails, and positional relationship with building B. Then, control unit 11 sets a virtual road boundary line LMm (m: a natural number) on road R.

[0037] The control section 11 performs processing that treats the set virtual road boundary line LMm as a road boundary. The control section 11 recognizes the road surface between a pair of adjacent virtual road boundary lines LMm as a road R. The control section 11 can recognize the road surface between an adjacent road boundary line Ln and a virtual road boundary line LMm as a road R. The control section 11 can recognize a road R by matching map data acquired from the server device 20 and position data of the position sensor 2D.

[0038] The control section 11 recognizes a road sign H present in the vicinity of the road R. The control section 11 recognizes a signboard H1 of the road sign H and analyzes the content of the information displayed on the signboard H1. The control section 11 recognizes a graphic and a character displayed on the signboard H1 and recognizes the content of the information displayed on the signboard H1 from the graphic information and the character information. In the case where the content of the information displayed on the signboard H1 is a prescribed sign that restricts the travel of the vehicle 1, such as a sign that prohibits the entry of a vehicle, a sign that prohibits the passage, a sign that prohibits the passage in a prescribed direction, or the like, the control section 11 determines whether the vehicle 1 can travel on the road R.

[0039] In the content of the information displayed on the signboard H1, there are cases where there is content that indicates a travel regulation based on the classification of the vehicle, a travel regulation based on the weight of the vehicle, a travel regulation based on the width or the height of the vehicle, a travel regulation based on the load, or the like. At this time, the control section 11 compares the specification data of the vehicle 1 with the content of the travel regulation. In the case where the specification of the vehicle 1 conforms to the travel regulation, the control section 11 determines that the vehicle 1 cannot travel on the road R. In the case where the specification of the vehicle 1 does not conform to the travel regulation, the control section 11 determines that the vehicle 1 can travel on the road R.

[0040] The control section 11 analyzes the content of the information displayed on the auxiliary sign H2 in the case where an auxiliary sign H2 present in the road sign H is recognized. In the content of the information displayed on the signboard H1, there are cases where there is a travel regulation based on the classification of the vehicle, a travel regulation based on the date / time period, a terminal point, a starting point, a travel regulation within an interval region, or the like. At this time, the control section 11 compares the specification data of the vehicle 1, the current date and time data, the current position information data, and the content of the travel regulation. In the case where the specification, the date and time, and the position of the vehicle 1 conform to the travel regulation, the control section 11 determines that the vehicle 1 cannot travel on the road R. In the case where the specification, the date and time, and the position of the vehicle 1 do not conform to the travel regulation, the control section 11 determines that the vehicle 1 can travel on the road R.

[0041] When the control unit 11 identifies a designated sign indicating "No Entry" on road R, it identifies road R with the designated sign as a no-entry road. If the control unit 1 determines that vehicle 1 will enter a no-entry road, it executes driver assistance control by causing the input / output unit 3 to output a designated warning. This determination is based, for example, on position data acquired from position sensor 2D, video data acquired from camera 2A, or map data acquired from server device 20. If the control unit 1 predicts that vehicle 1 will enter a no-entry road, it can execute driver assistance control by causing the input / output unit 3 to output a designated warning a designated distance or time before vehicle 1 enters the no-entry road.

[0042] When the control unit 11 predicts that vehicle 1 will enter a prohibited road, it can perform the following driving assistance control: prevent vehicle 1 from entering the prohibited road a predetermined distance or a predetermined time before it will enter. This driving assistance control is performed by the control unit 11 controlling the drive unit 5, the braking unit 6, and the steering unit 7. The control unit 11 generates at least one of the following information as a warning of entering a prohibited road: character information, image information, sound information, light information, vibration information, etc., and outputs the at least one information from the input / output unit 3.

[0043] like Figure 3 As shown, road R sometimes branches into road 1 R1 and road 2 R2. The illustrated example shows a road environment where road R branches into road 1 R1 and road 2 R2, and a designated sign is present at each branch of road R. When road R branches into road 1 R1 and road 2 R2 with a designated sign, the control unit 11 determines that one of road 1 R1 and road 2 R2 is a prohibited road. This determination is based on the positional relationship between the designated sign and road 1 R1 and road 2 R2.

[0044] The control unit 11 identifies the first road R1 and the second road R2 based on the camera data M2. Within a road R branching into the first road R1 and the second road R2, the control unit 11 identifies the first road boundary and the second road boundary contained within the road boundary or virtual road boundary. For example, the control unit 11 identifies the first road R1 and the second road R2 based on road boundary lines Ln. The control unit 11 identifies the road surface between adjacent pairs of road boundary lines Ln as either the first road R1 or the second road R2.

[0045] In the absence of the road boundary line Ln, the control section 11 sets a virtual road boundary line LMm and identifies the road surface between an adjacent pair of virtual road boundary lines LMm as the first road Rl or the second road R2. The control section 11 identifies the road surface between an adjacent road boundary line Ln and a virtual road boundary line LMm as the road R. The control section 11 identifies the first road boundary of the first road Rl from the road boundary line Ln and / or the virtual road boundary line LMm. The control section 11 identifies the second road boundary of the second road R2 from the road boundary line Ln and / or the virtual road boundary line LMm.

[0046] In the presence of the road sign H on the road R branching from the first road Rl and the second road R2, the control section 11 determines whether the road sign H is a prescribed sign. The control section 11 analyzes the content of the information indicated by the road sign H. The control section 11 determines whether the prescribed sign belongs to either of the first road Rl and the second road R2, in the case where the road sign H is identified as a prescribed sign.

[0047] The control section 11, for example, identifies the installation position of the prescribed sign in the road R. The control section 11, for example, identifies the support pillar H3 of the signboard Hl supporting the road sign H from the captured image data M2. The control section 11 identifies the installation point H4 of the support pillar H3 on the road surface. The installation point H4 is the base end portion of the support pillar H3 on the road surface. The control section 11 can identify the installation point H4 of the support pillar H3 on the ground surface using the detection value of the laser radar device 2B and / or the radar device 2C. The control section 11 identifies the installation point H4 of the support pillar H3 of the prescribed sign on the ground surface as the installation position of the road sign H. In the case where the signboard Hl is installed in the air, the control section 11 can set the plumb projected position of the signboard Hl on the road surface as a virtual installation point.

[0048] The control section 11 identifies the first road boundary of the first road Rl and measures the first distance between the installation position of the prescribed sign and the first road boundary. The control section 11, for example, sets the length of the first perpendicular line Dl to the first road boundary adjacent to the installation position as the first distance. The control section 11 identifies the second road boundary of the second road R2 and measures the second distance between the installation position of the prescribed sign and the second road boundary. The control section 11, for example, sets the length of the second perpendicular line D2 to the second road boundary adjacent to the installation position as the second distance.

[0049] The control unit 11 compares a first distance between the location of the designated sign and the boundary of the first road, and a second distance between the location of the designated sign and the boundary of the second road. Based on the comparison result of the first and second distances, the control unit 11 determines that the road closer to the designated sign belongs to a road where entry is prohibited. In the illustrated example, the control unit 11 determines that the second distance to the boundary of the second road is shorter than the first distance to the boundary of the first road, and identifies the designated sign as belonging to the second road R2. The control unit 11 identifies the second road R2 to which the designated sign belongs as a road where entry is prohibited.

[0050] The control unit 11 performs driving assistance control to prevent the vehicle 1 from entering the second road R2 by controlling the input / output unit 3, the drive unit 5, the braking unit 6, and the steering unit 7, and performs driving assistance control to make the vehicle 1 travel on the first road R1.

[0051] Figure 4 The diagram illustrates the processing flow of the vehicle control method executed in the vehicle control device 10. The vehicle control method is executed according to a computer program installed on a computer mounted on the vehicle control device 10 for performing driver assistance controls of the vehicle. The computer program causes the control unit 11 of the vehicle control device 10 to perform the following processes.

[0052] The control unit 11 acquires video data from the camera 2A obtained by capturing images of the environment surrounding the vehicle 1 (S100). The control unit 11 analyzes the video data (S102). The control unit 11 identifies roads existing in the environment in the direction of the vehicle's travel (S104). The control unit 11 determines whether there is a prohibited sign on road R (S106). If the control unit 11 determines that a prohibited sign exists, it determines whether road R branches (S108).

[0053] If a road R with a designated sign branches into Road 1 R1 and Road 2 R2 (S108: Yes), the control unit 11 determines that one of Road 1 R1 and Road 2 R2 is a prohibited road (S110). This determination is based on the positional relationship between the designated sign and Road 1 R1 and Road 2 R2. If a road R with a designated sign does not branch (S108: No), the control unit 11 identifies the road R with the designated sign as a prohibited road and proceeds to S112. If the control unit 1 determines that vehicle 1 has entered a prohibited road, it executes driver assistance controls such as causing the input / output unit 3 to output a designated alarm (S112).

[0054] As described above, the vehicle control device 10 improves the accuracy of determining prohibited roads in road environments where there are designated road branches (R) marked with "No Entry". The vehicle control device 10 can determine whether a designated sign belongs to a specific road branch in a branch road environment and can identify the prohibited road. Furthermore, the vehicle control device 10 can use the positional relationship between the designated sign and the first road R1 and the second road R2 for determination processing. Therefore, even if only one designated sign exists at the branch of road R, it is possible to determine that either the first road R1 or the second road R2 is a prohibited road.

[0055] In the above embodiments, the computer programs executed in each component of the vehicle control device 10 can be provided in the form of a computer-readable portable recording medium (storage medium) such as a semiconductor memory, magnetic recording medium, or optical recording medium. The computer program can be provided as a program product.

Claims

1. A vehicle control device, characterized in that, It has a control unit for performing driver assistance controls of the vehicle. The control unit performs the following processing: Analyze video data obtained by capturing images of the environment surrounding the vehicle; Identify roads existing in the vehicle's direction of travel within the environment; Identify the designated signs on the road indicating no entry; Identify prohibited areas within the roads marked with the prescribed signs; If it is determined that the vehicle will enter the prohibited road, the output unit will output a prescribed alarm. and If the road branch with the prescribed sign is Road 1 and Road 2, then based on the positional relationship between the prescribed sign and Road 1 and Road 2, one of Road 1 and Road 2 is determined to be the prohibited road.

2. The vehicle control device according to claim 1, characterized in that, The control unit performs the following processing: Identify the location of the prescribed signs on the road; Identify the boundary of the first road; Identify the boundary of the second road; and Based on the comparison between the first distance between the setting location and the first road boundary and the second distance between the setting location and the second road boundary, the road belonging to the side closer to the setting location is determined to be the prohibited road.

3. The vehicle control device according to claim 2, characterized in that, The control unit performs the following processing: The designated location is identified as the point on the road surface where the prescribed sign is placed. The length of the first perpendicular line from the setting position to the first road boundary adjacent to the setting position is set as the first distance; and The length of the second perpendicular line from the setting position to the boundary of the second road adjacent to the setting position is set as the second distance.

4. The vehicle control device according to claim 2, characterized in that, The control unit performs the following processing: If a boundary line is identified on the road, the boundary line is set as the road boundary; If the boundary line cannot be identified on the road, a virtual road boundary is set based on the configuration of structures adjacent to the road; and Identify the first road boundary and the second road boundary contained within the road boundary.

5. A computer program installed in a computer, said computer being mounted on a vehicle control device for performing driver assistance control of a vehicle, characterized in that, The computer program causes the computer to perform the following processes: Analyze video data obtained by capturing images of the environment surrounding the vehicle; Identify roads existing in the vehicle's direction of travel within the environment; Identify the designated signs on the road indicating no entry; Identify prohibited areas within the roads marked with the prescribed signs; If it is determined that the vehicle will enter the prohibited road, the output unit will output a prescribed alarm. and If the road branch with the prescribed sign is Road 1 and Road 2, then based on the positional relationship between the prescribed sign and Road 1 and Road 2, one of Road 1 and Road 2 is determined to be the prohibited road.

Images