Vehicle control device and computer program

Abstract

The present invention provides a vehicle control device and a computer program capable of suppressing erroneous recognition as a pedestrian crosswalk when a road sign is detected. A vehicle control device is provided with a control unit that executes driving assistance control of a vehicle, and the control unit performs: analysis of imaging data obtained by imaging an environment around the vehicle; identifying a road present in the environment in a direction in which the vehicle travels; calculating a first distance in the width direction of the road; identifying a road surface identifier existing on the road; calculating a second distance in the width direction of a predetermined region including the road surface mark when it is recognized that the road surface mark is the predetermined pattern representing the pedestrian crosswalk; calculating a ratio of the second distance to the first distance; when the ratio is equal to or greater than a predetermined value, the driving assistance control unit recognizes a road surface mark as a pedestrian crosswalk and executes driving assistance control. And when the ratio is less than the predetermined value, recognizing the road surface mark as being not the pedestrian crosswalk, and not executing the driving assistance control.

Description

Technical Field

[0001] This invention relates to a vehicle control device and computer program for performing driving assistance. Background Technology

[0002] For example, Patent Document 1 describes a technique for identifying pedestrian crossings drawn on roads based on camera data. The technique described in Patent Document 1 identifies white lines on the road based on camera data of the environment in front of a vehicle, and identifies them as pedestrian crossings when white lines of a specified width appear repeatedly.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2006-309313 Summary of the Invention

[0004] Characters are drawn in various patterns on roads. According to the technology described in Patent Document 1, it is possible to misidentify the characters drawn on the road as pedestrian crossings.

[0005] The purpose of this invention is to provide a vehicle control device and computer program that can suppress the misidentification of road markings as pedestrian crossings.

[0006] One aspect of the present invention is a vehicle control device comprising a control unit that performs driver assistance control of the vehicle, wherein the control unit performs the following processing: analyzing camera data obtained by capturing images of the environment surrounding the vehicle; identifying a road in the environment existing in the direction of the vehicle's travel; calculating a first distance in the width direction of the road; identifying a road marking existing on the road; if the road marking is identified as a prescribed pattern representing a pedestrian crossing, calculating a second distance in the width direction including a prescribed area of ​​the road marking; calculating the ratio of the second distance to the first distance; if the ratio is greater than or equal to a predetermined value, identifying the road marking as the pedestrian crossing and performing the driver assistance control; and if the ratio is less than the predetermined value, identifying the road marking as not being the pedestrian crossing and not performing the driver assistance control.

[0007] Invention Effects

[0008] According to the present invention, it is possible to suppress the misidentification of pedestrian crossings when road markings are detected. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating the structure of the vehicle involved in the implementation method.

[0010] Figure 2 This is an example of camera data showing a road, including road markings.

[0011] Figure 3This diagram illustrates one example of a method for identifying road markings.

[0012] Figure 4 This diagram illustrates an example of a method for measuring the relative distance between a vehicle and road markings.

[0013] Figure 5 This is a diagram illustrating an example of a method for measuring road width.

[0014] Figure 6 It is a flowchart representing the processing flow of a driver assistance control method. Detailed Implementation

[0015] like Figure 1 As shown, vehicle 1 is configured to perform driving assistance. Vehicle 1 consists of a detection unit 2 and a vehicle control unit 10, enabling it to perform driving assistance functions such as Advanced Driver-Assistance Systems (ADAS). The detection unit 2 detects the environment surrounding vehicle 1. The detection unit 2 consists of multiple devices configured according to its purpose. The detection values ​​detected by the detection unit 2 are used by driving assistance or navigation devices, etc.

[0016] The detection unit 2 includes a camera 2A that captures images of the environment surrounding the vehicle 1. The camera 2A captures images of the environment surrounding the vehicle 1 and outputs video data. In this embodiment, the camera 2A captures images of a predetermined area in front of the vehicle 1. The video data from the camera 2A can be used, for example, for driver assistance or a dashcam of the vehicle 1. The shooting range and shooting direction of the camera 2A can vary depending on the vehicle 1.

[0017] The detection unit 2 includes a light detection and ranging (LiDAR) device 2B for detecting three-dimensional data around the vehicle 1. Within a specified period, the LiDAR device 2B illuminates a laser beam in front of or around the vehicle 1 within the detection area and measures reflected light from objects. The LiDAR device 2B is configured to have an adjustable detection area. The LiDAR device 2B scans the laser beam within the detection area and acquires measurement data. The LiDAR device 2B is configured to generate three-dimensional point cloud data of the area around the vehicle 1 based on the measurement data.

[0018] The measurements taken by lidar device 2B are used to detect other vehicles, pedestrians, bicycles, motorcycles, and other traffic participants or objects present around vehicle 1. Lidar device 2B also detects road structures present in the road environment.

[0019] The detection unit 2 includes, for example, a radar device 2C that scans radar waves to detect objects present around the vehicle 1. The radar device 2C is configured to complement the lidar device 2B in object detection. The radar device 2C illuminates millimeter-wave radar waves in the detection area and receives reflected waves from objects, thereby detecting the relative distance to the objects. The radar device 2C has an adjustable detection area.

[0020] The measurements taken by radar device 2C are used to detect other vehicles, pedestrians, bicycles, motorcycles, and other traffic participants, or objects present in the vicinity of vehicle 1. Radar device 2C detects road structures present in the road environment. The object recognition unit is composed of lidar device 2B and / or radar device 2C.

[0021] The detection unit 2 includes a position sensor 2D that detects the current position of vehicle 1. The position sensor 2D may be, for example, a Global Positioning System (GPS) sensor or a Global Navigation Satellite System (GNSS) sensor. The position sensor 2D can supplement the position of vehicle 1 using autonomous sensors (not shown), such as gyroscope sensors and accelerometers, which are used in autonomous navigation.

[0022] Vehicle 1 is equipped with an input / output unit 3 capable of accepting user operations and displaying information. The input / output unit 3 may be configured, for example, as a touch panel capable of accepting touch operations and displaying images. The input / output unit 3 may be configured with an input unit that accepts input operations such as physical switches and a display unit capable of displaying information from display devices such as liquid crystal displays (LCDs) or organic light-emitting diode (EL) displays.

[0023] The input / output unit 3 may include a speaker for outputting voice information, an indicator for outputting light information, and a vibrator for outputting vibration information. The input / output unit 3 functions as an output unit for outputting information in at least one of the following modes: characters, images, voice, light, and vibration.

[0024] Vehicle 1 is equipped with a communication unit 4 capable of communicating with network W. The communication unit 4 is configured as a communication interface capable of wireless communication. For example, the communication unit 4 communicates with wireless base stations located around vehicle 1 and communicates with various communication objects via network W. Vehicle 1, for example, communicates with a server device 20 that is connected to network W and obtains map information including the current location of vehicle 1.

[0025] Vehicle 1 includes a drive unit 5 that generates power for movement. Drive unit 5 is, for example, composed of an internal combustion engine that uses fuel. If vehicle 1 is an electric vehicle, drive unit 5 is composed of an electric motor. If vehicle 1 is a hybrid vehicle, drive unit 5 may be composed of a combination of an internal combustion engine and an electric motor. When driving assistance is activated, drive unit 5 is controlled by vehicle control device 10, which adjusts its speed.

[0026] Vehicle 1 is equipped with a braking unit 6 for decelerating the vehicle speed and controlling it to a stop. The braking unit 6 is, for example, a braking device that generates braking force. In the case that vehicle 1 is an electric vehicle, the braking unit 6 may be integrated with the drive unit 5. When driving assistance is performed, the braking unit 6 is controlled by the vehicle control unit 10.

[0027] Vehicle 1 is equipped with a steering unit 7 for operating the driving direction. The steering unit 7 consists of a power steering device that imparts a steering angle to the steering wheels based on the operation of the handlebars. When vehicle 1 is an electric vehicle, the steering unit 7 can be integrated with a drive unit 5 that can variablely control the left and right driving forces of the drive wheels. When driving assistance is performed, the steering unit 7 is controlled by the vehicle control device 10 to adjust the steering angle.

[0028] The vehicle control device 10 includes a control unit 11 that performs controls related to the driving of the vehicle 1. The control unit 11 integrates and performs controls for driving, driving assistance, navigation, and communication via the network W of the vehicle 1 based on detection values ​​detected by the detection unit 2. The control unit 11 is composed of at least one hardware processor, such as a central processing unit (CPU). The control unit 11 can be implemented using hardware (including a circuitry) such as a large-scale integrated circuit (LSI), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a graphics processing unit (GPU), or it can be implemented through a combination of software and hardware.

[0029] The vehicle control unit 10 includes a storage unit 12 for storing data or programs. The storage unit 12 is composed of a non-transitory storage medium such as a hard disk drive (HDD) or a solid-state drive (SSD). The storage unit 12 stores computer programs and data required for the control of the vehicle 1. The programs can be pre-stored in the storage unit 12 or stored on an externally connectable storage medium such as a DVD or CD-ROM, and installed in the storage unit 12 by installing the storage medium in the drive unit. The control unit 11 controls the drive unit 5, the braking unit 6, and the steering unit 7 based on the detection values ​​detected by the detection unit 2, and performs driver assistance controls such as Obstacle Anticipation Assist (OAA).

[0030] like Figure 2 As shown, vehicle 1 is configured to perform driver assistance control based on the recognition results of camera data M1 captured by camera 2A of the environment surrounding vehicle 1. Control unit 11 acquires the camera data M1 captured by camera 2A. Control unit 11 is configured to perform machine learning, such as deep learning, in advance, using the camera data of the driving road environment as teacher data, and is able to identify the environment surrounding vehicle 1 or objects, signs, road markings, etc., existing within the environment. Control unit 11 uses image recognition techniques such as semantic segmentation to extract objects contained in the camera data M1.

[0031] The control unit 11 classifies the pixels of the camera data capturing the road environment and segments the camera data by region. The control unit 11 labels the segmented regions and identifies objects. The control unit 11 is configured to calculate the distance of an object relative to the vehicle 1 in the depth direction based on the horizontal (x-direction) and vertical (y-direction) coordinates in the camera data. The control unit 11 calculates the distance between the object and the vehicle 1 in the camera data and calculates the size of the object based on the object's coordinates in the image.

[0032] Control unit 11 analyzes video data M1 captured of the environment surrounding vehicle 1. Control unit 11 identifies roads R existing in the direction of vehicle travel from the environment surrounding vehicle 1. Control unit 11 identifies road structures, road signs, pavement markings P, buildings B, and other traffic participants such as vehicles and pedestrians related to road R. For example, control unit 11 identifies roads R existing in the direction of vehicle 1 travel G within the environment contained in the video data M1.

[0033] For example, when a road boundary line Ln (n: a natural number) exists on road R, control unit 11 sets the road boundary line Ln as the road boundary and identifies road R. Control unit 11 identifies the road surface between a pair of adjacent road boundary lines Ln as road R. Control unit 11 identifies the lane boundary line C1 existing between a pair of road boundary lines Ln as the center line of road R or the lane boundary line. In the illustrated example, control unit 11 identifies the road surface between road boundary lines L1 and L2 as road R, and identifies the lane boundary line C1 existing between road boundary lines L1 and L2 as the center line of road R or the lane boundary line. Road boundary lines Ln and lane boundary lines C1 are represented, for example, by continuous white or yellow lines, dashed lines, etc.

[0034] If the control unit 11 cannot identify the boundary line in road R, it sets a virtual road boundary based on the configuration of structures adjacent to the road. If there is no road boundary line Ln on road R, the control unit 11 extracts characteristic features of structures existing in the road environment, such as the color of the pavement, the presence or absence of sidewalks, the presence or absence of road structures such as guardrails, and their positional relationship with building B, and sets a virtual road boundary line LMm (m: a natural number) on road R.

[0035] The control unit 11 performs the process of treating the set virtual road boundary line LMm as a road boundary. The control unit 11 identifies the road surface between a pair of virtual road boundary lines LMm as road R. The control unit 11 can identify the road surface between road boundary line Ln and virtual road boundary line LMm as road R. It can identify road R by comparing the road map information contained in the map information obtained from the server device 20 with the 2D position data of the position sensor.

[0036] Upon extracting a road surface marking P present on road R, the control unit 11 determines whether the road surface marking P is a pedestrian crossing. The control unit 11 performs a first determination process for identifying a pedestrian crossing from the camera data M1, using an image recognition method such as pattern matching. In the first determination process, the control unit 11 compares the extracted image of the road surface marking P with a template image of the pedestrian crossing. A database of template images is stored, for example, in the storage unit 12. The template image shows a prescribed stripe pattern that is characteristic of a pedestrian crossing. The database of template images contains, for example, multiple images of pedestrian crossings taken from a vehicle-mounted camera.

[0037] The control unit 11 compares the extracted image of the road marking P with the template image and calculates the similarity. If the road marking P extracted from the camera data M1 has a high similarity to the prescribed pattern representing a pedestrian crossing, the control unit 11 identifies the road marking P as a candidate for a pedestrian crossing. The control unit 11, for example, defines a prescribed area Q that includes the road marking P.

[0038] like Figure 3 As shown, in the camera data M2 of road R, there exists not only a first road surface marking P1 indicating a pedestrian crossing, but also other second road surface markings P2 indicating character information, etc. A portion of the second road surface marking P2 includes one or more defined regions Qr (r: a natural number) similar to the defined pattern indicating a pedestrian crossing. In this case, the control unit 11 may identify the second road surface marking P2 as a candidate for a pedestrian crossing during the first determination process. In addition to the first determination process, the control unit 11 also performs a second determination process to improve the recognition accuracy of the road surface marking P.

[0039] In the first determination process, if the control unit 11 identifies the first road marking P1 and the second road marking P2 as prescribed patterns indicating a pedestrian crossing, in the second determination process, it calculates a first distance D1 in the width direction of road R. The width direction of road R refers to the direction of the shorter side of road R that is approximately orthogonal to the length direction of road R. The first distance D1 includes one or more lanes and one or more oncoming lanes in the direction of travel G of vehicle 1. The control unit 11 calculates the first distance D1 in the vicinity of the first road marking P1 and the second road marking P2.

[0040] Control unit 11 calculates a second distance in the width direction of a defined area including road markings. The second distance may include one or more lanes and one or more oncoming lanes in the direction of travel G of vehicle 1. Control unit 11 sets, for example, a defined area QA of a first road marking P1 including a defined pattern. Control unit 11 sets a defined area Qr of a second road marking P2 including a defined pattern. Control unit 11 calculates a second distance DA1 in the width direction of the defined area QA. Control unit 11 calculates a second distance DBr in the width direction of the defined area Qr. Control unit 11 calculates the ratio of the calculated second distance to the first distance D1 according to the following formula (1).

[0041] Ratio = 2nd distance / 1st distance (1)

[0042] Since designated area QA is a pedestrian crossing, the second distance DA1 is approximately equal to the first distance D1, so the ratio is close to 1. Designated area Qr is not a pedestrian crossing, and the second distance DBr is shorter than the first distance D1, so the ratio is less than 1. Since the second distance DBr is road markings such as characters, it is highly likely that the second distance DBr is less than half the length of the first distance D1. The control unit 11 compares the calculated ratio with a preset threshold. The threshold is set to a value less than 1, such as 0.5.

[0043] If the calculated ratio is above the threshold (above the specified value), the control unit 11 identifies the road sign P as a pedestrian crossing. Upon identifying a pedestrian crossing, the control unit 11 performs driver assistance control for the pedestrian crossing. When the pedestrian crossing approaches within a specified distance of the vehicle 1, the control unit 11 controls the drive unit 5, brake unit 6, and steering unit 7 to perform driver assistance control actions such as deceleration, slowing down, stopping, and avoidance maneuvers for the pedestrian crossing. If the calculated ratio is below the threshold (below the specified value), the control unit 11 identifies the second road sign P2 as not a pedestrian crossing. If the second road sign P2 is identified as not a pedestrian crossing, the control unit 11 does not perform driver assistance control.

[0044] like Figure 4 As shown, when performing driver assistance control of a pedestrian crossing, the control unit 11 calculates the relative distance from vehicle 1 to the first road marking P1. When map information is available, the control unit 11 obtains map information, including the current position of vehicle 1, from the server device 20 and / or storage unit 12. The control unit 11 extracts road map information related to road R from the map information, and calculates the relative distance (third distance DA2) from vehicle 1 to the first road marking P1 based on the first position information of road marking P and the second position information of the current position contained in the road map information. The control unit 11 can also calculate the relative distance from vehicle 1 to the first road marking P1 based on the longitudinal (Y-direction) coordinates in the camera data M3 captured of the pedestrian crossing.

[0045] In road R, the depth dimension DA3 of the pedestrian crossing is set to a predetermined value. When the first road marking P1 is identified as a pedestrian crossing, the control unit 11 calculates the depth dimension DA3 of the predetermined area QA of the pedestrian crossing in the camera data M3. Based on the comparison between the calculated dimension DA3 and the predetermined value, the control unit 11 calculates the third distance DA2 from vehicle 1 to the first road marking P1. The control unit 11 calculates the arrival time to the pedestrian crossing based on the relative distance and the vehicle speed. For example, if the arrival time is below the predetermined value, the control unit 11 performs driver assistance control corresponding to the pedestrian crossing.

[0046] exist Figure 5An example of a method for calculating the first distance D1 in the width direction of road R based on camera data M4 is shown. For example, in calculating the first distance D1, control unit 11 identifies a first boundary and a second boundary representing the boundary of road R. In the illustrated example, control unit 11 identifies road boundary line L1 and / or virtual road boundary line LM1 as the first boundary. Control unit 11 identifies road boundary line L2 and / or virtual road boundary line LM2 as the second boundary. Control unit 11 identifies road R through the first boundary and the second boundary. Control unit 11 identifies more than one lane boundary line C1 in road R.

[0047] The control unit 11 identifies lanes in road R based on lane boundary lines C1. The control unit 11 identifies the driving lane R1 in which vehicle 1 is traveling within the identified lanes. The control unit 11 sets a virtual trajectory V1 that extends the position of vehicle 1 relative to the driving direction when traveling between driving lanes R1. The virtual trajectory V1 represents the position of vehicle 1 in driving lane R1. The control unit 11 calculates a fourth distance DA4 relative to the width direction of the first boundary and the virtual trajectory V1 representing the position of vehicle 1. The control unit 11 calculates a fifth distance DA5 relative to the width direction of the second boundary and the virtual trajectory V1 representing the position of vehicle 1. The control unit 11 calculates a first distance D1, for example, by adding the fourth distance DA4 and the fifth distance DA5.

[0048] exist Figure 6 The diagram illustrates the processing flow of a driving assistance control method executed in the vehicle control unit 10. The driving assistance control method is executed according to a computer program installed in the computer mounted on the vehicle control unit 10. The computer program causes the vehicle control unit 10 to perform the following processes.

[0049] Control unit 11 acquires video data captured by camera 2A (S100). Control unit 11 analyzes the video data of the environment surrounding the vehicle (S102). Control unit 11 identifies a road R present in the environment in the direction of the vehicle's travel (S104). Control unit 11 determines whether a road marking P exists on road R (S106). If road marking P is not identified, control unit 11 returns to S100 to continue processing. If road marking P is identified, control unit 11 determines whether road marking P is a standard pattern indicating a pedestrian crossing (S108).

[0050] If the road marking P is not a prescribed pattern, the control unit 11 returns to S100 and continues processing. If the road marking P is a prescribed pattern, the control unit 11 calculates the first distance D1 in the width direction of the road R (S110). If the road marking P is identified as a prescribed pattern indicating a pedestrian crossing, the control unit 11 calculates the second distance in the width direction of the prescribed area Q including the road marking P (S112). The control unit 11 calculates the ratio of the second distance to the first distance and determines whether the ratio is greater than or equal to a prescribed value (S114).

[0051] If the ratio is less than the specified value, the control unit 11 determines that the road marking P is not a specified pattern, does not perform driving assistance, and returns the processing to S100. If the ratio is greater than or equal to the specified value, the control unit 11 identifies the road marking P as a pedestrian crossing (S116). The control unit 11 performs driving assistance control for the pedestrian crossing (S118).

[0052] As described above, the vehicle control device 10 can suppress the misidentification of road marking P as a pedestrian crossing when it is detected. According to the vehicle control device 10, even if the road marking P is determined to be a prescribed pattern in the first determination process, the misidentification of road marking P as a pedestrian crossing can be prevented by performing the second determination process. According to the vehicle control device 10, the execution of driver assistance control due to misidentification as a pedestrian crossing can be suppressed.

[0053] In the above embodiments, the computer programs executed in each component of the vehicle control device 10 can be provided in a computer-readable portable recording medium such as a semiconductor memory, magnetic recording medium, or optical recording medium. The computer program can be provided as a program product.

[0054] Symbol Explanation

[0055] 1-Vehicle, 2-Detection unit, 2A-Camera, 2B-LiDAR device, 2C-Radar device, 2D-Position sensor, 3-Input / Output unit, 4-Communication unit, 5-Drive unit, 6-Brake unit, 7-Steering unit, 10-Vehicle control unit, 11-Control unit, 12-Storage unit, 20-Server device, B-Building, C1-Lane boundary line, D1-First distance, DA1, DBr-Second distance, DA2-Third distance, DA3-Dimension, DA4-Fourth distance, DA5-Fifth distance, G-Direction, LMm-Virtual road boundary line, Ln-Road boundary line, M1, M2, M3, M4-Camera data, P-Road marking, P1-First road marking, P2-Second road marking, Q-Designated area, QA-Designated area, Qr-Designated area, R-Road, R1-Driving lane, W-Network.

Claims

1. A vehicle control device comprising a control unit for performing driver assistance control of the vehicle, characterized in that, The control unit performs the following processing: Camera data obtained by analyzing the environment around the vehicle being filmed; Identify roads that exist in the direction of vehicle travel within the environment; Calculate the first distance in the width direction of the road; Identify road markings present on the road; If the road marking is identified as a prescribed pattern representing a pedestrian crossing, a second distance in the width direction of the prescribed area including the road marking is calculated. Calculate the ratio of the second distance to the first distance; When the ratio is above a predetermined value, the road markings are identified as the pedestrian crossing, and the driving assistance control is executed. and If the ratio is less than a specified value, the road marking will be identified as not being a pedestrian crossing, and the driving assistance control will not be executed.

2. The vehicle control device according to claim 1, characterized in that, The control unit performs the following processing: When the road markings are identified as pedestrian crossings, the dimensions and coordinates of the designated area in the depth direction in the camera data are calculated. Based on the dimensions and coordinates, calculate the third relative distance from the vehicle to the pedestrian crossing; Based on the third distance and the speed of the vehicle, calculate the arrival time before reaching the pedestrian crossing; and If the arrival time is below a predetermined value, the driving assistance control corresponding to the pedestrian crossing is executed.

3. The vehicle control device according to claim 1, characterized in that, The control unit performs the following processing: Identify the first boundary and the second boundary representing the boundary of the road; Calculate the fourth relative distance in the width direction between the first boundary and the vehicle; Calculate the relative fifth distance in the width direction between the second boundary and the vehicle; and The first distance is calculated based on the fourth distance and the fifth distance.

4. The vehicle control device according to claim 1, characterized in that, The control unit performs the following processing: Obtain map information including the current location of the vehicle; Extract road map information related to the road from the map information; and Based on the first location information of the pedestrian crossing and the second location information of the current location contained in the road map information, the relative distance from the vehicle to the pedestrian crossing is calculated.

5. A computer program installed in a vehicle control device that performs driver assistance control of a vehicle, the computer program being characterized in that it causes the computer to perform the following processing: Camera data obtained by analyzing the environment around the vehicle being filmed; Identify roads that exist in the direction of vehicle travel within the environment; Calculate the first distance in the width direction of the road; Identify road markings present on the road; If the road marking is identified as a prescribed pattern representing a pedestrian crossing, a second distance in the width direction of the prescribed area including the road marking is calculated. Calculate the ratio of the second distance to the first distance; In a case where the ratio is equal to or greater than a predetermined value, the drive assist control is executed. and In a case where the ratio is less than the predetermined value, the drive assist control is not executed.

Images