Vehicle data generation server, data generation device, program, system

Abstract

To a map generation server (3), a driving assistance ECU (20) transmits, as a traffic signal response report, a data set that indicates a combination of lighting colors of a traffic signal when passage through an intersection was made or when a stop was made before the intersection and that indicates an ID of a lane which was traveled upon. On the basis of traffic signal response reports from a plurality of vehicles with respect to the same traffic signal, the map generation server (3) generates traffic signal response policy data that indicates, for each lane, a lighting pattern which allows passage / a lighting pattern which necessitates a stop. The traffic signal response policy data is distributed to a vehicle and reflected in control.

Description

Cross - reference to related applications

[0001] This application is based on Japanese Patent Application No. 2021 - 146928 filed in Japan on September 9, 2021, and the content of the basic application is incorporated herein by reference in its entirety.

Technical Field

[0002] The present disclosure relates to generating data for assisting vehicle control for a traffic signal with a arrow signal. Technology relates to

Background Art

[0003] Patent Document 1 discloses a configuration in which an in - vehicle device combines position information for each lighting unit constituting a traffic signal, lighting color thereof, lighting pattern information indicating a lighting shape, and a detection result of the lighting state of the traffic signal by an in - vehicle camera to recognize the lighting state of the traffic signal. The lighting shape indicates a circle, an arrow, a number, etc. Regarding a lighting unit with an arrow - shaped lighting shape, information about the direction of the arrow may also be included. A green arrow signal, which is a lighting unit with an arrow - shaped lighting shape that lights green, is often lit in parallel with a red round signal, which is a lighting unit with a round shape that lights red, as a sign that permits limited / exceptional passage in some directions. In the present disclosure, a traffic signal with a green arrow signal is also referred to as a traffic signal with an arrow signal.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

[0005] Even when the red light of a traffic signal with an arrow indicator located in front of the vehicle is on and the accompanying green arrow is on, depending on the driving lane of the vehicle, it may be possible to pass without stopping. However, since the illuminated part of the traffic signal included in the captured image of the in-vehicle camera is small, even if the illuminated color can be determined, it is difficult to accurately determine the illuminated shape. That is, it is difficult to determine by image recognition whether the illuminated shape is an arrow and the direction of the arrow. In addition, the difficulty level can increase as the vehicle moves away from the traffic signal. In addition, in bad environmental conditions such as during rainfall or fog, the recognition accuracy of the shape of the illuminated part that is lit green may deteriorate compared to sunny days.

[0006] Under such circumstances, as data for assisting vehicle control for a traffic signal with an arrow indicator, there is a demand for a mechanism that can generate data that allows the vehicle to determine whether to stop before an intersection even when the shape of the illuminated part is unknown.

[0007] Incidentally, if the vehicle could use the illuminated pattern information disclosed in Patent Document 1, it might be possible to determine whether to stop for a traffic signal with a green arrow even from the arrangement pattern of the illuminated part specified by image recognition. However, the illuminated pattern information disclosed in Patent Document 1 includes detailed information such as where on the traffic signal, in what shape, and what color it emits light. With such illuminated pattern data, the data size can become large. Also, data management can become complicated. From the perspective of reducing communication load, it is preferable that the data used in the vehicle is simpler data. Moreover, in Patent Document 1, there is no mention at all about how to create detailed illuminated pattern information.

[0008] The present disclosure has been made based on the above-mentioned viewpoints, and one of its purposes is to determine whether to stop before an intersection based on the illuminated state of the traffic signal Technology for generating data useful for judgment and provide it.

[0009] The vehicle data generation server disclosed herein is a vehicle data generation server that generates data for vehicle control with respect to traffic lights. The server includes a report acquisition unit (G1) that acquires, as a traffic signal response report, a data set indicating, from a plurality of vehicles, information indicating the lane in which the vehicle is traveling, a combination of the lit colors of the traffic lights observed by the vehicle, and the behavior of the vehicle with respect to the combination of the lit colors; a traffic signal response policy generation unit (G21) that generates, as traffic signal response policy data, passable pattern data indicating a passable combination of lit colors for each lane for each traffic light based on the traffic signal response report acquired by the report acquisition unit; and a transmission processing unit (G3) that transmits the traffic signal response policy data generated by the traffic signal response policy generation unit to an external device. The passable pattern data is a data set indicating a passable combination of lit colors for each lane, and does not include data indicating the shape of each lighting unit. - It is a data set that does not include data indicating the shape of each lighting unit.

[0010] The above server generates and transmits, as traffic signal response policy data, a data set indicating a passable / stopping combination of lit colors for each lane. By comparing the number of its own lane, the recognition result of the lit color of the traffic light, and the traffic signal response policy data, the vehicle can determine whether it can currently pass through the intersection. At this time, since it is not necessary to recognize the shape of the lighting unit such as the direction of the arrow, it is possible to determine whether passage is possible from a relatively far distance. Also, even if it is a camera or an image recognition device with relatively low resolution, as long as the combination of lit colors can be specified, it is possible to determine whether passage is possible. Note that the above traffic signal response policy data indicates whether passage is possible for each lane depending on the combination of colors, and does not necessarily need to include shape information of the lighting unit or location information in the housing. That is, as data related to traffic lights, it has the advantage that the data size can be suppressed compared to the lighting pattern information disclosed in Patent Document 1.

[0011] The present disclosure's The data generation device includes a behavior acquisition unit that acquires information indicating the behavior of the host vehicle or other vehicles, a lighting state acquisition unit that acquires information indicating the lighting state of a traffic signal from a camera, and a report generation unit that generates a traffic signal response report, which is a data set indicating the lighting state of the traffic signal acquired by the lighting state acquisition unit and the behavior of the host vehicle or other vehicles with respect to the lighting state acquired by the behavior acquisition unit. The report generation unit is configured to generate, as the traffic signal response report, a data set that does not include data indicating the shape of the lighting unit in the traffic signal. Data generation device 。

[0012] The above Data generation device is the above vehicle data generation server - No. 1 machine response policy data Generate a data set as a material for generating is. According to the above Data generation device, The vehicle data generation server efficiently generates traffic signal response policy data it becomes possible.

[0013] The program of the present disclosure includes instructions for causing a computer to acquire information indicating the behavior of a vehicle, acquire data indicating the lighting state of a traffic signal output from a camera, and generate a data set indicating the lighting state of the traffic signal and the behavior of the vehicle with respect to the lighting state, the data set not including data indicating the shape of the lighting unit in the traffic signal The system of the present disclosure includes a lighting state acquisition unit that acquires information indicating the lighting state of a traffic signal using a camera, a data generation unit that generates a data set indicating the lighting state of the traffic signal and the behavior of the vehicle with respect to the lighting state, the data set not including data indicating the shape of the lighting unit, and a traffic signal response policy generation unit that generates, as traffic signal response policy data, a data set indicating a combination of passable lighting colors for each lane for each traffic signal and not including data indicating the shape of each lighting unit, based on the data set generated by the data generation unit. A traffic signal passability determination unit that determines whether the lighting state of the traffic signal corresponds to a passable lighting state for the vehicle, based on the traffic signal response policy data generated by the traffic signal response policy generation unit, lane information, and the lighting state acquired by the lighting state acquisition unit, and a response unit that performs vehicle control according to the determination result of the traffic signal passability determination unit In addition, Patent The reference signs in parentheses described in the claims indicate the correspondence with the specific means described in the embodiments described below as one aspect, and do not limit the technical scope of the present disclosure.

Brief Description of Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Embodiments for Carrying Out the Invention

[0015] Hereinafter, embodiments of the vehicle control system 1 according to the present disclosure will be described with reference to the drawings. In the following, an area where left-hand traffic is legalized will be taken as an example for the description. The present disclosure can be appropriately modified and implemented to conform to the laws and customs of the area where the vehicle control system 1 is used. For example, in an area where right-hand traffic is legalized, the left and right in the description regarding left turns / right turns at intersections can be interchanged for implementation.

[0016] Also, the green light of the traffic signal 9 hereinafter is assumed to indicate a lit state permitting passage, and the yellow and red lights are assumed to indicate lit states instructing a stop. The expression of green as a lit color can be understood as blue in Japan. Also, the expression of yellow as a lit color in the present disclosure can be understood as amber color in some regions such as the United Kingdom.

[0017] The traffic signal 9 can include a traffic signal 9A with an arrow lamp (arrow light), which is a lighting device that displays an arrow. In the present disclosure, mainly for the traffic signal 9A with an arrow lamp, an embodiment assuming a traffic signal 9 with a green arrow lamp that adds a green arrow will be described. The green arrow lamp is a lighting device that limitedly permits passage in the direction indicated by the green arrow. The traffic signal 9 with a green arrow lamp is also referred to as an arrow-type traffic signal in Japan. Also, the green arrow lamp can also be referred to as a blue arrow lamp. The green arrow lamp corresponds to a lighting device that displays a green arrow. Note that as arrow lamps, in addition to the green arrow lamp, there are also a yellow arrow lamp that displays a yellow arrow and a red arrow lamp that displays a red arrow. The present disclosure can also be appropriately applied to the traffic signal 9 equipped with a yellow arrow lamp or a red arrow lamp.

[0018] <Overview of the overall configuration> FIG. 1 is a diagram showing an example of a schematic configuration of a map-linked system Sys including a vehicle control system 1 according to the present disclosure. As shown in FIG. 1, the map-linked system Sys includes a vehicle control system 1 installed in a vehicle Ma, a map generation server 3, and a map distribution server 4. In FIG. 1, only one vehicle Ma equipped with the vehicle control system 1 is shown, but there may be a plurality of vehicles Ma equipped with the vehicle control system 1. That is, there may be a plurality of vehicles constituting the map-linked system Sys. MGS shown in FIG. 1 is an abbreviation of Map Generation Server. Also, MDS is an abbreviation of Map Distribution / Delivery Server.

[0019] The vehicle control system 1 can be installed in various vehicles Ma capable of traveling on the road. The vehicle Ma may be, in addition to a four-wheel automobile, a two-wheel automobile, a three-wheel automobile, etc. A bicycle with a motor can also be included in the two-wheel automobiles. The vehicle Ma may be an owner's car owned by an individual, or a vehicle provided for a car-sharing service or a vehicle rental service (so-called rental car). Also, the vehicle Ma may be a service car. Service cars include taxis, route buses, shared buses, etc. Taxis and buses may be robot taxis etc. without a driver on board.

[0020] The vehicle control system 1 transmits the lighting state of traffic lights observed during driving and the position information of various features to the map generation server 3. The map generation server 3 generates map data used in the vehicle control system 1 based on information provided from a plurality of vehicles, and provides part or all of it to the map distribution server 4. The vehicle control system 1 downloads necessary map data from the map distribution server 4 by performing wireless communication with the map distribution server 4, and uses it for driving support, autonomous driving, and navigation.

[0021] <Configuration of Map Data> Here, first, an example of the map data used by the vehicle control system 1, in other words, the map data distributed by the map distribution server 4 will be described. Note that the map data handled by the map distribution server 4 is basically the same as the map data generated by the map generation server 3. However, the map distribution server 4 may generate distribution data according to the application based on the map data provided by the map generation server 3 and distribute it to the vehicle. The map data generated by the map generation server 3 and the map data distributed to the vehicle do not have to be exactly the same. In the present embodiment, a server that generates map data (traffic signal data) and a server that distributes map data to the vehicle are provided separately, but the embodiment is not limited to this. The map generation server 3 and the map distribution server 4 may be integrated as one map server.

[0022] The map data includes road structure data and ground feature data. The road structure data is so-called network data indicating the connection relationship of roads, and includes, for example, node data and link data. The node data is data about nodes at intersections, points where the number of lanes increases or decreases, and points where roads branch / merge. The link data is data about road links that are road sections connecting nodes. The link data includes data such as lane information, curvature, and gradient provided by the road link. The road link can also be called a road segment. The data related to the road structure may be described in terms of lanes. The road structure data may include lane network data indicating the connection relationship at the lane level. Each road link and each lane link are assigned a link ID which is a unique identifier.

[0023] The ground object data can be classified into road edge data, road surface marking data, and three-dimensional object data. The road edge data indicates the position of the road edge. The road surface marking data is data indicating the installation position and type of the road surface marking. The road surface marking refers to the paint drawn on the road surface for traffic regulations or instructions on the road. The road surface marking can be called road surface paint on one side. For example, the lane demarcation lines indicating the boundaries of lanes, crosswalks, stop lines, guiding strips, safety zones, regulatory arrows, etc. are included in the road surface markings. The lines, symbols, and characters applied to the road surface correspond to the road surface markings. In addition, the road surface markings can include not only paint but also differences in the color of the road surface itself and lines, symbols, and characters formed by road studs, stones, etc.

[0024] The three-dimensional object data represents the position and type of a predetermined three-dimensional structure installed along the road. The three-dimensional structures installed along the road are, for example, traffic signs, commercial billboards, poles, guardrails, curbstones, utility poles, traffic lights, etc. The traffic sign refers to a signboard to which at least one of symbols, character strings, and patterns acting as, for example, regulatory signs, guiding signs, warning signs, and indication signs is attached. In the map data, data related to traffic signs and traffic lights 9 is recorded as three-dimensional object data.

[0025] The traffic light data included in the map data includes the center coordinates of the housing, the array type, the size information, the green arrow light information, and the passable pattern data. The array type indicates whether the three-color light parts are arranged vertically or horizontally. The array type corresponds to information indicating whether it is a vertical traffic light or a horizontal traffic light, or the installation posture. The size information indicates the lengths in the horizontal and vertical directions. The arrow light information indicates the presence or absence, number, and direction of the green arrow lights. The green arrow light information indicates, for example, whether it includes a green arrow light or not, or the number of green arrow lights provided. In addition, in the traffic light 9 to which a green arrow light is attached, the green arrow light information also includes the direction of the green arrow light. The passable pattern data is data indicating the combination of passable lighting colors for each lane. The passable pattern data will be described separately later.

[0026] Data related to various features is associated with network data. For example, features installed on a specific lane, such as traffic lights, or features for a specific lane, are associated with the corresponding link data or node data. Some or all of the above features installed along the road, and predetermined road surface markings such as stop lines, are used as landmarks described later. That is, the map data includes data on the installation positions and types of landmarks.

[0027] The above map data is divided into a plurality of patches and managed (generated / updated / distributed). Each patch corresponds to map data of a different area. For example, the map data is stored in units of map tiles obtained by dividing the map recording area into a rectangular shape. A map tile corresponds to a sub-concept of a patch. Each map tile is assigned a tile ID, which is a unique identifier. The map data for each patch or each map tile is a part of the entire map recording area, in other words, local map data. A map tile corresponds to partial map data. The map distribution server 4 distributes partial map data corresponding to the position of the vehicle control system 1 based on a request from the vehicle control system 1.

[0028] The recording range of each individual patch does not have to be rectangular. The recording range of a patch may be hexagonal, circular, or the like. Each patch may be set to partially overlap with adjacent patches. That is, each patch may be set to overlap with other patches near the boundary. In addition, the division mode of the map data may be defined by the data size. In other words, the map recording area may be divided and managed within a range defined by the data size. In that case, each patch is set so that the data volume is less than a predetermined value. According to such an aspect, the data size in one delivery can be made to be a certain value or less.

[0029] The above-mentioned map data is updated as needed by, for example, integrally processing probe data uploaded from a plurality of vehicles. Note that the map data handled by the map cooperation system Sys in the present embodiment is a probe data map (hereinafter referred to as PD map) generated and updated by integrating probe data observed by a plurality of vehicles. As another aspect, the map data handled by the map cooperation system Sys may be a high-precision map (hereinafter referred to as HD map) generated based on fixed-point survey results, high-precision GPS survey results, or data measured by a dedicated probe car equipped with LiDAR or the like. LiDAR is an abbreviation for Light Detection and Ranging or Laser Imaging Detection and Ranging. LiDAR may include a ToF (Time-Of-Flight) camera that generates a distance image. The map data handled by the map cooperation system Sys may be navigation map data, provided that it includes feature data such as traffic lights 9 and landmarks.

[0030] <Configuration of Vehicle Control System 1> As shown in FIG. 2, the vehicle control system 1 includes a front camera 11, a vehicle state sensor 12, a locator 13, a V2X in-vehicle device 14, an HMI system 15, a driving actuator 16, and a driving support ECU 20. Note that ECU in the member name is an abbreviation for Electronic Control Unit and means an electronic control device. Also, HMI is an abbreviation for Human Machine Interface. V2X is an abbreviation for Vehicle to X (Everything) and refers to a communication technology that connects a vehicle to various things. Note that the "V" in V2X refers to an automobile as the host vehicle, and the "X" may refer to various entities other than the host vehicle, such as pedestrians, other vehicles, road facilities, networks, and servers.

[0031] Note that the host vehicle in the present disclosure refers to vehicle Ma on which the vehicle control system 1 is mounted, as viewed from the vehicle control system 1. In the present disclosure, the occupant sitting in the driver's seat of vehicle Ma (i.e., the driver's seat occupant) is also referred to as the user. The concept of the driver's seat occupant includes an operator who has the authority to remotely operate vehicle Ma. In addition, the front-rear, left-right, and up-down directions in the following description are defined based on the host vehicle. Specifically, the front-rear direction corresponds to the longitudinal direction of the host vehicle. The left-right direction corresponds to the width direction of the host vehicle. The up-down direction corresponds to the vehicle height direction.

[0032] The various devices or sensors constituting the vehicle control system 1 are connected, as nodes, to an in-vehicle network Nw, which is a communication network constructed inside the vehicle. Nodes connected to the in-vehicle network Nw can communicate with each other. Note that specific devices may be configured to be able to communicate directly without going through the in-vehicle network Nw. As the standard of the in-vehicle network Nw, various standards such as Controller Area Network (CAN is a registered trademark) and Ethernet (registered trademark) can be adopted.

[0033] The front camera 11 is a camera that images the front of the vehicle at a predetermined viewing angle. The front camera 11 is disposed, for example, at the upper end inside the vehicle compartment of the windshield, the front grille, the rooftop, or the like. As shown in FIG. 3, the front camera 11 includes a camera main body 111 and a camera ECU 112. The camera main body 111 is a module including at least an image sensor and a lens. The camera main body 111 generates imaging image data at a predetermined frame rate, such as 30 fps or 60 fps. The camera ECU 112 is an ECU that detects a predetermined detection target by performing recognition processing on the image frame generated by the camera main body 111. The camera ECU 112 is realized using an image processing chip including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and the like.

[0034] The camera ECU 112 detects a predetermined object based on image information including color, brightness, contrast related to color or brightness, etc. The camera ECU 112 includes an identifier E1 as a functional block. The identifier E1 is configured to identify the type of an object based on the feature vector of the image generated by the camera main body 111. For the identifier E1, for example, a CNN (Convolutional Neural Network) or DNN (Deep Neural Network) applying deep learning can be used.

[0035] The detection object of the camera ECU 112 is appropriately designed. For example, the camera ECU 112 detects the road edge, a predetermined road marking, and traffic signs. The road markings set as detection targets include lane dividing lines, stop lines, arrow paints indicating the traveling direction at intersections, etc. The camera ECU 112 can recognize the curvature, width, etc. of the road based on the regression curve of the detection points indicating the lane dividing lines and the road edge.

[0036] Also, the camera ECU 112 can detect moving objects such as pedestrians and other vehicles. Other vehicles include bicycles (so-called cyclists), motorized bicycles, and motorcycles. The camera ECU 112 identifies the own lane, which is the lane in which the own vehicle is traveling, based on the recognition result of the dividing lines existing on the left and right sides of the own vehicle, and recognizes other vehicles existing in front of the own vehicle on the own lane as preceding vehicles. Then, the distance and relative speed from the preceding vehicle are identified.

[0037] Furthermore, the front camera 11 is configured to be able to detect the traffic signal 9. When the front camera 11 recognizes the traffic signal 9, it recognizes at least the color of the lit part (that is, the lit color). In the present disclosure, the lit part refers to the part that emits light, that is, the lit lamp part among the plurality of lamp parts provided in the traffic signal 9. The lamp part refers to the device itself that can emit light, that is, the lamp device.

[0038] The recognition result of the traffic signal 9 by the front camera 11 includes relative position information of the traffic signal with respect to the host vehicle and lighting state information indicating the lighting state. The lighting state information mainly indicates the combination of lighting colors. The combination of lighting colors includes not only cases where multiple colors such as red and green are included, but also variations where only one lighting color such as only red or only green is present. Also, when the red light, the green arrow for straight-ahead, and the green arrow for left turn are lit simultaneously, the lighting state information may include information indicating the number of each color, such as one red and two greens.

[0039] When the camera ECU 112 can identify the shape of the lighting part of the traffic signal 9, it can output the recognition shape information. As the shape of the lighting part, a circle and an arrow are assumed. When the shape of the lighting part is determined to be an arrow, the direction in which the arrow points is also obtained. The information indicating the lighting state of the traffic signal 9 may include the color and the shape of the lighting part as a set. When the camera ECU 112 cannot identify the shape of the lighting part, a predetermined value indicating "unknown" may be inserted into the data field indicating the shape of the lighting part. In addition, the camera ECU 112 may recognize the center coordinates of the housing of the traffic signal, the arrangement type, the size information, the green arrow information, etc., and output the recognition result to the driving support ECU 20.

[0040] When the camera ECU 112 detects a plurality of traffic lights 9, it uses a flag or the like to distinguish and output the traffic light 9 for the host vehicle from the other traffic lights 9. The traffic light 9 for the host vehicle is the traffic light 9 for the host vehicle lane, in other words, the traffic light 9 that the host vehicle should follow. The traffic light 9 for oncoming vehicles or the traffic light 9 for intersecting vehicles does not correspond to the traffic light 9 for the host vehicle. Note that the intersecting vehicle refers to a vehicle traveling on another road connected to the road on which the host vehicle is traveling. For example, a vehicle coming from the side at an intersection corresponds to the intersecting vehicle. In an area where traffic lights 9 are provided for each lane, the traffic light 9 on the host vehicle lane corresponds to the traffic light 9 for the host vehicle, and the traffic light 9 for the adjacent lane does not correspond to the traffic light 9 for the host vehicle. In an area where one traffic light 9 is provided as a traffic light 9 for a plurality of lanes, among the traffic lights 9 that exist on the extension line of the host vehicle travel route and whose housing faces the host vehicle, the nearest traffic light may correspond to the traffic light 9 for the host vehicle.

[0041] When the camera ECU 112 detects a plurality of traffic lights 9, it preferentially adopts the traffic light 9 existing in the front direction of the host vehicle or the traffic light 9 existing above the host vehicle lane as the traffic light 9 for the host vehicle. Further, when the camera ECU 112 detects a plurality of traffic lights 9, it preferentially adopts the traffic light 9 that is in front of the host vehicle and whose housing faces the direction of the host vehicle as the traffic light 9 for the host vehicle. When a plurality of traffic lights 9 for the host vehicle are detected, the nearest traffic light 9 is adopted as the traffic light 9 for the host vehicle to be used for control. Note that the determination as to whether it is the traffic light 9 for the host vehicle lane may be made by the driving support ECU 20 instead of the camera ECU 112.

[0042] Some or all of the features detected by the front camera 11 are used as landmarks in the driving assistance ECU 20. A landmark refers to a feature that can be used as a landmark for identifying the position of the host vehicle on a map. As landmarks, for example, at least one of a signboard corresponding to a traffic sign such as a regulatory sign or a guide sign, a traffic signal 9, a pole, a guide board, a stop line, a lane line, etc. can be adopted. In the present disclosure, a linear landmark that extends continuously along a road, such as a lane line or a road edge, is referred to as a continuous landmark. In contrast to the continuous landmark, a landmark that is discretely arranged along a road, such as a traffic sign, a stop line, a fire hydrant, a manhole, etc., is referred to as a discrete landmark. The discrete landmark corresponds to a feature that is scattered.

[0043] The camera ECU 112 outputs a signal indicating the relative position, type, moving speed, and configuration of the detected object for each detected object. The output signal of the camera ECU 112 is input to the driving assistance ECU 20 via the in-vehicle network Nw. The detection result of the front camera 11 can also be read as a recognition result or an identification result.

[0044] Note that the functions of the camera ECU 112, such as object recognition processing based on image data, may be provided by another ECU such as the driving assistance ECU 20. In that case, the front camera 11 may provide the driving assistance ECU 20 with image data as observation data. The functional arrangement of the vehicle control system 1 can be changed as appropriate.

[0045] The vehicle state sensor 12 is a group of sensors that detect state quantities related to the driving control of the host vehicle. The vehicle state sensor 12 includes a vehicle speed sensor, a steering sensor, an acceleration sensor, a yaw rate sensor, an accelerator sensor, a brake sensor, etc. The vehicle speed sensor detects the vehicle speed of the host vehicle. The steering sensor detects the steering angle of the host vehicle. The acceleration sensor detects accelerations such as the longitudinal acceleration and lateral acceleration of the host vehicle. The yaw rate sensor detects the angular velocity of the host vehicle. The accelerator sensor is a sensor that detects the depression amount / depression force of the accelerator pedal. The brake sensor is a sensor that detects the depression amount / depression force of the brake pedal. Note that the types of sensors used by the vehicle control system 1 as the vehicle state sensor 12 may be appropriately designed, and it is not necessary to be equipped with all the sensors described above. The vehicle state sensor 12 also includes sensors that detect the operations of the driver. In addition, the vehicle state sensor 12 can include, for example, a rain sensor that detects rainfall and an illuminance sensor that detects the external brightness.

[0046] The locator 13 is a device that generates the position information of the host vehicle by composite positioning that combines a plurality of pieces of information. The locator 13 is configured using, for example, a GNSS receiver. The GNSS receiver is a device that sequentially detects the current position of the GNSS receiver by receiving the navigation signals transmitted from the positioning satellites that constitute the GNSS (Global Navigation Satellite System). For example, when the GNSS receiver can receive navigation signals from 4 or more positioning satellites, it outputs the positioning result every 100 milliseconds. As the GNSS, GPS, Galileo, IRNSS, QZSS, BeiDou, etc. can be adopted.

[0047] The locator 13 sequentially determines the position of the host vehicle by combining the positioning result of the GNSS receiver and the output of the inertial sensor. For example, when the GNSS receiver cannot receive GNSS signals, such as inside a tunnel, the locator 13 performs dead reckoning (i.e., autonomous navigation) using the vehicle speed, yaw rate, and acceleration information input from various vehicle state sensors 12. The position information as the positioning result is output to the in-vehicle network Nw and used by the driving support ECU 20 and the like. Part of the functions of the locator 13 may be provided by the driving support ECU 20.

[0048] The V2X in-vehicle device 14 is a device for the host vehicle to perform wireless communication with other devices. The V2X in-vehicle device 14 includes a cellular communication unit and a short-range communication unit as communication modules. The cellular communication unit is a communication module for performing wireless communication compliant with a predetermined wide-area wireless communication standard. As the wide-area wireless communication standard here, various standards such as LTE (Long Term Evolution), 4G, and 5G can be adopted. The communication module as the cellular communication unit may also be called a TCU (Telematics Control Unit) or a DCM (Data Communication Module).

[0049] In addition, the cellular communication unit may be configured to be able to directly perform wireless communication with other devices by a method compliant with the wide-area wireless communication standard in addition to communication via a wireless base station. The cellular communication unit may be configured to perform cellular V2X (PC5 / Uu). With the installation of the V2X in-vehicle device 14, the host vehicle becomes a connected car that can be connected to the Internet. For example, the driving support ECU 20 can download and use map data corresponding to the current position from the map distribution server 4 in cooperation with the V2X in-vehicle device 14.

[0050] The short-range communication unit included in the V2X in-vehicle device 14 is a communication module that performs short-range communication, which is wireless communication with a communication distance within several hundred meters. The short-range communication may be DSRC (Dedicated Short Range Communications) compliant with the IEEE802.11p standard, or may be Wi-Fi (registered trademark). The short-range communication may be the aforementioned cellular V2X. Either the cellular communication unit or the short-range communication unit can be omitted. When the V2X in-vehicle device 14 does not have a cellular communication function, the driving support ECU 20 may acquire map data and the like from the roadside unit or other vehicles through the short-range communication function.

[0051] The HMI system 15 is a system that provides an input interface function for receiving user operations and an output interface function for presenting information to the user. The HMI system 15 includes a display 151, a speaker 152, and an HCU (HMI Control Unit) 153. As means for presenting information to the user, in addition to the display 151 and the speaker 152, a vibrator, a lighting device, etc. can be adopted.

[0052] The display 151 is a device that displays an image corresponding to a signal input from the HCU 153. The display 151 is, for example, a so-called center display provided at the uppermost part in the center in the vehicle width direction of the instrument panel. The display 151 is capable of full-color display. The display 151 is realized using, for example, a liquid crystal display or an OLED (Organic Light Emitting Diode) display. The display 151 may be a meter display provided in front of the driver's seat. Also, the display 151 may be a head-up display that projects a virtual image onto a part of the front windshield in front of the driver's seat. The speaker 152 is a device that outputs a sound corresponding to an input signal from the HCU 153. Expressions with sound include not only notification sounds but also voices, music, etc.

[0053] HCU153 is configured to integrally control the presentation of information to the user. HCU153 is implemented using a processor such as a CPU or GPU, a RAM (Random Access Memory), a flash memory, etc. HCU153 controls the display screen of display 151 based on the information provided by driving support ECU20 and the signals from an input device (not shown). The input device refers to a touch panel laminated on display 151, a steering switch, a voice input device, etc. HCU153 displays an icon image indicating the recognition state of traffic signal 9 on display 151 based on a request from driving support ECU20.

[0054] Travel actuator 16 is actuators for travel. Travel actuator 16 includes, for example, a brake actuator as a braking device, an electronic throttle, a steering actuator, etc. The steering actuator also includes an EPS (Electric Power Steering) motor. Travel actuator 16 is controlled by driving support ECU20. Note that other ECUs such as a steering ECU for performing steering control, a power unit control ECU for performing acceleration / deceleration control, and a brake ECU may be interposed between driving support ECU20 and the travel actuator.

[0055] Driving support ECU20 is an ECU that supports the driving operation of the driver in the driver's seat based on the detection result of front camera 11. For example, based on the detection result of front camera 11, driving support ECU20 controls travel actuator 16 to execute part or all of the driving operation on behalf of the driver in the driver's seat. Driving support ECU20 may be an automatic driving device that autonomously drives the host vehicle based on the input of an autonomous driving instruction by the user.

[0056] The driving support ECU 20 is mainly composed of a computer including a processor 21, a RAM 22, a storage 23, a communication interface 24, and a bus connecting these components. The processor 21 is hardware for arithmetic processing combined with the RAM 22. The processor 21 includes at least one arithmetic core such as a CPU. The processor 21 executes various processes by accessing the RAM 22. The storage 23 is a memory device using a non-volatile storage medium such as a flash memory or an EEPROM (registered trademark, Electrically Erasable Programmable Read-Only Memory). A driving support program is stored in the storage 23 as a program executed by the processor 21. The processor 21 executing the above program corresponds to the execution of a driving support method as a method corresponding to the driving support program. The communication interface 24 is a circuit for communicating with other devices via the in-vehicle network Nw. The communication interface 24 may be realized using analog circuit elements, ICs, or the like.

[0057] <Regarding the driving support ECU 20> Here, the functions and operations of the driving support ECU 20 will be described with reference to FIG. 4. The driving support ECU 20 provides functions corresponding to various functional blocks shown in FIG. 4 by the processor 21 executing the driving support program stored in the storage 23. That is, the driving support ECU 20 includes, as functional blocks, a provisional position acquisition unit F1, a map acquisition unit F2, a camera output acquisition unit F3, a vehicle state acquisition unit F4, a localization unit F5, an environment recognition unit F6, a control plan unit F7, a control execution unit F8, and a reporting process unit F9.

[0058] The provisional position acquisition unit F1 acquires own vehicle position information, which is the position coordinates of the own vehicle, from the locator 13. Note that the provisional position acquisition unit F1 may have the function of the locator 13. Further, the provisional position acquisition unit F1 can sequentially perform dead reckoning based on the output of a yaw rate sensor or the like starting from the own vehicle position calculated by the localization unit F5 described later.

[0059] The map acquisition unit F2 acquires map data corresponding to the current position of the host vehicle by wirelessly communicating with the map distribution server 4 via the V2X in-vehicle device 14. For example, the map acquisition unit F2 requests and acquires partial map data regarding a road that the host vehicle is scheduled to pass through within a predetermined time from the map distribution server 4. The map data acquired from the map distribution server 4 is stored, for example, in the map storage unit M1. The download of the map data is performed in a predetermined distribution unit such as map tiles.

[0060] The map storage unit M1 is realized by using, for example, a part of the storage area provided by the storage 23 or the RAM 22. The map storage unit M1 is realized by using a non-transitory, physical storage medium. As described above, the map data includes the installation position of the traffic signal 9 for each intersection and its passable pattern data. Since the passable pattern data corresponds to the signal response policy data, the map acquisition unit F2 corresponds to the response policy data reception unit.

[0061] The camera output acquisition unit F3 acquires the recognition result of the front camera 11 with respect to ground objects, other moving bodies, etc. Specifically, the camera output acquisition unit F3 acquires the position, moving speed, type, size, etc. of other moving bodies. Also, when the camera ECU 112 is configured to be able to identify the preceding vehicle, the camera output acquisition unit F3 acquires preceding vehicle information from the camera ECU 112. The preceding vehicle information may include the presence or absence of a preceding vehicle, the inter-vehicle distance from the preceding vehicle, the relative speed, etc.

[0062] In addition, when the camera ECU 112 recognizes the traffic signal 9, the camera output acquisition unit F3 acquires information about the traffic signal for the host vehicle. For example, the camera output acquisition unit F3 acquires from the camera ECU 112 the recognition result regarding the position and lighting state of the traffic signal 9 particularly for the host vehicle. In addition, the camera output acquisition unit F3 acquires the relative position coordinates and type of landmarks such as traffic signs, lane dividing lines, road edges, etc. from the front camera 11. Either or both of the camera output acquisition unit F3 and the camera ECU 112 correspond to the lighting state acquisition unit.

[0063] The vehicle state acquisition unit F4 acquires the traveling speed, traveling direction, time information, weather, illuminance outside the vehicle, wiper operation speed, shift position, etc. from the vehicle state sensor 12 and the like via the in-vehicle network Nw. Further, the vehicle state acquisition unit F4 acquires operation information which is information indicating the driving operation state by the driver. For example, the vehicle state acquisition unit F4 acquires the depressed state of the brake pedal and the depressed state of the accelerator pedal as operation information. The depressed state includes the presence or absence of depression and the depression amount / depression force.

[0064] The localization unit F5 executes a localization process based on the landmark information and the map information acquired by the camera output acquisition unit F3. The localization process refers to a process of specifying the detailed position of the host vehicle by collating the position of a landmark or the like specified based on the image captured by the front camera 11 with the position coordinates of a ground feature registered in the map data. As a preparation process for localization, the localization unit F5 can convert the relative position coordinates of the landmark acquired from the camera ECU 112 into position coordinates in the global coordinate system (hereinafter also referred to as observation coordinates). The observation coordinates of the landmark are calculated, for example, by combining the current position coordinates of the host vehicle and the relative position information of the ground feature with respect to the host vehicle. Note that the camera ECU 112 may perform the calculation of the observation coordinates of the landmark using the current position coordinates of the host vehicle.

[0065] The localization unit F5 performs association between the landmark registered in the map and the landmark observed by the front camera 11 based on the observation coordinates for each landmark. The association (collation) between the observed landmark and the landmark registered in the map can be performed using the position coordinates and the type information. Further, when collating the landmarks, it is preferable to use feature amounts such as shape, size, color, etc. to adopt a landmark with a higher degree of feature matching.

[0066] When the association between the observed landmark and the landmark on the map is completed, the localization unit F5 performs vertical position estimation using the distance information from the observed discrete landmarks. The vertical position estimation corresponds to the process of specifying the vehicle position in the road extension direction. For example, the localization unit F5 sets the position shifted in the reverse direction of the traveling direction by the observed distance of the host vehicle with respect to the landmark as the vehicle position in the road extension direction from the position coordinates of the landmark on the map corresponding to the observed discrete landmark. For example, in a situation where the distance to the road sign in front of the host vehicle is specified as 40 m as a result of image recognition, it is determined that the host vehicle exists at a position shifted 40 m rearward of the vehicle from the position coordinates of the road sign registered in the map data. By performing such vertical position estimation, the detailed remaining distance to the feature points on the road, in other words, the POI, such as intersections, curve entrances / exits, tunnel entrances / exits, the end of traffic jams, etc., is specified.

[0067] In addition, as a lateral position estimation process, the localization unit F5 specifies the lateral position of the host vehicle with respect to the road based on the distances from the left and right road edges / section lines recognized by the front camera 11. For example, when the distance from the left road edge to the vehicle center is specified as 1.75 m as a result of image analysis, it is determined that the host vehicle exists at a position shifted 1.75 m to the right from the coordinates of the left road edge. The localization unit F5 can specify the ego-lane ID, which is the identifier of the host vehicle lane, based on the distances from the left and right road edges recognized by the front camera 11, or the number / types of section lines existing on the side of the host vehicle. The ego-lane ID indicates, for example, which lane the host vehicle is traveling in from the leftmost or rightmost road edge. The ego-lane ID can also be called the ego-lane number. The ego-lane can also be called the ego-lane. The localization unit F5 corresponds to the ego-lane recognition unit. Note that the function of specifying the ego-lane number may be provided by another ECU such as the camera ECU 112. The ego-lane recognition unit may be configured to acquire the ego-lane number determined by another ECU. The configuration of acquiring the ego-lane number determined by another ECU also corresponds to the configuration of recognizing which lane the ego-lane corresponds to from the road edge.

[0068] The localization unit F5 sequentially performs localization processing at a predetermined position estimation cycle. The default value of the position estimation cycle may be 200 milliseconds or 400 milliseconds. For example, as long as the localization unit F5 can recognize (in other words, capture) discrete landmarks, it sequentially performs vertical position estimation processing. Even when the localization unit F5 cannot recognize discrete landmarks, as long as it can recognize at least one of the partition line and the road edge, it sequentially performs horizontal position estimation processing. The position of the host vehicle as a result of the localization processing is expressed in the same coordinate system as the map data, for example, latitude, longitude, and altitude. The host vehicle position information calculated by the localization unit F5 is provided to the provisional position acquisition unit F1, the environment recognition unit F6, and the like.

[0069] The environment recognition unit F6 mainly recognizes the surrounding environment, which is the environment around the host vehicle, based on the recognition results of the front camera 11 obtained by the camera output acquisition unit F3 and the like. The surrounding environment here includes the current position of the host vehicle, the host vehicle lane, road type, speed limit, relative positions such as the traffic signal 9, etc. When there is a traffic signal 9 in front of the host vehicle, the lighting state of the traffic signal 9 is also included in the surrounding environment. The surrounding environment can also include the positions and moving speeds of other moving bodies, the shapes and sizes of surrounding objects, and the like. The environment recognition unit F6 may be integrated with the camera output acquisition unit F3.

[0070] In addition, the environment recognition unit F6 uses the passable pattern data of the traffic signal ahead included in the map data to determine whether the lighting state of the traffic signal ahead corresponds to the passable pattern. Specifically, the environment recognition unit F6 determines whether the lighting state of the traffic signal corresponds to a passable pattern based on the passable pattern data of the traffic signal ahead included in the map data, the host vehicle lane number, and the lighting state of the traffic signal recognized by the front camera 11. The environment recognition unit F6 corresponds to the passability determination unit. Note that the control planning unit F7 may also have the determination function. The functional arrangement can be changed as appropriate.

[0071] The environment recognition unit F6 may acquire detection results from each of a plurality of surrounding monitoring sensors, and recognize the positions and types of objects existing around the host vehicle by combining them. The surrounding monitoring sensors are sensors that recognize objects outside the vehicle, and refer to millimeter-wave radars, LiDARs, etc. Cameras that image the outside of the vehicle, such as the front camera 11, also correspond to the surrounding monitoring sensors.

[0072] For example, the environment recognition unit F6 may recognize the surrounding environment by using both the recognition result from the front camera 11 and the detection result from the distance measurement sensor. More specifically, the environment recognition unit F6 may specify the inter-vehicle distance, relative speed, etc. with the preceding vehicle by using the results of the front distance measurement sensor. The distance measurement sensor corresponds to a surrounding monitoring sensor that detects an object within the detection range by transmitting and receiving a probing wave, such as a millimeter-wave radar, LiDAR, or sonar. The front distance measurement sensor refers to a distance measurement sensor that includes the front of the host vehicle within the detection range.

[0073] In addition, the environment recognition unit F6 may identify the surrounding environment by using the information of other vehicles received by the V2X in-vehicle device 14 from other vehicles, traffic information received from the roadside unit through vehicle-road communication, etc. The traffic information that can be obtained from the roadside unit can include road construction information, traffic regulation information, traffic jam information, weather information, speed limits, etc. The environment recognition unit F6 can recognize the driving environment by integrating the information indicating the external environment input from a plurality of devices.

[0074] The control planning unit F7 generates a plan for vehicle control to assist the user's driving operation using the driving environment recognized by the environment recognition unit F6 and the map data. For example, when it is confirmed that a traffic signal 9 exists in front of the host vehicle, the control planning unit F7 creates a plan for vehicle control according to the lighting state of the traffic signal 9. For example, when the lighting state of the traffic signal 9 at the time when the host vehicle reaches 100 m in front of the traffic signal 9 corresponds to a stop pattern, a driving plan is created to decelerate and stop at a position a predetermined distance in front of the traffic signal 9. The stop pattern corresponds to a lighting pattern in which entry into the intersection is prohibited. If there is no preceding vehicle, or if the distance between the host vehicle and the preceding vehicle is greater than or equal to a predetermined value, the stop position as a response to the lighting state of the traffic signal 9 can be the position of the stop line shown in the map data.

[0075] Also, when the traffic signal 9 corresponds to a stop pattern in a situation where a preceding vehicle exists, the control planning unit F7 may update the control plan as needed so as to stop behind the preceding vehicle at a predetermined distance. When the lighting state of the traffic signal 9 is a passable pattern, a control plan for passing through the intersection is formulated. The passable pattern is a lighting state that permits the host vehicle to enter and pass through the intersection. The expression "passable" can be rephrased as "enterable". Also, the expression "not passable" can be rephrased as "entry into the intersection prohibited" or "passage prohibited". The lighting state that permits entry into and passage through the intersection includes, in addition to the case where a round green light is lit, the case where a green arrow light corresponding to the traveling direction of the host vehicle lane is lit.

[0076] The control plan as the system response to the lighting state of the traffic signal 9 is generated based on the lighting state of the traffic signal 9 when the host vehicle reaches a predetermined distance (for example, 100 m or 50 m) in front of the traffic signal 9, and can be updated at any time based on changes in the lighting state and the like. For the sake of convenience, vehicle control that supports driving when passing through the road where the traffic signal 9 is provided is referred to as traffic signal passing support. Traffic signal passing support includes automatic adjustment of the driving speed, for example, execution of brake control to stop in front of the traffic signal 9. Note that the traffic signal passing support may be a process of notifying the user of the presence of the traffic signal 9 and the lighting state of the traffic signal 9 in cooperation with the HMI system 15. The control plan for the traffic signal passing support can be updated at any time based on changes in the lighting state of the traffic signal 9.

[0077] In addition, the control plan unit F7 may create a control plan including a control schedule for the steering amount to drive in the center of the recognized host vehicle lane, or generate a route along the behavior or driving trajectory of the recognized preceding vehicle as a driving plan. The driving support ECU 20 can perform preceding vehicle following control to control the driving of the host vehicle so as to maintain a predetermined distance from the preceding vehicle and follow it. The driving plan may include acceleration / deceleration schedule information for speed adjustment on the calculated route and steering angle control schedule information.

[0078] The control execution unit F8 is configured to output a control signal corresponding to the control plan determined by the control plan unit F7 to the driving actuator 16 and / or the HCU 153 to be controlled. For example, when deceleration is planned, a control signal for realizing the planned deceleration is output to the brake actuator and the electronic throttle. In addition, a control signal for outputting an image or voice indicating the execution state of the traffic signal passing support is output to the HCU 153. The control plan unit F7, the control execution unit F8, and the notification processing unit Fa correspond to the response unit.

[0079] The report processing unit F9 is configured to transmit a data set in which the recognition result related to the lighting state of the traffic signal 9 for the host vehicle is associated with the host vehicle behavior data indicating the behavior of the host vehicle to the map generation server 3 as a traffic signal response report. The operation of the report processing unit F9 will be described below.

[0080] The notification processing unit Fa executes a process of notifying the driver of the recognition result of the traffic signal 9 and the determination result of passability corresponding to the recognition result. The above notification can be realized by image display on the display 151 and voice message output from the speaker 152. As images associated with the recognition result of the lighting state of the traffic signal 9, the entry prohibition image Im1 indicating that one should stop, in other words, entry is prohibited, and the passable image Im2 indicating that passage is possible can be selectively displayed on the display 151 by the notification processing unit Fa. The notification processing unit Fa performs image display regarding the recognition result of the traffic signal 9 and the determination result of passability on the condition that the remaining distance Drm to the intersection where the traffic signal 9 is installed is less than the control continuation determination distance Dcn described later. Note that various notification processes by the notification processing unit Fa are carried out in accordance with the plan of the control planning unit F7. The driving support ECU 20 may include the notification processing unit Fa as a part of the control execution unit F8.

[0081] The entry-prohibited image Im1 and the passable image Im2 may each include a recognition result image Ims indicating the recognition result of the traffic signal lighting state and a determination result image Imk indicating passability. For example, as shown in FIG. 5, the entry-prohibited image Im1 includes a stop instruction mark Imk1 and a red signal icon Ims1. Also, as shown in FIG. 6, the passable image Im2 includes a passable mark Imk2 and a green signal icon Ims2. The stop instruction mark Imk1 and the passable mark Imk2 correspond to the determination result image Imk. The red signal icon Ims1 and the green signal icon Ims2 correspond to the recognition result image Ims. The notification processing unit Fa may be configured to select and display, as the recognition result image Ims, an image that matches the recognized actual shape / arrangement type of the traffic signal 9 from among the display image databases prepared in advance. For example, when a green arrow signal is recognized, an icon image of the traffic signal 9 including the green arrow signal may be selectively displayed. Note that the character string included in the passable mark Imk2 is not limited to "PASSABLE" and may be, for example, "GO" or the like. Also, the text included in these images may be converted into the official language of the region of use. The determination result image Imk may be a diagram (so-called pictogram) or the like that represents passability without including text.

[0082] The information that the driving support ECU 20 should present to the driver as an image indicating the system operating state related to intersection passage support is (1) that there is a traffic signal 9 ahead and (2) the determination result of whether to proceed or stop. The specific recognition results that can be presented by the recognition result image Ims are arbitrary elements. The notification processing unit Fa may display, in parallel with the determination result image, an icon image that mimics only the shape or arrangement type of the traffic signal 9 instead of the traffic signal 9 image reflecting the recognized lighting state.

[0083] <Regarding the operation flow of the reporting processing unit F9> Next, the traffic signal response report process executed by the report processing unit F9 will be described using the flowchart shown in FIG. 7. The flowchart shown in FIG. 7 is executed at a predetermined cycle (for example, every 200 milliseconds) while the driving power source of the vehicle is on. The driving power source is, for example, the ignition power source in an engine vehicle. In an electric vehicle such as an electric vehicle or a plug-in hybrid vehicle, the system main relay corresponds to the driving power source. In this embodiment, as an example, the traffic signal response report process includes steps S101 to S106. Note that the flowcharts in the present disclosure are all examples, and the number of steps, the processing order, the execution conditions, etc. can be changed as appropriate.

[0084] Note that the localization unit F5 independently of, in other words, in parallel with the flowchart shown in FIG. 7, the driving support ECU 20 sequentially performs the localization process. Specifically, the localization unit F5 sequentially performs the localization process using landmarks. By executing the localization process, the detailed position of the host vehicle on the map is determined.

[0085] First, step S101 is a step in which the environment recognition unit F6 recognizes the driving environment based on signals from the front camera 11 and the like. In step S101, the environment recognition unit F6 acquires traffic signal information, preceding vehicle information, recognition results of lane lines, and the like. The traffic signal information includes the presence or absence of the traffic signal 9, and when the traffic signal 9 exists, the remaining distance to the traffic signal 9, the lighting state, and the like. The preceding vehicle information includes the presence or absence of a preceding vehicle, and when a preceding vehicle exists, the inter-vehicle distance, relative speed, lighting state of the lighting device, and the like with respect to the preceding vehicle. The lighting state of the lighting device refers to the lighting state of a turn signal, a brake lamp, or the like. Also, in step S101, the behavior of the host vehicle such as the vehicle speed and yaw rate of the host vehicle, and the operation information of the driver are acquired.

[0086] In step S102, the localization unit F5 specifies the host vehicle position coordinates and specifies the host vehicle lane ID based on the input signal from the front camera 11. Note that step S102 may be integrated with step S101.

[0087] In step S103, the environment recognition unit F6 determines whether the traffic signal 9 for the host vehicle is detected by the front camera 11. If the traffic signal 9 for the host vehicle is not detected, step S103 is determined to be negative and this flow ends. On the other hand, if the traffic signal 9 for the host vehicle is detected, step S104 is executed. Note that map data may be used to identify whether the detected traffic signal 9 is the traffic signal 9 for the host vehicle. The environment recognition unit F6 may determine whether the traffic signal 9 detected by the front camera 11 is the traffic signal 9 for the host vehicle based on information such as the position, size, arrangement type, and presence or absence of auxiliary lights of the traffic signal 9 shown in the map data.

[0088] In step S104, the camera output acquisition unit F3 acquires the recognition result for the lighting state of the traffic signal 9 for the host vehicle. For example, it acquires the color of the lighting part. If a plurality of lighting parts are lit, it acquires the lighting color of each. In addition, the camera output acquisition unit F3 can acquire the shape of the lighting part, such as round or arrow-shaped, if possible. Additionally, the camera output acquisition unit F3 can acquire the position of the lighting part with respect to the housing, if possible.

[0089] In step S105, the report processing unit F9 determines whether the transmission condition, which is a condition for transmitting the traffic signal response report, is satisfied. If the transmission condition is satisfied, the report processing unit F9 transmits the traffic signal response report as step S106. The traffic signal response report is a data set indicating whether the host vehicle / other vehicle has stopped or passed with respect to the lighting state of the traffic signal 9 for the host vehicle, that is, how it has responded.

[0090] The traffic signal response report is a data set that shows the combination of colors of the lighting unit at traffic signal 9 and the behavior of the host vehicle with respect to it. For example, as shown in FIG. 8, the traffic signal response report may include target information, reporting source, lighting state information, host vehicle behavior information, and leading vehicle information. The target information is information for the map generation server 3 to identify which traffic signal 9 the report is about. For example, the target information is represented by a traffic signal ID, which is a unique identification number assigned to each traffic signal 9. The target information may be represented by a combination of the position coordinates of the traffic signal 9 and the traveling direction. The reporting source information only needs to include information that allows the map generation server 3 to identify from which vehicle traveling in which lane the report is. For example, the reporting source information may be represented by the host vehicle lane ID. The reporting source information preferably includes a road link ID or a traveling direction in addition to the lane ID where the vehicle as the reporting source was located.

[0091] The lighting state information is information about the combination of colors of the lighting parts of the traffic signal 9. The lighting state information may include the number of lighting parts. If the shape of the lighting part is recognized, the lighting state information may include the shape information of the lighting part. When the reporting processing unit F9 is unable to obtain the shape of the lighting part due to environmental factors such as rainfall, it may report that the shape is unknown. Note that the traffic signal response report may include information indicating the color and direction of the lit green arrow lamp when the traffic signal 9 for the host vehicle is equipped with a green arrow lamp and the green arrow lamp is lit. The host vehicle behavior data included in the traffic signal response report indicates the behavior of the host vehicle with respect to the intersection, in other words, the lighting state of the traffic signal. The host vehicle behavior data included in the traffic signal response report indicates, for example, whether the vehicle stopped before the intersection or was able to pass through the intersection without stopping. The reporting processing unit F9 may also report the behavior of the host vehicle with respect to the lighting state of the traffic signal in detail, such as whether the vehicle stopped in front of the intersection for a predetermined number of seconds or more, was able to pass through the intersection without a temporary stop, or passed through the intersection after a temporary stop. Here, the temporary stop is a stop for checking the traffic situation and can be, for example, a stop of less than 5 seconds. The host vehicle behavior data may include time-series data such as vehicle speed, the amount of depression of the brake pedal, and the amount of depression of the accelerator pedal within a predetermined time from the stop time or the intersection passing time. Instead of / In parallel with the time-series data of the amount of depression of the brake / accelerator pedal, time-series data of acceleration may be included.

[0092] The traffic signal response report may include preceding vehicle information such as the inter-vehicle distance from the preceding vehicle and the lighting state of the brake lamp of the preceding vehicle. As another aspect, the traffic signal response report may include relative position information of the lighting part with respect to the housing. That is, it may include information about which part is lit in what color. Furthermore, the traffic signal response report may include configuration information such as the array type and the presence or absence of a green arrow lamp as reference information regarding the traffic signal 9. The array type indicates whether it is vertical or horizontal.

[0093] As the transmission condition of the signal response report, it is possible to adopt that the remaining distance to Signal 9 is equal to or less than a predetermined reporting distance. The reporting distance can be, for example, 10 m, 15 m, 20 m, 50 m. The reporting distance is set to a value at which it is expected that the recognition accuracy regarding the lighting state of Signal 9 is equal to or more than a predetermined value. The transmission condition is set so as to suppress the transmission of information that can become noise, in other words, information of low usefulness / unnecessary information, when generating the signal response policy data described later.

[0094] Note that even if the remaining distance to Signal 9 is equal to or more than the reporting distance, the reporting processing unit F9 may transmit a signal response report based on the fact that the host vehicle has stopped in front of Signal 9 or that the driver's braking operation has been detected. Further, the reporting processing unit F9 may also transmit a signal response report when a driver operation contrary to the automatic control content is detected while the automatic speed adjustment control by the driving support ECU 20 is being executed. The driver operation during the preceding vehicle following control is also referred to as a so-called override operation. For example, when the driver's depression of the accelerator is detected during automatic deceleration toward a stop in front of a signal, a signal response report may be transmitted. Further, a signal response report may be transmitted triggered by the detection of the driver's braking operation during the execution of the preceding vehicle following control. The reporting processing unit F9 may also transmit a signal response report triggered by detecting a change in the lighting state of Signal 9 in a state where the remaining distance to the signal / intersection is equal to or less than a predetermined value. As the reporting event which is an event (trigger) for transmitting the signal response report, it is possible to adopt a driver operation, a stop / start of a preceding vehicle, a change in the lighting state, etc.

[0095] The report processing unit F9 may be configured to transmit a traffic signal response report on the condition that the green arrow light is on or that a plurality of lighting units are on. Further, the report processing unit F9 may be configured to transmit a traffic signal response report only when passing through the traffic signal 9A with arrow lights. The report processing unit F9 may transmit a series of driving behavior data of the host vehicle related to passing through one traffic signal 9 in one data set, or may transmit it in a plurality of divided data sets. Further, the report processing unit F9 may transmit, as a traffic signal response report, a data set indicating the lighting state of the traffic signal 9 when the preceding vehicle or the host vehicle starts. The report processing unit F9 may transmit, as a traffic signal response report, a data set indicating the lighting state of the traffic signal 9 when the preceding vehicle or the host vehicle stops.

[0096] Furthermore, the report processing unit F9 may transmit a data set indicating the lighting state of the traffic signal 9 for the adjacent lane, the adjacent lane ID, and the driving behavior of other vehicles traveling on the adjacent lane to the map generation server 3. According to the configuration of transmitting not only the information related to the host vehicle lane but also the information related to the adjacent lane in this way, it becomes possible to more efficiently collect data indicating appropriate vehicle behavior according to the lighting state of the traffic signal 9 in the map generation server 3.

[0097] In addition, the report processing unit F9 periodically or based on an instruction from the map generation server 3 uploads probe data for updating the road structure and feature information in the map data, in addition to the data indicating the driving behavior of the host vehicle / other vehicles according to the lighting state of the traffic signal 9. The probe data may include the position information of the vehicle and the position information of the observed features. Note that the traffic signal response report can also be regarded as a kind of probe data. The probe data and the traffic signal response report may be integrated. A data set including the lighting state of the traffic signal and the information indicating the vehicle behavior with respect thereto, and the information indicating the traveling position of the host vehicle in the road width direction may correspond to the traffic signal response report. For example, the probe data transmitted when existing within a predetermined distance from the traffic signal may correspond to the traffic signal response report.

[0098] <Regarding the configuration of the map generation server 3> Here, the configuration of the map generation server 3 will be described. As shown in FIG. 9, the map generation server 3 includes a communication device 31, a server processor 32, a server memory 33, a server storage 34, a report DB 35, and a map DB 36. The DB in the member name is an abbreviation for Database.

[0099] The communication device 31 is a communication module for performing data communication with each vehicle via a wide - area communication network such as the Internet. The communication device 31 is configured to be able to communicate with communication facilities that constitute a wide - area communication network using, for example, optical fibers. As a result, the map generation server 3 can perform data communication with vehicles connected to the wide - area communication network. The communication device 31 outputs the data received from the vehicle to the server processor 32, and transmits the data input from the server processor 32 to the vehicle specified by the server processor 32. Note that the expression of the vehicle as a communication partner of the map generation server 3 can be read as the vehicle control system 1, more specifically, the driving support ECU 20.

[0100] The server processor 32 is configured to execute various processes based on the signals / data input from the communication device 31. The server processor 32 is communicably connected to each of the communication device 31, the server memory 33, the server storage 34, the report DB 35, and the map DB 36. The server processor 32 is an arithmetic core that executes various arithmetic processes, and is realized using, for example, a CPU or a GPU. The server memory 33 is a volatile memory such as a RAM. The server memory 33 temporarily stores the arithmetic data by the server processor 32. The server storage 34 is a rewritable non - volatile memory. A predetermined map generation program is stored in the server storage 34. By the server processor 32 executing the map generation program, various functional units described later are realized. Note that the server processor 32 executing the map generation program corresponds to the execution of a map generation method that is a method corresponding to the program.

[0101] Report DB35 is a database for temporarily storing the traffic signal response reports transmitted from vehicles. Probe data can also be stored in Report DB35. Report DB35 is realized using a rewritable non-volatile storage medium. Report DB35 is configured such that the server processor 32 can write, read, delete, etc. data thereto.

[0102] Map DB36 is a database in which the map data described at the beginning is stored. Map DB36 is realized using a rewritable non-volatile storage medium. Map DB36 is configured such that the server processor 32 can write, read, delete, etc. data thereto.

[0103] The map generation server 3 includes, as functional blocks, a report reception unit G1, a map update unit G2, and a transmission processing unit G3. The map update unit G2 includes, as a sub-function, a traffic signal response policy generation unit G21. Note that the traffic signal response policy generation unit G21 may be provided independently of the map update unit G2. Also, the map update unit G2 as a configuration independent of the traffic signal response policy generation unit G21 is an optional element and may be omitted. The map generation server 3 corresponds to a vehicle data generation server.

[0104] The report reception unit G1 acquires the traffic signal response reports and probe data uploaded from the vehicle via the communication device 31. The report reception unit G1 stores the traffic signal response reports and the like acquired from the communication device 31 in the report DB35. The report reception unit G1 can store the received traffic signal response reports separately for each corresponding traffic signal 9 or for each lane on which the reporting source was traveling. The data stored in the report DB35 can be referred to by the map update unit G2, the traffic signal response policy generation unit G21, etc. The report reception unit G1 corresponds to a report acquisition unit. The map update unit G2 performs a process of updating the map data based on the probe data transmitted from a plurality of vehicles. For example, for the same feature, the position of the feature is determined by integrating the observed coordinates reported from a plurality of vehicles, and the map data is updated. The map update unit G2 updates the map data, for example, at a predetermined cycle.

[0105] The traffic signal response policy generation unit G21 is configured to generate a passable pattern for each traffic signal 9 and each lane based on the traffic signal response reports provided from a plurality of vehicles. The process of generating the passable pattern for each lane is also referred to as the traffic signal response policy generation process. The traffic signal response policy generation process can be executed for the traffic signal 9 to which a green arrow light is assigned. Details of the traffic signal response policy generation process will be described separately later.

[0106] The transmission processing unit G3 is configured to transmit the map data including the traffic signal data to the map distribution server 4. The transmission of the map data to the map distribution server 4 may be performed based on a request from the map distribution server 4, or may be performed periodically. Further, the transmission processing unit G3 may transmit some or all of the map data to the map distribution server 4 based on the occurrence of a predetermined transmission event. For example, the transmission processing unit G3 may transmit the data of the patch in which the recording content based on the probe data (that is, map update) has been changed to the map distribution server 4. As another aspect, the transmission processing unit G3 may be configured to distribute the map data based on a request from a vehicle. The map distribution server 4 and the driving support ECU 20 etc. correspond to external devices for the map generation server 3.

[0107] The map distribution server 4 is a server that distributes the map data provided from the map generation server 3 to the vehicle that is the request source in patch units based on a request from the vehicle. For example, the map acquisition unit F2 of the vehicle requests the map distribution server 4 for the map data regarding the current position and the area to be passed within a predetermined time. The map distribution server 4 distributes the corresponding patch map data based on the request from the vehicle. Note that the map distribution server 4 may be configured to distribute only some of the various items included in the map data based on a request from the vehicle. For example, the map distribution server 4 may distribute only the traffic signal data as the map data related to passing through an intersection to the vehicle in association with the corresponding link / node data based on a request from the vehicle.

[0108] <Regarding response pattern generation> The traffic signal response policy generation unit G21 will explain the traffic signal response policy generation process using the flowchart shown in FIG. 10. The flowchart shown in FIG. 10 is executed, for example, at a predetermined generation cycle. The generation cycle is set to an arbitrary period such as one day, one week, or one month. The traffic signal response policy generation process includes steps S201 to S205 as an example. Note that the number of steps and the processing procedure included in the traffic signal response policy generation process can be changed as appropriate. The traffic signal response policy generation process can be implemented for each traffic signal 9. For convenience, the traffic signal 9 to be processed is also described as the target traffic signal. Note that the traffic signal response policy generation process may be executed only for the traffic signal 9 including a green arrow signal.

[0109] Step S201 is a step of reading out the traffic signal response report for the target traffic signal from the report DB 35. Step S201 may be a step of collecting the traffic signal response reports for the target traffic signal from a plurality of vehicles. The process of receiving the traffic signal response reports transmitted from each vehicle is carried out at any time.

[0110] Step S202 is a step of determining whether or not reports for the target traffic signal have accumulated to a specified number or more. The specified number here can be, for example, 10 or 20. Note that step S202 can also be a step of determining whether or not traffic signal response reports have accumulated to a specified number or more for each lane.

[0111] If reports for the target traffic signal have accumulated to a specified number or more, the process proceeds to step S203. Note that subsequent processing is omitted for lanes in which the number of received traffic signal response reports is less than the specified value. That is, the determination of the passable pattern is postponed for lanes in which the number of received reports is less than the default value.

[0112] In step S203, based on the traffic signal response reports for each lane that have been collected, data indicating the passable pattern for each lane, that is, passable pattern data, is generated. For a road having the lane structure shown in FIG. 11, FIG. 13 shows an example of the passable pattern data when the traffic signal 9 shown in FIG. 12 is provided. FIG. 13 is an example of the passable pattern data when the traffic signal 9 with a green arrow signal AG for right turn shown in FIG. 12 is provided for the road having the lane configuration shown in FIG. 11. The road shown in FIG. 11 is a three-lane road on one side, where the first lane is set as a left-turn only lane, the second lane is set as a straight-ahead lane, and the third lane is set as a right-turn only lane.

[0113] CG in FIG. 12 indicates a green round light which is a round light part that lights up green, and CY indicates a yellow round light which is a round light part that lights up yellow. CR indicates a red round light which is a round light part that lights up red. In FIG. 12, a right-facing green arrow signal AG is shown as an example. The right-facing green arrow signal AG is a green arrow signal for right turn. The state where the right-facing green arrow signal AG is lit indicates that right turn is possible.

[0114] As the lighting pattern, the traffic signal 9 shown in FIG. 12 can cyclically take on a state where only the green round light is lit, a state where only the yellow round light is lit, a state where only the red round light is lit, and a state where the red round light and the green arrow signal are lit, as shown in (A) to (D) of FIG. 12. For such a lighting pattern of the traffic signal 9, the traffic signal response policy generation unit G21 generates the passable pattern data shown in FIG. 13 based on reports from vehicles for each lane.

[0115] {G} in the lighting pattern shown in FIG. 13 indicates a state where only the green round light CG is lit, and {Y} indicates a state where only the yellow round light CY is lit. {R} as the lighting pattern indicates a state where only the red round light CR is lit. {R, G} indicates a state where the red round light CR and the green arrow signal AG are lit. G shown in the figure means green, Y means yellow, and R means red.

[0116] Also, the passable lanes {1, 2, 3} shown in FIG. 13 indicate that the first, second, and third lanes are passable. {3} indicates that only the third lane is passable. The {} (empty set) shown in FIG. 13 indicates that there are no passable lanes, that is, vehicles in any lane cannot pass. Whether a lane is passable according to the lighting state of each lane is determined by the behavior of vehicles in each lane associated with the lighting state.

[0117] Note that the configuration of the passable pattern data is not limited to the configuration shown in FIG. 13. For example, as shown in FIG. 14, it may be configured as data indicating a passable lighting state for each lane. FIGS. 13 and 14 are only different in expression form and are substantially equivalent.

[0118] When the generation of the passable pattern data for the target traffic signal is completed by the traffic signal response policy generation unit G21 (step S203), the data set is stored in the map DB36 as a part of the traffic signal data in the map data (step S204). In the map data, the passable pattern data for each traffic signal 9 is associated with the traffic signal 9 in the map data using the traffic signal ID and the like. Also, the corresponding traffic signal 9 itself is associated with network data such as node data and link data. That is, the passable pattern data is stored in a manner associated with the network data. Step S205 is a step in which the map data including the passable pattern data generated by the transmission processing unit G3 is transmitted to an external device such as the map distribution server 4. Step S205 can be executed at an arbitrary timing.

[0119] Note that the traffic signal response policy generation unit G21 of the present embodiment generates passable pattern data as traffic signal response policy data, which is a dataset indicating the response policy for each lane according to the lighting state of the traffic signal 9, but is not limited thereto. As shown in FIGS. 15 and 16, the traffic signal response policy generation unit G21 may generate stop pattern data as traffic signal response policy data. The stop pattern data is a dataset indicating the lighting pattern for which each lane should stop. The map distribution server 4 may distribute the stop pattern data instead of the passable pattern data as part of the map data.

[0120] A combination of lighting colors not defined as a passable pattern corresponds to a stop pattern. That is, the stop pattern data corresponds to the reverse of the passable pattern data. FIG. 15 shows the configuration of the stop pattern data corresponding to FIG. 13 and indicates the lane numbers that should stop for each lighting pattern. FIG. 16 shows another expression form of the stop pattern data and indicates the combination of lighting colors for which each lane should stop. Since the stop pattern is a lighting pattern that prohibits entry into the intersection, it can be rephrased as an entry prohibition pattern.

[0121] In the present disclosure, when the passable pattern data and the stop pattern data are not distinguished, it is also described as traffic signal response policy data. The traffic signal response policy data can also be called lane-by-lane response policy data. The traffic signal response policy data corresponds to vehicle data that supports the execution of vehicle control, that is, data for vehicle control. The explanations regarding the passable pattern data are also applicable to the stop pattern data as appropriate.

[0122] In the traffic signal response policy data, data regarding a single-color lighting pattern in which only one of green, yellow, and red is lit may be omitted. For example, the passable pattern data shown in FIG. 13 can be omitted to a data set that includes only data regarding a pattern in which red and green are lit simultaneously as shown in FIG. 17. The single-color lighting pattern corresponds to a state where only one lighting unit is lit. On the other hand, in the present disclosure, a pattern in which a red round light or a yellow round light and at least one green arrow are lit is referred to as a mixed-color lighting pattern.

[0123] The traffic signal response policy data may be configured to include only the mixed-color lighting pattern, in other words, only the lighting pattern for the green arrow signal. For the single-color lighting pattern, the driving support ECU 20 may simply follow the lit color, because there is little need to distribute it as map data. On the other hand, in the mixed-color lighting pattern, for example, when the vehicle is far from the traffic signal and the direction of the green arrow signal is unknown, it cannot be determined whether the host vehicle should stop. Considering such circumstances, a data set indicating passability for each lane in the mixed-color lighting pattern can be relatively useful information for vehicle control planning / execution. According to a configuration in which the traffic signal response policy generation unit G21 generates a data set including only data indicating passability for each lane with respect to the mixed-color lighting pattern as the traffic signal response policy data, the distribution data size can be suppressed.

[0124] Also, the traffic signal response policy generation unit G21 may be configured to generate traffic signal response policy data only for the traffic signal 9A with an arrow signal. The distribution data size can also be suppressed by a configuration in which traffic signal response policy data is not generated for a standard traffic signal 9 that does not have an arrow signal. According to the above system configuration, since the driving support ECU 20 can acquire traffic signal response policy data for the traffic signal 9A with an arrow signal, it becomes easier to determine the passability of an intersection where the traffic signal 9A with an arrow signal is provided.

[0125] Incidentally, there may be areas / intersections where even when only the red round lights are on, the rightmost lane can turn right, or the leftmost lane can turn left. Also, there are intersections where specific signs such as "NO TURN ON RED" limit the application of rules different from the basic rules of the area (hereinafter referred to as exception rules). For intersections where exception rules are applied by traffic lights or signs, it is preferable for the traffic signal response policy generation unit G21 to generate, as traffic signal response policy data, a data set indicating passable / unpassable lane numbers for each lighting pattern. According to a configuration in which traffic signal response policy data is generated / distributed only for traffic lights to which exception rules are applied, the size of the map data to be distributed can be suppressed.

[0126] <Supplement to Traffic Signal Response Policy Data> The number and lighting patterns of the green arrow lights provided in traffic signal 9 are diverse. For example, as shown in FIG. 18, there may be a traffic signal 9 to which a left-turn green arrow light AG1, a straight-ahead green arrow light AG2, and a right-turn green arrow light AG3 are assigned as green arrow lights AG. If such a traffic signal 9 can take the first pattern shown in FIG. 18(A) and the second pattern shown in (B) as mixed-color lighting patterns, the traffic signal response policy generation unit G21 can generate passable pattern data as shown in FIG. 19 based on reports from vehicles in each lane. The first pattern is a pattern in which the red round light CR, the left-turn green arrow light AG1, and the straight-ahead green arrow light AG2 are lit simultaneously. The second pattern is a pattern in which the red round light CR and the right-turn green arrow light AG3 are lit.

[0127] Since the number of lit green arrow lights is different between the first pattern and the second pattern, even without more detailed information such as which part of the traffic signal 9 is lit, it is possible to distinguish the lanes that can be traveled. Therefore, for a traffic signal 9 that can adopt such a lighting pattern, even with a simple data set that does not include the position information of the lighting part in the housing, the vehicle can appropriately determine whether it can pass according to the lighting state. Note that (A) and (B) in FIG. 19 have the same content, only different in the expression form. Both (A) and (B) in FIG. 19 show the lane numbers that can be traveled by the combination of the number for each lighting color. The traffic signal response policy data shown in FIG. 19 can be expressed in the forms such as FIG. 23, FIG. 15, FIG. 16, and FIG. 17.

[0128] By the way, as another lighting pattern of the traffic signal 9 equipped with a plurality of green arrow lights, as shown in FIG. 20, there may be a pattern in which, together with the red round light, a plurality of green arrow lights AG are lit one by one. That is, there may be a pattern in which the red round light CR and the left-turn green arrow light AG1 are lit, a pattern in which the red round light CR and the straight-ahead green arrow light AG2 are lit, and a pattern in which the red round light CR and the right-turn green arrow light AG3 are lit. In such a case, it is not possible to distinguish whether each lane can pass only based on the information that red and green are lit. This is because although the combination of the number for each lighting color is the same, the lane numbers that can be traveled are different depending on the lighting position of the green light. When it is not possible to distinguish whether each lane can pass only by the combination of the number for each lighting color, as shown in FIG. 21, a predetermined special value ( "X" in the figure) may be inserted into the data field indicating the lane numbers that can be traveled corresponding to the lighting pattern that cannot be distinguished. The special value is a value (code) indicating that the lanes that can be traveled are unknown. The special value suggests that the own vehicle lane may be passable. When the lighting pattern recognized by the front camera 11 corresponds to the lighting pattern corresponding to the special value, the driving support ECU 20 may not perform automatic deceleration control or the like, but may request the driver to confirm the direction of the arrow and then perform a driving operation.

[0129] <Example of vehicle control using passable pattern data> Here, an example of vehicle control using passable pattern data, in other words, an operation example of the driving support ECU 20, will be described with reference to the flowchart shown in FIG. 22. In the present disclosure, the process corresponding to the flowchart shown in FIG. 22 is also referred to as a traffic signal passing support process. The traffic signal passing support process includes steps S301 to S314 as an example. The traffic signal passing support process is executed at a predetermined cycle such as 200 milliseconds while the driving power source is on. The traffic signal passing support process is executed on the condition that the driving support function by the driving support ECU 20 is enabled by the driver. In the present embodiment, as an example, the driving support by the driving support ECU 20 includes control for automatically adjusting the traveling speed according to the inter-vehicle distance from the preceding vehicle, but is not limited thereto. The driving support may be limited to a proposal of a driving operation according to the driving environment without performing running control.

[0130] The traffic signal passing support process shown in FIG. 22 can be implemented in parallel with or in combination with the various processes described above, such as the traffic signal response report process and the process related to map download. Here, the case where passable pattern data is distributed to the vehicle will be described, but the case where stop pattern data is distributed can be implemented in the same manner.

[0131] First, in step S301, the environment recognition unit F6 acquires information indicating the driving environment based on signals from various devices, as in step S101. In step S302, the localization unit F5 identifies the own vehicle position coordinates and the own vehicle lane ID based on the input signal from the front camera 11. Step S302 may be integrated with step S301. In step S303, the environment recognition unit F6 determines, as in step S103, whether the traffic signal 9 for the own vehicle is detected by the front camera 11. If the traffic signal 9 for the own vehicle is not detected, step S303 is determined to be negative and this flow ends. On the other hand, if the traffic signal 9 for the own vehicle is detected, step S304 is executed.

[0132] In step S304, the environment recognition unit F6 acquires the remaining distance (Drm) to the intersection corresponding to the traffic signal 9 detected in step S303. The remaining distance to the intersection may be acquired as an image recognition result from the front camera 11, or may be specified by comparing the position information of the intersection shown in the map data with the own vehicle position information. The remaining distance to the intersection can be, for example, the remaining distance to the stop line provided in front of the intersection.

[0133] In step S305, the camera output acquisition unit F3 acquires the combination of the colors of the lit parts as the recognition result of the lit state of the traffic signal 9 for the own vehicle. For example, when there is only one lit part, its color is acquired. When there are multiple lit parts, the combination of their colors and the number for each color are acquired. If, as shown in (A) of FIG. 18, a red round light, a green arrow light for left turn, and a green arrow light for straight-ahead are lit, the environment recognition unit F6 acquires that the combination of the lit colors is one red light and two green lights. In the present embodiment, the shape recognition of the lit part is an arbitrary element. Although it is preferable to be able to specify even the shape of the lit part, such as the direction of the arrow, when the shape of the lit part is unknown, subsequent processing can be executed assuming that the shape is unknown.

[0134] In step S306, the environment recognition unit F6 determines whether or not the remaining distance (Drm) to the intersection is less than a predetermined control continuation determination distance (Dcn). The control continuation determination distance is, for example, 50 m or 75 m. The control continuation determination distance may be changed according to the scale of the road, the speed limit, the current vehicle speed, etc. The control continuation determination distance can be set longer as the vehicle speed is higher. For example, the control continuation determination distance is set to a value that allows the vehicle to stop at a predetermined deceleration until it reaches the intersection. More specifically, assuming the current speed is Vo and the deceleration is a, Drm can be set to the value obtained by adding a predetermined margin ε to Vo^2 / (2a). The margin ε can be set to, for example, 10 m, 15 m, 20 m, etc. The margin is set so as to ensure the time necessary for the driver to take over the driving operation related to acceleration and deceleration.

[0135] When the remaining distance to the intersection is less than the control continuation determination distance, that is, when the relationship Drm < Dcn holds, the processing after step S307 is executed. On the other hand, when the remaining distance to the intersection is greater than or equal to the control continuation determination distance, that is, when the relationship Drm ≥ Dcn holds, this flow ends. In this case, this flow is re-executed from step S301 after a predetermined time.

[0136] In step S307, it is determined whether the lighting state of the traffic signal 9 corresponds to the single-color lighting pattern. Step S307 can generally be understood as a process of determining whether only one lighting part of the recognized traffic signal 9 exists. Note that a pattern in which a plurality of green arrow lights are lit without the red or yellow round light being lit, that is, a pattern in which only a plurality of green arrow lights are lit simultaneously, can also be included in the single-color lighting pattern.

[0137] When the lighting state of the traffic signal 9 is not multicolor, the process proceeds to step S308. The control planning unit F7 plans control according to the lighting color, and the control execution unit F8 executes control according to the plan. For example, when the lighting color is red, the driving support ECU 20 starts deceleration control for stopping. When the lighting color is green, the following control of the preceding vehicle is continued. Note that when the following control of the preceding vehicle is set off by the driver's operation, only information presentation such as the display of the passable image Im2 can be performed. However, when a right or left turn is planned, deceleration control for stopping at the stop line is started.

[0138] When the lighting color is yellow, in principle, deceleration control for stopping before the intersection is executed. However, when it is recognized that the lighting color is yellow and the vehicle already exists within the intersection at that time, driving control for passing through the intersection is performed. Also, when it is recognized that the lighting color is yellow and the remaining distance is less than the braking distance at that time, or when it is determined that it is impossible to stop before the intersection with a reasonable deceleration, driving control for passing through the intersection is also performed. Note that since this flow is repeatedly executed at a predetermined interval, the recognition result of the traffic signal lighting state and the control plan according to the recognition result can also be dynamically updated at any time.

[0139] In conjunction with the above determination result, the notification processing unit Fa displays a determination result image Imk or the like on the display 151. When the system is operating normally, performing a notification using sound may cause annoyance to the driver. Therefore, it is preferable not to perform a notification using sound, such as a notification sound, unless it corresponds to a specific error state. Of course, the output conditions of the notification sound and voice message may be configured to be changeable by the driver via a predetermined setting screen.

[0140] When the recognized lighting state of the traffic signal 9 corresponds to the mixed-color lighting pattern, the environment recognition unit F6 compares the combination of recognized lighting colors recognized as step S309 with the passable pattern of the host vehicle lane. As a result of the comparison, when the combination of recognized lighting colors matches the passable pattern of the host vehicle lane, the environment recognition unit F6 determines that the intersection can be passed as it is, and outputs a passable signal, which is a signal indicating that fact, to the control plan unit F7. The passable signal may be a message signal indicating that the intersection can be passed (entered). Based on the fact that the passable signal is input from the environment recognition unit F6, the control plan unit F7 creates a plan for passing through the intersection. Then, the control execution unit F8 continues the control support according to the planned route based on the plan created by the control plan unit F7 (step S313).

[0141] For example, when the environment recognition unit F6 determines that the intersection can be passed and straight-ahead travel at the intersection is planned, the control execution unit F8 continues the preceding vehicle following control. When the notification processing unit Fa determines that the intersection can be passed as it is and straight-ahead travel at the intersection is planned, in conjunction with the vehicle control as step S313, it displays a passable image Im2 on the display 151. At that time, no special voice message or notification sound is output.

[0142] On the other hand, even when it is determined by the environment recognition unit F6 that the traffic signal lighting state is a pattern in which the host vehicle can pass, if a right or left turn at an intersection is planned, the control planning unit F7 temporarily suspends the preceding vehicle following control and starts deceleration control to stop at the stop line. In accordance with this, the notification processing unit Fa performs voice notification prompting confirmation of the traffic conditions at the right turn destination / left turn destination. That is, the driving support ECU 20 performs driving support related to right and left turns. Note that by suspending the preceding vehicle following control, it is possible to reduce the risk of being pulled by the preceding vehicle and automatically accelerating / starting / entering the intersection.

[0143] Also, when the recognized combination of lighting colors is not defined as a passable pattern for the host vehicle lane, the environment recognition unit F6 determines that entry into the intersection is not possible and outputs a predetermined non-passable signal to the control planning unit F7. The non-passable signal may be a message indicating that entry into the intersection is prohibited. Based on the input of the non-passable signal from the environment recognition unit F6, the control planning unit F7 creates a plan for deceleration control towards stopping. Then, the control execution unit F8 starts deceleration control towards stopping (step S312).

[0144] When the non-passable signal is output, the control planning unit F7 can temporarily suspend the preceding vehicle following control at a predetermined timing. The preceding vehicle following control may be suspended at the timing when automatic deceleration towards stopping is started, or a time difference may be provided. The preceding vehicle following control may be continued until the host vehicle completely stops, the distance from the traffic signal becomes equal to or less than a predetermined value, or the stop line is reached. The notification processing unit Fa, in conjunction with the vehicle control as step S312, displays the entry prohibited image Im1 on the display 151. Also in this case, since the system itself is operating normally, no special voice message or notification sound is output. Note that instead of starting automatic deceleration control, the control execution unit F8 may execute notification processing prompting the driver to perform a deceleration operation.

[0145] On the other hand, when the recognized combination of lighting colors does not correspond to any of the lane passable patterns defined in the passable pattern data (step S310 NO), the environment recognition unit F6 determines that it is impossible to determine the passability of the intersection ahead. In this case, the environment recognition unit F6 outputs a determination impossible signal to the control plan unit F7. The determination impossible signal may be a message indicating that it is impossible to determine whether the host vehicle can enter the current intersection.

[0146] Based on the fact that a determination impossible signal is input from the environment recognition unit F6, the control plan unit F7 interrupts the driving support related to passing through the intersection ahead (step S311). For example, it ends the control for automatically adjusting the driving speed, such as following the preceding vehicle control or decelerating towards a stop. In this case, the notification processing unit Fa outputs an audio message indicating the end of the support related to speed control from the speaker 152 and displays a text message with the same content on the display 151. A message indicating the end of the support related to speed control is, for example, "Since the lighting state of the traffic signal could not be recognized normally, the control is interrupted." Note that a warning sound may be output instead of / parallel to the audio message. This notification corresponds to the control cancellation notification process.

[0147] The above describes the case when the vehicle is in motion, but the present disclosure can also be applied when the vehicle is stopped in front of the intersection. When the traffic signal 9 changes from a red light to a green light, a passable mark Imk2 etc. may be displayed together with a notification sound. Also, when the recognized lighting state corresponds to a single-color lighting pattern, the environment recognition unit F6 outputs a passable signal or a non-passable signal according to the lighting color.

[0148] The above described the operation of the driving support ECU20 on the premise that the host vehicle lane can be specified. However, as an error that can occur in the driving support ECU20, a failure to specify the host vehicle lane ID etc. can also be considered. The environment recognition unit F6 can also output a determination impossible signal when the state of the unknown host vehicle lane ID continues for a predetermined time. The determination impossible signal may include information indicating the cause.

[0149] When an indeterminable signal resulting from an unknown vehicle lane ID is output, the notification processing unit Fa performs an operation request process, and then the control planning unit F7 ends the automatic control related to speed adjustment and lane keeping. The operation request process is a process of outputting a message requesting a driving operation according to the lighting state of the traffic light 9 by voice and image. When the control is ended due to a failure to identify the vehicle lane ID, the notification processing unit Fa outputs a voice message such as "The driving lane is unclear, so control will be interrupted" from the speaker 152 as the operation request process. A similar text message may be displayed on the display 151. The notification processing unit Fa can transmit necessary information to the driver while reducing the risk of causing annoyance to the driver by notifying the driver using sound only when an error occurs in the system.

[0150] Also, possible errors that may occur in the driving assistance ECU 20 include failure to acquire map data. The environment recognition unit F6 may also output a determination impossible signal when map data ahead of the vehicle cannot be acquired. Even when a determination impossible signal resulting from non-acquisition of map data is input, the control planning unit F7 transfers authority to the driver and terminates automatic control related to speed adjustment and lane keeping.

[0151] Furthermore, as described with reference to Fig. 21, in a pattern including the lighting of the green arrow light, even if it corresponds to a type in which it is impossible to distinguish whether each lane is passable or not only by the combination of lighting colors, the environment recognition unit F6 may determine that it is impossible to determine whether or not the lane is passable and output an impossible-to-determine signal. In this case as well, the notification processing unit Fa executes the operation request processing, and the control planning unit F7 stops the automatic control of the speed adjustment (in other words, acceleration / deceleration). Note that stopping the automatic control of acceleration / deceleration corresponds to stopping the automatic adjustment of the speed, in other words, stopping the control to follow the preceding vehicle.

[0152] Incidentally, in the present embodiment, operation request processing is executed based on the output of a determination impossible signal in a state where the remaining distance Drm to the intersection is less than the control continuation determination distance Dcn. The control continuation determination distance Dcn is set to be longer than the non-urgent braking distance Dstp. The non-urgent braking distance Dstp in the present disclosure is the distance required for stopping when decelerating at a basic deceleration α which is a predetermined acceleration within a range that does not give discomfort to the driver. The basic deceleration α can be set to 1.0 m / s^2, 1.25 m / s^2, 1.5 m / s^2, etc. As described above, when the applied deceleration is α, Dstp is determined by Dstp = Vo^2 / (2α). The deceleration start point, which is the point corresponding to the timing at which deceleration start is required, is a point in front of the intersection by a distance equal to or greater than the non-urgent braking distance Dstp. The above configuration corresponds to a configuration that makes a request for control end and takeover of driving operation due to a recognition error of the traffic signal 9 a predetermined time before the timing at which deceleration start is required. According to this configuration, the driver can recognize, judge, and operate the lighting state of the traffic signal with a time margin.

[0153] <Effect of the above configuration> Since each lighting unit provided in the traffic signal 9 is smaller than an object such as a preceding vehicle, as shown in FIG. 23, the camera ECU 112 cannot determine the direction of the arrow of the lit green arrow signal until the vehicle approaches the traffic signal 9 to a certain extent. Particularly in a bad environment such as during rainfall, the shape is blurred by raindrops or the like, making it difficult to recognize the direction of the arrow. Assuming that the distance at which the direction / shape of the green arrow signal can be recognized is the shape recognizable distance Da, depending on the environment, the shape recognizable distance Da can be larger than the non-urgent braking distance Dstp. That is, the direction of the green arrow signal may not be specified until after passing the point where braking should start.

[0154] As a first comparison configuration for performing intersection passage support using the recognition result of the traffic signal, a configuration in which braking starts after recognizing the direction of the green arrow signal can be considered. However, in the first comparison configuration, the start of deceleration may be delayed, so a relatively large deceleration may be applied to stop in front of the intersection. In the first comparison configuration, due to the above action, discomfort may be given to the driver.

[0155] Also, as another comparative configuration, i.e., the second comparative configuration, a configuration can be considered in which the lighting of the green arrow light is ignored and braking toward a stop is started when a red light is recognized. However, in the second comparative configuration, deceleration can be performed even when deceleration is not required by the green arrow light. Even when the host vehicle plans to go straight at an intersection and the traffic signal is lit with a red light and a straight-ahead green arrow light, deceleration control toward a stop before the intersection can be performed in the second comparative configuration.

[0156] On the other hand, as shown in FIG. 23, even if it is difficult to specify the shape of the lighting unit, it is relatively recognizable from a distance that the green lighting device is lit. Assuming that the distance at which it is possible to recognize that the green arrow light is lit is the lighting recognizable distance Db, the lighting recognizable distance Db is larger than the shape recognizable distance Da. Pb in FIG. 23 indicates a point at which the green light provided by the green arrow light can be recognized by image recognition. Pa in FIG. 23 indicates a point at which the direction of the green arrow light can be recognized by image recognition.

[0157] As described above, the present disclosure was created by focusing on the fact that even if it is impossible to identify whether it is a green arrow light and the direction of the arrow, it is relatively recognizable from a distance that the green lighting unit is lit. The server constituting the above map cooperation system Sys distributes, to the vehicle, as signal response policy data, a data set indicating a combination of lit colors that are passable / should stop for each lane. According to this configuration, for a traffic signal 9 in which it is possible to determine passability only from the combination of lit colors, the driving support ECU 20 can determine passability even if it is impossible to identify the shape of the lighting unit. That is, regarding an intersection / traffic signal 9 in which it is uniquely determined from the combination of lit colors whether to stop for each lane, the driving support ECU 20 can determine whether to stop (i.e., earlier) before being able to recognize the direction of the arrow. Therefore, it is possible to decelerate gently and reduce the risk of unnecessary deceleration. The present disclosure is suitable for a traffic signal 9 / intersection in which passability for each lane is uniquely determined according to the number of lit green arrow lights as shown in FIGS. 12 and 18.

[0158] In addition, the map generation server 3 generates the traffic signal response policy data from the combinations of the lit colors observed by each of the plurality of vehicles. In the observation data reported by the vehicle, the shape of the lit portion is an arbitrary element and is not essential. Also, in terms of the recognition degree of the lit color, it can be expected that it can be observed even with commercially available vehicles. Therefore, according to the above configuration, it is possible to generate the traffic signal response policy data based on reports from commercially available general-purpose vehicles without using a (i.e., special) probe vehicle equipped with a high-performance sensor. That is, according to the configuration of the present disclosure, it may be possible to generate and update the data for control support related to traffic signals at a lower cost than the lighting pattern information disclosed in Patent Document 1.

[0159] In general, in a bad environment for a camera such as rainy weather, the shape of the lit portion of the traffic signal 9 may be difficult to identify. According to the configuration of the present disclosure, since it is possible to determine whether or not to proceed even in such a bad environment, it is possible to reduce the risk of unnecessary deceleration or support interruption due to misrecognition / failure to recognize the lit state. In other words, it is possible to improve the continuous ability of driving support. Also, in the server, since database-izing the shape of the lit portion is an arbitrary element, the processing load can be reduced. Further, according to the present disclosure, an effect of suppressing the size of the distribution data can also be expected.

[0160] Furthermore, the above-described driving support ECU 20 notifies the driver of the recognition / judgment result of the system in an image even during normal operation of the system. According to this configuration, since the driver can understand the operating state (recognition state) of the system, a sense of security can be enhanced. Also, even when it is not possible to determine whether or not to proceed according to the lit pattern of the traffic signal 9, at least the fact that the traffic signal 9 exists is notified to the driver, and the driving operation is entrusted to the driver. Since the timing of this notification is performed in a state where the remaining distance Drm to the intersection is still longer than the emergency braking distance Dstp, the driver can perform the determination of the lit state and the driving operation with a time margin.

[0161] In addition, the passability of each lane according to the lighting state may also vary depending on the signs added to the side of the traffic signal 9. For example, in the United States, when the red light is on, in principle, the right-turn lane on the far right is passable (right-turn possible), but there are also intersections where right-turns are prohibited when the red light is on due to specific signs. The configuration disclosed in Patent Document 1 cannot handle such exception patterns. That is, in the configuration disclosed in Patent Document 1, even if the lighting state of the traffic signal 9 can be recognized, it is not possible to determine whether it is actually possible to pass through the intersection. On the other hand, the traffic signal response policy data generated in the configuration of the present disclosure is obtained by statistically processing the actual behavior of vehicles according to the lighting state of the traffic signal 9, and thus reflects the exception rules based on the auxiliary signs. Therefore, even for intersections where exception rules are applied by auxiliary signs or the like, it is possible to accurately determine the passability according to the lighting state.

[0162] As described above, the embodiments of the present disclosure have been explained. However, the present disclosure is not limited to the above-described embodiments, and various supplementary matters and modification examples described hereinafter are also included in the technical scope of the present disclosure. Furthermore, various changes can be made and implemented without departing from the gist thereof, other than the following. For example, the following various supplementary matters and modification examples can be appropriately combined and implemented within a range where no technical contradiction occurs. Note that members having the same function as the members described above may be given the same reference numerals, and the description thereof may be omitted. In addition, when only a part of the configuration is mentioned, the above description can be applied to other parts.

[0163] <Supplementary Note (1)> The control using the passable pattern data / stop pattern data by the driving support ECU 20 may be applied only while the direction of the green arrow signal cannot be specified. When the direction / shape of the green arrow signal is determined, the passability determination based on the passable pattern data / stop pattern data may be discarded, and a control plan may be created and executed according to the actual direction of the green arrow signal. That is, the control using the passable pattern data / stop pattern data may be adopted as a provisional control policy until the direction of the green arrow signal can be specified.

[0164] <Supplementary Explanation (2)> In the above description, a configuration has been disclosed in which the driving support ECU 20 transmits a data set including the own vehicle lane ID as information indicating the running position in the road width direction as a traffic signal response report. However, the configuration of the traffic signal response report is not limited to this. The data uploaded as the traffic signal response report does not necessarily have to include the lane number. The traffic signal response report only needs to include information indicating the running position of the reporting vehicle in the road width direction. Information indicating the lane in which the own vehicle (reporting source) is running corresponds to information indicating the running position in the road width direction.

[0165] As information indicating the running position in the road width direction, for example, information indicating the relative position with surrounding features can be adopted. More specifically, relative position information with predetermined features such as a direction signboard, a regulatory arrow applied to the road surface as a road surface marking, paint as a guiding strip, and the road edge can be used. The traffic signal response report may include information on surrounding features that can identify the running lane instead of / parallel to the own vehicle lane ID. Accordingly, the driving support ECU 20 does not necessarily have to be able to identify the own vehicle lane ID when transmitting the traffic signal response report. Step S102 is an optional element.

[0166] When the traffic signal response report does not include the own vehicle lane ID / When the own vehicle lane ID is unknown, the map generation server 3 may identify the lane number in which the reporting source was running from the relative position information of the surrounding features included in the traffic signal response report. That is, the map generation server 3 may be equipped with a function to identify the running lane number. As a preparation process for step S201, the map generation server 3 may perform a process of identifying the running lane number of the reporting source based on the relative position information of the surrounding features that may be included in the traffic signal response direction. According to the configuration in which the map server 3 identifies the running lane of the reporting source based on the relative position information of the surrounding features, it becomes possible to handle situations where it is difficult to identify the own vehicle lane ID in the driving support ECU 20. Note that a situation where it is difficult to identify the own vehicle lane ID in the driving support ECU 20 is, for example, a situation where the view of the front camera 11 is blocked by surrounding vehicles and the recognition results for the road edge and the outer dividing line of the adjacent lane are insufficient.

[0167] <Supplementary Note (3)> In the above-described embodiment, the aspect in which the own-lane ID is specified by analyzing the image generated by the front camera 11 has been described. However, the means for specifying the own-lane ID is not limited to this. The own-lane ID may be specified by analyzing the image of a rear camera, which is an in-vehicle camera mounted to photograph the rear of the vehicle, or the image of a side camera, which is an in-vehicle camera mounted to photograph the side of the vehicle. Further, the own-lane ID may be specified based on the detection results of a LiDAR, a millimeter-wave radar, or the like.

[0168] Furthermore, the own-lane ID may be specified based on the GNSS positioning result. When the condition that the GNSS positioning error is expected to be less than 10 cm is satisfied, the processor 21 may specify the own-lane ID based on the GNSS positioning result output from the locator 13. The case where the condition that the GNSS positioning error is expected to be less than 10 cm is satisfied is, for example, the case where the in-vehicle GNSS receiver can receive signals from a quasi-zenith satellite. Also, when the running lane ID is received from a vehicle set as the preceding vehicle by vehicle-to-vehicle communication, the running lane ID may be adopted as the own-lane ID.

[0169] In addition, the own-lane ID may be specified based on information from radio / optical beacons arranged to form communication areas for each lane. The radio / optical beacons correspond to roadside units arranged above the road. Also, the own-lane ID may be specified based on signals from magnetic markers embedded in the road surface. The magnetic marker is a communication device (radio tag) embedded in the road surface. The magnetic marker transmits absolute position coordinates or lane numbers spontaneously or based on an inquiry from a vehicle. For example, a battery-free type radio ID tag can be adopted as the magnetic marker. As described above, the information indicating the running position of the own vehicle in the road width direction can be specified based on information input from various in-vehicle devices such as peripheral monitoring sensors and communication devices.

[0170] <Supplementary Note (4)> In the above description, the driving support ECU 20 has been described as having a configuration in which, as a traffic signal passing support process, when the remaining distance to the intersection is less than a predetermined value, it collates the recognized lighting state with the passable pattern data. However, it is not limited to this. The driving support ECU 20 may periodically perform the collation of the recognized lighting state with the passable pattern data when a traffic signal for the host vehicle is recognized by the front camera 11 regardless of the remaining distance to the intersection. However, when the remaining distance or remaining time until reaching the intersection ahead is equal to or greater than a predetermined threshold value, even if the environment recognition unit F6 outputs a determination impossible signal, control suspension or the like is not performed. This is because as the vehicle approaches the intersection, the lighting state of the traffic signal 9 may switch to a pattern capable of determining passability. For example, before the remaining distance Drm to the intersection becomes less than the control continuation determination distance Dcn, it is also possible that the lighting state of the traffic signal transitions from a mixed-color lighting pattern where determination is impossible to a single-color lighting pattern. The control planning unit F7 preferably suspends the control related to speed adjustment and issues a notification of control suspension only when the environment recognition unit F6 outputs a determination impossible signal in a situation where the remaining distance or remaining time until reaching the intersection ahead is less than the threshold value.

[0171] <Supplementary Note (5)> In the above, a control example has been described in which when the yellow light of the traffic signal is lit, the response is substantially the same as when the red light is lit. However, it is not limited to this. In Japan, the lighting state of the traffic signal does not transition from yellow to green. However, as other regions, there may be regions that pass through yellow once before transitioning from red to green. That is, there may also be regions where the lighting color of the traffic signal transitions from yellow to green. In regions where the lighting color of the traffic signal may transition from yellow to green, deceleration at the time of recognizing the yellow light may be unnecessary deceleration. For such reasons, the driving support ECU 20 may perform the same system response as when the green light is lit when only the yellow light is lit. Specifically, when the driving support ECU 20 recognizes that only the yellow light is lit, deceleration toward a stop or the like may be postponed, and following control of the preceding vehicle or control to maintain traveling at the set target speed may be continued.

[0172] The response policy when only the yellow light is on may be dynamically changed according to the area where the vehicle is used. For example, the driving support ECU 20 may be configured to apply a traffic signal response policy according to the driving area based on the country code preset at a dealership or the like, or the position coordinates specified by GNSS.

[0173] <Modification example> The driving support ECU 20 may upload a data set including the position information of the lit part in the housing as a traffic signal response report. According to this configuration, the traffic signal response policy generation unit G21 can define the passable pattern for each lane including not only the combination of lit colors but also the position information of the lit part. As a result, it becomes possible to set the passable / stop pattern for each lane even for a traffic signal that could not distinguish the passability for each lane only by the number of combinations of lit colors.

[0174] The position of the lit part in the traffic signal may be expressed in XY coordinates with a predetermined position of the housing, such as the upper left or upper right corner of the housing, as the origin. Also, as shown in FIG. 24, the housing may be divided into a plurality of areas corresponding to the area where the light unit can be arranged, and the position of the lit part may be expressed by the number of each area. In FIG. 24, as an example, the case where the housing is divided into 6 areas in 2 rows and 3 columns to express the lit part is illustrated. Areas L11 to L13 are a group of areas corresponding to the relatively upper row (the first row). Areas L21 to 23 are a group of areas corresponding to the relatively lower side (the second row). The area numbers can be assigned in order, for example, from the upper left to the lower right. The rule for assigning area numbers may be designed as appropriate. Similarly, when the traffic signal 9 is of the vertical two-column type as shown in FIG. 25, the position of the lit part can be expressed by the row number and the column number.

[0175] In the above configuration, the driving support ECU 20 transmits a traffic signal response report including the position information of the lit portion in the housing in addition to the color of the lit portion, as shown in FIG. 26. Based on the traffic signal response report, the traffic signal response policy generation unit G21 generates a data set indicating the passable lanes for each lighting pattern, in other words, the passable pattern for each lane, according to the combination of the lit portion and its color, as shown in FIG. 27 for example. Note that a data set indicating the stop pattern for each lane can be generated in the same manner. According to the configuration for generating and distributing the data set, the driving support ECU 20 can determine whether it is possible to pass through the intersection relatively far away (early) even for a traffic signal 9 / intersection having a lighting pattern as shown in FIG. 20.

[0176] In the above, the mode of expressing the lit portion using the area number / position coordinates determined based on the corner of the housing or the like has been described, but the expression form of the lit portion is not limited to this. The green arrow lamp is often lit in parallel with the red lamp. Considering such circumstances, the position information of the lit green arrow lamp may be expressed based on the red light. For example, assuming the lighting pattern of FIG. 20, the passable pattern for each lane can be expressed as shown in FIG. 28. In the actual environment, there may be scenes where the housing is difficult or impossible to recognize, such as at night. In the configuration where the lit portion is defined based on the housing, when the housing is unclear at night or the like, the lit portion cannot be specified, and it may be impossible to determine whether passage is possible. On the other hand, according to the configuration of expressing the position of the green arrow lamp based on the red lamp, it is suitable for an environment where the housing itself is difficult to detect, such as at night. This is because it is highly likely that the red lamp can be recognized even in a scene where the housing cannot be recognized due to assimilation with the background.

[0177] In FIG. 28, for the sake of explanation, the lit portion based on the red lamp is shown in text, but it can be expressed by a predetermined code (number) indicating the relative position programmatically. FIG. 28 shows the passable pattern when a traffic signal 9 having the lighting pattern shown in FIG. 20 is provided for a road having the lane configuration shown in FIG. 11.

[0178] <Supplementary Note (1)> The present disclosure also includes the following technical ideas.

[0179] [Technical Idea (1)] A traffic signal response policy generation unit that generates passable pattern data indicating a combination of passable lighting colors for each lane for each traffic signal based on traffic signal response reports provided by a plurality of vehicles, A vehicle data generation server including a transmission processing unit that transmits the traffic signal response policy data generated by the traffic signal response policy generation unit to an external device.

[0180] [Technical Idea (2)] The vehicle data generation server according to the above Technical Idea (1), The traffic signal response policy generation unit generates passable pattern data for each traffic signal as part of map data indicating the connection relationship of the road using a plurality of nodes and links, The transmission processing unit is configured to transmit the passable pattern data for each traffic signal to an external device in association with the data of the node or link where the corresponding traffic signal is installed.

[0181] [Technical Idea (3)] The vehicle data generation server according to the above Technical Idea (1) or (2), The transmission processing unit is configured to transmit, in response to a request from a vehicle, the passable pattern data for each traffic signal existing in a range corresponding to the position of the vehicle to the vehicle.

[0182] [Technical Idea (4)] The vehicle data generation server according to any one of the above Technical Ideas (1) to (3), The transmission processing unit transmits, as data related to a traffic signal, data indicating whether the traffic signal is a traffic signal with an arrow lamp, which is a lighting device that displays an arrow, to an external device. A vehicle data generation server configured to transmit a data set with passable pattern data only for a traffic signal with a torch.

[0183] [Technical idea (5)] An acquisition unit that acquires information indicating the position of the host vehicle lane in the road width direction based on an input from an in-vehicle device, A lighting state acquisition unit that acquires data indicating the lighting state of a traffic signal corresponding to the host vehicle lane based on an input from the same or a different device as the above in-vehicle device, Based on the fact that the host vehicle has stopped before an intersection or has passed through an intersection, a data set indicating the information indicating the host vehicle lane, the combination of the lighting colors of the traffic signal for the vehicle acquired by the lighting state acquisition unit, and the behavior of the host vehicle is transmitted as a traffic signal response report to a predetermined server. A vehicle control device comprising a reporting processing unit.

[0184] [Supplementary remarks (2)] The devices, systems, and methods described in this disclosure may be implemented by a dedicated computer configured to program a processor to execute one or more functions embodied by a computer program. Also, the devices and methods described in this disclosure may be implemented using dedicated hardware logic circuits. Furthermore, the devices and methods described in this disclosure may be implemented by one or more dedicated computers configured by a combination of a processor that executes a computer program and one or more hardware logic circuits. For example, some or all of the functions provided by the driving assistance ECU 20 / map generation server 3 may be implemented as hardware. Modes of implementing a certain function as hardware include modes implemented using one or more ICs or the like. As the processor (computing core), a CPU, MPU, GPU, DFP (Data Flow Processor), or the like can be adopted. Also, some or all of the functions provided by the driving assistance ECU 20 / map generation server 3 may be implemented by combining multiple types of arithmetic processing units. Some or all of the functions provided by the driving assistance ECU 20 / map generation server 3 may be implemented using a system-on-chip (SoC), FPGA, ASIC, or the like. FPGA is an abbreviation for Field-Programmable Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit.

[0185] Also, the computer program may be stored as instructions executable by a computer in a computer-readable non-transitory tangible storage medium. As the storage medium for the program, an HDD (Hard-disk Drive), SSD (Solid State Drive), flash memory, or the like can be adopted. Forms such as a program for causing a computer to function as the driving assistance ECU 20 / map generation server 3 and a non-transitory physical recording medium such as a semiconductor memory recording this program are also included in the scope of this disclosure.

Claims

1. A vehicle data generation server that generates data for controlling a vehicle with respect to a traffic signal, a report acquisition unit (G1) that acquires, as a traffic signal response report, a data set indicating, from a plurality of vehicles, information indicating a lane in which the vehicle is traveling, a combination of lighting colors of the traffic signal observed by the vehicle, and the behavior of the vehicle with respect to the combination of lighting colors; a traffic signal response policy generation unit (G21) that generates, as traffic signal response policy data, passable pattern data indicating a combination of lighting colors that can be passed for each lane for each traffic signal, based on the traffic signal response report acquired by the report acquisition unit; and a transmission processing unit (G3) that transmits the traffic signal response policy data generated by the traffic signal response policy generation unit to an external device. The vehicle data generation server, wherein the passable pattern data is a data set indicating passable lanes for each combination of lighting colors and does not include data indicating the shape of each lighting unit.

2. The vehicle data generation server according to claim 1, wherein the traffic signal response policy generation unit generates the passable pattern data as the traffic signal response policy data, and the passable pattern data is a data set indicating passable lanes by a combination of numbers for each lighting color.

3. The vehicle data generation server according to claim 1 or 2, wherein the passable pattern data indicates passable lanes when a red light, which is a lighting unit that lights red, and a green arrow light, which is a lighting unit that displays a green arrow, are lit, and the passable pattern data is a data set indicating passable lanes based on the relative position of the green lighting unit with respect to the red lighting unit.

4. The vehicle data generation server according to claim 1 or 2, wherein the traffic signal response report includes information regarding the position and lighting color of the lighting unit in the traffic signal, and the passable pattern data is a data set indicating passable lanes by a combination of the position and color of each lighting unit in the traffic signal.

5. The vehicle data generation server according to claim 1 or 2, The traffic signal response policy generation unit generates the passable pattern data for a traffic signal with a beacon that is a lighting device for displaying an arrow, while being configured not to generate the passable pattern data for a standard traffic signal that does not include the beacon. A vehicle data generation server.

6. The vehicle data generation server according to claim 1 or 2, The traffic signal response policy generation unit is configured to generate stop pattern data indicating a combination of lighting colors to be stopped for each lane instead of the passable pattern data. A vehicle data generation server.

7. A behavior acquisition unit that acquires information indicating the behavior of the host vehicle or other vehicles, A lighting state acquisition unit that acquires information indicating the lighting state of a traffic signal from a camera, A report generation unit that generates a traffic signal response report, which is a data set indicating the lighting state of the traffic signal acquired by the lighting state acquisition unit and the behavior of the host vehicle or other vehicles with respect to the lighting state acquired by the behavior acquisition unit, and The report generation unit is configured to generate a data set that does not include data indicating the shape of the lighting unit in the traffic signal response report. A data generation device.

8. The report generation unit is configured to be able to generate a data set including information indicating that the shape of the lighting unit was unknown as the traffic signal response report. The data generation device according to claim 7.

9. The data indicating the shape of the lighting unit is data indicating whether the lighting unit is round or arrow-shaped. The data generation device according to claim 7.

10. The data indicating the shape of the lighting unit is data indicating the direction of the arrow in the beacon. The data generation device according to claim 7.

11. The report generation unit, Determines whether the traffic signal response report is a data set that satisfies a predetermined condition for ensuring the usefulness of the data, The data generation device according to claim 7, wherein the traffic signal response report that does not satisfy the predetermined condition is discarded.

12. The predetermined condition includes being generated when the remaining distance to the traffic signal is less than a predetermined value. The data generation device according to claim 11.

13. The traffic signal response report includes traffic signal information indicating the remaining distance to the traffic signal. The data generation device according to any one of claims 7 to 12.

14. A computer, acquires information indicating the behavior of a vehicle, acquires data indicating the lighting state of a traffic signal output from a camera, generates a data set that is a data set indicating the lighting state of the traffic signal and the behavior of the vehicle with respect to the lighting state and does not include data indicating the shape of the lighting unit in the traffic signal, and a program including instructions for causing the computer to execute the above operations.

15. A system having: a lighting state acquisition unit that acquires information indicating the lighting state of a traffic signal using a camera; a data generation unit that generates a data set that is a data set indicating the lighting state of the traffic signal and the behavior of the vehicle with respect to the lighting state and does not include data indicating the shape of the lighting unit; a traffic signal response policy generation unit that generates, as traffic signal response policy data, a data set indicating a combination of passable lighting colors for each lane for each traffic signal and not including data indicating the shape of each lighting unit, based on the data set generated by the data generation unit; a passability determination unit that determines whether the lighting state of the traffic signal corresponds to a lighting state in which the vehicle can pass, based on the traffic signal response policy data generated by the traffic signal response policy generation unit, lane information, and the lighting state acquired by the lighting state acquisition unit; and a response unit that performs vehicle control according to the determination result of the passability determination unit.

Images