Data generation device, data storage device

Abstract

A server 3 is provided with: a data generation unit 9e which generates signal specification data having a data structure that includes lane information specifying a lane in which a vehicle travels, signal information specifying each of a plurality of signals provided at an intersection to which the lane is connected, and reliability degree information indicating reliability degrees respectively set for the plurality of signals, different reliability degrees being respectively set for the plurality of signals according to the lane in which the vehicle travels, and that is capable of specifying a reliable signal on the basis of the reliability degrees; and a data storage unit 9f which stores the signal specification data.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Application No. 2022-033550, filed on March 4, 2022, the contents of which are incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to a data generating device and a data storage device. [Background technology]

[0003] For example, in the field of autonomous driving, when there are multiple traffic lights at an intersection, there may be cases where the vehicle cannot determine which traffic light to follow based on camera images alone. For this reason, as disclosed in Patent Documents 1 and 2, for example, it is considered to generate map data that links the lane in which the vehicle is traveling with the traffic lights at the intersections to which the lane is connected. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-005629 [Patent Document 2] Japanese Patent Application Publication No. 2019-191318 Summary of the Invention

[0005] However, although the methods of Patent Documents 1 and 2 link lanes and traffic lights, the accuracy of the linking is not guaranteed. As a result, there is a possibility that the wrong traffic light may be linked to the lane in which the vehicle is traveling, and there is room for further improvement in order to realize safer automated driving of vehicles.

[0006] The present disclosure has been made in consideration of the above-mentioned circumstances, and its purpose is to provide a data generation device that is capable of generating traffic light identification data that is capable of identifying a reliable traffic light among multiple traffic lights, even when multiple traffic lights are installed at an intersection where lanes on which vehicles are traveling are connected, and a data storage device that is capable of storing the traffic light identification data.

[0007] In one aspect of the present disclosure, a data generation device includes a data generation unit that generates data including lane information that identifies the lane in which a vehicle is traveling, traffic light information that identifies each of a plurality of traffic lights located at an intersection to which the lanes connect, and reliability information that indicates the reliability set for each of the plurality of traffic lights, and generates traffic light identification data having a data structure in which different reliability levels are set for each of the plurality of traffic lights depending on the lane in which the vehicle is traveling, and which can identify traffic lights to be trusted based on the reliability levels.

[0008] In one aspect of the present disclosure, a data storage device includes a data storage unit that stores data, and the data storage unit includes, as the data, lane information that identifies the lane in which the vehicle is traveling, traffic light information that identifies each of a plurality of traffic lights located at an intersection to which the lanes are connected, and reliability information that indicates the reliability set for each of the plurality of traffic lights, and stores traffic light identification data having a data structure in which different reliability levels are set for each of the plurality of traffic lights depending on the lane in which the vehicle is traveling, and which can identify traffic lights that should be trusted based on the reliability levels. [Brief explanation of the drawings]

[0009] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: [Figure 1] FIG. 1 is a functional block diagram illustrating an example of the configuration of a map generation system according to this embodiment. [Figure 2]FIG. 2 is a diagram illustrating an example of the configuration of a traffic light linking data table according to this embodiment. [Figure 3] FIG. 3 is a flowchart illustrating an example of a method for generating a traffic light linking data table according to this embodiment. [Figure 4] FIG. 4 is a diagram visually illustrating an example of a state in which a travel path and a traffic light are temporarily linked according to this embodiment. [Figure 5] FIG. 5 is a diagram visually illustrating a plurality of different examples of states in which a travel path and a traffic light are temporarily associated with each other according to this embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of a temporary linking process according to this embodiment. [Figure 7] FIG. 7 is a diagram visually illustrating an example of the stop information assignment process according to this embodiment. [Figure 8] FIG. 8 is a diagram for explaining an example of an advantage of the stop information assignment process according to this embodiment. [Figure 9] FIG. 9 is a diagram (part 1) for explaining an example of a method for determining a vehicle traveling direction by the vehicle traveling direction information assignment process according to the present embodiment; [Figure 10] FIG. 10 is a diagram (part 2) for explaining an example of a method for determining a vehicle traveling direction by the vehicle traveling direction information assignment process according to the present embodiment; [Figure 11] FIG. 11 is a diagram visually illustrating an example of a method for determining whether or not a traffic jam has occurred by the traffic jam determination information assignment process according to this embodiment. [Figure 12] FIG. 12 is a diagram (part 1) visually illustrating an example of a determination process for classifying a recognized traffic signal as a local signal or a non-local signal according to the present embodiment. [Figure 13] FIG. 13 is a diagram (part 2) visually illustrating an example of a determination process for classifying a recognized traffic signal as a local signal or a non-local signal according to the present embodiment. [Figure 14] FIG. 14 is a diagram (part 3) visually illustrating an example of a determination process for classifying a recognized traffic signal as a local signal or a non-local signal according to the present embodiment. [Figure 15]FIG. 15 is a diagram (part 4) visually illustrating an example of a determination process for classifying a recognized traffic signal as a local signal or a non-local signal according to the present embodiment; [Figure 16] FIG. 16 is a diagram (part 5) visually illustrating a determination example of classifying a recognized traffic signal into a local signal or a non-local signal according to the present embodiment; [Figure 17] FIG. 17 is a diagram visually illustrating an example of the integration process according to this embodiment. [Figure 18] FIG. 18 is a diagram illustrating an example of the configuration of an integrated table according to this embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of the configuration of a traffic light extraction table according to this embodiment. [Figure 20] FIG. 20 is a diagram illustrating an example of the configuration of a reliability table according to this embodiment. [Figure 21] FIG. 21 is a diagram illustrating an example of the configuration of a reinforced version reliability table according to this embodiment. [Figure 22] FIG. 22 is a diagram illustrating an example of the configuration of an extracted data table according to this embodiment. [Figure 23] FIG. 23 is a diagram schematically illustrating an example of one record in the traffic light linking data table according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of a data generation device and a data storage device according to the present disclosure will be described below with reference to the drawings. The map generation system 1 illustrated in FIG. 1 is configured so that an onboard device 2 mounted on a vehicle and a server 3 located on the network side can communicate data via a communication network 4 including, for example, the Internet. The vehicle on which the onboard device 2 is mounted may be a vehicle with an autonomous driving function or a vehicle without an autonomous driving function. The onboard device 2 and the server 3 have a multiple-to-one relationship, and the server 3 can communicate data with multiple onboard devices 2. The server 3 is an example of a data generation device and a data storage device.

[0011] The vehicle-mounted device 2 includes a control unit 5, a data communication unit 6, a probe data storage unit 7, and a map data storage unit 8. The control unit 5 is mainly composed of a microcomputer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and I / O (Input / Output). The microcomputer executes computer programs stored in a non-transitory physical storage medium to perform processes corresponding to the computer programs and control the overall operation of the vehicle-mounted device 2.

[0012] The control unit 5 includes an information input unit 5a, a probe data generation unit 5b, a communication control unit 5c, and a travel control unit 5d. The information input unit 5a inputs surrounding information related to the vehicle's surroundings, travel information related to vehicle travel, and location information related to the vehicle's position. The information input unit 5a inputs, as surrounding information, a camera image of the vehicle's traveling direction captured by an onboard camera, sensor information detected by a sensor such as a millimeter-wave sensor, radar information detected by a radar, and lidar information detected by a lidar (LiDAR: Light Detection and Ranging, Laser Imaging Detection and Ranging). The camera image includes traffic lights, traffic signs, signboards, stop lines painted on the road, lane markings, crosswalks, and the like.

[0013] The information input unit 5a inputs, as driving information, vehicle speed information detected by a vehicle speed sensor, etc. The information input unit 5a receives, as position information, navigation signals, i.e., positioning signals, transmitted from positioning satellites constituting the GNSS (Global Navigation Satellite System). The information input unit 5a is a device that receives navigation signals from the GNSS positioning satellites and thereby sequentially detects the current position of the information input unit 5a and, ultimately, the current position of the vehicle equipped with the on-board device 2. For example, when the GNSS is able to receive positioning signals from four or more positioning satellites, it outputs positioning results every 100 milliseconds. Note that, as position information, GPS (Global Positioning System), GLONASS, Galileo, IRNSS, QZSS, Beidou, etc. can also be used.

[0014] When surrounding information, driving information, and position information are input to the information input unit 5a, the probe data generation unit 5b generates probe data from the various input information and stores the generated probe data in the probe data storage unit 7. The probe data is data including surrounding information, driving information, and position information, and indicates the positions, colors, characteristics, and relative positional relationships of traffic lights, traffic signs, billboards, stop lines painted on the road surface, lane markings, crosswalks, etc. The probe data also includes data indicating various information about the road on which the vehicle is traveling, such as road shape, road characteristics, and road width.

[0015] The communication control unit 5c reads out the probe data stored in the probe data storage unit 7 and causes the data communication unit 6 to transmit the read-out probe data to the server 3, for example, when a predetermined time has elapsed or when the vehicle has traveled a predetermined distance. Alternatively, instead of using the time or the vehicle's travel distance as the trigger, the communication control unit 5c may use the reception by the data communication unit 6 of a probe data transmission request transmitted from the server 3 as a trigger, as long as the server 3 is configured to transmit a probe data transmission request to the in-vehicle device 2 at predetermined intervals. When causing the data communication unit 6 to transmit the probe data to the server 3, the communication control unit 5c may cause the data communication unit 6 to transmit the probe data to the server 3 in segment units, which are area units predetermined for managing maps, or may cause the data communication unit 6 to transmit the probe data to the server 3 in area units unrelated to segment units.

[0016] When the map data transmitted from the server 3 is received by the data communication unit 6, the driving control unit 5d stores the received map data in the map data storage unit 8, reads map data including necessary information according to the vehicle's position from the map data storage unit 8, and controls the vehicle's driving in accordance with the read map data. The driving control unit 5d may store a wide range of map data in advance in the map data storage unit 8 and read local map data according to the vehicle's position from the wide range of map data one by one to control the vehicle's driving, or may send a map data transmission request according to the vehicle's position from the data communication unit 6 to the server 3 and obtain local map data according to the vehicle's position from the server 3 one by one.

[0017] The server 3 includes a control unit 9, a data communication unit 10, a probe data storage unit 11, and a map data storage unit 12. The control unit 9 is mainly composed of a microcomputer having a CPU, ROM, RAM, and I / O. The microcomputer executes computer programs stored in a non-transitory physical storage medium to perform processing corresponding to the computer programs and control the overall operation of the server 3. The computer programs executed by the microcomputer include a map generation program, etc.

[0018] The control unit 9 includes a probe data acquisition unit 9a, a traffic light information identification unit 9b, a stop line information identification unit 9c, a lane identification unit 9d, a traffic light linking data table generation unit 9e, and a traffic light linking data table storage unit 9f. Hereinafter, the traffic light linking data table generation unit 9e may be simply referred to as the "data generation unit 9e," and the traffic light linking data table storage unit 9f may be simply referred to as the "data storage unit 9f."

[0019] When the probe data transmitted from the vehicle-mounted device 2 is received by the data communication unit 10, the probe data acquisition unit 9a stores the received probe data in the probe data storage unit 11 and acquires the probe data by reading out the necessary information of the probe data from the probe data storage unit 11. The probe data acquisition unit 9a acquires the probe data from a plurality of vehicles by receiving the probe data transmitted from each of the vehicle-mounted devices 2 mounted in the plurality of vehicles by the data communication unit 10.

[0020] The traffic light information identification unit 9b identifies traffic light information for each of the multiple traffic lights at the intersection based on the probe data acquired by the probe data acquisition unit 9a. Traffic light information is information managed by associating one or more traffic lights at an intersection where lanes on which vehicles travel connect with each other, with a traffic light ID that can identify each of the traffic lights. The traffic light information includes traffic light position that can identify the location of the traffic light, traffic light size that can identify the size of the traffic light, light direction, light color, traffic light type that can identify the type of traffic light, arrow direction information, etc.

[0021] The traffic light position is expressed, for example, by three-dimensional coordinates indicating the center of the traffic light. The traffic light size is expressed, for example, by the position coordinates of the traffic light's center, the position coordinates of its endpoints, the width (horizontal) dimension, and the height (vertical) dimension. The light direction is expressed by a normal vector perpendicular to the direction in which the traffic lights are arranged, which is the direction of the traffic light's normal vector. The light color is expressed as a color, such as blue or green, indicating permission to enter the intersection area, a color, such as yellow, indicating permission to proceed while paying attention to other traffic, or a color, such as red, indicating that entry into the intersection area is prohibited. The traffic light type is classified based on the shape of the traffic light, such as vertical or horizontal, or the number of lamps equipped on the traffic light. The arrow direction information, if the traffic light is equipped with an arrow lamp, is information on the direction of the arrow of the arrow lamp, such as the left turn direction, right turn direction, or straight ahead direction.

[0022] The stop line information identification unit 9c identifies stop line information for each of the multiple stop lines at the intersection based on the probe data acquired by the probe data acquisition unit 9a. The stop line information is information managed in association with a stop line ID that can identify the stop line, and includes a stop line position that can identify the position of the stop line, a stop line size that can identify the size of the stop line, a stop line type that can identify the type of the stop line, etc.

[0023] The stop line position is expressed, for example, by three-dimensional coordinates indicating the center of the stop line. The stop line size is expressed, for example, by the position coordinates of the center of the stop line, the position coordinates of its endpoints, the dimension in the width direction (road width), the dimension in the depth direction (lane direction), etc. The stop line type is classified, for example, by whether or not there is a crosswalk parallel to the stop line.

[0024] The lane identification unit 9d identifies lane information related to the lane in which the vehicle is traveling based on the probe data acquired by the probe data acquisition unit 9a. In this case, the lane identification unit 9d statistically processes multiple data groups indicating the vehicle's traveling trajectory and lane markings to identify the lane center line and identify the traveling lane. That is, the lane identification unit 9d identifies the lane center line and identifies the traveling lane, for example, by excluding data outside a predetermined range from the multiple data groups indicating the vehicle's traveling trajectory and lane markings and then averaging the data within the predetermined range. The lane information configured in this manner is information managed in association with a lane ID that can identify the lane in which the vehicle is traveling.

[0025] The data generator 9e is configured to generate various types of data, and can generate, for example, the traffic light linking data table T1 illustrated in FIG. 2. The traffic light linking data table T1 is an example of traffic light identification data and includes at least various types of ID information, such as the traffic light information and lane information described above. The traffic light linking data table T1 also includes linking reliability information, which is an example of reliability information. The reliability information is information indicating the reliability assigned to each of multiple traffic lights, in other words, the degree to which each traffic light can be trusted when controlling the autonomous driving of a vehicle. According to the traffic light linking data table T1, different reliability levels are assigned to each of multiple traffic lights depending on the lane in which the vehicle is traveling. Therefore, the traffic light linking data table T1 realizes a data structure that allows for identifying traffic lights that should be trusted when controlling the autonomous driving of a vehicle by comparing the reliability levels assigned to each traffic light.

[0026] The data storage unit 9f is configured to be able to store various types of data, and can store in the map data storage unit 12, for example, a traffic light linking data table T1 illustrated in FIG.

[0027] For example, according to the traffic light linking data table T1 shown in Fig. 2, various types of ID information such as traffic light information, lane information, stop line information, etc. are unique within the map data. By using such traffic light linking data table T1, it is possible to control vehicles passing through or attempting to stop at an intersection, for example, in the following manner.

[0028] That is, for example, assume a case where a vehicle traveling in a lane with lane information "100" enters an intersection as illustrated in FIG. 2. In this case, the vehicle-mounted device 2 refers to the traffic light linking data table T1 to identify multiple traffic lights, three in this case, with traffic light information "10000," "10001," and "10002," as traffic lights linked to the lane with lane information "100." When multiple traffic lights are linked in this way, the vehicle-mounted device 2 follows the traffic light with the highest reliability information among the multiple traffic lights. According to the traffic light linking data table T1 illustrated in FIG. 2, the traffic light with the highest reliability among the three traffic lights is the traffic light with traffic light information "10000." Therefore, the vehicle-mounted device 2 controls the vehicle's travel according to the traffic light with traffic light information "10000."

[0029] Furthermore, the vehicle-mounted device 2 references the passability information stored in the traffic light linking data table T1. The vehicle-mounted device 2 then recognizes the light state of a traffic light whose traffic light information is "10000" using a camera image and compares this recognized state with the traffic light recognition state information stored in the traffic light linking data table T1. The vehicle-mounted device 2 then references the passability information corresponding to the signal recognition state that matches the light state of the traffic light whose traffic light information is "10000". The vehicle-mounted device 2 then controls the vehicle to pass through the intersection if the referenced passability information stores the number "1", and controls the vehicle to stop at the stop line just before the intersection if the referenced passability information stores the number "0".

[0030] As described above, based on the traffic light linking data table T1, it is possible to determine which of a plurality of traffic lights a vehicle should follow when entering or passing through an intersection.

[0031] Furthermore, for example, when an intersection has a complex shape or traffic lights are arranged in a complex manner, even if a reliable traffic light can be identified from multiple traffic lights, there may be cases where it is impossible to determine whether the vehicle can proceed or stop at that traffic light when that traffic light is lit. The traffic light linking data table T1 includes signal recognition status information indicating the traffic light status and pass permission information indicating whether the vehicle can pass. Therefore, it is possible to determine whether the vehicle can proceed or stop based on the traffic light status.

[0032] Furthermore, according to the traffic light linking data table T1, different reliability levels are set for each of the multiple traffic lights depending on the lane in which the vehicle is traveling. Therefore, the vehicle's travel can be controlled according to the most reliable traffic light among the multiple traffic lights recognized by the onboard device 2. As a result, even if a reliable legitimate traffic light is not accurately linked, the safety of automated driving of the vehicle can be sufficiently ensured by having the onboard device 2 follow the most reliable traffic light among the multiple traffic lights actually recognized. In this case, even if the onboard device 2 identifies the most reliable traffic light among the multiple traffic lights recognized, if the reliability is lower than a predetermined reference value, the onboard device 2 may not follow that traffic light.

[0033] Next, an example of a generation method for generating the traffic light linking data table T1 shown in Fig. 2 will be described in detail. In this embodiment, the traffic light linking data table T1 is generated by the server 3. Fig. 3 shows an example of a main flow of the generation method. That is, the generation method includes a probe data acquisition process (step A1), a provisional linking process (step A2), an integration process (step A3), a final linking process (step A4), a passability information assignment process (step A5), and a database update process (step A6). Here, the provisional linking process (step A2) is an example of provisional processing, and the final linking process (step A4) is an example of final processing.

[0034] In the probe data acquisition process (step A1), the server 3 acquires each piece of probe data transmitted from a plurality of onboard devices 2. Hereinafter, the probe data may be simply referred to as "PD." The probe data includes at least location information indicating the vehicle's location, travel path information indicating the vehicle's travel path, speed information indicating the vehicle's speed, yaw rate information indicating the vehicle's yaw angle or yaw rate, inter-vehicle distance information indicating the inter-vehicle distance between the host vehicle and a preceding vehicle, position information of traffic lights identified by analyzing camera images, light information of traffic lights identified by analyzing camera images, and shape information of traffic lights identified by analyzing camera images. The light information is information indicating the light status of traffic lights. Note that, at this stage of the probe data acquisition process, it is not necessary to accurately identify the lane in which the vehicle is traveling or the traffic light recognized by the camera images.

[0035] In the provisional linking process (step A2), the server 3 performs provisional linking for each piece of probe data acquired in the probe data acquisition process. Note that "linking" in this disclosure can also be referred to as associating two or more different types of information with each other, or so-called information pairing. In the provisional linking process, the server 3 generates information indicating whether the travel trajectory included in the acquired probe data is linked to a traffic light. Figure 4 visually illustrates a state in which travel trajectory R1 and traffic light A are provisionally linked. In this case, travel trajectory R1 is provisionally linked to traffic light A, but is not provisionally linked to traffic light B.

[0036] Furthermore, at this stage of the temporary linking process, there may be cases where the temporary linking between the travel trajectory and the traffic light is incorrect due to factors such as the recognition status of the camera image and the influence of the external environment, and such incorrect cases are also tolerated. That is, as illustrated in FIG. 5, even if the vehicles are traveling in the same lane, different traffic lights may be temporarily linked to the travel trajectory R1 depending on factors such as the recognition status of the camera image and the influence of the external environment. In this case, according to PD-1, the travel trajectory R1 is temporarily linked to traffic lights A1 and B1. According to PD-2, the travel trajectory R2 is temporarily linked only to traffic light A2, and is not temporarily linked to traffic light B2. According to PD-3, the travel trajectory R3 is temporarily linked to traffic lights A3 and B3.

[0037] Next, the contents of the temporary linking process for generating the temporary linking information as described above will be described in detail. As shown in Fig. 6, the temporary linking process includes a stop information assignment process (step B1), a vehicle travel direction information assignment process (step B2), a congestion determination information assignment process (step B3), and a local signal determination process (step B4).

[0038] In the stop information assignment process (step B1), as illustrated in FIG. 7, the server 3 determines whether the vehicle has made a stopping motion within X meters before the target traffic light based on the vehicle's speed information, etc. If the vehicle has made a stopping motion within X meters before the target traffic light, the server 3 assigns stop information. Note that the distance X can be appropriately changed and set, for example, taking into account the situation around the vehicle. Furthermore, the distance X can be set to a different distance for each traffic light, for example.

[0039] In addition, in the stop signal issuing process, the server 3 issues stop signal only when the angle between the direction of the vehicle's traveling direction and the direction of the normal vector of the traffic light is equal to or greater than a predetermined angle, in order to exclude traffic lights at the destination of a right or left turn. Note that the predetermined angle can be changed as appropriate. In the example of FIG. 7, the predetermined angle is set, for example, within a range of 150 degrees to 180 degrees.

[0040] That is, according to FIG. 7(a), the vehicle exhibits a stopping behavior within Xa meters before the target traffic light A, and the angle Ka between the direction Z1 of the vehicle's direction of travel and the direction Za of the normal vector of traffic light A is "180 degrees," that is, within a predetermined angle range. Also, the vehicle exhibits a stopping behavior within Xb meters before the target traffic light B, and the angle Kb between the direction Z1 of the vehicle's direction of travel and the direction Zb of the normal vector of traffic light B is "180 degrees," that is, within a predetermined angle range. Therefore, the server 3 assigns information indicating that the vehicle has stopped in front of traffic lights A and B, i.e., stop information, to traffic lights A and B. That is, the server 3 associates the stop information with traffic lights A and B.

[0041] 7(b), the vehicle exhibits a stopping behavior within Xc meters before the target traffic light C, but the angle Kc formed by the direction Z1 of the vehicle's traveling direction and the direction Zc of the normal vector of traffic light C is outside the predetermined angle range. Therefore, the server 3 does not assign information indicating that the vehicle has stopped before the traffic light, i.e., stop information, to traffic light C. In other words, the server 3 does not associate stop information with traffic light C.

[0042] This type of processing has the following advantages: In an actual vehicle driving environment, as shown in Fig. 8, traffic light B may be cut off depending on the vehicle's position, that is, it may be outside the camera's capture range. Also, there may be cases where traffic light B cannot be recognized due to an obstacle such as a large vehicle traveling alongside the vehicle.

[0043] More specifically, Figure 8(a) illustrates a state in which a vehicle has not yet exhibited stopping behavior before entering an intersection, and in this state, the vehicle can recognize the light status of traffic light B, in this case, the red light status, through the camera. However, at this stage, the vehicle has not yet exhibited stopping behavior. Therefore, even if the camera can recognize the red light status of traffic light B, no stopping information is provided.

[0044] On the other hand, Figures 8(b) and 8(c) show examples of situations in which a vehicle is exhibiting stopping behavior in front of an intersection, but in these situations, traffic light B cannot be recognized due to the influence of the camera's field of view, etc. Therefore, even though the vehicle is exhibiting stopping behavior and traffic light B has a red light on, traffic light B is not recognized by the camera, and therefore stop information cannot be assigned to traffic light B.

[0045] In contrast, the above-described stop information assignment process determines whether a vehicle exhibited stopping behavior within X meters before the target traffic light. In other words, rather than determining whether a vehicle exhibited stopping behavior at a specific point when the traffic light was photographed by the camera, it determines whether a vehicle exhibited stopping behavior within a predetermined range before the target traffic light. Therefore, even if a vehicle exhibits stopping behavior in a situation where the target traffic light is not recognized by the camera, it is possible to assign stop information to the target traffic light. This improves the coverage rate of the process of assigning stop information, which is one element of linking. In other words, it is possible to avoid not assigning stop information to traffic lights that should be assigned stop information.

[0046] In the vehicle travel direction information assignment process (step B2), the server 3 assigns travel direction information indicating the travel direction of the vehicle at the intersection where the traffic light is recognized. The travel direction of the vehicle may be determined based on information about the vehicle's rotation angle using, for example, vehicle yaw rate information, or based on information about road paint painted on the lane in which the vehicle is traveling, or based on the status of features such as lane markings, or by appropriately combining multiple pieces of information. In addition, the probe data may include, for example, right turn information indicating that the vehicle has turned right, left turn information indicating that the vehicle has turned left, and turn signal information indicating the operation status of the vehicle's turn signals, and the travel direction of the vehicle may be determined based on this information.

[0047] For example, when determining the vehicle's direction of travel based on yaw rate information, as illustrated in Figure 9, the server 3 can calculate the vehicle's rotation angles Y2, Y3 based on yaw rate information within a specified range before and after the recognition point P1 at which the vehicle recognized traffic light A, and identify the vehicle's direction of travel based on the calculated angle information Y2, Y3.

[0048] Furthermore, when determining the direction of travel of a vehicle based on road paint information, the server 3 can determine that the direction of travel of the vehicle is a right turn if the road paint information is road paint indicating a right turn, can determine that the direction of travel of the vehicle is a left turn if the road paint information is road paint indicating a left turn, and can determine that the direction of travel of the vehicle is a straight ahead direction if the road paint information is road paint indicating a straight ahead.

[0049] 9, the road painting on the lane in which the vehicle is traveling indicates a right turn, so the server 3 can determine that the vehicle is traveling in a right turn direction.

[0050] This type of processing has the following advantages: There are many traffic lights that are always in the red light state and controlled only by the arrow light. For such traffic lights, linking can be performed based on the recognition status of the arrow light by the camera and the direction of travel of the vehicle, thereby improving the accuracy of linking.

[0051] For example, in the state shown in Figure 10(a), traffic light A has its red light on and an arrow light that allows straight-ahead driving. In this case, if it can be determined that the vehicle is traveling straight based on the vehicle's traveling direction information, traffic light A can be recognized as a reliable traffic light because the vehicle is obeying the arrow light even when traffic light A is red.

[0052] 10(b), traffic light A has its red light on and an arrow light that allows a right turn. In this case, if it can be determined that the vehicle is turning right based on the vehicle's traveling direction information, traffic light A can be recognized as a reliable traffic light because the vehicle is obeying the arrow light even when traffic light A is red.

[0053] In this way, by taking into consideration the direction of travel of the vehicle at the intersection, it is possible to prevent reliable traffic lights from being overlooked and not recognized.

[0054] In the congestion determination information assignment process (step B3), the server 3 assigns congestion information indicating whether or not congestion has occurred before or after the vehicle passes through the intersection or within the intersection. Whether or not congestion has occurred can be determined based on information such as speed information indicating the speed of the vehicle and inter-vehicle distance information indicating the distance between the vehicle and a preceding vehicle. As illustrated in FIG. 11, when congestion occurs, the vehicle speed repeatedly increases and decreases, and the inter-vehicle distance repeatedly increases and decreases. Therefore, based on the speed information and inter-vehicle distance information, it is possible to determine with sufficient accuracy whether or not congestion has occurred.

[0055] When a traffic jam occurs, there may be cases where the traffic light status and vehicle behavior do not match, such as when a vehicle is stopped even when the traffic light is green. Therefore, even when the traffic light status and vehicle behavior do not match, if the server 3 can determine that a traffic jam has occurred based on the traffic congestion information, it can recognize the traffic light in question as a reliable traffic light.

[0056] In this way, by taking into consideration whether or not there is congestion before, after, or within the intersection, it is possible to prevent a reliable traffic light from being overlooked without being recognized. As will be described later, the server 3 can classify traffic lights as either own or non-own signals by taking congestion information into consideration, and can also set an invalid value for a traffic light if there is congestion.

[0057] In the own signal determination process (step B4), the server 3 classifies the recognized traffic signal as either an own signal or a non-own signal by comprehensively considering the stop information provided by the stop information providing process, the vehicle traveling direction information provided by the vehicle traveling direction information providing process, the congestion information provided by the congestion information providing process, as well as the traffic signal position information, lighting information, signal shape information, etc. contained in the acquired probe data. An own signal is a traffic signal corresponding to the lane in which the vehicle is traveling, and can be defined as a traffic signal that the own vehicle should refer to. On the other hand, a non-own signal is a traffic signal that does not correspond to the lane in which the vehicle is traveling, that is, a traffic signal other than the own signal, and can be defined as a traffic signal that the own vehicle does not need to refer to or has little need to refer to.

[0058] The server 3 may manage traffic lights by assigning, for example, a numerical value "1" to traffic lights assigned to its own signal and by assigning, for example, a numerical value "0" to traffic lights assigned to non-own signals. Furthermore, the server 3 may comprehensively assess various information and identify all traffic lights that are determined to have a relatively high probability of being its own signal as its own signal, and identify all traffic lights that are determined to have a relatively low probability of being its own signal as non-own signals. Furthermore, the server 3 may set a predetermined valid value to traffic lights that it has determined to be its own signal, and a predetermined invalid value to traffic lights that it has determined to be non-own signals.

[0059] Next, an example of a determination to classify a recognized traffic light as a local signal or a non-local signal will be described. FIG. 12 shows a situation in which a vehicle turns right at an intersection, does not stop when passing through the intersection, and there is no traffic congestion. Furthermore, when the vehicle passes through the intersection, traffic light A has its red light on and an arrow light that allows a right turn. In this situation, it can be determined that the vehicle is following the behavior of traffic light A. Therefore, the server 3 identifies traffic light A as a local signal.

[0060] FIG. 13 illustrates a situation in which a vehicle is attempting to proceed straight through an intersection, but is stopped in front of the intersection due to a traffic jam. Furthermore, as the vehicle passes through the intersection, traffic light A has its red light on and an arrow light indicating that the vehicle is permitted to proceed straight. In this situation, it can be determined that the vehicle is not following the behavior of traffic light A. However, in this case, traffic congestion occurs at the intersection. Therefore, the server 3 sets an invalid value for traffic light A. In other words, the server 3 does not assign traffic light A to either the own signal or a non-own signal. Furthermore, the server 3 does not link traffic light A. By setting an invalid value for a traffic light and not linking it when traffic congestion occurs, it is possible to avoid using information from an invalid traffic light and improve the accuracy of the linking. In this case, the server 3 may also assign traffic light A to either the own signal or a non-own signal.

[0061] FIG. 14 shows a situation in which a vehicle is trying to go straight through an intersection but has stopped before the intersection. In this case, there is no traffic congestion. The vehicle recognizes traffic lights A and B, with traffic light A showing a red light and traffic light B showing a green light. In this situation, the vehicle follows the behavior of traffic light A but does not follow the behavior of traffic light B. Therefore, the server 3 identifies traffic light A as the subject signal and traffic light B as a non-subject signal.

[0062] FIG. 15 illustrates an example of an intersection in a foreign country other than Japan. In this example, a vehicle turns right at the intersection, does not stop, and is not congested. The vehicle recognizes traffic lights A and B, both of which have their green lights on. In this example, the vehicle follows the behavior of both traffic lights A and B. Traffic light A has three vertical lights, and traffic light B has five vertical lights. Therefore, traffic light B can be determined to be a right-turn-only traffic light in a foreign country other than Japan. Therefore, it can be inferred that the vehicle followed the behavior of traffic light B, which is a right-turn-only traffic light. Therefore, the server 3 identifies traffic light A as a non-subject signal and traffic light B as a subject signal.

[0063] FIG. 16 also illustrates an intersection where different traffic lights should be referenced for each lane. In this case, a vehicle turns right at the intersection, does not stop when passing through the intersection, and no congestion occurs. The vehicle recognizes traffic lights A and B, both of which have their green lights on. In this situation, the vehicle follows the behavior of both traffic lights A and B. Here, traffic light A corresponds to the left lane Ra, and traffic light B corresponds to the right lane Rb. Therefore, it can be assumed that a vehicle traveling in the right lane Rb follows the behavior of traffic light B, which corresponds to lane Rb. Therefore, the server 3 identifies traffic light A as a non-subject signal and traffic light B as a subject signal.

[0064] This concludes the detailed description of the temporary linking process (step A2). Next, the integration process (step A3) will be described in detail.

[0065] The integration process (step A3) is a process executed to generate most of the traffic light linking data table T1, and the server 3 integrates multiple pieces of allocation result data obtained by the above-described temporary linking process. More specifically, at the stage when the above-described temporary linking process is completed, it may not be clear which traffic lights are the same, which lanes are the same, or which lane is the vehicle's driving lane, i.e., the lane in which the vehicle is traveling. Therefore, for example, when multiple pieces of allocation result data have been obtained by the temporary linking process, the server 3 integrates the multiple pieces of allocation result data to clearly identify the same traffic lights, the same lanes, and the same driving lane.

[0066] 17, the server 3 integrates the multiple data items illustrated in FIG. 5, in this case PD-1, PD-2, and PD-3, to identify traffic lights A1, A2, and A3 as the same traffic light A and assign the traffic light information, for example, "10000." The server 3 also identifies traffic lights B1, B2, and B3 as the same traffic light B and assigns the traffic light information, for example, "10001."

[0067] 17, the server 3 identifies the travel loci R1 and R2 as the same travel locus by integrating, for example, the multiple data illustrated in FIG. 5, in this case, PD-1, PD-2, and PD-3, and assigns, for example, "100" as lane information indicating the lane along the travel locus. The server 3 also identifies the travel locus R3 as the same travel locus and assigns, for example, "101" as lane information indicating the lane along the travel locus.

[0068] 18, the server 3 generates an integrated table T2 for the recognized traffic lights, including traffic light information, lane information, various information such as stop information added by the temporary linking process described above, and original PD information indicating the referenced probe data. In this case, the server 3 stores lane information indicating the lane through which the vehicle passed or the lane immediately before the stop line where the vehicle stopped, but this is not limited to this. The integrated table T2 may also include various other information such as stop information, traveling direction information, traffic congestion information, and lighting information.

[0069] In this linking process (step A4), the server 3 creates the traffic light linking data table T1 shown in FIG. 2 based on the integrated table T2 created by the above-described integration process. The server 3 can then set the reliability information included in the traffic light linking data table T1 based on the information generated by the provisional linking process. The server 3 can also set a reliability for each combination of lane information and traffic light information. The server 3 can also set the reliability by statistically processing the information generated by the provisional linking process.

[0070] 19 shows an example of a reliability setting method by the server 3, illustrating a traffic light extraction table T3 in which data for which "10000" is set as traffic light information is extracted from the integrated table T2 described above. In this case, a total of 20 pieces of data are extracted, of which 19 pieces of data are tentatively linked as the own signal and 1 piece of data is tentatively linked as a non-own signal. Therefore, in this case, the server 3 assigns a reliability of 0.95 to the traffic light for which "10000" is set as traffic light information.

[0071] By performing such statistical processing, the server 3 generates a reliability table T4, for example, as shown in FIG. 20. The server 3 performs the above-described statistical processing for all combinations of lane information and traffic light information to set the reliability. Note that the method for setting the reliability is not limited to the above-described method, i.e., the method for setting the reliability based on the proportion of the own signal. For example, the server 3 may quantify the probability that the signal can be determined to be the own signal and the probability that the signal can be determined to be a non-own signal, and set the reliability based on the numerical value.

[0072] Furthermore, as illustrated in FIG. 21, the server 3 generates a reinforced reliability table T5 by adding various types of information such as stop line information, pedestrian crossing information, and road surface paint information to the reliability table T4 described above.

[0073] In the passability information assignment process (step A5), the server 3 further assigns passability information. The passability information is information indicating whether or not a vehicle can pass through an intersection. The passability information may be indicated, for example, by a frequency or a rate, similar to the reliability, or may be indicated by a binary or multi-value, such as a numerical value "1" if passable and a numerical value "0" if not passable. The passability information may also be expressed in a format using concrete characters or abstract characters. The passability information may also be expressed in a format in which, for example, individual reliability is assigned to light information.

[0074] Next, we will explain an example of a method for generating passability information by, for example, the server 3. That is, the server 3 creates passability information based on various information generated by the above-mentioned temporary linking process, integration process, final linking process, etc. Fig. 22 shows an example of an extracted data table T6 in which data in which "10000" is set as traffic light information and "100" is set as lane information is extracted from the various information generated by the above-mentioned temporary linking process, integration process, final linking process, etc.

[0075] In this case, in the data in which the recognition light information stores a red light state and an arrow light state that allows a right turn, the vehicle is stopped in all data. Hereinafter, for convenience, this pattern will be referred to as pattern 1. On the other hand, in the data in which the recognition light information stores a red light state and an arrow light state that allows a straight ahead, the vehicle is not stopped in two of the three data. Hereinafter, this pattern will be referred to as pattern 2.

[0076] According to pattern 1, vehicles are stopped in all data. Therefore, as illustrated in FIG. 23, the server 3 sets "impassable" as the passability information for the data of pattern 1. On the other hand, according to pattern 2, vehicles are not stopped in two of the three data. Therefore, as illustrated in FIG. 23, the server 3 sets "passable" as the passability information for the data of pattern 2, for example, by majority vote. The server 3 then assigns passability information to the multiple traffic lights recognized by the vehicle, and ultimately generates a traffic light linking data table T1 as illustrated in FIG. 2.

[0077] In the database update process (step A6), the server 3 updates the traffic light linking data table T1 stored in the map data storage unit 12 to the newly generated traffic light linking data table T1. If the traffic light linking data table T1 is not stored in the map data storage unit 12, the server 3 stores, or registers, the traffic light linking data table T1 generated in this process in the map data storage unit 12.

[0078] Furthermore, when updating the traffic light linking data table T1 stored in the map data storage unit 12, the server 3 may update the entire traffic light linking data table T1, or may update only a portion of the traffic light linking data table T1, such as the difference from the previous data, or only specific items. Furthermore, when updating the traffic light linking data table T1, the server 3 may refer to the reliability in the previous traffic light linking data table T1 and adjust the reliability to be updated this time.

[0079] The server 3 then distributes the traffic light linking data table T1 stored in the map data storage unit 12 to each of the multiple on-board devices 2. Upon receiving the latest traffic light linking data table T1 from the server 3, the on-board device 2 stores the traffic light linking data table T1 in the map data storage unit 8 and controls the automatic driving of the vehicle based on the traffic light linking data table T1.

[0080] According to an embodiment of the present disclosure, the server 3 is configured to be able to generate and store a traffic light linking data table T1 in which the reliability of traffic lights as described above is set. The traffic light linking data table T1 includes at least lane information identifying the lane in which the vehicle is traveling, traffic light information identifying each of multiple traffic lights at intersections connecting the lanes, and reliability information indicating the reliability set for each of the multiple traffic lights. Different reliability levels are set for the multiple traffic lights depending on the lane in which the vehicle is traveling, and the data structure allows for identifying a reliable traffic light by comparing the reliability levels. Therefore, based on this traffic light linking data table T1, even if there are multiple traffic lights at an intersection connecting the lanes in which the vehicle is traveling, it is possible to identify a reliable traffic light among the multiple traffic lights, thereby achieving safer automated driving of vehicles than before.

[0081] Furthermore, according to the server 3, the data generator 9e causes the traffic light linking data table T1 to include lighting information indicating the lighting status of traffic lights and passability information indicating whether the vehicle can pass through the intersection. Based on this traffic light linking data table T1, it is possible to control the automated driving of a vehicle while checking not only the reliability of the traffic lights but also the lighting status of the traffic lights and whether it is OK to pass through the intersection, thereby realizing even safer automated driving of a vehicle.

[0082] Furthermore, according to the server 3, the data generation unit 9e is configured to be able to execute a temporary linking process that classifies multiple traffic signals recognized by the vehicle into a local signal corresponding to the lane in which the vehicle is traveling or a non-local signal that does not correspond to the lane in which the vehicle is traveling, and a final linking process that sets reliability through statistical processing based on the proportion of traffic signals classified as local signals among the multiple traffic signals recognized by the vehicle. That is, the data generation unit 9e is configured to set the reliability of traffic signals through at least two stages: the temporary linking process and the final linking process. This configuration example allows the reliability of traffic signals to be set with even greater accuracy.

[0083] Furthermore, according to the server 3, the data generation unit 9e is configured to execute an integration process prior to the main linking process, which integrates multiple pieces of allocation result data obtained by the provisional linking process. In this integration process, the data generation unit 9e integrates traffic light information related to the same traffic light and also integrates lane information related to the same lane. That is, the data generation unit 9e is configured not only to integrate traffic light information but also to integrate lane information, which is information other than traffic light information. According to this configuration example, by integrating information other than traffic light information, the accuracy of information integration can be improved compared to simply integrating traffic light information alone, and traffic lights can be identified and their reliability can be assigned with even greater accuracy.

[0084] Furthermore, according to the server 3, the data generator 9e can generate traffic light information for traffic lights that are actually recognized by analyzing vehicle camera images, rather than traffic light information stored in existing map data. According to this configuration example, for example, traffic light information can be generated for newly installed traffic lights that are not stored in existing map data, as long as the traffic light is actually recognized by the vehicle, and data can be generated according to actual road conditions that are not reflected in existing map data.

[0085] Furthermore, in the server 3, the data generation unit 9e may be configured to make the generated reliability unreliable if the reliability is lower than a predetermined reference value. This makes it possible to prevent a traffic light with an excessively low reliability from being trusted when controlling the automatic driving of a vehicle. The predetermined reference value may be an appropriate value expressed in an appropriate format, such as 0.5 or 50 percent. The setting to make the reliability unreliable may be performed by the in-vehicle device 2 rather than the server 3.

[0086] The present disclosure is not limited to the above-described embodiments, and modifications and extensions may be made as appropriate without departing from the spirit and scope of the present disclosure. For example, the various functions of the server 3 may be configured to be provided by each of the multiple on-board devices 2. That is, the map generation system 1 according to the present disclosure may be configured to be implemented by a single on-board device 2. Furthermore, the on-board camera is not limited to a front camera that captures images in front of the vehicle, but may also include a side camera that captures images to the sides of the vehicle and a rear camera that captures images behind the vehicle. Furthermore, the present disclosure can be applied not only to autonomous driving of automobiles, but also to data for controlling autonomous driving of moving objects other than automobiles, such as bicycles.

[0087] Although the present disclosure has been described with reference to the embodiments, it is understood that the present disclosure is not limited to the embodiments or structures. The present disclosure also encompasses various modifications and modifications within the scope of equivalents. In addition, various combinations and forms, as well as other combinations and forms including only one element, more than one element, or less than one element, are also within the scope and spirit of the present disclosure.

[0088] The controller and methods described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the controller and methods described herein may be implemented by a special-purpose computer configured with a processor comprising one or more dedicated hardware logic circuits. Alternatively, the controller and methods described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored in a computer-readable non-transitory tangible storage medium as instructions executed by a computer.

Claims

1. A data generating unit (9e) for generating data is provided, The data generation unit generates the data as follows: The data includes lane information that identifies the lane in which the vehicle is traveling, traffic light information that identifies each of a plurality of traffic lights provided at an intersection where the lane is connected, and reliability information that indicates the reliability set for each of the plurality of traffic lights, and different reliability levels are set for each of the plurality of traffic lights depending on the lane in which the vehicle is traveling, and traffic light identification data is generated having a data structure that can identify a traffic light to be trusted based on the reliability levels, A provisional process of classifying a plurality of traffic lights recognized by the vehicle into a local signal corresponding to the lane in which the vehicle is traveling, or a non-local signal not corresponding to the lane in which the vehicle is traveling; a main process for setting the reliability by statistically processing the allocation result data obtained by the provisional process; A data generating device that executes the above.

2. The data generating device according to claim 1 , wherein the data generating unit further includes, in the traffic light identification data, lighting information indicating the lighting state of the traffic light and passability information indicating whether or not a vehicle can pass through the intersection.

3. The data generation unit An integration process is performed before the main process to integrate the plurality of allocation result data obtained by the provisional process; The data generating device according to claim 1 , wherein the integration process integrates the traffic light information relating to the same traffic light and also integrates the lane information relating to the same lane.

4. The data generating device according to claim 1 , wherein the data generating unit makes the reliability unreliable when the reliability is lower than a predetermined reference value.

5. The data generating device according to claim 1 , wherein the data generating unit is capable of generating traffic light information relating to a traffic light recognized by a vehicle.

6. A data generation device as described in claim 1, which functions as a data storage device that stores the traffic light identification data.

Images