Information processing device, vehicle control device, and vehicle

Abstract

This information processing device according to one embodiment of the present disclosure can execute the following two steps. (1) In road data and traffic data acquired by an acquisition unit, when one of a plurality of traffic elements constituting a traffic context is missing, numerical expressions of a plurality of non-missing elements among the plurality of traffic elements other than a missing element, which is a missing traffic element, are read from a knowledge graph, and a calculation is performed using numerical expressions of the plurality of non-missing elements that have been read out, whereby a first numerical expression of the missing element is derived; (2) a label of a second numerical expression having a high degree of similarity to the first numerical expression is used as a label of the missing element by comparing the first numerical expression with a plurality of second numerical expressions in the knowledge graph.

Description

Information processing device, vehicle control device, and vehicle

[0001] The present disclosure relates to an information processing device, a vehicle control device, and a vehicle.

[0002] A driving assistance method is known that predicts dangerous events and warns the driver by combining knowledge data such as ontology and knowledge graphs with logical inference.

[0003] JP 2016-091039 A

[0004] An information processing device according to a first aspect of the present disclosure includes a storage unit, an acquisition unit, and a processing unit. The storage unit stores a knowledge graph represented numerically. The acquisition unit is capable of acquiring road data around a vehicle and traffic data about the vehicle and traffic participants around the vehicle. The processing unit is capable of creating a numerical representation of a traffic context corresponding to the road data and traffic data acquired by the acquisition unit, using the knowledge graph. The processing unit is capable of performing the following two operations: (1) When one of multiple traffic elements constituting a traffic context is missing in the road data and traffic data acquired by the acquisition unit, reading, from the knowledge graph, numerical representations of multiple non-missing elements other than the missing element, which is the missing traffic element, among the multiple traffic elements, and performing a calculation using the numerical representations of the multiple non-missing elements read out, thereby deriving a first numerical representation of the missing element; and (2) Comparing the first numerical representation with multiple second numerical representations in the knowledge graph, and setting the label of the second numerical representation that has a high similarity to the first numerical representation as the label of the missing element.

[0005] A vehicle control device relating to a second aspect of the present disclosure includes a traffic context acquisition unit capable of acquiring multiple traffic contexts including completed traffic contexts in which missing elements have been completed by an information processing device relating to the first aspect of the present disclosure, and a control unit capable of performing at least one of notification control and driving control based on the multiple traffic contexts acquired by the traffic context acquisition unit.

[0006] A vehicle relating to a third aspect of the present disclosure includes an alarm device and a driving device, a traffic context acquisition unit capable of acquiring multiple traffic contexts including a completed traffic context in which missing elements have been completed by an information processing device relating to the first aspect of the present disclosure, and a control device capable of performing at least one of alarm control for the alarm device and driving control for the driving device based on the multiple traffic contexts acquired by the traffic context acquisition unit.

[0007] The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate one embodiment and, together with the description, serve to explain the principles of the disclosure.

[0008] FIG. 1 is a diagram illustrating an example of a traffic condition ahead of a vehicle (surrounding traffic condition X). FIG. 2 is a diagram illustrating an example of a traffic condition ahead of a vehicle (surrounding traffic condition X). FIG. 3 is a diagram illustrating an example of external recognition data obtained from a sensor or the like in the surrounding traffic condition X in FIGS. 1 and 2. FIG. 4 is a diagram illustrating an example of a traffic context of the surrounding traffic condition X in FIGS. 1 and 2. FIG. 5 is a diagram illustrating an example of functional blocks of a vehicle according to an embodiment of the present disclosure. FIG. 6 is a diagram illustrating an example of the concept of a known traffic scenario. FIG. 7 is a diagram illustrating an example of a knowledge graph embedded in a knowledge space. FIG. 8 is a diagram illustrating an example of a known traffic condition A. FIG. 9 is a diagram illustrating an example of a traffic context of the known traffic condition A in FIG. 8. FIG. 10 is a diagram illustrating an example of a known traffic condition B. FIG. 11 is a diagram illustrating an example of a traffic context of the known traffic condition B in FIG. 10. FIG. 12 is a diagram illustrating an example of a known traffic condition C. FIG. 13 is a diagram illustrating an example of a traffic context of the known traffic condition C in FIG. 12. FIG. 14 is a diagram illustrating an example of a surrounding traffic situation Y following the surrounding traffic situation X in FIGS. 1 and 2. FIG. 15 is a diagram illustrating an example of the traffic context of the surrounding traffic situation Y in FIG. 14. FIG. 16(A) is a diagram illustrating an example of the concept of a vector representation of a sentence including missing elements in FIGS. 4 and 15. FIG. 16(B) is a diagram illustrating an example of the concept of a vector representation of a sentence included in the knowledge graph in FIG. 6. FIG. 17 is a diagram illustrating an example of the concept of a surrounding traffic scenario. FIG. 18 is a diagram illustrating an example of vector values ​​of a surrounding traffic scene (scene 1) included in a surrounding traffic scenario and example vector values ​​of multiple traffic elements included in the surrounding traffic scene (scene 1). FIG. 19 is a diagram illustrating an example of vector values ​​of a surrounding traffic scene (scene 2) included in a surrounding traffic scenario and example vector values ​​of multiple traffic elements included in the surrounding traffic scene (scene 2). FIG. 20 is a diagram illustrating an example of a driving assistance procedure for the vehicle of FIG. 1.

[0009] Some exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that the following description illustrates one specific example of the present disclosure and should not be construed as limiting the present disclosure. For example, each element, including numerical values, shapes, materials, parts, the position of each part, and the connection method of each part, is merely an example and should not be construed as limiting the present disclosure. Furthermore, in the following exemplary embodiments, components not described in independent claims based on the highest concept of the present disclosure are optional and may be provided as needed. The drawings are schematic and are not intended to be drawn to scale. Throughout this specification and the drawings, components having substantially the same function and configuration are designated by the same reference numerals, and redundant description will be omitted. Furthermore, components not directly related to one embodiment of the present disclosure are not shown in the drawings.

[0010] <1. Background> A driving assistance method is known that predicts dangerous events and warns the driver by combining knowledge data such as ontologies and knowledge graphs with logical inference. In such driving assistance methods, logical inference works when the observed traffic situation is included in the traffic situation described in the knowledge data. However, if a major traffic element of the surrounding traffic situation cannot be detected because it is blocked by an obstacle or due to the performance limit of a recognition sensor, the logical inference may not work due to the missing element included in the observed traffic situation.

[0011] Therefore, after extensive research, the inventors of the present application have come up with a technology that allows logical inference to be performed even when the observed traffic situation includes missing elements. Below, we will explain the background of this newly conceived technology by giving an example of a traffic situation in which a recognition sensor cannot detect a major traffic element due to obstruction by an obstacle.

[0012] 1 and 2 show an example of traffic conditions (surrounding traffic conditions X) ahead of a vehicle 100a. FIG. 1 illustrates the surrounding traffic conditions X as seen from the driver of the vehicle 100a, and FIG. 2 illustrates the surrounding traffic conditions X as seen from above the road on which the vehicle 100a is traveling. The vehicle (host vehicle) 100a is traveling on a road La with one lane in each direction. The road La is composed of a traveling lane La1 in which the vehicle 100a is traveling and an oncoming lane La2 that runs along the traveling lane La2 via a center line. An intersection IS is provided on the road La ahead of the vehicle 100a. The road La intersects with a road Lb at the intersection IS. The road Lb is, for example, a road with one lane in each direction. Pedestrian crossings CW1 are provided on the road La both before and after the intersection IS in relation to the vehicle 100a. Further, a traffic light TL1 is provided on the road La at the rear of the intersection IS in relation to the vehicle 100a, and a traffic light TL2 is provided on the front side of the intersection IS in relation to the vehicle 100a. A crosswalk CW2 is provided on the road Lb on both the left and right sides of the intersection IS.

[0013] A large truck (vehicle 100b) is traveling ahead of vehicle 100a in driving lane La1. Meanwhile, vehicles 100c and 100d are traveling in oncoming lane La2. Vehicle 100b forms a blind spot area BA ahead of vehicle 100b (i.e., at intersection IS) as seen from the driver of vehicle 100a. Traffic light TL1 and vehicle 100d are located in blind spot area BA and cannot be seen by the driver of vehicle 100a.

[0014] 3 shows an example of data (external environment recognition data) acquired by an acquisition unit 311 (described later) of the vehicle 100a in a surrounding traffic situation X. The external environment recognition data may include, for example, the following various types of data. The external environment recognition data does not include data indicating the presence of the vehicle 100c. Furthermore, in the external environment recognition data, the presence of traffic lights TL1 and TL2 is indicated by at least data obtained from map data, but data regarding the lighting status of traffic lights TL1 and TL2 is non-observation data Non-Obs, which indicates that the data is unknown.

[0015] (1) Name, position, and speed of vehicle 100a (SbjCar) (2) Names of vehicles 100b (ObjCar1), 100c (ObjCar2), and 100d (ObjCar3) (3) Relative positions and relative speeds of vehicles 100b (ObjCar1), 100c (ObjCar2), and 100d (ObjCar3) with respect to vehicle 100a (SbjCar) (4) Names and structure of roads La and Lb (number of lanes, presence or absence of a central divider, etc.) (5) Names of intersections IS, traffic lights, and crosswalks (6) Relative positions of intersections IS, traffic lights, and crosswalks with respect to vehicle 100a (SbjCar) (7) Name and type of intersection IS (presence or absence of a time difference, presence or absence of pedestrian-vehicle separation, etc.) (8) Name, type (whether or not there is a right or left turn sign, etc.) and lighting status (green, yellow, red, etc.) of the traffic light

[0016] 4 shows an example of a traffic context obtained by interpreting the surrounding traffic situation X in the scenario creation unit 312 (described later) of the vehicle 100a. The traffic context of the surrounding traffic situation X is composed of multiple sentences (subject, predicate, object), for example, as shown in FIG.

[0017] 4, "Scene1, HasTime, 0 sec" means that "the transition time of scene 1 is 0 seconds." "SbjCar, isRunningOn, subjectLane" means that "vehicle 100a is traveling on traveling lane La1." "subjectLane, nextRoadSegment, SignalIntersection" means that "traveling lane La1 has a traffic light intersection IS as a road segment ahead of vehicle 100a."

[0018] "ObjCar1, isRunningOn, subjectLane" means "vehicle 100b is traveling in lane La1." "ObjCar1, HasSpeed, Constant" means "vehicle 100b is traveling at a constant speed." "ObjCar1, ApproachTo, SignalIntersection" means "vehicle 100b is approaching intersection IS." "ObjCar1, createBlindIn, SignalIntersection" means "vehicle 100b has a blind spot at intersection IS."

[0019] "ObjCar2, isRunningOn, OppositeDirectionLane" means that "vehicle 100c is traveling in the oncoming lane La2." "ObjCar2, HasSpeed, Constant" means that "vehicle 100c is traveling at a constant speed." "ObjCar2, EnterIn, SignalIntersection" means that "vehicle 100c is entering intersection IS."

[0020] However, in the external recognition data, data regarding the lighting status of traffic lights is non-observation data (Non-Obs). Therefore, data regarding the lighting status of traffic lights is missing in the traffic context of the surrounding traffic situation X. In the traffic context of the surrounding traffic situation X, the missing data state is expressed by "???", for example, as shown in FIG. 4. Note that FIG. 4 exemplifies a sentence including "???", such as "SignalIntersection, ???, Carmoving". "SignalIntersection, ???, Carmoving" means that the intersection IS is in a state where traffic lights La1 and La2 are lit with ??? and a vehicle is moving. Hereinafter, the "???" in "SignalIntersection, ???, Carmoving" will be referred to as a missing element ME. Furthermore, "SignalIntersection" and "Carmoving" in "SignalIntersection, ???, Carmoving" are referred to as a non-missing element UME.

[0021] As described above, the traffic context of the surrounding traffic situation X includes a missing element ME. Therefore, there is a possibility that logical inference using the traffic context of the surrounding traffic situation X will not work. Therefore, the inventors of the present application have devised a technology that enables logical inference to work even when a traffic context including a missing element ME is obtained. An information processing device, a vehicle control device, and a vehicle for realizing this technology will be described in detail below.

[0022] 2. Embodiment [Configuration Example] A vehicle 1 according to an embodiment of the present disclosure will be described. Fig. 5 shows a schematic configuration example of the vehicle 1 according to the present embodiment. The vehicle 1 corresponds to a specific example of a "vehicle" according to an embodiment of the present disclosure.

[0023] The vehicle 1 is capable of traveling by being driven by a prime mover 50 (engine or motor). As shown in Fig. 5 , the vehicle 1 includes, for example, a sensor unit 10, a communication unit 20, a control unit 30, a storage unit 40, the prime mover 50, a brake 60, an EPS (Electric Power Steering) motor 70, and a notification unit 80.

[0024] The sensor unit 10 is configured to include various sensors mounted on the vehicle 1. The sensor unit 10 is configured to include, for example, an accelerator opening sensor, a vehicle speed sensor, an acceleration sensor, an angular velocity sensor, a steering angle sensor, a steering torque sensor, and a brake torque sensor. The sensor unit 10 may also include sensors other than those described above.

[0025] The accelerator position sensor is capable of detecting the accelerator position from the amount of depression of the accelerator pedal, and is capable of outputting time-series data (accelerator position data) regarding the detected accelerator position to the control unit 30.

[0026] The vehicle speed sensor is capable of detecting the speed (vehicle speed) of the vehicle 1. The vehicle speed sensor is capable of outputting time series data (vehicle speed data) about the detected vehicle speed to the control unit 30. The acceleration sensor is capable of detecting acceleration applied to the vehicle 1. The acceleration sensor is capable of outputting time series data (acceleration data) about the detected acceleration in three directions to the control unit 30. The angular velocity sensor is capable of detecting the angular velocity of the vehicle 1. The angular velocity sensor is capable of outputting time series data (angular velocity data) about the detected three angular velocities (yaw angular velocity, roll angular velocity, and pitch angular velocity) to the control unit 30.

[0027] The steering angle sensor is capable of detecting the steering angle of the steering wheel of the vehicle 1 (steering angle). The steering angle sensor is capable of outputting time-series data (steering angle data) about the detected steering angle to the control unit 30. The steering torque sensor is capable of detecting the steering torque generated by the driver's steering wheel operation. The steering torque sensor is capable of outputting time-series data (steering torque data) about the detected steering torque to the control unit 30. The brake torque sensor is capable of detecting the braking force (torque of the brakes 60) relative to the brake pressure of the vehicle 1. The brake torque sensor is capable of outputting time-series data (braking force data) about the detected braking force to the control unit 30.

[0028] The sensor unit 10 further includes a stereo camera mounted on the vehicle 1 and a driving environment detection unit. The stereo camera is an autonomous sensor that senses the real space around the vehicle 1. The stereo cameras are, for example, arranged at symmetrical positions on either side of the central part in the width direction of the vehicle 1, and are capable of capturing stereo images of the area in front of the vehicle 1 from different viewpoints. The stereo cameras are capable of outputting image data Da (a pair of stereo image data) obtained by capturing images to the control unit 30.

[0029] The stereo camera is capable of generating distance image data Db calculated from the amount of displacement between corresponding objects based on image data Da (a pair of stereo image data) obtained by capturing images. The driving environment detection unit is capable of, for example, calculating lane markings that divide the road around the vehicle 1 based on the distance image data Db. The driving environment detection unit is also capable of calculating the road curvature of the markings that divide the left and right sides of the road (driving lane) on which the vehicle 1 is traveling, and the width between the left and right markings (vehicle width). The driving environment detection unit is also capable of performing predetermined pattern matching on the distance image data Db to detect lanes and three-dimensional objects such as structures present around the vehicle 1.

[0030] Here, the detection of a three-dimensional object by the driving environment detection unit includes, for example, detecting the type of the three-dimensional object, the distance to the three-dimensional object, the speed of the three-dimensional object, and the relative speed between the three-dimensional object and the vehicle (host vehicle). Examples of three-dimensional objects to be detected include traffic lights, intersections, road signs, stop lines, other vehicles, pedestrians, bicycles, and buildings. Examples of buildings include detached houses, apartment complexes (condominiums), commercial facilities, factories, and signs. The driving environment detection unit is capable of outputting driving environment information around the vehicle 1, including the thus acquired information on the three-dimensional object, to the control unit 30.

[0031] The communication unit 20 can acquire data to supplement data that cannot be obtained from the image data Da and the distance image data Db, for example, through vehicle-to-vehicle communication, road-to-vehicle communication, and satellite communication. The communication unit 20 can output the acquired data to the control unit 30.

[0032] The communication unit 20 can acquire data (e.g., vehicle position and vehicle speed) obtained by other vehicles through, for example, vehicle-to-vehicle communication. The communication unit 20 can receive positioning signals transmitted from multiple positioning satellites through, for example, satellite communication.

[0033] The communication unit 20 is capable of, for example, acquiring road map data around the vehicle 1 from a control device that is capable of successively integrating and updating road map data transmitted from each vehicle via road-to-vehicle communication and transmitting the updated road map data to each vehicle. The road map data is, for example, made up of high-precision road map data (dynamic map) and includes static information and quasi-static information that mainly constitute road information, and quasi-dynamic information and dynamic information that mainly constitute traffic information.

[0034] Static information that makes up road information includes, for example, information that requires updates within one month, such as roads, structures on roads, structures around roads, lane information, road surface information, and permanent traffic control information. "Roads" include, for example, road locations and shapes, intersections, and road attributes (e.g., national roads, prefectural roads, city roads, private roads, priority roads, non-priority roads, general roads, and expressways). "Structures on roads" include, for example, traffic signs, crosswalks, traffic lights, convex mirrors, pedestrian bridges, bus stops, and garbage collection stations. "Structures around roads" include, for example, various buildings and parks.

[0035] The semi-static information that constitutes the road information is composed of information that needs to be updated every hour, such as traffic regulation information due to road construction or events, wide-area weather information, and traffic congestion forecasts.

[0036] The semi-dynamic information that constitutes traffic information is composed of information that requires updating within one minute, such as the actual traffic congestion situation at the time of observation, driving restrictions, temporary driving obstructions such as fallen objects and obstacles, actual accident conditions, and narrow-area weather information.

[0037] The dynamic information that constitutes the traffic information is composed of information that needs to be updated every second, such as information sent and exchanged between mobile bodies, information on currently displayed traffic signals, information on pedestrians and bicycles at intersections, information on vehicles traveling on roads, etc. Such road map data is maintained and updated periodically until the next information is received from each vehicle, and the updated road map data is transmitted to each vehicle as appropriate via the communication unit 20.

[0038] The storage unit 40 is configured, for example, by a non-volatile memory, such as an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, a resistance change memory, etc. For example, as shown in Fig. 5, the storage unit 40 stores a road map DB 41, a knowledge graph DB 42, and a knowledge space DB 43. The storage unit 40 stores, for example, data obtained by calculations performed by the risk prediction unit 31 (described later).

[0039] The road map DB 41 is a large-capacity storage medium such as an HDD, and stores high-precision road map data (dynamic map). This high-precision road map data includes, for example, static information and quasi-static information that mainly constitute road information, and quasi-dynamic information and dynamic information that mainly constitute traffic information.

[0040] The knowledge graph DB 42 and knowledge space DB 43 are knowledge data that structure traffic rules, inference rules, and general knowledge, respectively, in a manner that can be used to analyze traffic conditions. Traffic conditions represent the relationship between the driving environment and the behavior of each traffic participant. Traffic rules refer to rules that traffic participants must follow in order to participate in traffic in a compliant manner. Inference rules refer to rules that can be used to derive conclusions about unknown things based on known facts. General knowledge refers to the experience and collective knowledge that people implicitly know.

[0041] The knowledge graph DB 42 stores knowledge data that describes multiple known traffic scenarios in a graph structure. A known traffic scenario refers to an expected traffic scenario or a past traffic scenario. A traffic scenario refers to a summary of traffic conditions that change from moment to moment. In the knowledge graph DB 42, known traffic scenarios are described as condition terms, and possible events (e.g., dangerous events) are described as result terms. In a traffic scenario, an identifier (ID) is assigned to each traffic participant, and the position and speed of each traffic participant are associated with each traffic participant. Traffic participants may include, for example, the subject vehicle (SbjCar), other vehicles (ObjCar), bicycles (Bicycles), and pedestrians (Pedestrians). In a traffic scenario, the lane and lane type of each vehicle are also associated with each vehicle.

[0042] A known traffic scenario includes multiple known traffic scenes in an integrated time series. A known traffic scene refers to an expected traffic scene or a past traffic scene. A traffic scene refers to an organized traffic situation at a certain moment. In a traffic scenario, multiple traffic scenes are associated with a temporal order. A known traffic scenario includes multiple traffic elements. Traffic elements may include, for example, object types, road structures, object speeds, object positions, or object states. Object types may include traffic participants (e.g., vehicles (passenger cars), motorcycles, bicycles, and pedestrians). Road structures may include, for example, intersections, traffic lights, and driving lanes. Object speeds may include, for example, a constant speed, deceleration, and acceleration. Object positions may include, for example, identifiers indicating that an object is moving in a certain location, creating a blind spot in a certain location, approaching a certain location or object, stopping in a certain location, indicating a positional relationship with a certain location or object, being on the verge of a collision with an object, and entering a certain location. The state of the object may include, for example, the lighting state of a traffic light. The traffic element may further include, for example, a transition time between scenes. The transition time between scenes refers to the difference between the time of a second scene and the time of a second scene in two consecutive scenes (a first scene and a second scene) over time.

[0043] A certain known traffic scenario (hereinafter referred to as "known traffic scenario I") in the knowledge graph DB 42 includes a plurality of known traffic scenes (scene a, scene b, scene c) in an integrated time series, as shown in FIG. 6, for example. In the known traffic scenario I, the plurality of known traffic scenes (scene a, scene b, scene c) are linked to a temporal order. In the known traffic scenario I, each known traffic scene (scene a, scene b, scene c) includes a plurality of traffic elements. The known traffic scene (scene a) includes a plurality of traffic elements (traffic elements a1, a2, a3, a4, etc.), as shown in FIG. 6, for example. The known traffic scene (scene b) includes a plurality of traffic elements (traffic elements b1, b2, b3, b4, etc.), as shown in FIG. 6, for example. The known traffic scene (scene c) includes a plurality of traffic elements (traffic elements c1, c2, c3, c4, etc.), as shown in FIG. 6, for example.

[0044] In the knowledge graph DB 42, each known traffic scenario is composed of a directed graph including multiple nodes (entities) and multiple edges (relationships) that associate two nodes (entities). A node (entity) corresponds to a known traffic scene or traffic element. In each known traffic scenario, each entity and each relation is associated with a label, and two entities and one relation that associates these two entities form a sentence (subject, predicate, object).

[0045] The knowledge space DB 43 has knowledge data in which a plurality of known traffic scenarios contained in the knowledge graph DB 42 are numerically expressed (e.g., spatially expressed). The knowledge space DB 43 has a knowledge graph 43A in which the known traffic scenarios I contained in the knowledge graph DB 42 are expressed as spatial vectors, as shown in FIG. 7, for example. The knowledge graph 43A includes a plurality of identifiers (IDs), as shown in FIG. 7, for example, and is table data in which attributes, labels, and spatial vectors are associated with each identifier (ID). The identifiers (IDs) are used to identify each entity and each relation in the known traffic scenario I, and one ID is assigned to each entity and each relation.

[0046] Examples of labels that can be assigned to entities and relations include a label indicating an object type, a label indicating a road structure, a label indicating an object speed, a label indicating an object position, and a label indicating an object state.

[0047] Examples of labels indicating object types include the following: Label indicating a main vehicle (SbjCar) Label indicating a secondary vehicle (ObjCar1, ObjCar2, ...) Label indicating a motorcycle (Motorcycle1, Motorcycle2, ...) Label indicating a pedestrian (Pedestrian1, Pedestrian12, ...) Label indicating a bicycle (Bicycle1, Bicycle2, ...)

[0048] Examples of labels indicating road structures include the following: A label indicating an intersection (Intersection) A label indicating an intersection with a traffic light (SignalIntersection) A label indicating an intersection without a traffic light (noSignalIntersection) A label indicating a traffic light (TrafficSignal) A label indicating the lane in which the main vehicle is traveling (SubjectLane) A label indicating the lane whose traveling direction is opposite to that of the main vehicle (OppositeDirectionLane) A label indicating the lane that intersects with the lane in which the main vehicle is traveling (CrossRoad)

[0049] The labels indicating the object speed include, for example, the following labels: A label indicating a constant speed (Constant) A label indicating deceleration (Deceleration)

[0050] Examples of labels indicating object positions include the following: Label indicating where an object is moving (IsRunningOn) Label indicating that an object has speed (HasSpeed) Label indicating that an object is creating a blind spot (CreateBlindIn) Label indicating that an object is approaching (ApproachTo) Label indicating that an object is stopped (stopAt) Label indicating the relative position of an object or road structure (nextRoadSegment) Label indicating that an object is on the verge of collision (NearCrachTo) Label indicating that an object is entering (EnterIn) Label indicating that an object is moving away (Leave)

[0051] Examples of labels that indicate the state of an object include the following: Labels that indicate the lighting state of a traffic light (Light_turns_green, Light_turns_yellow, Light_turns_red)

[0052] In the knowledge space DB 43, a set of (subject, predicate, object) is converted into a numerical expression using an "embedding algorithm" so that a specific operation is established for the set of (subject, predicate, object) defined in the knowledge graph DB 42. Examples of the "embedding algorithm" include TransE and RESCAL. Here, the numerical expression is, for example, a space vector as shown in FIG. 7. When the numerical expression is a space vector, the "specific operation" means that when the entity corresponding to the subject is a first vector value, the relation corresponding to the predicate is a second vector value, and the entity corresponding to the object is a third vector value, then "first vector value + second vector value = third vector value."

[0053] For example, in a set (subject, predicate, object) called (SbjCar IsRunningOn subjectLane), the vector value of SbjCar is [0,1,2], the vector value of IsRunningOn is [1,1,1], and the vector value of subjectLane is [1,2,3]. In this case, [0,1,2] + [1,1,1] = [1,2,3] holds.

[0054] The "embedded algorithm" is stored in, for example, the storage unit 40. The calculation process for deriving a numerical expression using the "embedded algorithm" is executed, for example, in the risk prediction unit 31 described below. The numerical expression may be, for example, a two-dimensional vector or a vector of four or more dimensions. The numerical expression may also be an expression other than a vector, for example, a matrix.

[0055] The knowledge space DB 44 may include a set of (subject, predicate, object) indicating the transition time of a certain known traffic scene. For example, a set of (subject, predicate, object) indicating the transition time of a certain known traffic scene may include (Scene, HasTime, 0 sec) indicating the transition time of a known traffic scene (scene a). "Scene, HasTime, 0 sec" means "the transition time of scene a is 0 seconds."

[0056] 8 shows an example of a known traffic situation (hereinafter referred to as "known traffic situation A"). The known traffic scene (scene a) in the knowledge graph DB 42 is generated based on a traffic context (see FIG. 9) obtained by interpreting data acquired by the data acquisition unit 311 (described later) in the known traffic situation A shown in FIG. 8.

[0057] In known traffic situation A, a vehicle (host vehicle) 100a is traveling on a road La with one lane in each direction. The vehicle (host vehicle) 100a is an example of a vehicle 1. The road La is composed of a driving lane La1 in which the vehicle 100a is traveling and an oncoming lane La2 that runs along the driving lane La2 via a center line. An intersection IS is provided on the road La ahead of the vehicle 100a. The road La intersects with a road Lb at the intersection IS. The road Lb is, for example, a road with one lane in each direction. A crosswalk CW1 is provided on the road La both before and behind the intersection IS in relation to the vehicle 100a. A traffic light TL1 is also provided on the road La behind the intersection IS in relation to the vehicle 100a, and a traffic light TL2 is provided before the intersection IS in relation to the vehicle 100a. A crosswalk CW2 is provided on the road Lb on both the left and right sides of the intersection IS.

[0058] A passenger car (vehicle 100e) is traveling ahead of vehicle 100a in driving lane La1. Meanwhile, vehicles 100c and 100d are traveling in oncoming lane La2. From the driver of vehicle 100a's perspective, there is no blind spot area BA (as shown in FIG. 2 ) ahead of vehicle 100e (i.e., at intersection IS). Therefore, the driver of vehicle 100a can see traffic light TL1 and vehicle 100d.

[0059] 9 shows an example of a traffic context obtained by interpreting known traffic situation A. The traffic context of known traffic situation A is composed of multiple sentences (subject, predicate, object), for example, as shown in FIG.

[0060] 9, "Scenea, HasTime, 0sec" means that "the transition time of scene a is 0 seconds." "SbjCar, isRunningOn, subjectLane" means that "vehicle 100a is traveling on traveling lane La1." "subjectLane, nextRoadSegment, SignalIntersection" means that "traveling lane La1 has a traffic light intersection IS as a road segment ahead of vehicle 100a."

[0061] "ObjCar1, isRunningOn, subjectLane" means that "the vehicle 100e is traveling in the traveling lane La1." "ObjCar1, HasSpeed, Constant" means that "the vehicle 100e is traveling at a constant speed." "ObjCar1, ApproachTo, SignalIntersection" means that "the vehicle 100e is approaching the intersection IS."

[0062] "ObjCar2, isRunningOn, OppositeDirectionLane" means that "vehicle 100c is traveling in the oncoming lane La2." "ObjCar2, HasSpeed, Constant" means that "vehicle 100c is traveling at a constant speed." "ObjCar2, EnterIn, SignalIntersection" means that "vehicle 100c is entering intersection IS."

[0063] "ObjCar3, isRunningOn, OppositeDirectionLane" means that "vehicle 100d is traveling in the oncoming lane La2." "ObjCar3, HasSpeed, Constant" means that "vehicle 100d is traveling at a constant speed." "ObjCar3, ApproachTo, SignalIntersection" means that "vehicle 100d is approaching intersection IS." "SignalIntersection, Light_turns_green, Carmoving" means that "intersection vehicle 100d is in a moving state with traffic lights La1 and La2 lit green."

[0064] 10 shows an example of a known traffic situation (hereinafter referred to as "known traffic situation B") following known traffic situation A. Known traffic situation B is, for example, a traffic situation one second after known traffic situation A. A known traffic scene (scene b) in the knowledge graph DB 42 is generated based on a traffic context (see FIG. 11) obtained by interpreting data acquired by a data acquisition unit 311 (described later) in known traffic situation B shown in FIG. 10.

[0065] In known traffic situation B, vehicle 100a travels in driving lane La1 and is therefore slightly closer to intersection IS than in scene a. Vehicle 100e travels in driving lane La1 and therefore enters intersection IS, passing through crosswalk CW1 at the back of intersection IS as seen from the driver of vehicle 100a. Vehicle 100c continues to travel in oncoming lane La2. Vehicle 100d has entered intersection IS and is turning right at intersection IS.

[0066] Fig. 11 shows an example of a traffic context obtained by interpreting known traffic situation B. The traffic context of known traffic situation B is composed of multiple sentences (subject, predicate, object), for example, as shown in Fig. 11 .

[0067] 11, "Sceneb, HasTime, 1sec" means that "the transition time of scene b is 1 second." "SbjCar, isRunningOn, subjectLane" means that "vehicle 100a is traveling on traveling lane La1." "subjectLane, nextRoadSegment, SignalIntersection" means that "traveling lane La1 has a traffic light intersection IS as a road segment ahead of vehicle 100a."

[0068] "ObjCar1, isRunningOn, subjectLane" means that "the vehicle 100e is traveling in the traveling lane La1." "ObjCar1, HasSpeed, Constant" means that "the vehicle 100e is traveling at a constant speed." "ObjCar1, EnterIn, SignalIntersection" means that "the vehicle 100e is entering the intersection IS."

[0069] "ObjCar2, isRunningOn, OppositeDirectionLane" means that "vehicle 100c is traveling in the oncoming lane La2." "ObjCar2, HasSpeed, Constant" means that "vehicle 100c is traveling at a constant speed." "ObjCar2, leave, SignalIntersection" means that "vehicle 100c has left intersection IS."

[0070] "ObjCar3, TurnRightIn, SignalIntersection" means that "vehicle 100d is turning right at intersection IS." "SignalIntersection, Light_turns_green, Carmoving" means that "intersection vehicle 100d is in a moving state with traffic lights La1 and La2 lit green."

[0071] 12 shows an example of a known traffic situation (hereinafter referred to as "known traffic situation C") following known traffic situation B. Known traffic situation C is, for example, a traffic situation one second after known traffic situation B. A known traffic scene (scene c) in the knowledge graph DB 42 is generated based on a traffic context (see FIG. 13) obtained by interpreting data acquired by a data acquisition unit 311 (described later) in known traffic situation C shown in FIG. 12 (various data acquired from the sensor unit 10, various data acquired from the outside via the communication unit 20, and various control signals to various devices of the vehicle 1).

[0072] In known traffic situation C, vehicle 100a is traveling in lane La1 and is entering intersection IS. Vehicle 100c is traveling in oncoming lane L2 and is moving away from intersection IS. Vehicle 100d is turning right at intersection IS and is approaching crosswalk CW2 on road Lb.

[0073] Fig. 13 shows an example of a traffic context obtained by interpreting a known traffic situation C. The traffic context of the known traffic situation C is composed of, for example, multiple sentences (subject, predicate, object) as shown in Fig. 13 .

[0074] 13, "Scenec, HasTime, 2sec" means that "the transition time of scene c is 2 seconds." "SbjCar, isRunningOn, subjectLane" means that "vehicle 100a is traveling on traveling lane La1." "subjectLane, nextRoadSegment, SignalIntersection" means that "traveling lane La1 has a traffic light intersection IS as a road segment ahead of vehicle 100a."

[0075] "ObjCar2, isRunningOn, OppositeDirectionLane" means that "vehicle 100c is traveling in the oncoming lane La2." "ObjCar2, HasSpeed, Constant" means that "vehicle 100c is traveling at a constant speed." "ObjCar2, leave, SignalIntersection" means that "vehicle 100c has left intersection IS."

[0076] "ObjCar3, TurnRightIn, SignalIntersection" means "vehicle 100d is turning right at intersection IS." "ObjCar3, NearCrashTo, SbjCar" means "vehicle 100d is about to collide with vehicle 100a." "SignalIntersection, Light_turns_green, Carmoving" means "intersection vehicle 100d is in a moving state with traffic lights La1 and La2 lit green."

[0077] The control unit 30 is capable of controlling the entire vehicle 1. The control unit 30 is, for example, a so-called ECU (Electronic Control Unit) and is configured to include, for example, one or more processors and one or more memories. The control unit 30 may be configured to include, for example, a CPU (Central Processing Unit). In this case, the control unit 30 is capable of controlling the entire vehicle 1 by, for example, executing a program stored in a storage unit. The control unit 30 corresponds to a specific example of a "vehicle control device" according to an embodiment of the present disclosure.

[0078] The control unit 30 includes, for example, a locator unit. The locator unit is capable of acquiring the position coordinates of the vehicle 1 based on the positioning signal received through the communication unit 20. The locator unit is capable of estimating the vehicle's position on a road map by map-matching the acquired position coordinates with route map data. Based on the acquired position coordinates of the vehicle 1, the locator unit is capable of acquiring map data of a predetermined range including the vehicle 1 from map data stored in a road map DB (database) 41 (described later).

[0079] In an environment where it is not possible to receive valid positioning signals from positioning satellites due to reduced sensitivity, such as when driving inside a tunnel, the locator unit can switch to autonomous navigation, which estimates the vehicle's position based on the vehicle speed, angular velocity, and longitudinal acceleration detected by sensor unit 10, and estimate the vehicle's position on a road map.

[0080] As described above, the locator unit estimates the position of vehicle 1 (vehicle position) on a road map based on the positioning signal received through communication unit 20 or information detected by sensor unit 10, and is then able to determine the road type, etc. of the road on which vehicle 1 is traveling based on the estimated vehicle position on the road map.

[0081] The locator unit is capable of updating the road map data stored in the road map DB 41 to the latest version using road map data acquired through external communication (roadside-to-vehicle communication and vehicle-to-vehicle communication) via the communication unit 20. This information update is performed not only for static information but also for quasi-static information, quasi-dynamic information, and dynamic information. As a result, the road map data includes road information and traffic information acquired through communication with the outside of the vehicle, and information on moving objects such as vehicles traveling on roads is updated in approximately real time.

[0082] The locator unit verifies road map data based on the traveling environment information recognized as described above, and is able to update the road map data stored in the road map DB 41 to the latest version. This information update is performed not only on static information, but also on semi-static information, semi-dynamic information, and dynamic information. As a result, information on moving objects such as vehicles traveling on roads recognized as described above is updated in real time.

[0083] The control unit 30 has a risk prediction unit 31, for example, as shown in FIG. 5 . The risk prediction unit 31 is capable of estimating whether or not the vehicle 1 is in a dangerous traffic situation. If the estimation result indicates that the vehicle 1 is in a dangerous traffic situation, the risk prediction unit 31 is capable of predicting risk based on a dangerous traffic scenario that the vehicle 1 is likely to face. The risk prediction unit 31 is capable of outputting the content of the predicted risk (risk event) to the driving control unit 32. The risk prediction unit 31 corresponds to a specific example of an "information processing device" according to an embodiment of the present disclosure.

[0084] 5, the risk prediction unit 31 includes a data acquisition unit 311, a scenario creation unit 312, a similarity determination unit 313, and an intensity setting unit 314. The data acquisition unit 311 corresponds to a specific example of an "acquisition unit" according to an embodiment of the present disclosure. The scenario creation unit 312, the similarity determination unit 313, and the intensity setting unit 314 correspond to a specific example of a "processing unit" and a "traffic context acquisition unit" according to an embodiment of the present disclosure.

[0085] The data acquisition unit 311 is capable of periodically acquiring data on the situation or state of the vehicle 1. Specifically, the data acquisition unit 311 is capable of acquiring various data acquired from the sensor unit 10, various data acquired from the outside via the communication unit 20, and various control signals for various devices of the vehicle 1. The data acquisition unit 311 is further capable of acquiring map data of the surroundings of the vehicle 1 from the road map DB 41 of the storage unit 40.

[0086] The road data around the vehicle 1 included in the various data obtained from the sensor unit 10, the various data obtained from the outside via the communication unit 20, the various control signals for the various devices of the vehicle 1, and the map data around the vehicle 1 obtained from the road map DB 41 includes, for example, the names (labels), positions, number of lanes, presence or absence of a center divider, etc. of roads La and Lb, the names (labels), positions, types, etc. of intersections IS, the names (labels), positions, lighting status, etc. of traffic lights TL1 and TL2, and the names (labels) and positions of pedestrian crossings CW, etc. These data correspond to a specific example of "road data around the vehicle" according to an embodiment of the present disclosure.

[0087] Traffic data about the vehicle 1 and traffic participants around the vehicle 1, which are included in the various data obtained from the sensor unit 10, the various data obtained from the outside via the communication unit 20, the various control signals for the various devices of the vehicle 1, and the map data about the vehicle 1 obtained from the road map DB 41, are, for example, the name (label), position, speed, etc. of the vehicle 1, and the names (labels), positions, speeds, etc. of the traffic participants around the vehicle 1. These data correspond to specific examples of "traffic data about the vehicle and traffic participants around the vehicle."

[0088] The scenario creation unit 312 , the similarity determination unit 313 , and the intensity setting unit 314 are capable of performing information processing using the data acquired by the data acquisition unit 311 .

[0089] The scenario creation unit 312 is capable of generating external environment recognition data such as that shown in Fig. 3 based on the road data and traffic data acquired by the data acquisition unit 311. The scenario creation unit 312 is further capable of interpreting a traffic context such as that shown in Fig. 4 based on the generated external environment recognition data.

[0090] The scenario creation unit 312 is capable of generating new external environment recognition data based on the road data and traffic data acquired by the data acquisition unit 311 one second after the surrounding traffic condition X, for example, and interpreting a new traffic context based on the newly generated external environment recognition data. Assume that the vehicle 1 (vehicle 100a) is in a certain traffic condition (hereinafter referred to as "surrounding traffic condition Y") one second after the surrounding traffic condition X, as shown in FIG. 14. At this time, the scenario creation unit 312 is capable of interpreting a traffic context based on the data acquired by the data acquisition unit 311 in the surrounding traffic condition Y. The scenario creation unit 312 is capable of generating a traffic context such as that shown in FIG. 15, for example.

[0091] 15, "Scene2, HasTime, 1sec" means that "the transition time of scene 2 is 1 second." "SbjCar, isRunningOn, subjectLane" means that "vehicle 100a is traveling on traveling lane La1." "subjectLane, nextRoadSegment, SignalIntersection" means that "traveling lane La1 has a traffic light intersection IS as a road segment ahead of vehicle 100a."

[0092] "ObjCar1, isRunningOn, subjectLane" means that "vehicle 100b is traveling in lane La1." "ObjCar1, HasSpeed, Constant" means that "vehicle 100b is traveling at a constant speed." "ObjCar1, EnterIn, SignalIntersection" means that "vehicle 100b is entering intersection IS." "ObjCar1, createBlindIn, SignalIntersection" means that "vehicle 100b has created a blind spot at intersection IS."

[0093] "ObjCar3, isRunningOn, OppositeDirectionLane" means that "vehicle 100d is traveling in the oncoming lane La2." "ObjCar3, TurnRightIn, SignalIntersection" means that "vehicle 100d is turning right at intersection IS." "SignalIntersection, ???, Carmoving" means that at intersection IS, traffic lights La1 and La2 are lit with ??? and a car is moving." In "SignalIntersection, ???, Carmoving," "???" is a missing element ME, and "SignalIntersection" and "Carmoving" are non-missing elements UME. In this way, the traffic context of the surrounding traffic situation Y includes a missing element ME, just like the traffic context of the surrounding traffic situation X.

[0094] The scenario creation unit 312 is capable of creating a numerical representation of the traffic context obtained in this manner using the knowledge space DB 43. The scenario creation unit 312 is capable of determining whether one of the multiple traffic elements constituting the traffic context is missing. When the scenario creation unit 312 detects a missing element ME in a sentence included in the traffic context, it is capable of reading, from the knowledge space DB 43, the numerical representations of multiple non-missing elements UME other than the missing element ME among the multiple traffic elements constituting the sentence including the missing element ME. The scenario creation unit 312 is capable of deriving a first numerical representation of the missing element ME by performing a calculation using the numerical representations of the multiple non-missing elements UME that have been read. The scenario creation unit 312 is capable of comparing the derived first numerical representation of the missing element ME with multiple second numerical representations in the knowledge space DB 43 and setting the label of the second numerical representation that has a high similarity to the first numerical representation as the label of the missing element ME.

[0095] The scenario creation unit 312 may be capable of determining the similarity of the second numerical expression to the first numerical expression of the missing element ME as a completion accuracy that is an index of the likelihood of completion. In this case, when the completion accuracy is equal to or greater than a predetermined threshold, the scenario creation unit 312 may be capable of determining the label of the missing element ME as the label of the second numerical expression that has a high similarity to the first numerical expression of the missing element ME.

[0096] The scenario creation unit 312 is capable of determining, for example, whether a missing element ME is included in each sentence included in the traffic context of the surrounding traffic condition X or the surrounding traffic condition Y. For example, when the scenario creation unit 312 detects a missing element ME in one sentence (SignalIntersection, ???, Carmoving) included in the traffic context of the surrounding traffic condition X or the surrounding traffic condition Y, the scenario creation unit 312 is capable of reading out the numerical expressions of the two non-missing elements UME (SignalIntersection, Carmoving) from the knowledge space DB 43. For example, the scenario creation unit 312 is capable of deriving a first numerical expression of the missing element ME by performing a calculation using the numerical expressions of the two non-missing elements UME (SignalIntersection, Carmoving) that have been read out. For example, as shown in Figure 16(A), the scenario creation unit 312 is able to obtain the numerical representation of the missing element ME (space vector [3,0,0]) by subtracting the numerical representation of SignalIntersection (space vector [2,4,1]) from the numerical representation of Carmoving (space vector [5,4,1]). For example, as shown in Figure 16(B), the scenario creation unit 312 is able to use the label (Light_Turns_green) of the numerical representation (space vector [3,0,0]) that has a high degree of similarity in relation to the numerical representation of the missing element ME (space vector [3,0,0]) among the multiple numerical representations in the knowledge space DB 43 as the label of the missing element ME.

[0097] The scenario creation unit 312 can generate two time-series surrounding traffic scenes (scene 1, scene 2) having a graph structure from the two time-series traffic contexts in which the missing elements ME have been supplemented in this way. Surrounding traffic scene 1 is a traffic scene corresponding to surrounding traffic situation X. Surrounding traffic scene 2 is a traffic scene corresponding to surrounding traffic situation Y. The scenario creation unit 312 can further create surrounding traffic scenario II by integrating the two generated time-series surrounding traffic scenes (scene 1, scene 2), for example, as shown in FIG. 17 .

[0098] The surrounding traffic scenario II includes a plurality of integrated time-series surrounding traffic scenes (scene 1, scene 2), as shown in FIG. 17 . In the surrounding traffic scenario II, the surrounding traffic scenes (scene 1, scene 2) are associated with a time sequence. In the surrounding traffic scenario II, each surrounding traffic scene (scene 1, scene 2) includes a plurality of traffic elements. For example, as shown in FIG. 17 , the surrounding traffic scene (scene 1) includes a plurality of traffic elements (traffic elements α1, α2, α3, α4, etc.). For example, as shown in FIG. 17 , the surrounding traffic scene (scene 2) includes a plurality of traffic elements (traffic elements β1, β2, β3, β4, etc.).

[0099] A surrounding traffic scenario is composed of a directed graph including multiple nodes (entities) and multiple relations (edges) that associate two nodes (entities). In the surrounding traffic scenario, each entity and each relation is associated with a label, and two entities and one relation that associates these two entities form a sentence (subject, predicate, object).

[0100] The scenario creation unit 312 is capable of, for example, reading out, for each traffic element (e.g., α1, α2, α3, α4, etc.) included in the surrounding traffic scene (scene 1) from the knowledge graph 43A, etc. included in the knowledge space DB 43, a numerical representation (e.g., a spatial vector V1) associated with the same label as the label of each traffic element (e.g., α1, α2, α3, α4, etc.) included in the surrounding traffic scene (scene 1). The scenario creation unit 312 is capable of, for example, reading out from the knowledge space DB 43 a plurality of vector values ​​([1, 1, 1], [2, 2, 2], [3, 3, 3], [4, 4, 4], etc.) as shown in FIG. 18 . The scenario creation unit 312 is capable of, for example, calculating a new numerical representation (e.g., a spatial vector V2) based on the plurality of read numerical representations (e.g., a spatial vector V1) and setting the newly calculated numerical representation (e.g., a spatial vector V2) as a numerical representation (e.g., a spatial vector V3) corresponding to the surrounding traffic scene (scene 1). For example, as shown in FIG. 18, the scenario creation unit 312 can calculate the centroid vector ([2.5, 2.0, 2.5]) of the multiple vector values ​​([1, 0, 1], [2, 2, 2], [3, 3, 3], [4, 4, 4], etc.) that have been read out, and use the calculated centroid vector ([2.5, 2.5, 2.5]) as the spatial vector V3 corresponding to the surrounding traffic scene (scene 1).

[0101] The scenario creation unit 312 is capable of, for example, reading out, for each traffic element, a numerical representation (e.g., a space vector V4) associated with the same label as the label of each traffic element (e.g., β1, β2, β3, β4, etc.) included in the surrounding traffic scene (scene 2) from the knowledge graph 43A or the like included in the knowledge space DB 43. The scenario creation unit 312 is capable of, for example, calculating a new numerical representation (e.g., a space vector V5) based on the plurality of read numerical representations (e.g., a space vector V4) and setting the newly calculated numerical representation (e.g., a space vector V5) as a numerical representation (e.g., a space vector V6) corresponding to the surrounding traffic scene (scene 2). 19, the scenario creation unit 312 can calculate the centroid vector ([2.5, 3.0, 2.5]) of the read multiple vector values ​​([1, 1, 1], [2, 3, 2], [3, 4, 3], [4, 5, 4], etc.) and use the calculated centroid vector ([2.5, 3.0, 2.5]) as the space vector V6 corresponding to the surrounding traffic scene (scene 2). The surrounding traffic scenes (scene 1, scene 2) are expressed by the same space vectors as those in the knowledge space DB 43.

[0102] The similarity determination unit 313 is capable of calculating the similarity S between each known traffic scenario included in the knowledge graph DB 42 or the knowledge space DB 43 and the created surrounding traffic scenario.

[0103] The similarity determination unit 313 is capable of reading, for example, from the knowledge space DB 43, numerical representations (e.g., spatial vectors Va, Vb, and Vc) of each known traffic scene (scene a, scene b, and scene c) constituting the known traffic scenario A. The spatial vector Va is the spatial vector of the known traffic scene (scene a) and is the spatial vector of the first scene in the known traffic scenario A. The spatial vector Vb is the spatial vector of the known traffic scene (scene b) and is the spatial vector of the second scene in the known traffic scenario A. The spatial vector Vc is the spatial vector of the known traffic scene (scene c) and is the spatial vector of the third scene in the known traffic scenario A.

[0104] The similarity determination unit 313 is capable of calculating the similarity S1 between each known traffic scene and each surrounding traffic scene by comparing the numerical representation of each known traffic scene read from the knowledge space DB 43 with the numerical representation of each surrounding traffic scene created by the scenario creation unit 312.

[0105] The similarity determination unit 313 is capable of calculating the similarity S1 between each known traffic scene (scene a, scene b, scene c) and each surrounding traffic scene (scene 1, scene 2) by comparing, for example, the numerical representation (e.g., spatial vectors Va, Vb, Vc) of each known traffic scene (scene a, scene b, scene c) read from the knowledge space DB 43 with the numerical representation (e.g., spatial vectors V3, V6) of each surrounding traffic scene (scene 1, scene 2) created by the scenario creation unit 312. The similarity determination unit 313 is capable of calculating, for example, the similarity S1 between each of the spatial vectors Va, Vb, Vc and the spatial vector V3, and the similarity S1 between each of the spatial vectors Va, Vb, Vc and the spatial vector V6.

[0106] The similarity determination unit 313 is capable of calculating the similarity S1 by, for example, applying a first-order norm, a second-order norm, or cosine similarity to the spatial vectors Va, Vb, and Vc and the spatial vectors V2 and V4. The similarity determination unit 313 is capable of, for example, deriving a difference between the first-order norm of each of the spatial vectors Va, Vb, and Vc and the first-order norm of each of the spatial vectors V2 and V4, and setting the derived difference as the similarity S1. The similarity determination unit 313 may also be capable of, for example, deriving a difference between the second-order norm of each of the spatial vectors Va, Vb, and Vc and the second-order norm of each of the spatial vectors V2 and V4, and setting the derived difference as the similarity S1. The similarity determination unit 313 may be capable of, for example, deriving the difference between the cosine similarity of each spatial vector Va, Vb, and Vc and the cosine similarity of each spatial vector V2 and V4, and setting the derived difference as the similarity S1.

[0107] The similarity determination unit 313 calculates the similarity S2 based on the calculated similarity S1 for each known traffic scene, and can use the calculated similarity S2 as the similarity S of the surrounding traffic scenario to the known traffic scene.

[0108] For example, the similarity determination unit 313 can calculate a similarity S1 between each known traffic scene (scene a, scene b, scene c) in the known traffic scenario A and a surrounding traffic scene (scene 1) in the surrounding traffic scenario II, and determine the largest similarity S1 among the calculated similarities S1 as similarity Sm1, and designate the known traffic scene having similarity Sm1 as scene m1. Scene m1 is, for example, scene a. For example, the similarity determination unit 313 can calculate a similarity S1 between each known traffic scene (scene a, scene b, scene c) in the known traffic scenario A and a surrounding traffic scene (scene 2) in the surrounding traffic scenario II, and determine the largest similarity S1 among the calculated similarities S1 as similarity Sm2, and designate the known traffic scene having similarity Sm2 as scene m2. Scene m2 is, for example, scene b. For example, the similarity determination unit 313 can calculate a similarity S2 using similarities Sm1 and Sm2. For example, the similarity determination unit 313 can determine the average value of the similarity Sm1 and the similarity Sm2 as the similarity S2. For example, the similarity determination unit 313 can determine the similarity S2 as the similarity S of the surrounding traffic scenario II to the known traffic scene A. For example, the similarity determination unit 313 can calculate the similarity S2 for other known traffic scenarios in the same manner as above, and determine the calculated similarity S2 as the similarity S of the surrounding traffic scenario to the known traffic scene.

[0109] The similarity determination unit 313 is capable of determining that the known traffic scenario with the highest similarity S value among the multiple known traffic scenarios included in the knowledge graph DB 42 or the knowledge space DB 43 is the dangerous traffic scenario that the vehicle 1 (vehicle 100a) is likely to be facing. The similarity determination unit 313 is capable of storing the dangerous traffic scenario in the memory unit 40, and storing the similarity S of the dangerous traffic scenario (hereinafter referred to as the "maximum similarity Smax") in the memory unit 40 as the similarity 44.

[0110] The intensity setting unit 314 is capable of predicting the risk that the vehicle 1 (vehicle 100a) is likely to face based on the content of the result term (e.g., hazardous event) of the dangerous traffic scenario obtained by the judgment. The intensity setting unit 314 is capable of storing the content of the risk obtained by the prediction as a risk content 45 in the memory unit 40. The intensity setting unit 314 is further capable of calculating the degree of risk obtained by the prediction based on the magnitude of the maximum similarity Smax of the dangerous traffic scenario. The intensity setting unit 314 is capable of setting the calculated degree of risk as the intensity of at least one of the danger warning control and the danger avoidance control. The intensity setting unit 314 is capable of storing the set intensity as an intensity 46 in the memory unit 40. The intensity setting unit 314 is capable of outputting the risk content 45 and the intensity 46 to the driving control unit 32, which will be described later.

[0111] The control unit 30 further includes a driving control unit 32, as shown in Fig. 5, for example. The driving control unit 32 is capable of controlling the driving of the vehicle 1 (for example, the torque of the prime mover 50, the amount of brake depression, and the steering angle of the steering wheel) and notifications to the driver of the vehicle 1. The driving control unit 32 is capable of performing driving control and notification control using the data acquired by the data acquisition unit 311 and the data obtained by the intensity setting unit 314.

[0112] The driving control unit 32 is capable of calculating a correction torque for correcting a required torque to be applied to an accelerator control unit 321 (described later) based on the data acquired by the data acquisition unit 311 and the data obtained by the intensity setting unit 314. The driving control unit 32 is capable of calculating a correction torque for correcting a required torque to be applied to a brake control unit 322 (described later) based on the data acquired by the data acquisition unit 311 and the data obtained by the intensity setting unit 314. The driving control unit 32 is capable of calculating a correction torque for correcting a steering assist torque generated by a steering control unit 323 (described later) based on the data acquired by the data acquisition unit 311 and the data obtained by the intensity setting unit 314. The driving control unit 32 is capable of generating notification data for notifying the driver of the vehicle 1 based on the data acquired by the data acquisition unit 311 and the data obtained by the intensity setting unit 314.

[0113] The traveling control unit 32 includes, for example, an accelerator control unit 321, a brake control unit 322, a steering control unit 323, and a notification control unit 324, as shown in FIG.

[0114] Accelerator control unit 321 is capable of controlling the torque of prime mover 50 based on a required torque corresponding to the amount of accelerator pedal depression by the driver of vehicle 1. Accelerator control unit 321 is further capable of controlling the torque of prime mover 50 based on a target torque obtained by adding a correction torque to the required torque. Prime mover 50 is configured to drive the steered wheels of vehicle 1, and is capable of driving the steered wheels of vehicle 1 in accordance with the required torque or target torque input from accelerator control unit 321.

[0115] The brake control unit 322 is capable of controlling the torque of the brake 60 based on a required torque corresponding to the amount of brake pedal depression by the driver of the vehicle 1. The brake control unit 322 is further capable of controlling the torque of the brake 60 based on a target torque obtained by adding a correction torque to the required torque. The brake 60 is configured to brake the steered wheels of the vehicle 1, and is capable of braking the steered wheels of the vehicle 1 in accordance with the required torque or target torque input from the brake control unit 322.

[0116] The steering control unit 323 is capable of deriving a steering assist torque that assists the steering torque generated by the driver's steering wheel operation and setting an EPS torque corresponding to the derived steering assist torque. The steering control unit 323 is capable of outputting a control signal to the EPS motor 70 so that the output torque of the EPS motor 70 becomes the set EPS torque. The steering control unit 323 is capable of outputting a control signal to the EPS motor 70 so that the output torque of the EPS motor 70 becomes the EPS torque taking into account the correction torque. The EPS motor 70 generates an output torque based on the input control signal and is capable of controlling the steering angle of the steering wheel.

[0117] The accelerator control unit 321, the brake control unit 322, and the steering control unit 323 may be configured to include, for example, a CPU, etc. In this case, the accelerator control unit 321, the brake control unit 322, and the steering control unit 323 are able to perform the various driving controls described above by, for example, executing control software stored in a storage unit.

[0118] The notification control unit 324 is capable of outputting notification data to the notification unit 80 for notifying the driver of the vehicle 1. The notification control unit 324 is capable of, for example, generating a video signal including the notification data and outputting it to the notification unit 80. The notification control unit 324 is capable of, for example, generating an audio signal including the notification data and outputting it to the notification unit 80. The notification unit 80 is configured to include, for example, a display panel and a speaker. When the video signal is input to the notification unit 80 from, for example, the control unit 30 (notification control unit 324), the notification unit 80 is capable of displaying an image corresponding to the input video signal on a display screen. When the audio signal is input to the notification unit 80 from, for example, the control unit 30 (notification control unit 324), the notification unit 80 is capable of outputting audio corresponding to the input audio signal from the speaker.

[0119] (Driving Assistance Procedure) Next, a driving assistance procedure in the vehicle 1 will be described with reference to Fig. 20. Fig. 20 shows an example of a driving assistance procedure in the vehicle 1.

[0120] The vehicle 1 (risk prediction unit 31) first acquires various data from the sensor unit 10, various data acquired from the outside via the communication unit 20, and various control signals for various devices of the vehicle 1. Based on the acquired data, the risk prediction unit 31 acquires position data of the vehicle 1 and traffic participants around the vehicle 1. The risk prediction unit 31 further acquires surrounding map data including the acquired position of the vehicle 1 from the road map data DB 41 in the storage unit 40. The risk prediction unit 31 also acquires stereo images Da or distance images Db from the stereo camera. From the map data and the stereo images Da or distance images Db, the risk prediction unit 31 acquires attribute data and position data of each structure constituting the road around the vehicle 1, and attribute data and position data of structures on the road around the vehicle 1. In this way, road data around the vehicle 1 and traffic data about the vehicle 1 and traffic participants around the vehicle 1 are acquired (step S101).

[0121] Next, the risk prediction unit 31 generates external environment recognition data based on the acquired road data and traffic data, and interprets a traffic context based on the generated external environment recognition data (step S102). The risk prediction unit 31 newly interprets a traffic context based on, for example, periodically obtained road data and traffic data.

[0122] The risk prediction unit 31 determines whether one of the multiple traffic elements constituting the traffic context is missing (step S103). When the risk prediction unit 31 detects a missing element ME in a sentence included in the traffic context (step S103; Y), it reads from the knowledge space DB 43 the numerical expressions of multiple non-missing elements UME other than the missing element ME among the multiple traffic elements constituting the sentence including the missing element ME. The risk prediction unit 31 derives a first numerical expression of the missing element ME by performing an operation using the numerical expressions of the multiple non-missing elements UME that have been read. The risk prediction unit 31 compares the derived first numerical expression of the missing element ME with multiple second numerical expressions in the knowledge space DB 43, and sets the label of the second numerical expression that has a high similarity to the first numerical expression as the label of the missing element ME. In this way, the risk prediction unit 31 complements the missing element ME (step S104).

[0123] The risk prediction unit 31 may use the similarity of the second numerical expression to the first numerical expression of the missing element ME as a completion accuracy, which is an index of the likelihood of completion. In this case, when the completion accuracy is equal to or greater than a predetermined threshold, the risk prediction unit 31 sets the label of the missing element ME to the second numerical expression that has a high similarity to the first numerical expression of the missing element ME.

[0124] The risk prediction unit 31 generates a plurality of time-series surrounding traffic scenes having a graph structure from the two time-series traffic contexts in which the missing elements ME have been supplemented (step S105). Subsequently, the risk prediction unit 31 integrates the generated time-series surrounding traffic scenes to create a surrounding traffic scenario (step S106).

[0125] Next, the risk prediction unit 31 calculates the similarity S between each known traffic scenario included in the knowledge space DB 43 and the created surrounding traffic scenario (step S107). The risk prediction unit 31 determines that the known traffic scenario with the highest similarity S value among the multiple known traffic scenarios included in the knowledge space DB 43 is the dangerous traffic scenario that the vehicle 1 is likely to be facing (step S108). The risk prediction unit 31 stores the dangerous traffic scenario in the memory unit 40, and also stores the similarity S of the dangerous traffic scenario (hereinafter referred to as the "maximum similarity Smax") in the memory unit 40 as the similarity 44.

[0126] The risk prediction unit 31 predicts the risk that the vehicle 1 is likely to face based on the content of the result term (e.g., a dangerous event) of the dangerous traffic scenario obtained by the judgment. The risk prediction unit 31 stores the content of the risk obtained by the prediction as a dangerous event in the memory unit 40. The risk prediction unit 31 further calculates the degree of risk obtained by the prediction based on the magnitude of the maximum similarity Smax of the dangerous traffic scenario. The risk prediction unit 31 sets the calculated degree of risk as the strength of at least one of the risk warning control and the risk avoidance control (step S109). The risk prediction unit 31 outputs the dangerous event and its strength to the driving control unit 32 (step S110).

[0127] The driving control unit 32 performs driving control and notification control using the data acquired by the data acquisition unit 311 and the data obtained by the intensity setting unit 314. The driving control unit 32 calculates a correction torque for correcting a required torque to be applied to the accelerator control unit 321 based on the data acquired by the data acquisition unit 311 and the data obtained by the intensity setting unit 314. The driving control unit 32 calculates a correction torque for correcting a required torque to be applied to the brake control unit 322 based on the data acquired by the data acquisition unit 311 and the data obtained by the intensity setting unit 314. The driving control unit 32 corrects the steering assist torque generated by the steering control unit 323 based on the data acquired by the data acquisition unit 311 and the data obtained by the intensity setting unit 314. The driving control unit 32 generates notification data for notifying the driver of the vehicle 1 based on the data acquired by the data acquisition unit 311 and the data obtained by the intensity setting unit 314.

[0128] The accelerator control unit 321 controls the torque of the prime mover 50 based on a target torque obtained by adding a correction torque to the required torque. The prime mover 50 drives the steered wheels of the vehicle 1 in accordance with the target torque input from the accelerator control unit 321. The brake control unit 322 controls the torque of the brake 60 based on the target torque obtained by adding a correction torque to the required torque. The brake 60 brakes the steered wheels of the vehicle 1 in accordance with the target torque input from the brake control unit 322. The steering control unit 323 outputs a control signal to the EPS motor 70 so that the output torque of the EPS motor 70 becomes EPS torque taking the correction torque into consideration. The EPS motor 70 generates an output torque based on the input control signal and controls the steering angle of the steering wheel.

[0129] The notification control unit 324 generates a video signal including the notification data and outputs it to the notification unit 80. The notification control unit 324 further generates an audio signal including the notification data and outputs it to the notification unit 80. When the video signal is input from, for example, the control unit 30 (notification control unit 324), the notification unit 80 displays an image corresponding to the input video signal on the display screen. When the audio signal is input from, for example, the control unit 30 (notification control unit 324), the notification unit 80 outputs audio corresponding to the input audio signal from a speaker. In this manner, driving assistance in the vehicle 1 is performed.

[0130] [Effects] Next, effects of the vehicle 1 according to the embodiment of the present disclosure will be described.

[0131] In this embodiment, when one of the multiple traffic elements constituting a traffic context is missing from the acquired road data and traffic data, a numerical value table of multiple non-missing elements UME other than the missing traffic element ME is read from the knowledge space DB 43, and a calculation is performed using the numerical representations of the multiple non-missing elements UME to derive a first numerical representation of the missing element ME. The first numerical representation is then compared with multiple second numerical representations in the knowledge space DB 43, and the label of the second numerical representation that is most similar to the first numerical representation becomes the label of the missing element. In this way, in this embodiment, the missing element ME is complemented using the knowledge space DB 43. This allows logical inference to be performed even when a traffic context including a missing element ME is obtained.

[0132] In this embodiment, when one of the multiple traffic elements constituting the traffic context is missing in the acquired road data and traffic data, the spatial vectors of multiple non-missing elements UME are read from the knowledge space DB 43, and a calculation is performed using the spatial vectors of the multiple non-missing elements UME to derive a first spatial vector of the missing element ME. Then, the first spatial vector is compared with multiple second spatial vectors in the knowledge space DB 43, and the label of the second spatial vector that has a high similarity to the first spatial vector becomes the label of the missing element ME. In this way, in this embodiment, the missing element ME is complemented using the knowledge space DB 43. As a result, logical inference can be performed even when a traffic context including a missing element ME is obtained.

[0133] In this embodiment, the road data includes labels and position data of roads and road structures around the vehicle 1, the traffic data includes labels and position / speed data of the vehicle 1 and traffic participants, and the multiple traffic elements include labels and numerical representations of roads and road structures, as well as labels and numerical representations of the vehicle 1 and traffic participants. Thus, missing elements ME are complemented using space vector operations. This allows logical inference to be performed even when a traffic context including missing elements ME is obtained.

[0134] In this embodiment, the similarity S of the second numerical representation (spatial vector) to the first numerical representation (spatial vector) of the missing element ME may be used as a completion accuracy, which is an index of the likelihood of completion. In this case, when the completion accuracy is equal to or greater than a predetermined threshold, the label of the second numerical representation (spatial vector) that has a high similarity to the first numerical representation (spatial vector) of the missing element ME becomes the label of the missing element ME. This makes it possible to perform logical inference even when a traffic context including the missing element ME is obtained.

[0135] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0136] Furthermore, for example, the present disclosure can be configured as follows: (1) An information processing device comprising: a storage unit that stores a knowledge graph represented numerically; an acquisition unit that is capable of acquiring road data around a vehicle and traffic data about the vehicle and traffic participants around the vehicle; and a processing unit that is capable of creating, by using the knowledge graph, a numerical representation of a traffic context corresponding to the road data and the traffic data acquired by the acquisition unit, wherein the processing unit is capable of executing the following when one of a plurality of traffic elements constituting the traffic context is missing in the road data and the traffic data acquired by the acquisition unit: reading, from the knowledge graph, numerical representations of a plurality of non-missing elements other than the missing element that is the missing traffic element among the plurality of traffic elements, and performing a calculation using the read numerical representations of the plurality of non-missing elements; and comparing the first numerical representation with a plurality of second numerical representations in the knowledge graph, and setting the label of the second numerical representation that has a high similarity to the first numerical representation as the label of the missing element. (2) The information processing device according to (1), wherein the numerical representation is a spatial vector, and the processing unit is capable of: when one of the plurality of traffic elements is missing in the road data and the traffic data acquired by the acquisition unit, reading spatial vectors of the plurality of non-missing elements from the knowledge graph and performing an operation using the read spatial vectors of the plurality of non-missing elements to derive a first spatial vector of the missing element; and comparing the first spatial vector with a plurality of second spatial vectors in the knowledge graph and setting the label of the second spatial vector that has a high similarity to the first spatial vector as the label of the missing element.(3) The information processing device according to (2), wherein the road data includes labels and position data of roads and structures on the roads around the vehicle, the traffic data includes labels and position / speed data of the vehicle and the traffic participants, and the multiple traffic elements include labels and numerical representations of the roads and structures on the roads, and labels and numerical representations of the vehicle and the traffic participants. (4) The information processing device according to (2), wherein the processing unit is capable of setting a similarity of the second spatial vector to the first spatial vector as a completion accuracy that is an index of likelihood of completion, and when the completion accuracy is equal to or greater than a predetermined threshold, setting a label of the second spatial vector that has a high similarity to the first spatial vector as the missing data. (5) A vehicle control device comprising: a traffic context acquisition unit capable of acquiring multiple traffic contexts including completed traffic contexts in which the missing elements are completed by the information processing device according to any one of claims 1 to 4; and a control unit capable of performing at least one of notification control and driving control based on the multiple traffic contexts acquired by the traffic context acquisition unit. (6) A vehicle comprising: an alarm device and a driving device; a traffic context acquisition unit capable of acquiring a plurality of traffic contexts including a completed traffic context in which the missing elements have been completed by the information processing device described in any one of claims 1 to 4; and a control device capable of performing at least one of alarm control for the alarm device and driving control for the driving device based on the plurality of traffic contexts acquired by the traffic context acquisition unit.

[0137] The control unit 30 shown in FIG. 5 can be implemented by circuitry including at least one semiconductor integrated circuit, such as at least one processor (e.g., a central processing unit (CPU)), at least one application-specific integrated circuit (ASIC), and / or at least one field-programmable gate array (FPGA). The at least one processor can be configured to perform all or a portion of the various functions of the control unit 30 shown in FIG. 5 by reading instructions from at least one non-transitory, tangible computer-readable medium. Such media can take various forms, including, but not limited to, various magnetic media such as hard disks, various optical media such as CDs or DVDs, and various semiconductor memories (i.e., semiconductor circuits) such as volatile or non-volatile memories. Volatile memories can include DRAM and SRAM. Non-volatile memories can include ROM and NVRAM. An ASIC is an integrated circuit (IC) specialized to perform all or a portion of the various functions of the control unit 30 shown in FIG. 5. An FPGA is an integrated circuit designed to be configurable after manufacture to perform all or part of the various functions of the control unit 30 shown in FIG.

Claims

1. An information processing device comprising: a memory unit that stores a knowledge graph expressed numerically; an acquisition unit that is capable of acquiring road data around a vehicle and traffic data about the vehicle and traffic participants around the vehicle; and a processing unit that is capable of creating a numerical representation of a traffic context corresponding to the road data and the traffic data acquired by the acquisition unit, using the knowledge graph, wherein the processing unit is capable of: when one of a plurality of traffic elements constituting the traffic context is missing in the road data and the traffic data acquired by the acquisition unit, reading from the knowledge graph numerical representations of a plurality of non-missing elements among the plurality of traffic elements other than the missing element that is the missing traffic element, and performing an operation using the read numerical representations of the plurality of non-missing elements to derive a first numerical representation of the missing element; and comparing the first numerical representation with a plurality of second numerical representations in the knowledge graph, and setting the label of the second numerical representation that has a high similarity to the first numerical representation as the label of the missing element.

2. The information processing device according to claim 1, wherein the numerical representation is a spatial vector, and the processing unit is capable of: when one of the plurality of traffic elements is missing in the road data and the traffic data acquired by the acquisition unit, reading spatial vectors of the plurality of non-missing elements from the knowledge graph and performing an operation using the read spatial vectors of the plurality of non-missing elements to derive a first spatial vector of the missing element; and comparing the first spatial vector with a plurality of second spatial vectors in the knowledge graph and setting the label of the second spatial vector that has a high similarity to the first spatial vector as the label of the missing element.

3. The information processing device of claim 2, wherein the road data includes label and position data of roads and structures on the roads around the vehicle, the traffic data includes label and position speed data of the vehicle and traffic participants, and the plurality of traffic elements includes labels and numerical representations of the roads and structures on the roads, and labels and numerical representations of the vehicle and traffic participants.

4. The information processing device according to claim 2, wherein the processing unit is capable of: determining the similarity of the second spatial vector to the first spatial vector as a completion accuracy that is an index of the likelihood of completion; and, when the completion accuracy is equal to or greater than a predetermined threshold, determining the label of the second spatial vector that has a high similarity to the first spatial vector as the missing data.

5. A vehicle control device comprising: a traffic context acquisition unit capable of acquiring multiple traffic contexts including a completed traffic context in which the missing elements have been completed by an information processing device described in any one of claims 1 to 4; and a control unit capable of performing at least one of notification control and driving control based on the multiple traffic contexts acquired by the traffic context acquisition unit.

6. A vehicle comprising: an alarm device and a driving device; a traffic context acquisition unit capable of acquiring a plurality of traffic contexts including a completed traffic context in which the missing elements have been completed by an information processing device described in any one of claims 1 to 4; and a control device capable of performing at least one of alarm control for the alarm device and driving control for the driving device based on the plurality of traffic contexts acquired by the traffic context acquisition unit.

Images