Training data generation device, training data generation method, training data generation program, and program

Abstract

To provide a technique for generating learning data of a risk prediction model in order to specifically predict risk during driving of a vehicle.SOLUTION: As a learning data generation device, a risk prediction device 3 comprises: a travel-related information acquisition unit 13 for acquiring travel-related information including at least a position, a speed, an acceleration, a direction of travel and a steering angle of a vehicle 2 based on detection of the acceleration of the vehicle 2 exceeding a predetermined value; a risk factor type determination unit 16 for determining a risk factor type including a type of a mobile identified from an image picked up based on the detection of the acceleration; and a learning data generation unit 33 for generating learning data associating a risk factor type label indicating the risk factor type with the travel-related information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a training data generation device, a training data generation method, a training data generation program, and a trained model. [Background technology]

[0002] Patent Document 1 discloses a risk prediction device that predicts risk according to the driver of a vehicle. Specifically, the risk prediction device accumulates driving state information when a risk is detected, and generates a risk prediction model that predicts risk using the accumulated driving state information. When the risk prediction device predicts a risk using the risk prediction model, it warns the driver by, for example, sounding a beep. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-3707 Summary of the Invention [Problem to be solved by the invention]

[0004] A driver using the risk prediction device can know that a risk is approaching, but cannot know specifically what kind of risk is approaching.

[0005] An object of the present disclosure is to provide a technology for generating learning data for a risk prediction model for specifically predicting risks when driving a vehicle. [Means for solving the problem]

[0006] According to the present disclosure, there is provided a learning data generation device including: an acquisition unit that acquires driving-related information including at least the position, speed, acceleration, direction of travel, and steering angle of the vehicle based on detection of acceleration of the vehicle that exceeds a predetermined value; a determination unit that determines a risk factor type including the type of moving object identified from an image captured based on the detection of the acceleration; and a generation unit that generates learning data in which a risk factor type label indicating the risk factor type is associated with the driving-related information.

[0007] Furthermore, according to the present disclosure, there is provided a learning data generation method including: an acquisition step of acquiring driving-related information including at least the position, speed, acceleration, direction of travel, and steering angle of the vehicle based on detection of acceleration of the vehicle exceeding a predetermined value; a determination step of determining a risk factor type including the type of moving object identified from an image captured based on the detection of the acceleration; and a generation step of generating learning data in which a risk factor type label indicating the risk factor type is associated with the driving-related information.

[0008] Furthermore, according to the present disclosure, there is provided a learning data generation program that causes a computer to execute an acquisition step of acquiring driving-related information including at least the position, speed, acceleration, direction of travel, and steering angle of the vehicle based on detection of acceleration of the vehicle that exceeds a predetermined value; a determination step of determining a risk factor type including the type of moving body identified from an image captured based on the detection of the acceleration; and a generation step of generating learning data in which a risk factor type label indicating the risk factor type is associated with the driving-related information. [Effects of the Invention]

[0009] According to the present invention, learning data for a risk prediction model for specifically predicting risks when driving a vehicle is generated. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram of a danger prediction system. [Figure 2] FIG. 2 is a functional block diagram of an in-vehicle processing unit of the danger prediction device. [Figure 3] FIG. 2 is a functional block diagram of a server processing unit of the risk prediction device. [Figure 4] 10 is a diagram illustrating an example of a determination made by a risk factor type determining unit. FIG. [Figure 5] 10 is a control flow of a danger prediction device. [Figure 6] 10 is a control flow of a danger prediction device. [Figure 7] 10 is a control flow of a danger prediction device. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of the present disclosure will be described below with reference to FIGS. 1 to 7. FIG. 1 is a schematic diagram of a risk prediction system 1. As shown in FIG. 1, the risk prediction system 1 includes at least one vehicle 2 and a risk prediction device 3. In this embodiment, the at least one vehicle 2 includes multiple vehicles 2. The multiple vehicles 2 include vehicle A, vehicle B, and vehicle C. In this embodiment, the risk prediction device 3 is realized by distributed processing performed by an on-board processing unit 3a implemented by a computer installed in each vehicle 2 and a server processing unit 3b implemented in a cloud server 4. The multiple on-board processing units 3a and the server processing unit 3b are configured to be able to communicate bidirectionally via, for example, a wide area network (WAN).

[0012] 2 shows a functional block diagram of the on-board processing unit 3a of the risk prediction device 3. The on-board processing unit 3a of the risk prediction device 3 mainly predicts risk using a risk prediction model. When the on-board processing unit 3a of the risk prediction device 3 predicts risk, it warns the driver of the vehicle 2 about specific risk factors that may cause a traffic accident.

[0013] 2 , in this embodiment, the in-vehicle processing unit 3a of the risk prediction device 3 includes an ECU (Electronic Circuit Unit) 11, a camera 5, a GPS (Global Positioning System) module 6, a speed sensor 7, an acceleration sensor 8, a steering angle sensor 9, a geomagnetic sensor 10, a speaker 12, and a communication unit 21. The ECU 11 includes a driving-related information acquisition unit 13, a driving-related information storage unit 14, an abnormal acceleration detection unit 15, a risk factor type determination unit 16, a driving-related information transmission unit 17, a risk prediction model reception unit 18, a risk prediction unit 19, and a warning unit 20.

[0014] The driving-related information acquisition unit 13, abnormal acceleration detection unit 15, risk factor type determination unit 16, driving-related information transmission unit 17, risk prediction model reception unit 18, risk prediction unit 19, and warning unit 20 are typically configured by a CPU (Central Processing Unit) that executes programs stored in a non-volatile storage medium such as a ROM (Read Only Memory).

[0015] The driving-related information storage unit 14 is typically configured by a non-volatile storage medium such as a flash memory or a hard disk.

[0016] 3 shows a functional block diagram of the server processing unit 3b of the risk prediction device 3. The server processing unit 3b of the risk prediction device 3 mainly generates learning data and also generates a risk prediction model. The server processing unit 3b of the risk prediction device 3 supplies the generated risk prediction model to the in-vehicle processing unit 3a of the risk prediction device 3.

[0017] 3, in this embodiment, the server processing unit 3b of the risk prediction device 3 includes a processing unit 30 and a communication unit 31. The processing unit 30 includes a driving-related information receiving unit 32, a learning data generating unit 33, a learning data storage unit 34, a risk prediction model generating unit 35, and a risk prediction model transmitting unit 36.

[0018] The driving-related information receiving unit 32, the learning data generating unit 33, the risk prediction model generating unit 35, and the risk prediction model transmitting unit 36 ​​are typically configured by a CPU (Central Processing Unit) that executes programs stored in a non-volatile storage medium such as a ROM (Read Only Memory).

[0019] The learning data storage unit 34 is typically configured by a non-volatile storage medium such as a flash memory or a hard disk.

[0020] In this manner, in this embodiment, the functions of the risk prediction device 3 are realized by distributed processing between the in-vehicle processing unit 3a and the server processing unit 3b. Therefore, for example, some or all of the functions of the in-vehicle processing unit 3a may be provided by the server processing unit 3b, and some or all of the functions of the server processing unit 3b may be provided by the in-vehicle processing unit 3a.

[0021] (In-vehicle processing unit 3a) First, the on-vehicle processing unit 3a will be described in detail with reference to FIG.

[0022] The camera 5 outputs image data obtained by capturing an image of the outside of the vehicle 2 to the ECU 11 of the danger prediction device 3. The camera 5 typically captures an image in front of the vehicle 2. However, instead of this, the camera 5 may capture an image of the side or rear of the vehicle 2.

[0023] The GPS module 6 is a specific example of a GNSS (Global Navigation Satellite System) module. The GPS module 6 outputs current location information of the vehicle 2 to the ECU 11 of the danger prediction device 3. The GNSS module may be a GLONASS (Global Navigation Satellite System) module, a Galileo module, a BeiDou module, or a QZSS (Quasi-Zenith Satellite System) module.

[0024] The speed sensor 7 detects the speed of the vehicle 2 and outputs speed information indicating the speed to the ECU 11 of the danger prediction device 3. The speed sensor 7 is typically a mechanical speedometer, an electric speedometer, or an electronic speedometer.

[0025] The acceleration sensor 8 detects the acceleration of the vehicle 2 and outputs acceleration information indicating the acceleration to the ECU 11 of the danger prediction device 3. The acceleration sensor 8 is typically a three-axis acceleration sensor or a six-axis acceleration sensor.

[0026] The steering angle sensor 9 detects the steering angle of the vehicle 2, and outputs steering angle information indicating the steering angle to the ECU 11 of the danger prediction device 3. The steering angle sensor 9 is typically a steering sensor (steering angle force meter).

[0027] The geomagnetic sensor 10 detects the azimuth angle of the traveling direction of the vehicle 2, and outputs azimuth angle information indicating the azimuth angle to the ECU 11 of the danger prediction device 3. The geomagnetic sensor 10 is typically a Hall sensor or a 3D Hall sensor.

[0028] The ECU 11 acquires direction indication information indicating an indication operation from a direction indicator (not shown). The ECU 11 also acquires map information from a navigation device (not shown) or an external server (not shown). The map information indicates multiple nodes, links connecting the nodes, and road attributes of each link. The road attributes refer to road shapes such as the number of lanes, the presence or absence of right-turn lanes or left-turn lanes, and the presence or absence of pedestrian crossings.

[0029] The speaker 12 is a specific example of output means used to issue a warning to the driver of the vehicle 2. Instead of the speaker 12, the output means may be a display.

[0030] The ECU 11 communicates with a communication unit 31 of the server processing unit 3b via a communication unit 21 configured as a communication module.

[0031] The driving-related information acquisition unit 13 acquires various data output from the camera 5, GPS module 6, speed sensor 7, acceleration sensor 8, steering angle sensor 9, and geomagnetic sensor 10.

[0032] The driving-related information acquisition unit 13 acquires image data from the camera 5. Based on the image data acquired from the camera 5, the driving-related information acquisition unit 13 acquires traffic control information indicating the presence or absence of traffic lights and traffic control by traffic lights. Typically, the driving-related information acquisition unit 13 detects the light color of the traffic lights and determines traffic control by traffic lights based on the detected light color. The driving-related information acquisition unit 13 may receive traffic control information via wireless communication from a traffic control device installed at an intersection.

[0033] The driving-related information acquisition unit 13 acquires the current location information of the vehicle 2 from the GPS module 6 .

[0034] The traveling-related information acquisition unit 13 acquires the speed information of the vehicle 2 from the speed sensor 7 and the acceleration information of the vehicle 2 from the acceleration sensor 8, respectively.

[0035] The driving-related information acquisition unit 13 acquires steering angle information from the steering angle sensor 9 .

[0036] The driving-related information acquisition unit 13 acquires azimuth angle information from the geomagnetic sensor 10 .

[0037] The travel-related information acquisition unit 13 acquires date and time information indicating the current date, day of the week, and time from a date and time management device (not shown).

[0038] The driving-related information acquisition unit 13 acquires map information of the vicinity of the current location of the vehicle 2 from the navigation device 40 or the external server 41. Typically, the driving-related information acquisition unit 13 identifies the road on which the vehicle will travel after passing the next intersection based on route information generated by a route generation unit (not shown), and determines whether or not there is a crosswalk that crosses the identified road by referring to the map information. In this way, the driving-related information acquisition unit 13 can acquire information indicating whether or not there is a crosswalk that crosses the lane on which the vehicle will travel after passing the intersection.

[0039] As described above, the driving-related information acquisition unit 13 acquires image data, traffic control information, current location information, speed information, acceleration information, steering angle information, azimuth angle information, date and time information, direction indication information, and map information. The driving-related information includes at least one of image data, traffic control information, current location information, speed information, acceleration information, steering angle information, azimuth angle information, date and time information, direction indication information, and map information. The driving-related information acquisition unit 13 sequentially acquires the driving-related information. Typically, the driving-related information acquisition unit 13 acquires the driving-related information once every 0.5 seconds.

[0040] The driving-related information storage unit 14 stores the driving-related information acquired by the driving-related information acquisition unit 13. The driving-related information storage unit 14 typically stores the driving-related information in a ring buffer format.

[0041] The abnormal acceleration detection unit 15 acquires driving-related information from the driving-related information storage unit 14 and detects abnormal acceleration of the vehicle 2 based on the acceleration information included in the driving-related information. The acceleration here typically refers to the acceleration in the front-rear direction of the vehicle 2. Detecting abnormal acceleration of the vehicle 2 typically means detecting acceleration in the front-rear direction that exceeds a predetermined value. The predetermined value can be set arbitrarily, for example, to 2G, 3G, or 5G.

[0042] The risk factor type determination unit 16 determines the type of risk factor associated with the driving-related information of the vehicle 2 when the abnormal acceleration detection unit 15 detects abnormal acceleration of the vehicle 2. In detail, the risk factor type determination unit 16 acquires driving-related information from the driving-related information storage unit 14 at a time before the abnormal acceleration detection unit 15 detects abnormal acceleration of the vehicle 2, and determines the type of risk factor associated with the acquired driving-related information.

[0043] More specifically, when the abnormal acceleration detection unit 15 detects abnormal acceleration of the vehicle 2, the risk factor type determination unit 16 acquires driving-related information at a second time point that is a first time before the first time point at which the information was detected from the driving-related information storage unit 14 via the abnormal acceleration detection unit 15, and determines the type of risk factor associated with the acquired driving-related information. Here, the first time period may be a fixed value such as 3 seconds or 5 seconds, or may be set according to the speed of the vehicle 2. In the latter case, the first time period may be set longer the higher the speed at which the abnormal acceleration detection unit 15 detects the abnormal acceleration of the vehicle 2.

[0044] For example, if the immediately preceding speed is A [km / h], the first time period may be set to A / 10 [sec]. In this case, if the speed at the time of detection is 40 [km / h], the first time period may be set to 4 [sec]. Alternatively, the first time period may be set to A×A / 1000 [sec]. In this case, if the speed at the time of detection is 40 [km / h], the first time period may be set to 1.6 [sec].

[0045] The risk factor type determining unit 16 determines the type of risk factor including the type of moving object identified from the image captured based on the detection by the abnormal acceleration detecting unit 15 of acceleration of the vehicle 2 exceeding a predetermined value.

[0046] That is, the risk factor type determination unit 16 performs known object detection processing such as Faster R-CNN (Regions with Convolutional Neural Networks), YOLO (You Look Only Once), and SSD (Single Shot Multibox Detector). The risk factor type determination unit 16 performs the above object detection processing on an image captured at the time when the abnormal acceleration detection unit 15 detects an acceleration of the vehicle 2 that exceeds a predetermined value. Specifically, the risk factor type determination unit 16 performs the above object detection processing on an image captured immediately before or immediately after the time when the abnormal acceleration detection unit 15 detects an acceleration of the vehicle 2 that exceeds the predetermined value. In this way, the risk factor type determination unit 16 detects all moving objects captured in the image. A moving object typically refers to an object moving on a public road, such as an oncoming four-wheeled vehicle, an oncoming two-wheeled vehicle, a bicycle, a pedestrian, a stroller, or a wheelchair.

[0047] For example, when the risk factor type determination unit 16 recognizes an oncoming four-wheeled vehicle and a bicycle in the image, it determines that the risk factor type is “oncoming four-wheeled vehicle and bicycle.” For example, when the risk factor type determination unit 16 recognizes an oncoming two-wheeled vehicle and a wheelchair in the image, it determines that the risk factor type is “oncoming two-wheeled vehicle and wheelchair.”

[0048] The risk factor type is specifically a label indicating the other party in the traffic accident, and includes the other party's means of transportation (walking, bicycle, etc.). Fig. 4 shows a specific example of the above determination by the risk factor type determination unit 16.

[0049] In Figure 4, "Image example" is image data of the driving-related information. "Date and time" is date and time information of the driving-related information. "Location information" is current location information of the driving-related information. Current location information can typically be expressed as longitude and latitude. "Vehicle information, related information" is speed information, acceleration information, steering angle information, azimuth angle information, and traffic control information of the driving-related information. "Risk factor type" is a risk factor type label that indicates the type of risk factor related to the driving-related information.

[0050] As shown in FIG. 4, when determining the type of risk factor associated with the driving-related information, the risk factor type determination unit 16 first performs a major classification determination, and then performs a minor classification determination.

[0051] The major classification determination is a specific example of the first classification determination. In this embodiment, the major classification determination includes determination of the path of the vehicle 2 at the intersection.

[0052] That is, the risk factor type determination unit 16 determines the route of the vehicle 2 at the intersection based on the driving-related information. The risk factor type determination unit 16 typically determines the route of the vehicle 2 at the intersection based on steering angle information or direction indication information.

[0053] The risk factor type determination unit 16 determines the course of the vehicle 2 at the intersection as a "right turn" if the steering angle information indicates that the steering angle is 45 degrees to the right or more. The risk factor type determination unit 16 determines the course of the vehicle 2 at the intersection as a "left turn" if the steering angle information indicates that the steering angle is 45 degrees to the left or more. The risk factor type determination unit 16 determines the course of the vehicle 2 at the intersection as a "straight forward" if the steering angle information indicates that the steering angle is between 45 degrees to the right and 45 degrees to the left. In FIG. 4, "major classification" indicates the determination result by the risk factor type determination unit 16.

[0054] The minor classification judgment is a specific example of the second classification judgment. In this embodiment, the minor classification judgment includes a judgment of the target of the risk factor. In FIG. 4, "minor classification" indicates the judgment result of the minor classification judgment.

[0055] That is, when the driving-related information includes map information, the risk factor type determining unit 16 performs a small classification determination based on the road attributes indicated by the map information.

[0056] In Figure 4, "road shape" refers to the road attributes of each link indicated by the map information. As mentioned above, the road attributes include the road attributes of the driving lane and the road attributes of the oncoming lane. The road attributes indicate the number of lanes, whether there is a right-turn lane or a left-turn lane, and whether there is a pedestrian crossing.

[0057] When the map information indicates that "there is a pedestrian crossing on the road on which the vehicle 2 will travel after turning right," as in No. 1 in Fig. 4, the risk factor type determination unit 16 determines that the means of transportation of the other party who may cause a traffic accident is walking. Therefore, the risk factor type determination unit 16 determines that the type of risk factor related to the driving-related information of No. 1 is "pedestrian turning right." "Pedestrian turning right" means that "the means of transportation of the other party who may cause a traffic accident when the vehicle 2 turns right is walking."

[0058] When the map information indicates that "there is one lane on the oncoming lane when turning right," as in No. 2 in Fig. 4, the risk factor type determination unit 16 determines that the other vehicle's means of transportation that could cause a traffic accident is a motorcycle. Therefore, the risk factor type determination unit 16 determines that the type of risk factor associated with the driving-related information of No. 2 is "oncoming motorcycle turning right." "Oncoming motorcycle turning right" means that "the other vehicle's means of transportation that could cause a traffic accident when vehicle 2 turns right is an oncoming motorcycle."

[0059] When the map information indicates that "there are two or more lanes on the oncoming lane side when turning right, or there is a right-turn lane on the oncoming lane side," as in No. 3 in Fig. 4, the risk factor type determination unit 16 determines that the other vehicle's means of transportation that could cause a traffic accident is a four-wheeled vehicle. Therefore, the risk factor type determination unit 16 determines that the type of risk factor associated with the driving-related information of No. 3 is "oncoming straight-moving vehicle turning right." "Oncoming straight-moving vehicle turning right" means that "the other vehicle's means of transportation that could cause a traffic accident when vehicle 2 turns right is an oncoming straight-moving vehicle."

[0060] In Figure 4, No. 4 to No. 6 indicate the same driving-related information. Because the driving-related information No. 4 to No. 6 does not include map information, the risk factor type determination unit 16 does not perform a subcategory determination. Therefore, the risk factor type determination unit 16 determines that the types of risk factors associated with the driving-related information No. 4 to No. 6 are "pedestrian moving straight," "oncoming motorcycle moving straight," and "oncoming vehicle moving straight." In this way, multiple risk factor type labels can be attached to one piece of driving-related information.

[0061] In Figure 4, No. 7 to No. 9 indicate the same driving-related information. Because the driving-related information No. 7 to No. 9 does not include map information, the risk factor type determination unit 16 does not perform a subcategory determination. Therefore, the risk factor type determination unit 16 determines that the types of risk factors associated with the driving-related information No. 7 to No. 9 are "pedestrian turning left," "oncoming motorcycle turning left," and "oncoming vehicle going straight while turning left."

[0062] The driving-related information transmitting unit 17 transmits the risk factor type label, which is the determination result by the risk factor type determining unit 16, to the server processing unit 3b together with the corresponding driving-related information.

[0063] That is, the traveling-related information transmission unit 17 transmits to the server processing unit 3b traveling-related information from a point in time prior to the point in time when the vehicle acceleration exceeding the predetermined value was detected and a risk factor type label indicating a risk factor type including the type of moving object identified from the image captured at the point in time when the vehicle acceleration exceeding the predetermined value was detected, in association with each other. The predetermined time can be set arbitrarily, for example, to 1 second, 5 seconds, or 30 seconds.

[0064] The risk prediction model receiving unit 18 receives the risk prediction model from the server processing unit 3b.

[0065] The risk prediction unit 19 predicts whether or not there is a risk to the vehicle 2 by using the driving-related information acquired by the driving-related information acquisition unit 13 and the risk prediction model received by the risk prediction model receiving unit 18. In this embodiment, the risk prediction unit 19 not only simply predicts whether or not there is a risk to the vehicle 2, but also predicts the type of risk factor approaching the vehicle 2.

[0066] When the risk prediction unit 19 predicts that there is a risk to the vehicle 2, the warning unit 20 issues a warning to the driver of the vehicle 2 by voice from the speaker 12 based on the type of risk factor predicted by the risk prediction unit 19.

[0067] (Server processing unit 3b) Next, the server processing unit 3b will be described in detail with reference to Fig. 3. A processing unit 30 of the server processing unit 3b communicates with the communication unit 21 of the in-vehicle processing unit 3a via a communication unit 31 configured with a communication module.

[0068] The driving-related information receiving unit 32 receives driving-related information and a risk factor type label corresponding to the driving-related information from the in-vehicle processing unit 3a.

[0069] The learning data generating unit 33 generates learning data using the driving-related information as an explanatory variable and the risk factor type label indicating the type of risk factor as a response variable.

[0070] That is, the learning data generating unit 33 generates learning data in which risk factor type labels indicating risk factor types are associated with driving-related information.

[0071] The learning data storage unit 34 stores the learning data generated by the learning data generation unit 33.

[0072] The risk prediction model generating unit 35 generates a risk prediction model based on the plurality of learning data stored in the learning data storage unit 34.

[0073] The risk prediction model is a trained model that receives driving-related information as input and outputs risk factor type labels. The risk prediction model can be generated by training a mathematical model used for classification with multiple training data. Typical mathematical models used for classification include decision trees, random forests, logistic regression, support vector machines, and neural networks. In this embodiment, the risk prediction model generation unit 35 generates the risk prediction model by training a neural network with multiple training data.

[0074] That is, when the risk prediction model generation unit 35 inputs the driving-related information of each learning data into the input layer of the neural network, it calculates multiple weighting coefficients that constitute the neural network so that a risk factor type label of each learning data is output from the output layer of the neural network. Specifically, the output layer of the neural network outputs a matching rate for each risk factor type label.

[0075] The risk prediction model transmitting unit 36 ​​transmits the risk prediction model generated by the risk prediction model generating unit 35 to the in-vehicle processing unit 3a.

[0076] Next, the control flow of the danger prediction system 1 will be described with reference to FIGS.

[0077] FIG. 5 shows a control flow of the in-vehicle processing unit 3a of the danger prediction device 3.

[0078] S100: First, the driving-related information acquisition unit 13 sequentially acquires driving-related information.

[0079] S110: Next, the driving-related information storage unit 14 stores the driving-related information acquired by the driving-related information acquisition unit 13 in a ring buffer system.

[0080] S120: Next, the abnormal acceleration detection unit 15 monitors the acceleration of the vehicle 2 based on the output value from the acceleration sensor 8. If the abnormal acceleration detection unit 15 detects abnormal acceleration of the vehicle 2, the process proceeds to step S130. On the other hand, if the abnormal acceleration detection unit 15 does not detect abnormal acceleration of the vehicle 2, the process returns to step S100.

[0081] S130: Next, the risk factor type determination unit 16 determines the type of risk factor associated with the traveling-related information of the vehicle 2 when the abnormal acceleration detection unit 15 detects the abnormal acceleration of the vehicle 2.

[0082] S140: Next, the driving-related information transmission unit 17 transmits the risk factor type label, which is the determination result by the risk factor type determination unit 16, to the server processing unit 3b together with the corresponding driving-related information, and ends the process.

[0083] FIG. 6 shows a control flow of the server processing unit 3b of the danger prediction device 3.

[0084] S200: First, the driving-related information receiving unit 32 receives driving-related information and risk factor type labels from a plurality of vehicles 2.

[0085] S210: Next, the learning data generating unit 33 generates learning data in which the driving-related information is used as an explanatory variable and the risk factor type label indicating the type of risk factor is used as a response variable.

[0086] S220: Next, the learning data storage unit 34 stores the learning data generated by the learning data generation unit 33.

[0087] S230: Next, the risk prediction model generation unit 35 determines whether sufficient learning data has been accumulated in the learning data accumulation unit 34. If it is determined that sufficient learning data has been accumulated in the learning data accumulation unit 34, the risk prediction model generation unit 35 proceeds to step S240. On the other hand, if it is determined that sufficient learning data has not been accumulated in the learning data accumulation unit 34, the risk prediction model generation unit 35 returns the process to S200.

[0088] S240: Next, the risk prediction model generation unit 35 generates a risk prediction model as a trained model by training a mathematical model based on the plurality of training data stored in the training data storage unit 34.

[0089] S250: Next, the risk prediction model transmitting unit 36 ​​transmits the risk prediction model generated by the risk prediction model generating unit 35 to the plurality of vehicles 2, and the process ends.

[0090] FIG. 7 shows a control flow of the in-vehicle processing unit 3a of the danger prediction device 3.

[0091] S300: First, the risk prediction model receiving unit 18 receives the risk prediction model from the server processing unit 3 b of the risk prediction device 3 and transmits it to the risk prediction unit 19 .

[0092] S310: Next, the driving-related information acquisition unit 13 acquires driving-related information.

[0093] S320: Next, the risk prediction unit 19 predicts whether or not there is a risk to the vehicle 2 using the driving-related information acquired by the driving-related information acquisition unit 13 and the risk prediction model received by the risk prediction model reception unit 18.

[0094] Specifically, the risk prediction unit 19 acquires a matching rate (probability) for each risk factor type label output to the output layer of the risk prediction model when driving-related information is input to the input layer of the risk prediction model. If there is a risk factor type label with a class matching rate of 80% or more among the multiple risk factor type labels, the risk prediction unit 19 predicts that there is a risk for the vehicle 2 corresponding to the risk factor type label.

[0095] In this embodiment, the multiple risk factor type labels are "pedestrian turning right," "oncoming two-wheeled vehicle turning right," "oncoming vehicle going straight and turning right," "pedestrian going straight," "oncoming two-wheeled vehicle going straight," "oncoming vehicle going straight and turning left," "pedestrian turning left," "oncoming two-wheeled vehicle turning left," and "oncoming vehicle going straight and turning left." These risk factor type labels are merely examples, and the total number of risk factor type labels may be more or less than these.

[0096] S330: If it is determined that there is a risk to the vehicle 2, the risk prediction unit 19 advances the process to step S340. On the other hand, if it is determined that there is no risk to the vehicle 2, the risk prediction unit 19 returns the process to step S310.

[0097] S340: When the risk prediction unit 19 predicts that there is a risk to the vehicle 2, the warning unit 20 issues a warning to the driver of the vehicle 2 by voice from the speaker 12 based on the type of risk factor (risk factor type label) predicted by the risk prediction unit 19. For example, when the class conformance rates for "pedestrian turning right" and "oncoming two-wheeled vehicle turning right" are simultaneously 80% or higher, the warning unit 20 typically issues a warning to the driver of the vehicle 2 by voice, such as "When turning right, be careful of pedestrians and oncoming two-wheeled vehicles." For example, when the class conformance rates for "pedestrian" and "oncoming two-wheeled vehicle" are simultaneously 80% or higher, the warning unit 20 may typically issue a warning to the driver of the vehicle 2 by voice, such as "Be careful of pedestrians and oncoming two-wheeled vehicles." Then, the warning unit 20 ends the processing.

[0098] The preferred embodiment of the present disclosure has been described above, and the above embodiment has the following features.

[0099] The risk prediction device 3 as a learning data generation device includes a driving-related information acquisition unit 13 (acquisition unit) that acquires driving-related information including at least the position, speed, acceleration, traveling direction, and steering angle of the vehicle 2 based on detection of acceleration of the vehicle 2 exceeding a predetermined value, a risk factor type determination unit 16 (determination unit) that determines the risk factor type including the type of moving object identified from an image captured based on the above acceleration detection, and a learning data generation unit 33 (generation unit) that generates learning data in which risk factor type labels indicating the risk factor types are associated with the driving-related information. According to the above configuration, learning data of a risk prediction model for specifically predicting risks when driving a vehicle is generated.

[0100] The driving-related information further includes road attributes of the oncoming lane.

[0101] The learning data generation method includes an acquisition step (S100) of acquiring driving-related information including at least the position, speed, acceleration, traveling direction, and steering angle of the vehicle 2 based on detection of acceleration of the vehicle 2 exceeding a predetermined value, a determination step (S130) of determining a risk factor type including the type of moving object identified from an image captured based on the detection of acceleration, and a generation step (S210) of generating learning data in which risk factor type labels indicating the risk factor types are associated with the driving-related information. According to the above method, learning data of a risk prediction model for specifically predicting risks when driving a vehicle is generated.

[0102] In the above embodiment, the acceleration information included in the traveling-related information, which is an explanatory variable of the learning data generated by the learning data generating unit 33, indicates the acceleration at a time point a predetermined time before the time point at which abnormal acceleration was detected. However, instead, the acceleration information may be discrete data that indicates the fluctuation of acceleration from a time point a predetermined time before the time point at which abnormal acceleration was detected as a reference time point on the time axis to a time point a predetermined time after the reference time point.

[0103] The program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media, such as magnetic recording media such as hard disk drives, optical recording media, magneto-optical recording media, and semiconductor memories. Semiconductor memories can be various types of rewritable read-only memories (ROMs) and random access memories (RAMs), and also include those provided as solid-state drives (SSDs). Optical recording media include Blu-ray Discs (BDs), digital versatile discs (DVDs), and compact discs (CDs). The program can also be supplied to a computer by various types of temporary computer-readable media via wired communication paths such as electric wires and optical fibers, or wireless communication paths. For example, temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. [Explanation of symbols]

[0104] 1. Hazard prediction system 2 vehicles 3. Hazard prediction device 3a In-vehicle processing unit 3b Server processing section 4. Cloud Server 5. Camera 6 GPS modules 7 Speed ​​Sensor 8 Accelerometer 9 Steering angle sensor 10 Geomagnetic Sensor 11 ECU 12 speakers 13 Driving-related information acquisition unit 14 Driving-related information storage unit 15 Abnormal acceleration detection unit 16 Risk Factor Classification Section 17 Driving-related information transmission unit 18 Risk prediction model receiver 19 Hazard Prediction Department 20 Warning part 21 Communications Department 30 Processing section 31 Communications Department 32 Driving-related information receiving unit 33 Learning data generation unit 34 Learning data storage unit 35 Risk prediction model generation unit 36 Risk prediction model transmission unit 40 Navigation Devices 41 External Server

Claims

1. an acquisition unit that acquires, based on detection of vehicle acceleration exceeding a predetermined value, travel-related information including at least the position, speed, acceleration, traveling direction, and steering angle of the vehicle at a time point that is a predetermined time prior to the time point at which the acceleration is detected; a determination unit that determines a risk factor type including a type of moving object identified from an image captured based on the detection of the acceleration; a generation unit that generates learning data in which risk factor type labels indicating the risk factor types are associated with the driving-related information; Equipped with The determination unit determining a route of the vehicle at the intersection, and then determining a means of transportation of the target of the risk factor, thereby attaching a risk factor type label of the route of the vehicle at the intersection and the means of transportation of the target of the risk factor to one piece of driving-related information; Training data generation device.

2. The driving-related information further includes road attributes of an oncoming lane. The training data generating device according to claim 1 .

3. A training data generation method executed by a computer that controls a training data generation device, comprising: an acquisition step of acquiring, based on detection of acceleration of the vehicle exceeding a predetermined value, driving-related information including at least the position, speed, acceleration, traveling direction, and steering angle of the vehicle at a time point prior to a time point at which the acceleration was detected; a determination step of determining a risk factor type including a type of moving object identified from an image captured based on the detection of the acceleration; a generating step of generating learning data in which risk factor type labels indicating the risk factor types are associated with the driving-related information; Including, In the determining step, determining a route of the vehicle at the intersection, and then determining a means of transportation of the target of the risk factor, thereby attaching a risk factor type label of the route of the vehicle at the intersection and the means of transportation of the target of the risk factor to one piece of driving-related information; Training data generation method.

4. an acquisition step of acquiring, based on detection of acceleration of the vehicle exceeding a predetermined value, driving-related information including at least the position, speed, acceleration, traveling direction, and steering angle of the vehicle at a time point prior to a time point at which the acceleration was detected; a determination step of determining a risk factor type including a type of moving object identified from an image captured based on the detection of the acceleration; a generating step of generating learning data in which risk factor type labels indicating the risk factor types are associated with the driving-related information; on the computer, In the determining step, determining a route of the vehicle at the intersection, and then determining a means of transportation of the target of the risk factor, thereby attaching a risk factor type label of the route of the vehicle at the intersection and the means of transportation of the target of the risk factor to one piece of driving-related information; Training data generation program.

5. A program executed by a computer that controls a training data generation device, A program for causing a computer to predict a risk to the vehicle using a trained model generated by training the computer using the training data according to claim 1.

Images