Information processing device, driving assistance system, and information processing method

Abstract

The invention provides an information processing device, a driving assistance system and an information processing method, which can extract driving experience information capable of performing automatic driving with minimum cost and the like from driving data, transmit the driving experience information extracted according to the driving environment and driving conditions of a vehicle, and provide driving assistance with high comfort and usability. The server (1) transmits, to the vehicle, driving experience information with the lowest cost among the costs of the plurality of driving actions calculated from the driving data of the plurality of vehicles in the travel section and the travel condition, thereby reducing the release of the automatic driving.

Description

Technical Field

[0001] The present invention relates to an information processing device, a driving assistance system, and an information processing method for assisting autonomous driving control of an autonomous vehicle, for example. Background Art

[0002] The market for autonomous driving is growing, and in order to achieve, for example, level 3 or higher autonomous driving, it is necessary to upgrade the system.

[0003] Patent document 1 discloses the following autonomous driving assistance device and method: a path suitable for each function of the autonomous driving vehicle (avoiding a right turn if it is difficult, etc.) is calculated using function achievement values ​​(success / failure counts) representing the degree of achievement of each of the multiple functions of the autonomous driving vehicle during autonomous driving.

[0004] Patent Document 2 discloses a driving assistance device and a method thereof for setting assistance information for driving assistance of a moving body according to the driving difficulty of the moving body. Prior art literature Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-008411 Patent Document 2: International Publication No. 2021 / 229671 Summary of the invention Problem that the invention aims to solve

[0006] However, in the automatic driving system that extracts driving experience information from the driving data of the vehicle and sends the extracted driving experience information to the vehicle as described above, if it is an automatic driving vehicle, it is necessary to request the driver to take over (the system returns the driving to the driver) when it is difficult to continue the automatic driving during driving. In this case, as long as the driver can respond immediately, an accident may occur if the response is delayed. In addition, in a complex driving environment, there is a possibility of frequent takeover requests, which increases the load on the driver and also leads to a decrease in system availability.

[0007] The present invention has been completed in view of the above-mentioned problems, and its purpose is to provide an information processing device, a driving assistance system and an information processing method that extracts driving experience information from driving data that can perform automatic driving at minimum cost, sends the driving experience information extracted based on the driving environment and driving conditions of the vehicle, and can provide driving assistance with high comfort and usability. Technical means of solving problems

[0008] The information processing device of the present invention for solving the above-mentioned problems is characterized in that it comprises: a driving data acquisition unit, which acquires driving data of a vehicle; a driving action extraction unit, which extracts driving actions from the driving data; an execution time acquisition unit, which acquires the execution time required for the driving action; a cost acquisition unit, which acquires the cost of the driving action based on the unit price corresponding to the driving action and the execution time; and a driving experience information storage unit, which stores driving experience information, wherein the driving experience information associates the driving action with the cost, interval and driving conditions corresponding to the driving action, and the driving experience information storage unit stores the driving experience information based on multiple costs acquired from the driving data of multiple vehicles under the interval and the driving conditions. Effects of the Invention

[0009] According to the present invention, in order to enable the autonomous driving vehicle to use the autonomous driving function that is as easy to execute as possible, for example, the server sends driving experience information that can be driven at the minimum cost to the vehicle based on past driving data, and the vehicle plans the driving trajectory based on the driving experience information and continues autonomous driving. As a result, the vehicle uses the autonomous driving function that is as easy to execute as possible, so the number of times the autonomous driving is released (= the number of takeover requests to the driver) can be reduced. In addition, if the number of takeover requests to the driver is reduced, the load on the driver is reduced and the availability of the system is also improved. At the same time, it also helps to reduce traffic accidents caused by delays in taking over. In addition, by driving at the minimum cost, the remaining computing resources can also be used for other tasks.

[0010] Other features related to the present invention will become apparent from the description of this specification and the accompanying drawings. In addition, other problems, structures and effects than those described above will become clear from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is an overall diagram conceptually showing the driving assistance system according to the first embodiment. Figure 2A This is a block diagram illustrating the internal structure of the server in Example 1. Figure 2B This is a block diagram illustrating the internal structure of the data operation device of the server in Example 1. Figure 3 This is a diagram for explaining an example of event extraction in the event extraction unit in the first embodiment. Figure 4 This is a block diagram illustrating the internal structure of the vehicle control device in Example 1. Figure 5 This is a block diagram illustrating the internal structure of the recognition judgment device in Example 1. Fig. 6AThis is a diagram showing an example of setting the automatic driving function and unit price in Example 1. Figure 6B This is a diagram for explaining the setting of unit prices based on the difficulty level of the driving operation assistance and sensing functions in the first embodiment. Figure 7 This is a diagram showing an extraction example of the driving action extraction unit including the automatic driving function in the first embodiment. Figure 8 This is a diagram showing a calculation example of the cost calculation unit including the execution time and total cost of the automatic driving function in Example 1. Fig. 9 This is a diagram for explaining a calculation example in which the cost calculation unit in the first embodiment calculates the total cost. Fig.10 This is a diagram for explaining an example of driving experience information stored in the driving experience information database in the first embodiment. Fig.11 This is a flowchart for explaining data processing (at the time of analysis (at the time of generating driving experience information)) by the data operation device of the server in the first embodiment. Fig. 12A This is a flowchart for explaining the data processing (when the driving experience information is transmitted and updated) of the data operation device of the server in the first embodiment. Fig. 12B This is a flowchart for explaining data processing (when driving experience information is transmitted and updated) of a trajectory planning unit included in the recognition and determination device of the vehicle control device in the first embodiment. Fig.13 This is a diagram showing an example of setting information sources and weights for understanding the surrounding environment in the second embodiment. Fig.14A This is a diagram showing an example of setting weather and correction amount in the third embodiment. Fig. 14B This is a diagram showing an example of setting information sources, weather, and correction amounts for understanding the surrounding environment in the third embodiment. Fig.15A This is a diagram showing a setting example of the time period and the correction amount in the third embodiment. Fig. 15B This is a diagram showing an example of setting information sources, time periods, and correction amounts for understanding the surrounding environment in the third embodiment. Fig.16 This is a block diagram illustrating the internal structure of the data operation device of the server in Example 4. Fig.17 This is a diagram for explaining an example of setting conditions for constraint conditions in the fourth embodiment. DETAILED DESCRIPTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In all the drawings for describing the embodiments, the same reference numerals are given to parts having the same functions, and their repeated descriptions may be omitted.

[0013] [Example 1] Reference Figure 1 to Figure 12B Embodiment 1 of the present invention will be described.

[0014] Figure 1 This is a diagram conceptually showing the entire driving assistance system in the first embodiment.

[0015] The driving assistance system includes a server 1 and a plurality of vehicles Ve (Ve1, Ve2, Ve3, ..., VeN) as mobile bodies. The server 1 and the vehicle Ve can communicate bidirectionally through wireless communication and exchange information. The server 1 extracts driving experience information from the driving data sent by the vehicle Ve, and sends the driving experience information extracted based on the driving environment and driving conditions of the vehicle Ve to the vehicle Ve. The vehicle Ve uses the driving experience information sent from the server 1 to perform trajectory planning (setting of target trajectory or target speed) and perform advanced driving assistance system (ADAS) or automatic driving (AD).

[0016] Figure 2A This is a block diagram illustrating the internal structure of the server in Example 1.

[0017] The server 1 that transmits and receives information to and from the vehicle Ve includes a communication device 100 , a data operation device 110 , a map system 120 , and a storage device 130 .

[0018] The communication device 100 performs bidirectional communication of information between itself and the vehicle Ve by wireless communication. The communication device 100 transmits driving experience information to the vehicle Ve, and receives supply of feedback information including driving data including position information, path information, and the like from the vehicle Ve.

[0019] The data operation device 110 obtains feedback information including driving data of the vehicle Ve from the communication device 100. Then, based on the feedback information, the cost (described later) corresponding to the driving action of the driving data is obtained, and stored together with the driving action as driving experience information, and the driving experience information (with cost) extracted based on the driving environment and driving conditions of the vehicle Ve is provided to the communication device 100.

[0020] The map system 120 is a system for managing map data on the server 1. Alternatively, a method of acquiring map data from an external map system to the server 1 is also possible.

[0021] The storage device 130 stores data used in data processing by the data operation device 110 , data in the process of data processing, data of data processing results, and the like.

[0022] Figure 2B This is a block diagram illustrating the internal structure of the data operation device of the server in Example 1.

[0023] The data computing device 110 includes a driving data acquisition unit 111, an external environment data acquisition unit 112, an event extraction unit 113, a road information extraction unit 114, a driving action extraction unit 115, an execution time calculation unit 116, a cost calculation unit 117, a driving experience information extraction unit 118, and a driving experience information delivery unit 119.

[0024] The driving data acquisition unit 111 acquires driving data sent from the vehicle Ve via the communication device 100. The driving data includes speed information, acceleration / deceleration information, various internal sensor information (wiper ON / OFF, direction indicator signs, etc.), external sensor information (camera image information, etc.) and its recognition results, various vehicle control information (action signs of various automatic driving functions, etc.), route information (car navigation, etc.), location information, etc.

[0025] The external environment data acquisition unit 112 acquires external environment data sent from the vehicle Ve via the communication device 100. The external environment data acquisition unit 112 acquires weather information and the like as external environment data, but is not limited to weather information. In addition to weather information, road condition information including road surface conditions and the like may also be acquired. In addition, in the present embodiment, the external environment data refers to data sent from the vehicle Ve (which may be any of the cases where the vehicle itself identifies the data or the vehicle Ve acquires the data from an external supplier), but is not limited thereto. External environment information may also be acquired from an external service provider. The acquired data is associated with the driving data before and after the occurrence of the event described later, and is stored in the storage device 130 (analysis data 132).

[0026] The event extraction unit 113 detects an event from the driving data according to a predetermined rule, extracts the driving data before and after the event, and stores it in (the analysis data 132 of) the storage device 130. Before and after the event refers to a predetermined time before and after the event, a predetermined interval before and after the event, a predetermined distance before and after the event, etc. Figure 3 This is a diagram for explaining an example of event extraction by the event extraction unit 113 in Embodiment 1. Since it is difficult to analyze driving data of the entire trip, driving data before and after an event occurs is used as an analysis target.

[0027] The road information extraction unit 114 performs map matching based on the location information (or time series data of the location information) and the map information to obtain road information including the road ID or road section being traveled. The acquired data is associated with the driving data before and after the event and stored in the storage device 130 (analysis data 132). In addition, in this embodiment, as an example of map information, high-precision map information with lane unit information is used, but it is not necessary to be limited to this. In the case of using map information of road units such as a car navigation map, lane information can be assigned to the map information based on the attribute information and specifications of the road. In addition, here, the map information is obtained from the map data 131 of the storage device 130 on the server 1 via the map system 120, but in addition to the map data 131 stored in the storage device 130 of the server 1, map data obtained from an external system (map supplier, etc.) can also be used.

[0028] The driving action extraction unit 115 extracts driving actions including the automatic driving function from the driving data before and after the event (see Figure 7 ) and stored in the storage device 130 (analysis data 132).

[0029] The execution time calculation unit 116 calculates the execution time required for executing the driving action extracted by the driving action extraction unit 115 (refer to Figure 8 ) and stored in the storage device 130 (analysis data 132).

[0030] The cost calculation unit 117 calculates the unit price corresponding to the driving action extracted by the driving action extraction unit 115 based on the unit price (refer to Fig. 6A ) and the execution time obtained by the execution time calculation unit 116, and calculates the cost corresponding to the driving action (refer to Figure 8 ) and stored in the storage device 130 (analysis data 132).

[0031] The driving experience information extraction unit 118 performs statistical processing based on the driving data after event extraction, the corresponding road information, and the external environment data, and extracts driving experience information such as the normalized vehicle dynamics and the method of passing the same road ID more quickly from the analysis data 132. That is, in this embodiment, the driving experience information extraction unit 118 extracts driving experience information that associates the driving action extracted from the driving data after event extraction with the cost corresponding to the driving action, the section (road ID or road section) in the vehicle's driving path, and the driving conditions. Here, the driving experience information extraction unit 118 extracts the driving experience information with the minimum cost or the driving experience information with the highest cost occurrence frequency based on the multiple costs obtained from the multiple driving data in the section (road ID or road section) in the vehicle's driving path and the driving conditions. The extracted driving experience information is stored in the driving experience information database 133 of the storage device 130. Thus, the driving experience information that determines the driving action of the vehicle is generated and stored (saved).

[0032] The driving experience information delivery unit 119 retrieves and extracts the corresponding driving experience information from the driving experience information registered in the driving experience information database 133 based on the driving road information (road ID, etc.) determined by the road information extraction unit 114, the path information sent from the vehicle Ve, the external environment information (weather information, etc.), and the date and time information, and sends it to the vehicle Ve via the communication device 100. In addition, the determined driving experience information is not limited to the driving experience information corresponding to the road ID on which the vehicle is currently traveling, and the driving experience information of the next road ID passed can be determined and sent based on the path information. It can be arbitrarily determined in advance that the driving experience information of the next few road IDs will be sent to the vehicle Ve. Furthermore, in the case where the external environment information cannot be obtained, the corresponding driving experience information is determined based on the remaining information. In addition, in this embodiment, the path information is used, but it is not necessary to be limited to this. For example, it can be the current location information of the vehicle Ve, or it can be the location information of the place where the user wants to go. Furthermore, it is also possible to consider a method in which the driving data is sent from the vehicle Ve to the server, and the server searches for the path information or location information therefrom.

[0033] Thereby, the driving experience information extracted based on the driving environment and driving conditions of the vehicle Ve can be transmitted to the vehicle Ve.

[0034] Figure 4 This is a block diagram illustrating the internal structure of the vehicle control device in Example 1.

[0035] The vehicle control device 2 is mounted on the vehicle Ve. The vehicle control device 2 includes an on-vehicle wireless communication device 200, an external sensor 210, an internal sensor 220, a navigation system 230, an HMI device 240, a recognition judgment device 250, a vehicle motion control device 260, a steering control device 270, a throttle control device 280, and a brake control device 290.

[0036] The vehicle-mounted wireless communication device 200 is a communication device used for wireless communication with the server 1 or other vehicles, and performs the following processing: obtaining driving experience information from the server 1 and providing it to the cognitive judgment device 250, and sending the driving data and external environment information obtained from the cognitive judgment device 250 to the server 1.

[0037] The external sensor 210 is an environment recognition sensor for recognizing the surrounding environment of the vehicle Ve, and has a stereo camera 210-b and a LiDAR 210-a. In addition to the above, the environment recognition sensor can be a combination of a monocular camera and a laser radar, or an ultrasonic sensor can be used. In this embodiment, it is composed of a stereo camera 210-b.

[0038] The inner limit sensor 220 is a sensor for detecting the operating state of the vehicle Ve, and is composed of at least one sensor such as an acceleration sensor, a vehicle speed sensor, an engine rotation sensor, and a throttle opening sensor.

[0039] The navigation system 230 is a system for displaying a route to a destination on a map based on map information and vehicle position information, and for guiding the vehicle Ve to the destination.

[0040] The HMI device 240 is a device operated by an occupant of the vehicle Ve, and is operated when inputting a destination into the navigation system 230 , for example.

[0041] The recognition judgment device 250 performs a process of recognizing the surrounding environment around the vehicle using driving experience information, etc. The recognition judgment device 250 sends information on the target trajectory and target speed to the vehicle motion control device 260 based on the result of the recognition process.

[0042] The vehicle motion control device 260 outputs command values ​​for controlling the steering, throttle opening, and braking of the vehicle Ve to the steering control device 270, the throttle control device 280, and the brake control device 290. The steering control device 270, the throttle control device 280, and the brake control device 290 control the steering, throttle opening, and braking of the vehicle Ve according to the command values ​​from the vehicle motion control device 260.

[0043] The recognition judgment device 250, the vehicle motion control device 260, the steering control device 270, the throttle control device 280, and the brake control device 290 are composed of an electronic control unit (hereinafter referred to as ECU) having a microcomputer. The ECU may be provided with a dedicated ECU for each device, or may be a structure in which two or more devices are integrated into one ECU.

[0044] Figure 5 This is a block diagram illustrating the internal structure of the recognition judgment device in Example 1.

[0045] The recognition determination device 250 includes a recognition unit 251 , a driving action planning unit 252 , a trajectory planning unit 253 , and a driving experience information search unit 255 .

[0046] The recognition unit 251 acquires environmental data of the surrounding environment of the vehicle Ve from the external sensor 210. The recognition unit 251 performs a process of recognizing a target object around the vehicle using detection information detected by the external sensor 210.

[0047] The driving experience information retrieval unit 255 acquires the driving experience information 254 transmitted from the server 1 via the in-vehicle wireless communication device 200. The driving experience information retrieval unit 255 retrieves and determines appropriate driving experience information from the acquired driving experience information 254 (also referred to as driving experience information candidates) according to the current driving environment and driving conditions.

[0048] The driving action planning unit 252 performs a process of planning the driving action of the vehicle Ve based on the recognition result of the recognition unit 251, the detection result of the inner boundary sensor 220, the map information from the navigation system 230, and the search result of the driving experience information search unit 255. The driving action planning unit 252 determines whether to adopt the driving experience information determined by the driving experience information search unit 255 based on the recognition result of the recognition unit 251, the detection result of the inner boundary sensor 220, and the map information from the navigation system 230, and plans the driving action of the vehicle Ve.

[0049] The trajectory planning unit 253 plans the trajectory (target trajectory and target speed) of the vehicle Ve based on the external information, the driving action plan, the detection result of the inner boundary sensor 220, and the map information from the navigation system 230. When it is determined that the driving experience information determined by the driving experience information retrieval unit 255 is adopted, the trajectory planning unit 253 plans the trajectory of the vehicle Ve based on the external information, the driving action plan, the detection result of the inner boundary sensor 220, the map information from the navigation system 230, and the driving experience information. On the other hand, when it is determined that the driving experience information determined by the driving experience information retrieval unit 255 is not adopted, the trajectory planning unit 253 plans the trajectory of the vehicle Ve based on the external information, the driving action plan, the detection result of the inner boundary sensor 220, and the map information from the navigation system 230 (without using the determined driving experience information). In addition, the trajectory planning unit 253 sends the driving data to the server 1 via the vehicle-mounted wireless communication device 200.

[0050] Fig. 6A This is a diagram showing an example of setting the automatic driving function and unit price in Example 1. Figure 6B This is a diagram for explaining the setting of unit prices based on the difficulty level of the driving operation assistance and sensing functions in the first embodiment.

[0051] like Fig. 6A and Figure 6B As shown, the unit price of the autonomous driving function (driving action) is preset according to the difficulty of the driving operation assistance and sensing function of the autonomous driving function.

[0052] Figure 7 This is a diagram showing an extraction example of the driving action extraction unit including the automatic driving function in the first embodiment. Figure 8 This is a diagram showing a calculation example of the cost calculation unit including the execution time and total cost of the automatic driving function in Example 1. Fig. 9 This is a diagram for explaining an example of calculating the total cost by the cost calculation unit in the first embodiment.

[0053] like Figure 7 As shown in FIG. 1 , in the driving action extraction unit 115, information including the automatic driving function (driving action) is extracted. Figure 8 As shown, in the cost calculation unit 117, in addition to Figure 7 In addition, information including the total cost calculated using the execution time of the autonomous driving function is also determined.

[0054] like Fig. 9As shown, the cost calculation unit 117 calculates the cost of each element constituting the autonomous driving function (driving action) for each extracted event (=unit price of the autonomous driving function×execution time), adds up the calculated cost of each element, and calculates the total cost of the autonomous driving function (driving action). As described above, the driving experience information extraction unit 118 extracts the driving experience information with the minimum cost or the highest frequency based on the multiple costs (total costs) obtained from the multiple driving data under the interval (road ID or road interval) in the vehicle's driving path and the driving conditions, and stores (saves) it in the driving experience information database 133 of the storage device 130.

[0055] Fig.10 This is a diagram for explaining an example of driving experience information stored in the driving experience information database in the first embodiment.

[0056] like Fig.10 As shown, the driving experience information stored in the driving experience information database 133 includes the following: (a) "management information", (b1) (when the road section information can be obtained, (b2)) "search object", (c) "generation condition", (d) "execution condition", (e) "control information", etc. In addition, the "search object" of (b1) (or (b2)), the "day" and "time" of the "generation condition" of (c) are information used by the server 1 during the search, and the "weather" and "road surface conditions" of the "generation condition" of (c), and the "execution condition" of (d) are information used by the vehicle Ve during the search.

[0057] When narrowing down the driving experience information to be sent from the driving experience information database 133 to the vehicle Ve, the driving experience information delivery unit 119 determines the corresponding driving experience information based on the road information (road ID, etc.) and the external environment information. For example, the driving experience information delivery unit 119 searches for matching driving experience information for the "search object" of (b1) (or (b2)) and the day of the week information and time information of the "generation condition" of (c) based on the road information. In this case, the driving experience information delivery unit 119 sends the "ID" of the "management information" of (a), the "generation condition" of (c), the "execution condition" of (d), and the "control information" of (e) to the vehicle Ve, and confirms the information of (c) and (d) on the vehicle side. Alternatively, the information of (c) may be confirmed as a whole on the server side and then sent to the vehicle Ve. Then, the driving experience information delivery unit 119 sends the matching items to the vehicle Ve. At this time, in order to reduce the amount of transmission, only the "ID" of the "management information" of (a), the "execution condition" of (d), and the "control information" of (e) are sent. However, all the information from (a) to (e) may be sent to the vehicle Ve.

[0058] Fig.11 This is a flowchart for explaining data processing (at the time of analysis (at the time of generating driving experience information)) by the data operation device of the server in the first embodiment.

[0059] The data computing device 110 of the server 1 acquires the external environment data (SCA100) transmitted from the vehicle Ve by the external environment data acquisition unit 112. Also, the driving data acquisition unit 111 acquires the driving data (SCA101) transmitted from the vehicle Ve.

[0060] Then, the event extraction unit 113 detects an event from the driving data, and extracts driving data before and after the event occurs (SCA 102 ).

[0061] Furthermore, the road information extraction unit 114 acquires map data from the map system 120 (SCA 103 ), and acquires road information (SCA 104 ) based on the map data and the position information (or time series data of the position information).

[0062] Then, the driving action extraction unit 115 extracts the driving action (autonomous driving function) between road nodes for the road information (each road ID, etc.) based on the driving data (SCA 105 ).

[0063] In addition, the execution time of the extracted driving action is calculated by the execution time calculation unit 116 (SCA 106 ).

[0064] In addition, the cost calculation unit 117 calculates the cost (total cost) of each driving pattern based on the unit price of the driving action defined in advance (SCA 107).

[0065] Furthermore, the driving experience information extraction unit 118 extracts the driving mode with the lowest cost or the highest frequency (the lowest in the present embodiment) for each road information and driving condition (SCA108), and (associates) the extracted driving mode with the lowest cost or the highest frequency (the lowest in the present embodiment) with the corresponding road information and driving condition, and accumulates (stores) it as driving experience information in the driving experience information database 133 of the storage device 130 (SCA109).

[0066] Fig. 12A This is a flowchart for explaining data processing (when driving experience information is delivered and updated) by the data operation device of the server in the first embodiment.

[0067] The data computing device 110 of the server 1 acquires driving data (position information, route information, etc.) sent from the vehicle Ve via the driving data acquisition unit 111 (CTS100), and acquires external environment data (weather information, etc.) sent from the vehicle Ve via the external environment data acquisition unit 112 (CTS101).

[0068] Then, the driving experience information distribution unit 119 determines the corresponding driving experience information (also called driving experience information candidates) (CTS102) based on the driving data (position information, path information, etc.) and external environment data (weather information, etc.) sent from the vehicle Ve, and sends the determined driving experience information (driving experience information candidates) to the vehicle Ve (CTS103).

[0069] Fig. 12B This is a flowchart for explaining data processing (when driving experience information is delivered and updated) of a trajectory planning unit included in the recognition and judgment device of the vehicle control device in the first embodiment.

[0070] The recognition judgment device 250 transmits driving data (position information, route information, etc.) to the server 1 (VS101), and also acquires driving experience information transmitted from the server 1 (VS102).

[0071] In addition, the driving experience information retrieval unit 255 determines appropriate driving experience information from the acquired driving experience information (also referred to as driving experience information candidates) based on the current driving environment and driving conditions (VS103). For example, the vehicle Ve sends a plurality of driving experience information with different driving conditions (e.g., driving lanes) in a section of the vehicle's driving path, and the vehicle Ve side (when driving in the section) selects the driving experience information executed based on the external information of the vehicle from the plurality of driving experience information.

[0072] Then, the driving action planning unit 252 and the trajectory planning unit 253 plan the trajectory (target trajectory and target speed) of the vehicle Ve based on the determined driving experience information (VS104).

[0073] In the driving assistance system of the present embodiment described above, the vehicle Ve sends driving data to the server 1. Furthermore, the server 1 obtains the cost corresponding to the driving action of the driving data based on the driving data sent from the vehicle Ve, and stores it together with the driving action as driving experience information. Then, the server 1 extracts the driving experience information from the driving data sent from the vehicle Ve, and sends the driving experience information extracted based on the driving environment and driving conditions of the vehicle Ve to the vehicle Ve. The vehicle Ve uses the driving experience information sent from the server 1 to perform trajectory planning (setting of target trajectory or target speed) and perform advanced driving assistance system (ADAS) or automatic driving (AD).

[0074] According to this embodiment, in order to enable the autonomous driving vehicle to use the autonomous driving function that is as easy to execute as possible, for example, the server sends driving experience information that can be driven at the minimum cost to the vehicle based on past driving data, and the vehicle plans the driving trajectory based on the driving experience information and continues autonomous driving. As a result, the vehicle uses the autonomous driving function that is as easy to execute as possible, so the number of times the autonomous driving is released (= the number of takeover requests to the driver) can be reduced. In addition, if the number of takeover requests to the driver is reduced, the load on the driver is reduced and the availability of the system is also improved. At the same time, it also helps to reduce traffic accidents caused by delays in taking over. In addition, by driving at the minimum cost, the remaining computing resources can also be used for other tasks.

[0075] [Example 2] Below, refer to Fig.13 Embodiment 2 of the present invention will be described.

[0076] The present embodiment is characterized in that the driving experience information is generated in consideration of a weight (also referred to as a negotiation cost) based on the surrounding environment.

[0077] Fig.13 This is a diagram showing an example of setting information sources and weights for understanding the surrounding environment in the second embodiment.

[0078] like Fig.13 As shown, the weight is set in advance according to the difficulty of negotiation with the information source for understanding the surrounding environment. In this case, the cost of the automatic driving function (driving action) is calculated by the unit price of the automatic driving function×execution time×weight.

[0079] According to the present embodiment, the cost calculation unit 117 can obtain the cost including the weight based on the surrounding environment, generate and send driving experience information to the vehicle. As a result, the vehicle uses the automatic driving function that is as easy to execute as possible, so the number of times the automatic driving is released (= the number of takeover requests to the driver) can be reduced. In addition, if the number of takeover requests to the driver is reduced, the load on the driver is reduced and the availability of the system is improved. At the same time, it also helps to reduce traffic accidents caused by delays in taking over. In addition, by driving at the minimum cost, the remaining computing resources can also be used for other tasks.

[0080] [Example 3] Below, refer to Figures 14A to 15B Embodiment 3 of the present invention will be described.

[0081] The present embodiment is characterized in that the driving experience information is generated in consideration of weather and time period.

[0082] Fig.14A This is a diagram showing an example of setting weather and correction amount in the third embodiment. Fig. 14B This is a diagram showing an example of setting information sources, weather, and correction amounts for understanding the surrounding environment in the third embodiment. Fig.15A This is a diagram showing a setting example of the time period and the correction amount in the third embodiment. Fig. 15B This is a diagram showing an example of setting information sources, time periods, and correction amounts for understanding the surrounding environment in the third embodiment.

[0083] It can be assumed that the difficulty of understanding the surrounding environment and the difficulty of achieving functions will change depending on the weather and time period. Fig.14A , Fig.15A As shown, the correction amount is preset according to the weather or time period, and the unit price of the automatic driving function (driving action) is corrected.

[0084] in addition, Fig.14A , Fig.15A The correction values ​​listed in the table do not necessarily apply to the Fig.13 For example, when fog occurs, the correction amount is based on Fig.13 The correction amount when fog occurs can be considered, for example, the external sensor of the vehicle is +5 (for example, in the case of a camera, if fog occurs, it is difficult to see the front, so it is difficult to obtain correct information), the roadside sensor is +2, the vehicle-to-vehicle communication is +1, the building sensor is +3, and the pedestrians on the road are +5. Therefore, it is not like Fig.14A Instead of applying the correction as shown in Fig. 14B As shown, it is possible to consider setting the correction amount based on the information source for understanding the surrounding environment and the difficulty of obtaining information when considering the influence of weather. Fig. 14B The correction amount shown is an example. Similarly, for example, at night or late at night, the correction amount is based on Fig.13 The correction amount at night or late at night can be considered as +2 for the external sensor of the vehicle (for example, in the case of a camera, it is difficult to see the front at night or late at night, so it is difficult to obtain correct information), +1 for the roadside sensor, building sensor or pedestrian on the road, and 0 for vehicle-to-vehicle communication (since it is not related to brightness). Therefore, it is not like Fig.15A Instead of applying the correction as shown in Fig. 15B As shown, it is possible to consider setting the correction amount based on the information source for understanding the surrounding environment and the difficulty of obtaining information when considering the influence of the time period. Fig. 15B The correction amount shown is an example.

[0085] In this case, as an example, the cost of the autonomous driving function (driving action) is calculated by the unit price of the autonomous driving function×execution time×(weight+correction amount).

[0086] According to this embodiment, the cost calculation unit 117 can correct the unit price according to the weather information and time information, obtain the cost, and thus generate and send the driving experience information to the vehicle. As a result, the vehicle uses the automatic driving function that is as easy to execute as possible, so the number of times the automatic driving is released (= the number of takeover requests to the driver) can be reduced. In addition, if the number of takeover requests to the driver is reduced, the load on the driver is reduced and the availability of the system is also improved. At the same time, it also helps to reduce traffic accidents caused by delays in taking over. In addition, by driving at the minimum cost, the remaining computing resources can also be used for other tasks.

[0087] [Example 4] Below, refer to Fig.16 , Fig.17 Embodiment 4 of the present invention will be described.

[0088] The present embodiment is characterized in that the driving experience information is generated in consideration of constraints.

[0089] Fig.16 This is a block diagram illustrating the internal structure of the data operation device of the server in Example 4.

[0090] and Figure 2B Compared with the block diagram of the first embodiment shown in FIG. 1 , a constraint setting unit 161 is introduced, and constraint conditions are added as input information of the driving experience information extraction unit 118 .

[0091] Fig.17 This is a diagram for explaining an example of setting conditions for constraint conditions in the fourth embodiment.

[0092] When extracting driving experience information, the driving experience information extraction unit 118 extracts driving experience information that satisfies a predetermined constraint condition, and the driving experience information associates the driving action extracted from the driving data after event extraction with the cost corresponding to the driving action, the section (road ID or road section) in the vehicle's driving path, and the driving condition. In other words, the driving experience information extraction unit 118 extracts driving experience information that satisfies the predetermined constraint condition with the minimum cost or the highest frequency based on the sections (road ID or road section) in the vehicle's driving path and the multiple costs obtained from the multiple driving data under the driving conditions. The extracted driving experience information is stored in the driving experience information database 133 of the storage device 130.

[0093] In the driving assistance system of the above-mentioned embodiment, the vehicle Ve sends driving data to the server 1. And, based on the driving data sent from the vehicle Ve, the server 1 obtains the cost corresponding to the driving action of the driving data, and stores it together with the driving action as driving experience information. At this time, the driving experience information that satisfies the predetermined constraints is stored. Then, the server 1 extracts the driving experience information from the driving data sent by the vehicle Ve, and sends the driving experience information extracted based on the driving environment and driving conditions of the vehicle Ve to the vehicle Ve. The vehicle Ve uses the driving experience information sent from the server 1 to perform trajectory planning (setting of the target trajectory or target speed) and perform advanced driving assistance system (ADAS) or automatic driving (AD).

[0094] According to this embodiment, in order to enable the autonomous driving vehicle to use the autonomous driving function that is as easy to execute as possible, for example, the server sends driving experience information that can meet the pre-set specified constraints and can travel at the minimum cost to the vehicle based on past driving data, and the vehicle plans the driving trajectory based on the driving experience information and continues autonomous driving. As a result, the vehicle uses the autonomous driving function that is as easy to execute as possible, so the number of times the autonomous driving is released (= the number of takeover requests to the driver) can be reduced. In addition, if the number of takeover requests to the driver is reduced, the load on the driver is reduced and the availability of the system is also improved. At the same time, it also helps to reduce traffic accidents caused by delays in takeover response. In addition, by driving at the minimum cost, the remaining computing resources can also be used for other tasks.

[0095] In addition, the above embodiment shows an example of a case where the cost calculation process is performed on the server 1 side, but it is not limited to the server 1 side, and the calculation may be performed on the vehicle Ve side. That is, including the case where the process on the server 1 side is executed on the vehicle Ve side, there are the following cases. (1) The driving data is transmitted from the vehicle Ve to the server 1. The cost calculation unit 117 calculates the cost and the driving experience information extraction unit 118 extracts the driving experience information on the server 1 side (the above-mentioned embodiments 1 to 4). (2) On the vehicle Ve side, the cost calculation unit 117 performs cost calculation processing and sends the result to the server 1 , and the driving experience information extraction unit 118 extracts the driving experience information on the server 1 side. (3) On the vehicle Ve side, the cost calculation unit 117 performs cost calculation processing, and the driving experience information extraction unit 118 also extracts driving experience information on the vehicle Ve side. The extracted information is transmitted to the server 1.

[0096] [Summary of Examples 1 to 4] As described above, the information processing device of this embodiment includes: a driving data acquisition unit 111, which acquires the driving data of the vehicle; a driving action extraction unit 115, which extracts the driving action from the driving data; an execution time acquisition unit (execution time calculation unit 116), which acquires the execution time required for the driving action; a cost acquisition unit (cost calculation unit 117), which acquires the cost of the driving action based on the unit price corresponding to the driving action and the execution time; and a driving experience information storage unit (driving experience information database 133), which stores driving experience information, which associates the driving action with the cost, interval and driving conditions corresponding to the driving action, and the driving experience information storage unit (driving experience information database 133) stores the driving experience information based on multiple costs acquired from the driving data of multiple vehicles under the interval and the driving conditions (Example 1, etc.).

[0097] In addition, the driving experience information storage unit (driving experience information database 133) stores driving experience information with the smallest cost or driving experience information with the highest cost appearance frequency from among the plurality of costs (Example 1, etc.).

[0098] In addition, the cost acquisition unit (cost calculation unit 117) acquires the cost including a weight based on the surrounding environment (Example 2).

[0099] In addition, the cost acquisition unit (cost calculation unit 117) corrects the unit price based on at least one of weather information and time period information to acquire the cost (Example 3).

[0100] In addition, the cost acquisition unit (cost calculation unit 117) corrects the unit price based on at least one of weather information and time period information and an information source for understanding the surrounding environment, thereby acquiring the cost (Example 3).

[0101] In addition, the driving experience information storage unit (driving experience information database 133) stores driving experience information that satisfies predetermined constraints that are set in advance (Embodiment 4).

[0102] In addition, there is also a driving experience information distribution unit 119, which sends a plurality of driving experience information with different driving conditions in the section to the vehicle, and can select driving experience information executed according to external information of the vehicle from the plurality of driving experience information on the vehicle side (Example 1, etc.).

[0103] Furthermore, the driving assistance system of the present embodiment includes the information processing device and a vehicle, and performs driving assistance for the vehicle by executing the driving experience information stored in the driving experience information storage unit (driving experience information database 133) on the vehicle side.

[0104] In addition, the information processing method of the present embodiment includes: a driving data acquisition step for acquiring driving data of a vehicle; a driving action extraction step for extracting driving actions from the driving data; an execution time acquisition step for acquiring the execution time required for the driving action; a cost acquisition step for acquiring the cost of the driving action based on the unit price corresponding to the driving action and the execution time; and a driving experience information storage step for storing driving experience information, which associates the driving action with the cost, interval and driving conditions corresponding to the driving action. In the driving experience information storage step, the driving experience information is stored based on multiple costs acquired from the driving data of multiple vehicles under the interval and the driving conditions.

[0105] According to this embodiment, in order to enable the autonomous driving vehicle to use the autonomous driving function that is as easy to execute as possible, for example, the server sends driving experience information that can be driven at the minimum cost to the vehicle based on past driving data, and the vehicle plans the driving trajectory based on the driving experience information and continues autonomous driving. As a result, the vehicle uses the autonomous driving function that is as easy to execute as possible, so the number of times the autonomous driving is released (= the number of takeover requests to the driver) can be reduced. In addition, if the number of takeover requests to the driver is reduced, the load on the driver is reduced and the availability of the system is also improved. At the same time, it also helps to reduce traffic accidents caused by delays in takeover response. In addition, by driving at the minimum cost, the remaining computing resources can also be used for other tasks.

[0106] The embodiments of the present invention are described in detail above, but the present invention is not limited to the embodiments described, and various design changes can be made without departing from the spirit of the present invention described in the claims. For example, the embodiments described are detailed descriptions for easily understanding the present invention, and are not necessarily limited to having all the described structures. In addition, a part of the structure of some embodiments can be replaced by the structure of other embodiments, and the structure of other embodiments can be added to the structure of some embodiments. Furthermore, for a part of the structure of each embodiment, other structures can be added, deleted, or replaced. Explanation of symbols

[0107] 1: Server, 100: Communication device, 110: Data computing device, 111: Driving data acquisition unit, 112: External environment data acquisition unit, 113: Event extraction unit, 114: Road information extraction unit, 115: Driving action extraction unit, 116: Execution time calculation unit (execution time acquisition unit), 117: Cost calculation unit (cost acquisition unit), 118: Driving experience information extraction unit, 119: Driving experience information distribution unit, 120: Map system, 130: Storage device, 131: Map data, 132: Analysis data, 133: Driving experience Experience information database (driving experience information storage unit), 2: vehicle control device, 200: vehicle-mounted wireless communication device, 210: external sensor, 220: internal sensor, 230: navigation system, 240: HMI device, 250: cognitive judgment device, 251: identification unit, 252: driving action planning unit, 253: trajectory planning unit, 254: driving experience information, 255: driving experience information retrieval unit, 260: vehicle motion control device, 270: steering control device, 280: throttle control device, 290: braking control device, Ve: vehicle.

Claims

1. An information processing device, characterized in that: have: a driving data acquisition unit that acquires driving data of a vehicle; a driving action extraction unit that extracts a driving action from the driving data; an execution time acquisition unit, which acquires the execution time required for the driving action; a cost acquisition unit configured to acquire a cost of the driving action based on a unit price corresponding to the driving action and the execution time; as well as a driving experience information storage unit storing driving experience information that associates the driving action with the cost, section, and travel condition corresponding to the driving action, The driving experience information storage section stores the driving experience information based on a plurality of costs acquired from driving data of a plurality of vehicles under the section and the travel condition.

2. The information processing device according to claim 1, characterized in that The driving experience information storage unit stores, from among the plurality of costs, driving experience information having the smallest cost or driving experience information having the highest cost appearance frequency.

3. The information processing device according to claim 1, characterized in that The cost acquisition unit acquires the cost including a weight based on a surrounding environment.

4. The information processing device according to claim 1, characterized in that The cost acquisition unit amends the unit price according to at least one of weather information and time period information to acquire the cost.

5. The information processing device according to claim 1, characterized in that The cost acquisition unit acquires the cost by correcting the unit price based on at least one of weather information or time period information and an information source for understanding surrounding environment.

6. The information processing device according to claim 1, characterized in that The driving experience information storage unit stores driving experience information that satisfies a predetermined constraint condition that is set in advance.

7. The information processing device according to claim 1, characterized in that: A driving experience information delivery unit is provided, which transmits a plurality of driving experience information with different driving conditions in the section to the vehicle, and can select driving experience information executed according to external information of the vehicle from the plurality of driving experience information on the vehicle side.

8. A driving assistance system, characterized in that: have: The information processing device and vehicle according to claim 1, Driving assistance of the vehicle is performed by executing the driving experience information stored in the driving experience information storage unit on the vehicle side.

9. An information processing method, characterized in that: Include: A driving data acquisition step, acquiring driving data of the vehicle; A driving action extraction step, extracting driving actions from the driving data; An execution time acquisition step, acquiring the execution time required for the driving action; a cost acquisition step of acquiring the cost of the driving action according to the unit price corresponding to the driving action and the execution time; as well as a driving experience information storage step of storing driving experience information, wherein the driving experience information associates the driving action with the cost, the section, and the driving condition corresponding to the driving action, In the driving experience information storing step, the driving experience information is stored based on a plurality of costs acquired from driving data of a plurality of vehicles under the section and the travel condition.

Images