Vehicle control system and vehicle data collection method

Abstract

To efficiently collect data effective for training of an AI in charge of cognition of an automatic driving vehicle.SOLUTION: A vehicle control system comprises: a vehicle control determination section for creating a control signal for controlling a vehicle using data acquired from a sensor mounted on the vehicle; a near-miss detection section for detecting a near-miss that a passenger riding on the vehicle feels from at least any one of a biological change of the passenger and the control signal of the vehicle and outputs a near-miss detection signal; a determination mismatch detection section for detecting a mismatch of the control signal and the near-miss detection signal in timing when the near-miss is detected from the data acquired from the sensor, the control signal created by the vehicle control determination section and the near-miss detection signal detected by the near-miss detection section; and a data storage section for storing data acquired from the sensor mounted on the vehicle and the control signal created by the vehicle control determination section correspondingly to the timing in which the mismatch is detected by the determination mismatch detection section.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a vehicle control system and a vehicle data collection method. [Background technology]

[0002] In recent years, efforts have been made to collect driving data from vehicles and use it as training data for autonomous driving AI (Artificial Intelligence) in order to prevent accidents involving autonomous vehicles. By distributing trained AI via OTA (Over The Air), it becomes possible to constantly update the vehicle control system to the latest version.

[0003] Regarding a method for collecting driving data, Patent Document 1 states that "the recording device comprises an acquisition unit, a setting unit, and a selection unit. The acquisition unit acquires biometric information of the vehicle driver. The setting unit sets the importance of the vehicle's driving information based on the biometric information acquired by the acquisition unit. The selection unit selects the driving information to be stored in the memory unit based on the importance set by the setting unit" (see abstract). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-195193 Summary of the Invention [Problem to be solved by the invention]

[0005] The information that should be obtained from the vehicle as training data for autonomous driving AI is driving information when there is a discrepancy between humans and AI in their perception and judgment of driving conditions. If there is no discrepancy between humans and AI, the AI ​​is already able to perceive and judge in the same way as humans, and new training is not necessarily required.

[0006] However, the method of Patent Document 1 records driving information when the driver senses danger, making it impossible to determine whether there was a discrepancy in perception or judgment between humans and the AI. As a result, there was a possibility that the collected information would include information that was unnecessary as training data. Furthermore, since the discrepancy in perception and judgment between humans and the AI ​​is expected to decrease as the level of autonomous driving increases, there was a possibility that the proportion of information that did not require training would increase if driving information were acquired using the method of Patent Document 1, especially in a highly autonomous driving environment.

[0007] The present invention provides a vehicle control system and a vehicle data collection method that solve the above-mentioned problems of the conventional technology and enable the efficient collection of data that is effective for training the AI ​​responsible for recognition in self-driving vehicles. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, in the present invention, a vehicle control system is configured to include a vehicle control judgment unit that creates a control signal to control the vehicle using data acquired from a sensor attached to the vehicle; a near-miss detection unit that detects near-misses felt by the occupants from biochemical changes in the occupants of the vehicle while driving the vehicle or from the vehicle's control signal, and outputs a near-miss detection signal; a judgment mismatch detection unit that detects a mismatch between the control signal created by the vehicle control judgment unit and the near-miss detection signal at the timing when the near-miss is detected, based on the data acquired from the sensor attached to the vehicle, the control signal created by the vehicle control judgment unit, and the near-miss detection signal detected by the near-miss detection unit; and a data storage unit that stores the data acquired from the sensor attached to the vehicle corresponding to the timing when the mismatch is detected by the judgment mismatch detection unit and the control signal created by the vehicle control judgment unit.

[0009] In addition, in order to solve the above-mentioned problems, the present invention provides a vehicle data collection method for collecting vehicle data in a vehicle control system equipped with a vehicle control judgment unit, a near-miss detection unit, a judgment mismatch detection unit, and a data storage unit, in which a control signal for controlling the vehicle is created by the vehicle control judgment unit using data acquired from a sensor attached to the vehicle, a near-miss felt by the occupant is detected by the near-miss detection unit from biochemical changes in the occupant in the vehicle while driving the vehicle or from the vehicle's control signal, a mismatch between the control signal and the near-miss detection signal at the timing when the near-miss is detected by the judgment mismatch detection unit is detected from the data acquired from the sensor attached to the vehicle, the control signal created by the vehicle control judgment unit, and the near-miss detection signal detected by the near-miss detection unit, and the data acquired from the sensor attached to the vehicle corresponding to the timing when the mismatch was detected by the mismatch detection unit and the control signal created by the control unit are stored in the data storage unit. [Effects of the Invention]

[0010] This invention makes it possible to efficiently collect data that is effective for training the AI ​​responsible for the recognition of autonomous vehicles. Furthermore, the collected data is used to improve the AI's functions, thereby improving the safety of the vehicle control system. Problems, configurations, and effects other than those described above will become clear from the description of the following embodiments. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a schematic configuration of a vehicle control system according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the configuration of a biological change detection unit according to the first embodiment of the present invention. [Figure 3] FIG. 3 is a flowchart showing a processing flow in the vehicle control system according to the first embodiment of the present invention. [Figure 4] FIG. 10 is a block diagram showing a schematic configuration of a vehicle control system according to a modified example of the first embodiment of the present invention. [Figure 5]FIG. 10 is a front view of a display screen of a data output unit in the vehicle control system according to a modified example of the first embodiment of the present invention. [Figure 6] FIG. 10 is a block diagram showing a schematic configuration when data is transmitted from a vehicle control system according to a second embodiment of the present invention to a cloud for processing. [Figure 7] FIG. 10 is a block diagram showing a data flow when the travel data transmitted from the vehicle control system according to the second embodiment of the present invention is managed, learning data is created, and update data is transmitted. [Figure 8] FIG. 10 is a block diagram showing the internal configuration of a cloud that receives information from a vehicle control system according to a second embodiment of the present invention. [Figure 9] FIG. 10 is a flowchart showing a processing flow in a cloud that receives information from a vehicle control system according to a second embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart showing a processing flow in a vehicle control system according to a second embodiment of the present invention. [Figure 11] 10 is a table illustrating an example of learning data generated by a learning data generating unit according to Example 2 of the present invention. [Figure 12] FIG. 10 is a block diagram showing a configuration in which a vehicle control system according to a second embodiment of the present invention is realized by an add-on device. [Figure 13] FIG. 10 is a block diagram showing a schematic configuration of a vehicle control system according to a third embodiment of the present invention. [Figure 14] FIG. 10 is a block diagram showing a configuration in which a biological change detection unit is realized by a bus in Example 3 of the present invention. [Figure 15] FIG. 10 is a block diagram showing a schematic configuration of a vehicle control system according to a fourth embodiment of the present invention. [Figure 16] FIG. 10 is a block diagram showing a schematic configuration of a vehicle control system according to a fifth embodiment of the present invention. [Figure 17] FIG. 10 is a block diagram showing an example in which a vehicle control system according to a sixth embodiment of the present invention is applied to trucks traveling in a convoy. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present invention detects near misses (when a vehicle occupant feels danger) from changes in biometric information or changes in vehicle control, and if the vehicle control system is unable to detect a vehicle control signal corresponding to the near miss at the time the near miss factor occurs, it determines that there is a judgment mismatch between the occupant and the vehicle control system.If a judgment mismatch occurs, the vehicle control signal and driving data are recorded, and learning data is created to improve the accuracy of the vehicle control system.

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings. [Example]

[0014] In this embodiment, a vehicle control system that stores driving data when a mismatch is detected according to the present invention will be described. Figure 1 is a configuration diagram of this embodiment, Figure 2 is a configuration example of a biological change detection unit when this embodiment is implemented in a passenger vehicle, and Figure 3 is a process flow diagram for storing data in this embodiment.

[0015] First, a description will be given of Fig. 1. A vehicle control system 1 according to this embodiment includes a sensor group 2, a vehicle control determination unit 3, a buffer unit 5, a vehicle control detection unit 6, a biological change detection unit 7, a near-miss detection unit 8, a determination mismatch detection unit 9, a delay time setting unit 10, an information acquisition unit 11, and a data storage unit 12.

[0016] The sensor group 2 is a set of sensors for collecting information necessary for vehicle control, and includes GNSS (Global Navigation Satellite Systems) such as GPS (Global Positioning System), LiDAR (Light Detection And Ranging), radar, cameras, etc. The sensing results of each sensor (including RAW data from the camera) are input data to the vehicle control determination unit 3 and the buffer unit 5.

[0017] The vehicle control determination unit 3 determines the vehicle's status based on input data from the sensor group 2, calculates the vehicle control amount, and transmits control data (vehicle control values) to the vehicle drive unit 4 and buffer unit 5. The control data includes not only the amount of brake or accelerator pedal depression, but also information used to determine vehicle control such as the object recognition result and the distance to the object, and the result of a determination as to whether the vehicle needs to take emergency avoidance measures (such as issuing a warning or alert to the driver or applying emergency braking). Here, the vehicle control determination unit 3 is configured with trained AI (artificial intelligence).

[0018] The vehicle driving unit 4 is composed of a brake, an accelerator, a steering wheel, etc., and drives the vehicle based on the control data output from the vehicle control determination unit 3.

[0019] The buffer unit 5 temporarily stores information from the sensor group 2 and the vehicle control determination unit 3. When recording, the data is linked to time, and the information (input) from the sensor group 2 referenced by the vehicle control determination unit 3 and the control data (output) derived by the vehicle control determination unit 3 as a result are recorded together as driving data.

[0020] The vehicle control detection unit 6 detects when the vehicle is controlled significantly differently from normal and outputs the detected signal as a vehicle control detection signal. For example, by using time series analysis of the operation of the brake, accelerator, steering, etc., it is possible to detect unusual actions (such as sudden braking or sudden steering).

[0021] The biometric change detection unit 7 acquires biometric information of vehicle occupants, and if the acquired biometric information changes significantly from normal biometric information, it detects this change and outputs it as a biometric change detection signal. Occupants include the vehicle driver, passengers in the front or rear seats, and passengers in the case of buses. For example, biometric information includes heart rate, blood pressure, facial expressions, etc., and abnormal behavior can be detected using time series analysis. In the case of facial expressions, detection is possible by learning human facial expressions in advance and installing a camera to acquire and analyze the facial expressions of passengers.

[0022] When the near-miss detection unit 8 detects a vehicle control detection signal from the vehicle control detection unit 6 and / or a biochange detection signal from the biochange detection unit 7, it determines that the occupant has detected danger, i.e., that a near-miss has occurred, and outputs a near-miss detection signal. Note that the near-miss detection unit 8 may detect a wide range of near-misses based on either the vehicle control detection signal or the biochange detection signal, and output a near-miss detection signal.

[0023] When the judgment mismatch detection unit 9 detects a near-miss detection signal from the near-miss detection unit 8, it refers to the control data stored in the buffer unit 5, and the control data (judgment result when the near-miss cause occurred) from the time set in the delay time setting unit 10 back to the time when the near-miss detection signal was detected and the time before and after that (for example, several seconds).

[0024] If the vehicle control judgment unit 3 is unable to detect danger after referring to the control data stored in the buffer unit 5 a predetermined time prior to and including the time when the near miss was detected in the judgment mismatch detection unit 9 (if the vehicle control judgment unit 3 does not take any action such as issuing a warning signal to the driver or outputting control data such as emergency braking to the vehicle drive unit 4), it determines that there is a judgment mismatch between the human and the vehicle control system, and outputs to the information acquisition unit 11 a data acquisition trigger signal and information on the linking time (reference data time) of the data referenced in the buffer unit 5.

[0025] Upon receiving a data acquisition trigger signal from the judgment mismatch detection unit 9, the information acquisition unit 11 stores the control data and input data corresponding to the reference data time from the data stored in the buffer unit 5 in the data storage unit 12. The data stored in this data storage unit 12 may be data corresponding only to the reference data time, or may be data corresponding to a predetermined time range from the reference data time.

[0026] The functions of the components of this embodiment may be implemented by being incorporated into a central gateway of the vehicle, or may be implemented by being incorporated into an ECU (Electronic Control Unit).Furthermore, the functions of the components of this embodiment may be implemented by a retrofit device.

[0027] Next, a case where this embodiment is implemented in a passenger vehicle will be described using Figure 2. A passenger 130 rides in the vehicle 13. Biometric information of the passenger 130 is acquired from a biological change detection unit 7, which is realized by a smartphone 7-11, a smartwatch 7-12, a camera 7-13, a seat 7-14 with a heart rate measurement function, etc. The acquired information is transmitted to a near-miss detection unit 8 by means of wireless communication, wired communication, or the like. The operation when a near-miss detection signal is detected by the near-miss detection unit 8 is as described in Figure 1.

[0028] The flow of data processing in the vehicle control system 1 according to this embodiment will be described with reference to FIG.

[0029] First, data detected by each sensor in the sensor group 2 while the vehicle 13 is being driven is acquired (S301), and this acquired data is input to the vehicle control determination unit 3 to determine the vehicle situation and create vehicle control data (S302), and the data is stored in the buffer unit 5.

[0030] In S302, the vehicle control determination unit 3 processes the input data from each sensor using trained AI (artificial intelligence) to determine the vehicle situation and create control data for the vehicle. The control data created in the vehicle control determination unit 3 is sent to the vehicle drive unit 4 to control the vehicle (S303), and is also stored in the buffer unit 5 (S304).

[0031] On the other hand, if the vehicle control detection unit 6 detects that the vehicle 13 has been controlled significantly differently from normal while the vehicle is being driven, the signal output from the vehicle control detection unit 6 is received by the near-miss detection unit 8 as a vehicle control detection signal (S305), and if the biometric information of the occupant 130 of the vehicle 13 detected by the biometric change detection unit 7 has changed significantly differently from normal biometric information, the signal output from the biometric change detection unit 7 is received by the near-miss detection unit 8 as a biometric change detection signal (S306), and it is determined that a near-miss has occurred from the vehicle control detection signal and the biometric change detection signal received by the near-miss detection unit 8, and a near-miss detection signal is output (S307). Also, in S307, if either S305 or S306 is detected, it may be determined that a near-miss has occurred.

[0032] Next, in response to the near-miss detection signal output from the near-miss detection unit 8, the judgment mismatch detection unit 9 extracts, from the signals stored in the buffer unit 5 in S304, sensor data from the sensor group 2 and control data from the vehicle control determination unit 3 at a time preceding the time the near-miss detection signal was received by the predetermined delay time stored in the delay time setting unit 10 and before and after that time. If the vehicle control determination unit 3 has not detected a danger based on the extracted sensor data and control data (if the vehicle control determination unit 3 has not taken action such as issuing a warning to the driver or outputting a warning signal, or outputting control data such as emergency braking to the vehicle drive unit 4), a judgment mismatch between the human and the vehicle control system is detected (S308), and the information acquisition unit 11 collects driving data at the time of the detection of this judgment mismatch (S309). Next, the collected driving data at the time of the detection of the judgment mismatch is stored in the data storage unit 12 (S310).

[0033] According to this embodiment, by acquiring driving data only when there is a mismatch in judgment between humans and the vehicle control system, it is possible to efficiently collect data that is effective for training the AI ​​responsible for recognition in autonomous vehicles.

[0034] [Variations] 4 shows the configuration of a vehicle control system 1-1 according to a modified example of the first embodiment. The same components as those in the vehicle control system 1 according to the first embodiment are assigned the same part numbers.

[0035] The vehicle control system 1-1 of this modified example differs from Example 1 in that, as shown in Figure 4, a data output unit 14 is provided between the information acquisition unit 11 and the data storage unit 12, and information acquired by the information acquisition unit 11 is output to the data output unit 14, making it possible to select data to be stored in the data storage unit 12.

[0036] That is, this modified example differs from Example 1 in that it adds a display unit 141 that displays on the screen the information acquired by the information acquisition unit 11, and a data output unit 14 that has a save button 142 for selecting to save the displayed data and a delete button 143 for selecting to delete the displayed data, as shown in FIG. 5.

[0037] According to this modification, in addition to the effects described in the first embodiment, it is possible to delete data when a near-miss signal is detected due to a factor other than a near-miss by displaying the information acquired by the information acquisition unit 11 on the data output unit 14 and selecting the data to be stored in the data storage unit 12. In other words, it is possible to reduce the amount of data stored in the data storage unit 12 and improve the accuracy of the stored information (the probability that the stored data is data at the time of the near-miss occurrence). [Example]

[0038] In a second embodiment of the present invention, a method for collecting vehicle data and utilizing the collected data using the vehicle control system 1 described in embodiment 1 will be described. Fig. 6 is a block diagram showing the configuration of this embodiment, Fig. 7 is a block diagram showing the data flow of this embodiment, Fig. 8 is a block diagram showing the internal configuration of the cloud in this embodiment, Fig. 9 is a flow diagram showing the processing flow in the cloud in this embodiment, and Fig. 10 is a flow diagram showing the processing flow in the vehicle control system.

[0039] First, the overall configuration of a vehicle data collection system that utilizes a vehicle control system 1 according to this embodiment will be described with reference to FIG.

[0040] The vehicle 13 is equipped with a vehicle control system 1 having the configuration and functions described in the first embodiment, and a passenger 130 (see FIG. 2) rides in the vehicle 13. Although shown in a simplified manner in FIG. 6, the configuration and operation of the vehicle control system 1 are the same as those described in the first embodiment.

[0041] The transmission unit 15 transmits the driving data stored in the data storage unit 12 of the vehicle control system 1 at the time of the detection of the judgment mismatch to the information collection unit 16 (such as a cloud server).

[0042] The information collection unit 16 collects and organizes the driving data at the time of the detection of a judgment mismatch, which is transmitted from the transmission unit 15 and stored in the data storage unit 12, and surrounding information 17 at the time of the judgment mismatch detection. The purpose of collecting surrounding information 17 is because the cause of the judgment mismatch between humans (near-miss detection signal) and AI (control data) may exist other than the driving data. Even if the judgment mismatch detection unit 9 determines that there is a judgment mismatch between humans and AI, there is a possibility that it is not actually a judgment mismatch. Therefore, the information collection unit 16 performs processing to remove data that is not a judgment mismatch between humans and AI from the data received from the data storage unit 12. Details of this processing will be described later using FIG. 9.

[0043] Data for which a mismatch in judgment between humans and AI has been confirmed through the above process is sent to the learning data generation unit 18. The data storage unit 12 of the vehicle control system 1 contains time information and vehicle position information at the time of the mismatch in judgment between the passenger 130 and the vehicle control system 1, so it is possible to obtain and store surrounding information 17 at that time and position via the Internet, etc. The surrounding information 17 includes weather, temperature, traffic conditions, etc.

[0044] The learning data generation unit 18 is equipped with a defect database (defect DB) 19, which stores information that has actually been defective among the information collected and organized by the information collection unit 16. The defect database 19 also collects information from other vehicles 21, and is capable of integrating defect information. Using the information in the defect database 19, the learning data generation unit 18 creates learning data 20 for training the AI ​​stored in the vehicle control determination unit 3. The created learning data 20 is provided to the system design department 22.

[0045] The system design department 22 utilizes the learning data 20 received from the learning data generation unit 18 to train the AI ​​stored in the vehicle control determination unit 3 of the vehicle control system 1 and create update data. The created update data is distributed to each vehicle using OTA or the like and stored in the vehicle control determination unit 3.

[0046] According to this embodiment, the safety and reliability of the vehicle control system 1 can be improved by training the AI ​​using data on mismatches in judgment between humans and AI collected by the vehicle control system 1, and updating the AI ​​stored in the vehicle control judgment unit 3 by distributing update data.

[0047] In the configuration shown in Figure 6, the transmitter 15 is configured to be arranged outside the vehicle control system 1, but the transmitter 15 may also be incorporated inside the vehicle control system 1 to form a single unit that constitutes the vehicle control system 1.

[0048] Next, an overview of a service utilizing the present invention will be described using Figure 7. Here, the players in the service include a personal vehicle purchaser 23, a vehicle provision service provider 24, a vehicle user 25 who receives the vehicle provision service from the vehicle provision service provider 24, a vehicle operation company 26, a data management department 27, and a system design department 28 (corresponding to the system design department 22 in Figure 6). In Figure 7, the information provided by each player is indicated by arrows, and the value each player receives from this service is described.

[0049] First, the flow of information will be explained. Personal vehicle purchasers 23 and vehicle operators 26 provide driving data to a data management department 27. A vehicle provision service provider 24 obtains driving data from vehicles used by vehicle users 25 and provides it to the data management department 27. The data management department 27 aggregates, analyzes, and processes the driving data provided by each player to create learning data 20. The created learning data 20 is provided to a system design department 28. The system design department 28 uses the provided learning data 20 to create update data for the vehicle control system 1 installed in each vehicle. The created update data is provided to personal vehicle purchasers 23, vehicle provision service providers 24, and vehicle operators 26 using means such as OTA.

[0050] Next, we will explain the value each player receives from the service. By receiving update data from the system design department 28, personal vehicle purchasers 23 can enjoy the value of improved safety for their vehicles. Furthermore, vehicle provision service providers 24 and vehicle operation companies 26 can enjoy the value of improved vehicle safety as well as improved vehicle uptime due to a reduction in vehicle malfunctions. Vehicle users 25 can receive improved vehicle provision services with reduced vehicle malfunctions as well as improved vehicle safety. The system design department 28 can improve its market competitiveness by utilizing the learning data 20 to improve its products. Furthermore, the provision of update data enables the provision of safe vehicles, which can increase customer trust.

[0051] Next, the configuration of the information collection unit 16 that collects information will be described with reference to Fig. 8. The information collection unit 16 includes a receiving unit 161 that receives information transmitted from the transmitting unit 15 described in Fig. 6, a traveling data acquiring unit 162, a mismatch checking unit 163, a mismatch detection exclusion condition storage unit 164, a mismatch feature extracting unit 165, a surrounding environment information acquiring unit 166, a saving frequency specifying unit 167, and a transmitting unit 168, which are connected to each other by a data line 169.

[0052] The data organization flow on the cloud side having such a configuration will be explained with reference to FIG. First, the receiving unit 161 receives the driving data transmitted from the transmitting unit 15 and stored in the data storage unit 12 of the vehicle control system 1 when the judgment mismatch detection unit 9 detects a mismatch, and stores the data in the driving data acquisition unit 162 (S901).

[0053] Next, the mismatch check unit 163 checks whether or not there is actually a mismatch in the judgment between the human and the AI ​​for the running data stored in the running data acquisition unit 162 (S902).

[0054] This mismatch check unit 163 automatically performs this process using a method such as extracting common features between humans and AI when detecting mismatches using mismatch detection exclusion conditions that are preset in a mismatch detection exclusion condition storage unit 164, or information collected in a defect database 19 of the learning data generation unit 18 using a mismatch feature extraction unit 165.

[0055] An example of a mismatch detection exclusion condition that is preset in the mismatch detection exclusion condition storage unit 164 is that if the camera's RAW data or the recognition results do not show any objects that are likely to cause near misses, such as vehicles, people, or animals, then it is determined that there was no near miss.

[0056] Examples of features extracted by the mismatch feature extractor 165 include the time of the mismatch, the recognized object, and the weather.

[0057] If it is determined in S902 that there is a mismatch (YES in S902), information on the surrounding environment, such as weather, temperature, and traffic conditions, corresponding to the time when the mismatch between the human and AI was detected is collected (S903).

[0058] Next, before storing the information determined to have a mismatch in S902 in the defect database 19, a final check is made by a human to see if there is a mismatch (S904), and if it is determined that there is a mismatch (YES in S904), the data is stored in the defect database 19 (S905) and the process ends. This S904 step may be omitted if the accuracy of the mismatch check unit 163 in S902 is high, and the process may proceed directly from S903 to S905.

[0059] In the learning data generating unit 18, learning data 20 is created based on data relating to mismatches stored in a defect database 19.

[0060] 11 shows an example of learning data 20, which is learning data 1100 shown in a table format. In the learning data 1100, the following information is stored in association with one another: vehicle type 1101, which is information on the type of vehicle in which a near miss was detected; mismatch occurrence date and time 1102, which is information on the date and time when a mismatch in the near miss data occurred; vehicle control information 1103, which is the recognition result of the near miss detected by the near miss detection unit 8, the vehicle control value output from the vehicle control determination unit 3, the vehicle situation determination result obtained from an ECU (not shown), etc.; surrounding information 1104, which is information on the surroundings obtained via the Internet, such as weather, temperature, and traffic conditions; and information 1105, which is information on the presence or absence of a mismatch determined by the mismatch check unit 163.

[0061] Here, the driving data received from the vehicle in S901 is a mixture of small amounts of data such as location information and time information, and large amounts of data such as camera RAW data, and depending on the frequency of data reception, this may strain the communication between the vehicle 13 and the information collection unit 16. For this reason, only small amounts of data may normally be received, and if the possibility of a true mismatch increases in S902 or S904, the remaining large amounts of data may be additionally requested from the vehicle in order to obtain more detailed information.

[0062] If it is determined in S902 that there is no mismatch (NO in S902), it is determined whether additional data is required (S906), and if it is determined that additional data is required (YES in S906), the additional data is obtained from the data stored in the data storage unit 12 (S907), and the process proceeds to S902 again to determine whether there is a mismatch.

[0063] On the other hand, if it is determined that additional data is not necessary (NO in S906), the data acquired in S901 is deleted (S910) and the process ends.

[0064] Furthermore, if it is determined in S904 that there is no mismatch (NO in S904), it is determined whether or not additional data is required (S908), and if it is determined that additional data is required (YES in S908), the additional data is obtained from the data stored in the data storage unit 12 (S909), and the process proceeds to S904 again to determine whether or not there is a mismatch.

[0065] On the other hand, if it is determined that additional data is not necessary (NO in S908), the data acquired in S901 is deleted (S910) and the process ends.

[0066] Next, the data processing flow on the vehicle control system 1 side in this embodiment will be described with reference to Fig. 10. The processing from S1001 to S1009 is the same as the processing from S301 to S309 in the processing flow described with reference to Fig. 3 in the first embodiment, and therefore the description will be omitted.

[0067] For the information collected in S1009 of Figure 10, it is determined whether the frequency at which it is stored in the data storage unit 12 has reached a preset frequency (S1010), and if it is determined that the set frequency has not yet been reached (NO in S1010), the information collected in S1009 is stored in the data storage unit 12 (S1011), and this stored data is sent to the information collection unit 16 (S1012).

[0068] 6 has update data, the vehicle control determination unit 3 receives the update data (S1014), and uses this update data to make a vehicle control determination in S1002. By acquiring update data in this way and making a vehicle control determination, it is possible to maintain a high level of accuracy in near-miss determination, and to maintain a high level of reliability in the determination.

[0069] When transmitting data to the information collection unit 16 configured in the cloud in S1012, basic vehicle information such as the vehicle type, mileage, and years of use may be transmitted together with the driving data stored in the buffer unit 5.

[0070] Furthermore, the frequency of storage used in the determination in S1010 is the data of storage frequency stored in storage frequency designation unit 167 of information collection unit 16, but may be set as the number of times data is stored according to the mismatch occurrence situation. For example, if it is found as a result of data processing in information collection unit 16 that mismatch detection is rare for the vehicle model, the storage frequency is set low (save once every 10 times, etc.). Furthermore, the data of storage frequency stored in storage frequency designation unit 167 may be updatable via OTA or the like.

[0071] On the other hand, if it is determined in S1010 that the set frequency has been reached (YES), the information collected in S1009 is discarded (deleted) (S1013), and the process ends.

[0072] 12, a hardware configuration when the present invention is realized as a retrofit device on a vehicle 120 will be described. In relation to this embodiment, the hardware installed on the vehicle 120 to detect near-miss mismatches comprises a driving assistance system 1210, a telematics data collection device 1201, a near-miss detection device 1202, and a telematics control system 1203.

[0073] The data acquired by this configuration is transmitted wirelessly or otherwise to a cloud server 1220. Of these, the telematics data collection device 1201, near-miss detection device 1202, and telematics control system 1203 are retrofitted devices 1200. The telematics data collection device 1201 is a device capable of collecting information about the vehicle, and includes devices capable of collecting information directly from a CAN and devices capable of collecting information from an OBD port, but either method may be used.

[0074] The near-miss detection device 1202 collects biometric information from a smartwatch, smartphone, or the like, and transmits data when a mismatch is detected between the near-miss detection device 1202 and the telematics data collection device 1201 via wireless communication or the like from the telematics control system 1203 to the cloud server 1220. With this configuration, it becomes possible to collect information when a near-miss is detected from an existing vehicle.

[0075] According to this embodiment, data regarding mismatches in judgment between humans and vehicle control detected in individual vehicles is collected and learned in the cloud, and this learned data is sent to individual vehicles and reflected in the vehicle control judgment unit, thereby improving the reliability of the vehicle control judgment unit and making it possible to reduce mismatches in judgment between humans and vehicle control. [Example]

[0076] As a third embodiment of the present invention, a case where a mismatch is detected using a plurality of pieces of biometric information in a vehicle control system will be described. Fig. 13 is a configuration diagram of a vehicle control system 101 according to this embodiment. In the following explanation, the same components as those in the vehicle control system 1 described in the first embodiment are assigned the same reference numerals, and differences will be mainly described. Points that are not particularly explained are the same as in the first embodiment.

[0077] In this embodiment, instead of the biochange detection unit 7 of Example 1, it is configured with a plurality of biochange detection units 7-1, ... 7-n, and further provides a biochange aggregation unit 35 that receives information from the plurality of biochange detection units 7-1, ... 7-n, and is configured so that when a biochange is detected in more than one or a majority of the plurality of biochange detection units 7-1, ... 7-n, the biochange aggregation unit 35 outputs a biochange detection signal to the near-miss detection unit 8.

[0078] By adopting such a configuration, according to this embodiment, a biochange detection signal is output to the near-miss detection unit 8 only when a biochange is detected by more than one of the multiple biochange detection units 7-1, ....7-n, thereby increasing the probability that the mismatch detection timing is a near-miss compared to when a single piece of bioinformation is used.

[0079] FIG. 14 shows an example of facial expression detection on a bus 131. As a specific example of this embodiment, a camera 7-21 that reads the facial expressions 7-24 of passengers 130 is installed on the bus 131 as the biometric change detection unit 7. The results of the facial expressions 7-24 of passengers 130 detected by the camera 7-21 are collected by a biometric change collection unit 35, and if it is estimated that a certain number or more of passengers 130 are feeling unsafe, a biometric change detection signal is output to an information collection unit 16 configured in the cloud. In addition to the camera 7-21, devices such as smartphones 7-22 and smartwatches 7-23 owned by passengers 130 may also be used as the biometric change detection unit 7. As in this embodiment, mismatch detection is possible even in vehicles without drivers, such as autonomous buses. [Example]

[0080] As a fourth embodiment of the present invention, instead of the delay time setting unit 10 of the vehicle control system 1 described in the first embodiment, a predetermined delay time can be set to a predetermined delay time as shown in FIG. A vehicle control system 102 provided with a gap width setting unit 36 ​​will be described.

[0081] In the following description, the same components as those in the vehicle control system 1 described in the first embodiment are denoted by the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as those in the first embodiment.

[0082] This embodiment is characterized in that a predetermined delay time width setting unit 36 ​​and an estimation unit 37 are provided instead of the delay time setting unit 10 of the vehicle control system 1 described in Example 1. The predetermined delay time width setting unit 36 ​​is configured to set a certain width for the predetermined delay time used when the judgment mismatch detection unit 9 refers to the buffer unit 5 after the near-miss detection unit 8 outputs a near-miss detection signal.

[0083] After the near-miss detection unit 8 outputs a near-miss detection signal, the estimation unit 37 performs a time series analysis of the control data in the buffer unit 5 within a predetermined delay time width, estimates the timing (data time) at which the control data output from the vehicle control judgment unit 3 changes significantly compared to the others, and outputs this to the judgment mismatch detection unit 9 as the reference data time.

[0084] According to this embodiment, after a near miss is detected, by searching for timing within a certain range, compared to going back a fixed delay time, it is possible to save driving data that is more reliably synchronized with the timing of the near miss occurrence. [Example]

[0085] In the fifth embodiment, an operation will be described in a case where a delay adjustment unit that calibrates the delay time set by the predetermined delay time width setting unit 36 ​​is provided in the vehicle control system 102 described in the fourth embodiment.

[0086] 16 is a configuration diagram of a vehicle control system 103 in this example. In the following explanation, the same components as those in the fourth embodiment are denoted by the same reference numerals, and differences will be mainly explained. Points that are not specifically explained are the same as those in the fourth embodiment.

[0087] The vehicle control system 103 according to this embodiment differs from the vehicle control system 103 described in the fourth embodiment in that a delay adjustment unit 38 is provided in addition to the vehicle control system 103 described in the fourth embodiment. In addition to the operations described in the fourth embodiment, the estimation unit 37 estimates the reaction speed of the biometric information provider from the timing of the near-miss detection signal output by the near-miss detection unit 8 and the estimated timing, and outputs the result to the delay adjustment unit 38. The delay adjustment unit 38 feeds back the adjusted delay time to the predetermined delay time width setting unit 36 ​​based on the estimated reaction speed. The predetermined delay time width setting unit 36 ​​changes the value of the delay time width based on the adjusted delay time.

[0088] According to this embodiment, by estimating the biological reaction time and feeding it back to a predetermined delay time width, it becomes possible to store driving data that is synchronized with the timing of the near miss occurrence more reliably with each storage count. [Example]

[0089] In the sixth embodiment, an operation will be described in which trucks equipped with any one of the vehicle control systems 1 or 101, or 102 or 103 described in the first to fifth embodiments travel in a convoy.

[0090] 17 is a diagram showing the configuration of trucks 39a, 39b, and 39c according to this embodiment traveling in a convoy. In this embodiment, the leading truck 39a is equipped with a rear camera 40b, the middle truck 39b is equipped with a front camera 40a, and the trailing truck 39c is not equipped with a camera. In addition, manned vehicles 41a, 41b, and 41c are present in the vicinity, and a surveillance camera 42 is installed.

[0091] In this situation, trucks 39a, 39b, and 39c travel in a convoy. Platooning refers to the traveling of multiple trucks controlled by vehicle-to-vehicle communication. Trucks 39a, 39b, and 39c may or may not have people on board, but in this embodiment, the first truck a is described as being manned, and the middle truck 39b and the rear truck 39c are unmanned.

[0092] First, the leading truck 39a is manned, and therefore the vehicle control system 1 can operate as described in Example 1. On the other hand, the middle truck 39b and the trailing truck 39c are unmanned, and therefore the biochemical change detection unit 7 in the vehicle control system 1 cannot acquire biochemical change information, and therefore biochemical information from the surrounding manned vehicles 41a, 41b, and 41c is used instead.

[0093] Specifically, one method is for a passenger in a manned vehicle 41a, 41b, or 41c traveling in the lane next to trucks 39a, 39b, or 39c to notice a dangerous situation in the unmanned central truck 39b or the rear truck 39c, and receive information about a biological change as a near-miss detection signal via vehicle-to-vehicle communication.

[0094] When a near miss detection signal is received from a nearby manned vehicle 41 through vehicle-to-vehicle communication, if the unmanned central truck 39b or the unmanned rear truck 39c has not detected any danger, the data is stored.

[0095] As another method, if a passenger in the manned leading truck 39a notices a dangerous situation in the unmanned central truck 39b or the trailing truck 39c traveling behind them through a rearview mirror or the like, and their biometric information changes, this can be used as a near miss detection signal.

[0096] Furthermore, a near-miss detection signal may be detected using the front camera 40a equipped on the center truck 39b. Specifically, a near-miss detection signal may be detected when there is a difference in the recognition results of camera images capturing the same position, such as when the recognition result of the rear camera 40b of the lead truck 39a detects a danger, but the image recognition result of the front camera 40a of the center truck 39b does not detect a danger. In addition, images from a surveillance camera 42 capturing the area around the road on which the truck 39 is traveling may be used as a method of calculating the difference in the recognition results of similar camera images.

[0097] According to this embodiment, even when multiple vehicles traveling in a convoy are driven unmanned, near-miss detection signals corresponding to each vehicle can be detected, making it possible to store driving data synchronized with the timing of the near-miss occurrence.

[0098] The invention made by the inventor has been specifically described above based on the embodiments, but it goes without saying that the present invention is not limited to the above embodiments and can be modified in various ways without departing from the spirit of the invention. For example, the above embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the described configurations. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations. [Explanation of symbols]

[0099] 1, 1-1, 101, 102, 103... Vehicle control system 2... Sensor group 3... Vehicle control judgment unit 4... Vehicle drive unit 5... Buffer unit 6... Vehicle control detection unit 7... Biological change detection unit 8... Near-miss detection unit 9... Judgment mismatch detection unit 10... Delay time setting unit 11... Information acquisition unit 12... Data storage unit 13, 120... Vehicle 14... Data output unit 15... Transmission unit 16... Information collection unit 17... Surrounding information 18... Learning data generation unit 19... Defect database 20... Learning data 22, 28... System design department 23... Individual vehicle purchaser 24... Vehicle service provider 25... Vehicle user 26... Vehicle operation company 27 Data management department 35 Biological change aggregation unit 36 ​​Predetermined delay time width setting unit 37 Estimation unit 38 Delay adjustment unit 39a Leading truck 39b Center truck 39c Rear truck 40a Front camera 40b Rear camera 41 Surrounding manned vehicles 42 Surveillance camera 161 Receiving unit 162 Traveling data acquisition unit 163 Mismatch check unit 164 Mismatch detection exclusion condition storage unit 165 Mismatch feature extraction unit 166 Surrounding environment information acquisition unit 167 Storage frequency designation unit 168 Transmitting unit 1200 Retrofit device 1201 Telematics data collection device 1202 Near miss detection device 1203···Telematics control system 1210···Driving assistance system 1220···Cloud server.

Claims

1. a vehicle control determination unit that generates a control signal for controlling the vehicle using data acquired from a sensor mounted on the vehicle; a near-miss detection unit that detects a near-miss felt by a passenger in the vehicle from at least one of a biological change of the passenger in the vehicle while driving the vehicle and a control signal of the vehicle, and outputs a near-miss detection signal; a judgment mismatch detection unit that detects a mismatch between the control signal and the near-miss at the timing when the near-miss is detected based on the data acquired from the sensor attached to the vehicle, the control signal created by the vehicle control judgment unit, and the near-miss detection signal detected by the near-miss detection unit; and a data storage unit that stores the data acquired from the sensor attached to the vehicle corresponding to the timing at which the mismatch is detected by the determination mismatch detection unit and the control signal created by the vehicle control determination unit; Equipped with The vehicle control system is characterized in that the judgment mismatch detection unit detects the mismatch when the vehicle control judgment unit has not detected any danger at a time that is a predetermined time before the time when the occupant felt a near miss and before and after that time.

2. 2. The vehicle control system of claim 1, Further comprising a delay time setting unit, The vehicle control system is characterized in that, when the near miss detection unit detects the near miss, the judgment mismatch detection unit detects the mismatch between the control signal and the timing at which the near miss was detected using the data acquired from the sensor attached to the vehicle at a time including before and after the time set back by the delay time setting unit and the control signal created by the vehicle control judgment unit.

3. 3. The vehicle control system according to claim 1 or 2, A vehicle control system further comprising: a buffer unit that stores the data acquired from the sensor mounted on the vehicle and the control signal created by the vehicle control judgment unit; and an information acquisition unit that acquires from the buffer unit the data and the control signal stored in the buffer unit, the data and the control signal corresponding to the timing at which the mismatch was detected by the judgment mismatch detection unit, and stores the data and the control signal acquired by the information acquisition unit corresponding to the timing at which the mismatch was detected in the data storage unit.

4. 3. The vehicle control system according to claim 1 or 2, The vehicle control system further comprises a transmitting unit that transmits the information about the mismatch stored in the data storage unit to an external device.

5. 3. The vehicle control system according to claim 1 or 2, The vehicle control system is characterized in that the vehicle control determination unit receives learning data created externally and creates the control signal using the received learning data.

6. 3. The vehicle control system according to claim 1 or 2, A vehicle control system characterized in that the near-miss detection unit, the judgment mismatch detection unit, and the data storage unit are configured as retrofit devices for the vehicle.

7. A method for collecting vehicle data in a vehicle control system including a vehicle control determination unit, a near-miss detection unit, a mismatch detection unit, and a data storage unit, a control signal for controlling the vehicle by the vehicle control determination unit using data acquired from a sensor mounted on the vehicle; The near miss detection unit detects a near miss felt by the occupant from at least one of a biological change of the occupant riding in the vehicle while driving the vehicle or a control signal of the vehicle, Detecting a mismatch between the control signal and the near miss at the timing when the near miss is detected by the mismatch detection unit from the data acquired from the sensor attached to the vehicle, the control signal created by the vehicle control determination unit, and the near miss detected by the near miss detection unit; the data acquired from the sensor mounted on the vehicle corresponding to the timing at which the mismatch was detected by the mismatch detection unit and the control signal created by the vehicle control determination unit are stored in the data storage unit, A vehicle data collection method characterized in that the mismatch is detected when the vehicle control judgment unit is unable to detect danger at a time a predetermined time prior to and around the time when the occupant felt a near miss.

8. 8. The vehicle data collection method according to claim 7, further comprising: the vehicle control system further includes a delay time setting unit; A vehicle data collection method characterized in that, when the near miss detection unit detects the near miss, the mismatch detection unit detects the mismatch between the control signal and the timing at which the near miss was detected using the data acquired from the sensor attached to the vehicle at a time that is backdated by the delay time setting unit and the control signal created by the vehicle control determination unit.

9. 9. The vehicle data collection method according to claim 7 or 8, the vehicle control system further includes a buffer unit and an information acquisition unit; a buffer unit that stores the data acquired from the sensor mounted on the vehicle and the control signal created by the vehicle control determination unit; an information acquisition unit that acquires from the buffer unit the data and the control signal that correspond to the mismatch detected by the mismatch detection unit among the data and the control signal stored in the buffer unit; and a data storage unit that stores the data and the control signal that correspond to the timing at which the mismatch was detected and acquired by the information acquisition unit.

10. 9. The vehicle data collection method according to claim 7 or 8, the vehicle control system further includes a transmitter; The vehicle data collection method further comprises transmitting the information about the mismatch stored in the data storage unit from the transmission unit to an external device.

11. 9. The vehicle data collection method according to claim 7 or 8, The vehicle data collection method, wherein the vehicle control determination unit receives externally generated learning data, and generates the control signal using the received learning data.

Images