Information processing device, information processing method, and computer readable medium recording information processing program

Abstract

The present disclosure relates to an information processing device, an information processing method, and a computer-readable medium recording an information processing program. The information processing device of the present disclosure includes: a detection unit that detects an observed value related to a change in the acceleration of a vehicle caused by a driving operation; an inertial measurement unit that detects an actual value of the acceleration of the vehicle; and an inference unit that determines that the vehicle has collided when the difference between the actual value of the acceleration of the vehicle detected by the inertial measurement unit and the inferred value of the acceleration of the vehicle calculated based on the observed value, i.e., the acceleration difference, is greater than a predetermined threshold.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing method, and a computer-readable medium recording an information processing program for detecting impact generation involved in a collision of a vehicle based on a driving history of the vehicle or the like. Background Art

[0002] In the used car market, the valuation of a used car varies depending on whether or not the used car has an accident history. In addition, even in a relatively minor collision (hereinafter referred to as a "minor collision"), the vehicle may be damaged and must be repaired. In such a case, it is necessary to inspect the collision at a sales store or repair shop.

[0003] Although the airbag is deployed when the vehicle body collides, in reality, the airbag is deployed only in a small part of the collision that damages the frame of the vehicle body. Therefore, regardless of whether the airbag is activated or not, it is necessary to detect a minor collision with high accuracy.

[0004] Japanese Patent Application Laid-Open No. 2012-221407 discloses an invention of a vehicle evaluation index creation system for estimating vehicle damage using a vibration load on a normal vehicle body as an index.

[0005] However, the invention described in Japanese Patent Application Laid-Open No. 2012-221407 does not exclude acceleration components caused by sudden acceleration, sudden braking, etc., from acceleration components caused by minor collisions among the forces acting on the vehicle, and therefore it is difficult to detect minor collisions with high accuracy. Summary of the invention

[0006] The present disclosure provides an information processing device, an information processing method, and a computer-readable medium recording an information processing program that can detect a minor collision with high accuracy.

[0007] A first embodiment of the present disclosure is an information processing device, comprising: a detection unit, which detects an observed value related to a change in the acceleration of a vehicle caused by a driving operation; an inertial measurement unit, which detects an actual measured value of the acceleration of the vehicle; and an inference unit, which determines that the vehicle has collided when a difference between the actual measured value of the acceleration of the vehicle detected by the inertial measurement unit and an inferred value of the acceleration of the vehicle calculated based on the observed value, i.e., an acceleration difference, is greater than a predetermined threshold value.

[0008] The estimated value of acceleration calculated based on the observed value related to the change in the acceleration of the vehicle caused by the driving operation becomes the estimated value of the acceleration of the vehicle caused by the driving operation. In addition, the acceleration of the vehicle generated during the collision is different from the change in acceleration caused by the driving operation. Therefore, when the difference between the actual value of the acceleration of the vehicle detected by the inertial measurement unit and the estimated value of the acceleration of the vehicle calculated based on the observed value related to the change in the acceleration of the vehicle caused by the driving operation is greater than a predetermined threshold, it can be determined that the vehicle has collided.

[0009] According to the information processing device of the first aspect, since there is a discrepancy between the actual measurement value and the predicted value of the acceleration even in a minor collision, it is possible to estimate the collision of the vehicle with high accuracy.

[0010] A second aspect of the present disclosure is that, in the first aspect, the estimation unit may not use the estimated value in the collision judgment of the vehicle when the acceleration involved in the acceleration difference is the acceleration input to the vehicle from a road surface via a tire.

[0011] According to the information processing device of the second aspect, a vehicle collision can be estimated with high accuracy by not using an actual measurement value of acceleration input from the road surface via the tires, which may cause a judgment error, in collision judgment.

[0012] The third mode of the present disclosure is that in the above second mode, the following mode may be adopted, that is, the estimation unit calculates the wheel speed difference, which is the difference between the actual value of the wheel speed of each of the four wheels of the front and rear wheels of the vehicle detected by the wheel speed detection unit and the estimated value of the wheel speed of each of the four wheels of the front and rear wheels of the vehicle calculated based on the observed value, and in any of the cases where the acceleration difference changes after the change of the wheel speed difference and the case where the wheel speed difference of the front and rear wheels changes sequentially, the acceleration involved in the acceleration difference is judged as the acceleration input from the road surface through the tire. Thereby, the component that may cause the judgment error can be excluded from the acceleration.

[0013] A fourth aspect of the present disclosure is that, in the above-mentioned second aspect, the following aspect may also be adopted, namely, the estimation unit sets the acceleration related to the acceleration difference at the time when the intense running condition of the vehicle is recorded in the image data obtained by the camera unit as the acceleration input to the vehicle from the road surface via the tire.

[0014] A fifth aspect of the present disclosure is that, in the above-mentioned second aspect, the following aspect may also be adopted, namely, the inference unit judges the acceleration detected when the vertical displacement of the vehicle position information calculated based on information from the satellite is greater than a predetermined value as the acceleration input from the road surface via the tire.

[0015] The sixth method of the present disclosure is that in the above-mentioned second method, the following method can also be adopted, that is, the inference unit refers to a database of locations where there are undulations on the road surface, locations where the slope changes sharply, and locations where multiple vehicles experience acceleration changes, so as to judge the acceleration involved in the acceleration difference as the acceleration input from the road surface via the tires.

[0016] The seventh method of the present disclosure is that in the above method, the following method can also be adopted, that is, the inertial measurement unit can detect each of the acceleration in the front-to-back direction and the left-to-right direction of the vehicle, and the inference unit can calculate each of the inferred value of the acceleration in the front-to-back direction and the left-to-right direction of the vehicle based on the observed value, and when any one of the difference between the actual measured value of the acceleration in the front-to-back direction of the vehicle detected by the inertial measurement unit and the inferred value of the acceleration in the front-to-back direction of the vehicle, and the difference between the actual measured value of the acceleration in the left-to-right direction of the vehicle detected by the inertial measurement unit and the inferred value of the acceleration in the left-to-right direction of the vehicle is greater than a predetermined threshold value, it is judged that the vehicle has collided.

[0017] The information processing device of the seventh aspect can determine a vehicle collision when any one of the difference between the measured value and the estimated value of the acceleration in the longitudinal direction of the vehicle and the difference between the measured value and the estimated value of the acceleration in the lateral direction of the vehicle is greater than or equal to a predetermined threshold.

[0018] The eighth mode of the present disclosure is that in the above-mentioned seventh mode, the following mode may also be adopted, that is, the inference unit infers the damage direction of the vehicle based on the quotient of the difference between the actual measured value of the acceleration in the front-rear direction of the vehicle detected by the inertial measurement unit and the estimated value of the acceleration in the front-rear direction of the vehicle, and the difference between the actual measured value of the acceleration in the left-right direction of the vehicle detected by the inertial measurement unit and the estimated value of the acceleration in the left-right direction of the vehicle.

[0019] The information processing device of the eighth aspect can estimate the damage direction of the vehicle based on the acceleration in the front-rear direction and the acceleration in the left-right direction of the vehicle.

[0020] A ninth aspect of the present disclosure is that, in the above aspect, the following aspect may also be adopted, namely, the estimation unit constructs a model for estimating the acceleration of the vehicle based on observed quantities related to changes in acceleration of multiple vehicles obtained in advance and actual measured values ​​of the acceleration of the multiple vehicles.

[0021] The information processing device of the ninth aspect constructs an acceleration estimation model using machine learning based on so-called big data obtained from a plurality of vehicles. Therefore, it is possible to construct a model that estimates the acceleration of the vehicle caused by the driving operation with high accuracy.

[0022] A tenth aspect of the present disclosure is that, in the above aspect, the following aspect may be adopted, namely, the observed amount includes a measured value of the vehicle motion of the vehicle and a control signal related to the driving operation amount of the vehicle.

[0023] The information processing device of the tenth aspect can construct a model for accurately estimating the acceleration of the vehicle due to the driving operation by subjecting observed quantities related to changes in the acceleration of the vehicle due to the driving operation to machine learning.

[0024] The eleventh embodiment of the present disclosure is an information processing method, in which an observed quantity related to a change in acceleration of a vehicle caused by a driving operation is detected by a detection unit, and an actual measured value of the acceleration of the vehicle is detected by an inertial measurement unit. When the difference between the actual measured value of the acceleration of the vehicle and the estimated value of the acceleration of the vehicle calculated based on the observed quantity, that is, the acceleration difference, is greater than a predetermined threshold value, it is determined that the vehicle has collided.

[0025] The estimated value of acceleration calculated based on the observed value related to the change in the acceleration of the vehicle caused by the driving operation becomes the estimated value of the acceleration of the vehicle caused by the driving operation. In addition, the acceleration of the vehicle generated during the collision is different from the change in acceleration caused by the driving operation. Therefore, when the difference between the actual value of the acceleration of the vehicle detected by the inertial measurement unit and the estimated value of the acceleration of the vehicle calculated based on the observed value related to the change in the acceleration of the vehicle caused by the driving operation is greater than a predetermined threshold, it can be determined that the vehicle has collided.

[0026] According to the information processing method of the eleventh aspect, since there is a discrepancy between the actual measured value and the predicted value of the acceleration even in a minor collision, it is possible to estimate the collision of the vehicle with high accuracy.

[0027] A twelfth aspect of the present disclosure is that, in the above-mentioned eleventh aspect, the following aspect may be adopted, namely, when the acceleration involved in the acceleration difference is the acceleration input to the vehicle from the road surface via the tire, the estimated value is not adopted in the collision judgment of the vehicle.

[0028] According to the information processing method of the twelfth aspect, the collision of the vehicle can be estimated with high accuracy by not using the input from the road surface via the tires as the actual measurement value of the acceleration which may cause a judgment error in the collision judgment.

[0029] The thirteenth mode of the present disclosure is that in the above-mentioned twelfth mode, the following mode may also be adopted, that is, the wheel speed difference is calculated as the difference between the actual value of the wheel speed of each of the four wheels of the front and rear wheels of the vehicle detected by the wheel speed detection unit and the estimated value of the wheel speed of each of the four wheels of the front and rear wheels of the vehicle calculated based on the observed value, and in any of the cases where the acceleration difference changes after the change of the wheel speed difference and the case where the wheel speed difference of the front and rear wheels changes sequentially, the acceleration involved in the acceleration difference is judged as the acceleration input from the road surface through the tire. In this way, the component that may cause the judgment error can be excluded from the acceleration.

[0030] The fourteenth embodiment of the present disclosure is a computer-readable medium having recorded thereon an information processing program, wherein the information processing program enables the computer to function as an inference unit, namely: the inference unit determines that the vehicle has collided when a difference between an actual value of the acceleration of the vehicle detected by an inertial measurement unit and an inferred value of the acceleration of the vehicle calculated based on an observed value related to a change in the acceleration of the vehicle caused by a driving operation detected by a detection unit, i.e., an acceleration difference, is greater than a predetermined threshold value.

[0031] The estimated value of acceleration calculated based on the observed value related to the change in the acceleration of the vehicle caused by the driving operation becomes the estimated value of the acceleration of the vehicle caused by the driving operation. In addition, the acceleration of the vehicle generated during the collision is different from the change in acceleration caused by the driving operation. Therefore, when the difference between the actual value of the acceleration of the vehicle detected by the inertial measurement unit and the estimated value of the acceleration of the vehicle calculated based on the observed value related to the change in the acceleration of the vehicle caused by the driving operation is greater than a predetermined threshold, it can be determined that the vehicle has collided.

[0032] According to the computer-readable medium recording the information processing program of the fourteenth aspect, since there is a discrepancy between the measured value and the predicted value of the acceleration even in a minor collision, it is possible to estimate the collision of the vehicle with high accuracy.

[0033] A fifteenth aspect of the present disclosure is that, in the fourteenth aspect, the following aspect may be adopted, namely, when the acceleration involved in the acceleration difference is the acceleration input to the vehicle from the road surface via the tire, the estimated value is not used in the collision judgment of the vehicle.

[0034] According to the computer-readable medium recording the information processing program of the fifteenth aspect, the collision of the vehicle can be estimated with high accuracy by not using the input from the road surface via the tires as the measured value of acceleration which may cause a judgment error in the collision judgment.

[0035] The sixteenth mode of the present disclosure is that in the fifteenth mode, the following mode may be adopted, that is, the wheel speed difference is calculated as the difference between the actual value of the wheel speed of each of the four wheels of the front and rear wheels of the vehicle detected by the wheel speed detection unit and the estimated value of the wheel speed of each of the four wheels of the front and rear wheels of the vehicle calculated based on the observed value, and in any of the cases where the acceleration difference changes after the change of the wheel speed difference and the case where the wheel speed difference of the front and rear wheels changes sequentially, the acceleration involved in the acceleration difference is judged as the acceleration input from the road surface through the tire. In this way, the component that may cause the judgment error can be excluded from the acceleration.

[0036] According to the above aspects, the information processing device, the information processing method, and the computer-readable medium recording the information processing program of the present disclosure can detect a slight collision with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Exemplary embodiments will be described in detail based on the following drawings, in which:

[0038] Figure 1 It is a schematic diagram showing the structure of the information processing device involved in this exemplary embodiment.

[0039] Figure 2 is a block diagram showing the structure of a vehicle.

[0040] Figure 3 It is a block diagram showing an example of a specific structure of a computing server involved in this exemplary embodiment.

[0041] Figure 4 The figure is a functional block diagram of the CPU of a computing server.

[0042] Figure 5 Functional block diagram of the CPU of the server for calculating the learned acceleration inference.

[0043] Figure 6A functional block diagram showing the CPU of the calculation server after learning of wheel speed estimation.

[0044] Figure 7 A functional block diagram showing the CPU of a computation server after learning of road surface input estimation.

[0045] Figure 8 The flowchart shows an example of the processing in the computing server involved in this exemplary embodiment.

[0046] Fig. 9 This is a schematic diagram showing how accident determination is performed based on factors such as the acceleration difference, the FL wheel speed difference, the RL wheel speed difference, the pedal operation performed by the driver, and the latitude and longitude.

[0047] Fig.10 This is a schematic diagram of the case where accident judgment is performed based on the difference between the actual measured value and the estimated value obtained by RNN.

[0048] Fig.11 is a schematic diagram showing an example of values ​​detected by various sensors.

[0049] Fig. 12A This is an example of an image of a vehicle attempting to cross a height difference.

[0050] Fig. 12B This is an explanatory diagram showing a situation in which, after the left and right front wheels spin and speed up when the vehicle travels over a height difference, the left and right rear wheels spin and speed up.

[0051] Fig.13A This is an example of an image of a vehicle passing over a rough road.

[0052] Fig. 13B This is an explanatory diagram showing a situation where two local values ​​of the acceleration difference are observed and the left front wheel and the left rear wheel are accelerated before each local value.

[0053] Fig.14A This is an example of an image obtained by the vehicle's camera device showing a state in which the right front wheel of the vehicle spins when the accelerator pedal is depressed while turning right at a level difference between a sidewalk and a lane.

[0054] Fig. 14B This is an explanatory diagram showing a situation where the actual measurement value (solid line) and the estimated value (dashed line) of the wheel speed at the right front wheel diverge as a result of the right front wheel of the vehicle spinning and accelerating.

[0055] Fig.15A This is an example of an image acquired by the vehicle's camera device in a situation where the left rear wheel of the vehicle is unloaded (pitch).

[0056] Fig. 15B 1 is an explanatory diagram showing a situation where the actual measured value (solid line) and the estimated value (dashed line) of the wheel speed at the left rear wheel diverge as a result of the left rear wheel spinning.

[0057] Fig.16 It is a block diagram showing an example of operation of the information processing device according to the present exemplary embodiment.

[0058] Fig.17 It is a block diagram showing another example of the operation of the information processing device according to the present exemplary embodiment. DETAILED DESCRIPTION

[0059] Below, use Figure 1 The information processing device 100 according to this exemplary embodiment will be described. Figure 1 The information processing device 100 shown includes: a communication device 110, which obtains data from a plurality of vehicles 200, so-called networked vehicles, which are equipped with a function of always being connected to a network; a data storage device 120, which accumulates the data received by the communication device 110; a computing server 10, which requests the data storage device 120 for data necessary for machine learning using a neural network, and constructs an acceleration inference model for predicting the acceleration of the vehicle 200 by performing machine learning based on the data obtained from the data storage device 120, and predicts the acceleration of the vehicle 200 based on the constructed acceleration inference model and the data obtained from the data storage device 120; and an operator terminal 130, which is configured to be able to communicate with each vehicle 200 via the communication device 110 based on a notification from the computing server 10.

[0060] As described later, the data storage 120 is a data server with a database, and the computing server 10 is a computer capable of performing high-level computing processing at high speed. Although the data storage 120 and the computing server 10 can each be a single server, they can also be a cloud that can distribute the processing load. The data storage 120 and the computing server 10 can also be the same server. The operator terminal 130 is not an essential structure and can be omitted depending on the type of service involved in the vehicle 200.

[0061] Figure 21 is a block diagram showing the structure of a vehicle 200. The vehicle 200 is composed of a storage device 18, an image information processing unit 20, an input device 12, a computing device 14, and an output device 16. The storage device 18 stores data necessary for computing by the computing device 14 and computing results obtained by the computing device 14. The image information processing unit 20 calculates the operating status of the vehicle 200 based on the image information obtained by the camera 22. The input device 12 is input with information related to the operating status of the vehicle 200 calculated by the image information processing unit 20, the front and rear speed of the vehicle detected by the vehicle speed sensor 24, and the IMU (inertial measurement unit) 26. The calculation device 14 estimates the acceleration of the vehicle 200 based on the input data input from the input device 12 and the data stored in the storage device 18 as needed, and the output device 16 outputs the calculation result in the calculation device 14 to the V2X communication unit 34. In addition, in addition to the brake pedal sensor 32, the vehicle 200 may also be equipped with a master cylinder sensor for detecting the pressure in the master cylinder of the brake.

[0062] As described above, vehicle 200 is a so-called networked vehicle, but it may not be a networked vehicle, but may be a vehicle that has a post-installed communication device such as a so-called transaction log that analyzes and effectively uses the driving data sent from the onboard equipment mounted on vehicle 200, as well as various sensors that obtain driving data.

[0063] The imaging device 22 according to the present embodiment is a vehicle-mounted camera or the like, and acquires image data of the periphery of the vehicle 200. In addition, the vehicle speed sensor 24 may be configured to detect the speeds of the four wheels of the vehicle 200, respectively.

[0064] Figure 3The block diagram is a block diagram showing an example of a specific structure of a computing server 10 according to an exemplary embodiment of the present disclosure. The computing server 10 is configured to include a computer 40. The computer 40 includes a CPU (Central Processing Unit) 42, a ROM (Read Only Memory) 44, a RAM (Random Access Memory) 46, and an input / output port 48. As an example, the computer 40 is preferably a model capable of performing high-level arithmetic processing at high speed.

[0065] In the computer 40, the CPU 42, ROM 44, RAM 46, and input / output ports 48 are connected to each other via various buses such as an address bus, a data bus, and a control bus. To the input / output ports 48, as various input / output devices, a display 50, a mouse 52, a keyboard 54, a hard disk (HDD) 56, and a disk drive 60 for reading information from various disks (e.g., CD-ROM or DVD, etc.) 58 are connected.

[0066] Furthermore, the input / output port 48 is connected to a network 62, and is configured to be able to exchange information with various devices connected to the network 62. In the present exemplary embodiment, a data storage 120 as a data server connected to a database (DB) 122 is connected to the network 62, and is configured to be able to exchange information with the DB 122.

[0067] The DB 122 stores time series data of a plurality of vehicles 200 acquired via the communication device 110 . The data stored in the DB 122 may be registered via another device connected to the computer 40 or the network 62 other than the communication device 110 .

[0068] Although the present exemplary embodiment is described as a case where time series data of a plurality of vehicles 200 are stored in DB 122 connected to the data storage device 120, the following method may be adopted, i.e., the information of DB 122 may be stored in an external storage device such as HDD 56 built into the computer 40 or an external hard disk.

[0069] A program related to machine learning using a neural network is installed in the HDD 56 of the computer 40. In the present exemplary embodiment, the CPU 42 executes the program to start machine learning, and then constructs a learned model based on the machine learning. Furthermore, the constructed learned model is used to estimate the acceleration of the vehicle 200. In addition, the CPU 42 causes the display 50 to display the processing results obtained by the program.

[0070] There are several methods for installing the program involved in the machine learning of the present exemplary embodiment into the computer 40. For example, the program may be stored together with a setup program in a CD-ROM or DVD in advance, the disk may be placed in the disk drive 60, and the setup program may be executed for the CPU 42 to install the program into the HDD 46. Alternatively, the program may be installed into the HDD 46 by communicating with other information processing equipment connected to the computer 40 via a public telephone network or the network 62.

[0071] Figure 4 A functional block diagram of the CPU 42 of the computing server 10 is shown. Various functions that are realized by executing a program related to machine learning by the CPU 42 of the computing server 10 are described. The program related to machine learning has: a pre-processing function for calculating the cumulative value and average value of the data sent from the vehicle 200, a data screening function for screening the data for machine learning, a model generation function for generating a model for acceleration inference, a model for wheel speed inference, and a model for road surface input inference, a learning function for performing machine learning by providing the screened data as teaching data to a candidate model, an evaluation function for evaluating the performance of a learned model using the measured value (measured value) as teaching data, and a selection function for selecting a learned model with excellent performance. By executing the program related to machine learning having these functions by the CPU 42, the CPU 42 is able to perform machine learning. Figure 4 As shown, it functions as a pre-processing unit 72 , a data screening unit 74 , a model generating unit 76 , a learning unit 78 , an evaluating unit 80 , and a selecting unit 82 .

[0072] Figure 5This is a functional block diagram of the CPU 42 of the computing server 10 after learning acceleration inference. The CPU 42 after learning has: a pre-processing function for calculating the cumulative value and average value of the data sent from the vehicle 200, a data filtering function for filtering the data used for collision detection, an acceleration inference function for inferring the acceleration of the vehicle 200 using the learned model, a difference calculation function for calculating the difference between the measured value and the estimated value, a judgment function for judging the presence or absence of a collision, and a damage direction inference function for inferring the direction of damage caused by the collision. The CPU 42 executes the program related to collision detection having these functions, so that the CPU 42 is like Figure 5 As shown, it functions as a pre-processing unit 84, a data filtering unit 86, an acceleration estimation unit 88, a difference calculation unit 90, a determination unit 92, and a damage direction estimation unit 94.

[0073] Figure 6 This is a functional block diagram of the CPU 42 of the calculation server 10 after learning of wheel speed estimation. The CPU 42 after learning has: a pre-processing function for calculating the cumulative value and average value of the data sent from the vehicle 200, a data filtering function for filtering the data used for wheel speed estimation, a wheel speed estimation function for estimating the speed of each wheel of the vehicle 200 using the learned model, a difference calculation function for calculating the difference between the measured value and the estimated value, and a judgment function for judging the presence or absence of a collision and a road surface input as a force input from the road surface to the vehicle 200 via the tire. The CPU 42 executes a program related to collision detection having these functions, so that the CPU 42 is like Figure 6 As shown, it functions as a pre-processing unit 184 , a data filtering unit 186 , a wheel speed estimating unit 188 , a difference calculating unit 190 , and a determining unit 192 .

[0074] Figure 7 This is a functional block diagram of the CPU 42 of the calculation server 10 after learning the road input inference. The CPU 42 after learning has a pre-processing function for calculating the cumulative value and average value of the data sent from the vehicle 200, a data filtering function for filtering the data used for the road input inference, a road input inference function for inferring the input from the road to the vehicle 200 via the tires using the learned model, and a judgment function for judging the presence or absence of road input. The CPU 42 executes the program related to the collision detection having these functions, so that the CPU 42 can be used as a collision detection system. Figure 7 As shown, it functions as a pre-processing unit 284, a data filtering unit 286, a road surface input estimating unit 288, and a determining unit 290.

[0075] Figure 81 is a flowchart showing an example of processing in the computing server 10 according to the present exemplary embodiment. Figure 8 On the left side are threads involved in constructing a model for acceleration inference obtained by machine learning (hereinafter referred to as the “acceleration inference model”), a model for wheel speed inference (hereinafter referred to as the “wheel speed inference model”), and a model for inferring the presence or absence of road surface input (hereinafter referred to as the “road surface input inference model”). Figure 8 On the right side are the threads involved in collision detection using the learned acceleration inference model, wheel speed inference model, and road surface input inference model.

[0076] In step 600, in order to construct an acceleration estimation model, various driving data of a plurality of vehicles 200 are collected from the data storage 120. Since the various driving data of the vehicles 200 include data that requires pre-processing such as calculation of cumulative values ​​and average values, the data involved is processed in step 600.

[0077] In step 602, data screening is performed to select various driving data that can be used as teaching data for machine learning. Various driving data that can be used as teaching data include, for example, measured values ​​of vehicle motion mechanically associated with the front, rear, left, and right acceleration or wheel speed of the vehicle 200 represented by the vehicle speed, the operating state of vehicle motion control represented by the ABS (Antilock Braking System) that is the main cause of the increase or decrease of the front, rear, left, and right acceleration or wheel speed of the vehicle 200, control signals indicating driving operations performed by the driver such as pedals or steering wheels of the vehicle 200 that are the main cause of the increase or decrease of the front, rear, left, and right acceleration or wheel speed, and control signals indicating the operation of power units such as engines that are the main cause of the increase or decrease of the front, rear, left, and right acceleration or wheel speed. In addition, measured values ​​of the vehicle environment that indirectly affects the increase or decrease of the front, rear, left, and right acceleration and wheel speed, such as the outside temperature or wipers, and control signals indicating the operation of vehicle-mounted devices used according to the outside environment can also be used. Even if the measured value of the vehicle external environment is not a value obtained by the vehicle-mounted sensor, external data generally disclosed by the Meteorological Agency or the like may be used. In the data screening of step 602, various driving data of the vehicle 200 under relatively wide conditions such as geographical or meteorological conditions are collected as teaching data. In addition, by importing more significant acceleration changes caused by emergency braking of the vehicle 200, more extreme driving operations such as sudden steering, more extreme motion states of the vehicle 200 such as high vehicle speed, and control states that rarely occur such as ABS operation into the teaching data as much as possible, a robust acceleration inference model can be constructed.

[0078] The teaching data may include such acceleration data related to an accident. This is because, since accidents are extremely rare and are difficult to predict based on driving operations in principle, even if the acceleration data is included in the teaching data, it is believed that the impact on model construction is relatively small.

[0079] In step 604, a candidate model for acceleration inference is generated, and the selected various driving data are provided as teaching data to the candidate model to perform machine learning. In this exemplary embodiment, the model for which machine learning is performed is a model that infers the acceleration of the vehicle 200 caused by the driving operation, but does not infer the acceleration caused by the collision. In other words, the acceleration that cannot be inferred is regarded as a force from outside the vehicle, and the situation in which the acceleration involved is generated is judged as the possibility of a collision.

[0080] In step 604, a separate model may be constructed for each structure of the vehicle 200, such as the vehicle type, product year, grade, and tires used. In addition, a model may be constructed according to the country, season, and climate where the vehicle 200 is used. Alternatively, a candidate model for acceleration estimation may be constructed by adding the conditions related to the structure of the vehicle 200, such as the vehicle type, product year, grade, and tires used, and the country, season, and climate where the vehicle 200 is used to the various driving data described above.

[0081] In this exemplary embodiment, an artificial recurrent neural network (RNN) for processing time series data uses, for example, LSTM (Long Short-Term Memory). LSTM is a structure that can internally perform processes such as (1) inferring the acceleration of vehicle 200 based on the driving operation at the same time and the history of the driving operation, (2) inferring the acceleration of vehicle 200 based on the immediately preceding vehicle movement, and (3) inferring the acceleration of vehicle 200 based on the correspondence between the immediately preceding driving operation and the vehicle movement.

[0082] In LSTM, as vectors of output values ​​of various sensors at time t (hereinafter referred to as “sensor values”), sensor value vectors x and 1,t 、x 2,t . Respectively, the sensor value vector x 1,t The sensor value vector x is a vector that is composed of the measured values ​​of the vehicle motion mechanically associated with the front, rear, left, and right accelerations or wheel speeds of the vehicle 200, represented by the vehicle speed. 2,tIts elements include control signals indicating driving operations performed by the driver, such as pedals or a steering wheel of the vehicle 200, which are the main causes of increase or decrease in front, rear, left, and right acceleration or wheel speeds, and control signals indicating the operation of a power unit, such as an engine, which are the main causes of increase or decrease in front, rear, left, and right acceleration or wheel speeds.

[0083] Since the sensor value vector x 1,t 、x 2,t are time series data respectively, so the sensor value matrix x used to estimate the acceleration of the vehicle 200 at time t is calculated as follows t Define the sensor value matrix x t The m in is an arbitrary mask value.

[0084]

[0085] In this exemplary embodiment, according to the sensor value matrix x stored in t The time series data in the image is used to construct a model f for inferring the acceleration of the vehicle 200 using LSTM. In the model f of this exemplary embodiment, the model f for inferring the front and rear acceleration of the vehicle 200 has fb (x t ), and a model f for estimating the left and right acceleration of the vehicle 200 lr (x t ).

[0086] LSTM is an algorithm that predicts output based on input time series data, and constructs a model of output prediction value by updating past data with new data. t By applying it as teaching data, a model f for estimating the longitudinal acceleration of the vehicle 200 is constructed as follows: fb (x t ), and a model f for estimating the left and right acceleration of the vehicle 200 lr (x t ). In this exemplary embodiment, as Figure 1 As shown, since a sensor value matrix x is collected in large quantities from a plurality of vehicles 200 t The data involved makes it easier to build a model implemented by LSTM.

[0087] Acceleration inference model f fb (x t ), f lr (x t) can be constructed as a statistical model based on a non-neural network or based on the motion equation of the vehicle 200.

[0088]

[0089]

[0090] Fore and aft acceleration inference, Left and right acceleration estimation

[0091] In step 606, the constructed model f is adjusted using the new time series data as the measured values. fb (x t ), f lr (x t ) and outputs a model f whose error between the measured value and the estimated value is within the allowable range. fb (x t ), f lr (x t ) and the process ends. The model f output in step 606 fb (x t ), f lr (x t ) is used in the collision detection thread.

[0092] In step 610, various driving data of the plurality of vehicles 200 are collected from the data storage 120 in order to construct the wheel speed estimation model. If there is data that requires pre-processing such as calculation of cumulative values ​​and average values ​​among the various driving data of the vehicles 200, the data is processed in step 610.

[0093] In step 612, data screening is performed to select various driving data that can be used as teaching data for machine learning. The various driving data that can be used as teaching data are the same as the various driving data listed in step 602 above.

[0094] In step 614, a candidate model for wheel speed estimation is generated, and the selected various driving data are provided as teaching data to the candidate model to perform machine learning. In this exemplary embodiment, the model for which machine learning is performed is a model for estimating the wheel speed of each of the four wheels of the vehicle 200, namely, the front wheels and the rear wheels.

[0095] In step 614, similarly to step 604, an independent model may be constructed for each structure of the vehicle 200, such as the vehicle type, product year, grade, and tires used. In addition, a model may be constructed based on the country, season, and climate where the vehicle 200 is used. Alternatively, a candidate model for wheel speed estimation may be constructed by adding the conditions related to the structure of the vehicle 200, such as the vehicle type, product year, grade, and tires used, and the country, season, and climate where the vehicle 200 is used to the various driving data described above. In step 614, similarly to step 604, in the architecture of the RNN, for example, LSTM is used.

[0096] In step 614, similarly to step 604, the sensor value vectors x and x are defined as vectors of sensor values ​​at time t. 1,t 、x 2,t . Respectively, the sensor value vector x 1,t The sensor value vector x is a vector that is composed of the measured values ​​of the vehicle motion mechanically associated with the front, rear, left, and right accelerations or wheel speeds of the vehicle 200, represented by the vehicle speed. 2,t Its elements include control signals indicating driving operations performed by the driver, such as pedals or a steering wheel of the vehicle 200, which are the main causes of increase or decrease in front, rear, left, and right acceleration or wheel speeds, and control signals indicating the operation of a power unit, such as an engine, which are the main causes of increase or decrease in front, rear, left, and right acceleration or wheel speeds.

[0097] In step 614, similarly to step 604, the sensor value matrix x t Then, according to the sensor value matrix x stored in t The time series data in , uses LSTM to build a model f for inferring the wheel speed of vehicle 200 vx (x t ). Wheel speed inference model f vx (x t ) can be constructed as a statistical model based on a non-neural network or based on the motion equations of the vehicle 200.

[0098] In step 616, the constructed model f is adjusted using the new time series data as the measured values. vx (x t ) and outputs a model f whose error between the measured value and the estimated value is within the allowable range. vx (x t ) and the process ends. The model f output in step 616 vx (x t ) is used in the collision detection thread.

[0099] In step 620, various driving data of the plurality of vehicles 200 are collected from the data storage 120 in order to construct a road surface input estimation model. If there is data that requires pre-processing such as calculation of cumulative values ​​and average values ​​among the various driving data of the vehicles 200, the data involved is processed in step 620.

[0100] In step 622, data screening is performed to select various driving data that can be used as teaching data for machine learning. Although the various driving data that can be used as teaching data are the same as the various driving data listed in the aforementioned step 602, since the undulations of the road surface that generate the road surface input occur in the same place in most cases, data on the convex and concave positions of the road surface based on positioning data obtained from a GPS (Global Positioning System) and the like are included.

[0101] In step 624, a candidate model for road surface input inference is generated, and the selected various driving data are provided as teaching data to the candidate model to perform machine learning. In this exemplary embodiment, the model to which machine learning is performed is a model that infers road surface input based on changes in acceleration of the vehicle 200, changes in wheel speeds of the four wheels, namely, the front wheels and the rear wheels, and information on the positions of bumps and depressions on the road surface.

[0102] In step 624, similarly to step 604, an independent model may be constructed for each structure of the vehicle 200, such as the vehicle type, product year, grade, and tires used. In addition, a model may be constructed based on the country, season, and climate in which the vehicle 200 is used. Alternatively, a candidate model for road surface input inference may be constructed by adding the conditions related to the structure of the vehicle 200, such as the vehicle type, product year, grade, and tires used, and the country, season, and climate in which the vehicle 200 is used to the various driving data described above. In step 624, similarly to step 604, in the architecture of the RNN, for example, LSTM is used.

[0103] In step 624, similarly to step 604, a model for estimating the road surface input of vehicle 200 is constructed based on the time series data using LSTM. The road surface input estimation model may be constructed as a statistical model based on a non-neural network or based on the motion equation of vehicle 200.

[0104] In step 626, the performance of the constructed model is evaluated using new time series data as measured values, and a model is outputted in which the error between the measured value and the estimated value is within the allowable range, and the process ends. The model outputted in step 626 is used in the collision detection thread.

[0105] Fig. 9 The figure is a schematic diagram showing the situation of implementing accident judgment based on the difference between the measured value and the estimated value of acceleration (hereinafter referred to as "acceleration difference"), the difference between the measured value and the estimated value of FL (left front wheel) wheel speed (hereinafter referred to as "FL wheel speed difference"), the difference between the measured value and the estimated value of RL (right rear wheel) wheel speed (hereinafter referred to as "RL wheel speed difference"), the pedal operation performed by the driver, and latitude and longitude and other factors. Fig. 9 In this study, we studied the road surface input and collision identification logic using time series data at the moment of the accident and before and after the accident.

[0106] exist Fig. 9 In the time period 300, the acceleration difference, the FL wheel speed difference, and the RL wheel speed difference show significant changes. Moreover, after the acceleration difference shows significant changes at time t1 in the time period 300, the FL wheel speed difference and the RL wheel speed difference show significant changes at the same time at time t2. Fig. 9 As shown in FIG. 1 , if the difference between the measured value and the estimated value of the wheel speed (hereinafter referred to as "wheel speed difference") changes after the change in the acceleration difference, or if the wheel speed differences of the front and rear wheels or the four wheels change simultaneously, it is suspected that there is a collision caused by an accident. In addition, if the acceleration difference changes after the change in the wheel speed difference, or if the wheel speed differences of the front and rear wheels change sequentially, it can be determined as a road surface input.

[0107] In this exemplary embodiment, since when LSTM is used to implement machine learning, Figure 1 As shown, a large number of examples of road input can be collected from multiple vehicles 200, so the acceleration difference caused by the road input is inferred based on the driving operation and the wheel speed difference, and the acceleration caused by the road input is subtracted from the inferred acceleration difference, thereby being able to infer the acceleration caused by the collision.

[0108] In addition, if the vertical displacement of the position information of the vehicle 200 can be detected by GPS or the like, the acceleration involved in the acceleration difference when the displacement is greater than a predetermined value can also be determined as the road surface input. Furthermore, in the case where it is possible to grasp the places with undulations on the road surface and the places where the slope changes sharply, the acceleration involved in the acceleration difference when the vehicle 200 passes through the places can also be determined as the road surface input. Furthermore, in the case where multiple vehicles 200 have the same wheel speed changes and acceleration changes at the same place, the information of the place can also be databased, and the acceleration detected when the vehicle 200 passes through the place can be determined as the road surface input.

[0109] Fig.10 This is a schematic diagram of the accident judgment based on the difference between the actual measured value and the estimated value obtained by RNN. fb (x t ), f lr (x t ) is the acceleration caused by driving operation, so it is impossible to infer the model f from the acceleration. fb (x t ), f lr (x t ) is not caused by the driving operation, and is therefore estimated to be caused by the collision of the vehicle 200. Therefore, if the difference Δa between the actual measurement value and the estimated value obtained by the RNN is fb,t , Δa lr,t If it is large, it can be considered that the model f cannot be inferred from the acceleration. fb (x t ), f lr (x t ) to infer the acceleration.

[0110] In the present exemplary embodiment, the difference Δa between the actual measured value and the estimated value of the acceleration of the vehicle 200 is fb,t , Δa lr,t To implement accident judgment, the acceleration acting on the vehicle 200 includes a component caused by the road input. In this exemplary embodiment, in addition to using the acceleration inference model constructed by LSTM1 to infer the acceleration of the vehicle 200, the wheel speed inference model constructed by LSTM2 is used to infer the wheel speed of the vehicle 200 in the front and rear wheels, and the wheel speed difference Δv is calculated. vx,t , and further use the road input inference model constructed by LSTM3 to infer the road input, so as to implement the final accident judgment. Wheel speed difference Δvvx,t Alternatively, the accident determination may be performed based on the difference between the actual value and the estimated value of the vertical acceleration, yaw rate, and vehicle speed of the vehicle 200 .

[0111] Based on the above description, an example of processing in the collision detection thread is described. In step 700 of the collision detection thread, various driving data are collected from the data storage 120 for each vehicle 200. Since the various driving data of the vehicle 200 include data that needs to be pre-processed to calculate the cumulative value and the average value, the data involved will be processed in step 700.

[0112] In step 702, data screening is performed to select various driving data used in the acceleration estimation of the vehicle 200. The various driving data used in the acceleration estimation are observed quantities related to the acceleration change of the vehicle 200 caused by the driving operation, and include, for example, a measured value of the vehicle motion mechanically associated with the front, rear, left, and right acceleration or wheel speed of the vehicle 200 represented by the vehicle speed, an operation state of the vehicle motion control represented by the ABS that is the main cause of the increase or decrease of the front, rear, left, and right acceleration or wheel speed of the vehicle 200, a control signal indicating the driving operation performed by the driver such as the pedals or steering wheel of the vehicle 200 that is the main cause of the increase or decrease of the front, rear, left, and right acceleration or wheel speed, and a control signal indicating the operation of the power unit such as the engine that is the main cause of the increase or decrease of the front, rear, left, and right acceleration or wheel speed. In addition, a measured value of the vehicle environment that indirectly affects the increase or decrease of the front, rear, left, and right acceleration or wheel speed, such as the outside temperature or the wiper, and a control signal indicating the operation of the vehicle-mounted device used according to the vehicle environment may also be used. Even if the measured value of the vehicle external environment is not obtained by the vehicle-mounted sensor, the external data generally disclosed by the Meteorological Agency or the like may be used. In the data screening of step 602, various driving data of the vehicle 200 under relatively wide conditions such as geographical or meteorological conditions are collected as teaching data. In addition, relatively significant acceleration changes caused by emergency braking of the vehicle 200, relatively extreme driving operations such as sudden steering, relatively extreme motion states of the vehicle 200 such as high vehicle speed, and rare control states such as ABS operation are used as much as possible.

[0113] In step 702, various driving data may be filtered using logic that is simpler than RNN, thereby reducing the processing in the later stage and speeding up the processing. For example, the jerk, which is the time derivative of the acceleration, may be compared with a predetermined threshold, and data with the jerk less than the predetermined threshold may not be used as meaningful data.

[0114] In step 704, the acceleration inference model f constructed by the model construction thread is used. fb (x t ), f lr (x t ) is used to calculate the estimated value of the acceleration of the vehicle 200.

[0115] In step 706, the actual measured value of acceleration detected by the IMU 26 and the acceleration estimation model f are respectively compared in the following manner: fb (x t ), f lr (x t ) The difference Δa between the estimated acceleration values ​​obtained fb,t , Δa lr,t Perform calculations.

[0116]

[0117]

[0118] As described above, in this exemplary embodiment, the acceleration inference model f constructed by the LSTM-based RNN cannot be used. fb (x t ), f lr (x t ) is considered as a force from outside the vehicle. Therefore, the difference Δa fb,t , Δa lr,t When any one of the above is greater than the predetermined threshold, it is a case where a large acceleration is applied to the vehicle 200 from outside the vehicle, and it can be inferred that an accident caused by a collision has occurred. The predetermined threshold is determined based on various driving data collected from multiple vehicles 200. In addition, the predetermined threshold can be either fb,t , Δa lr,t In the difference Δa fb,t , Δa lr,t are different values.

[0119] Fig.11 , a schematic diagram showing an example of values ​​detected by various sensors. Fig.11 In the figure, the front and rear accelerations and the left and right accelerations are marked with the actual measured values ​​(solid lines) obtained by sensors such as the IMU 26 and the acceleration inference model f fb (x t ), f lr (x t ) obtained (dashed line).

[0120] exist Fig.11 In step 708, the measured value and the estimated value in the left and right accelerations are greatly different, and there is a possibility of collision in the case involved. fb,t , Δa lr,t When any one of the values ​​is greater than a predetermined threshold, it is inferred that the vehicle 200 has collided. The predetermined threshold is determined based on statistics of actual measured values ​​of the operating conditions of the vehicle 200. In addition, the following formula may be used to calculate the inferred input composite value a from outside the vehicle as needed: t Perform calculations and infer the input composite value a t When the combined value exceeds a predetermined upper limit, it is estimated that the vehicle 200 has collided.

[0121]

[0122] In step 710 , the wheel speed estimation model constructed using LSTM2 is used to estimate the wheel speed of each of the four wheels, namely the front and rear wheels, of the vehicle 200 .

[0123] In step 712, the acceleration involved in the road input is inferred using the road input inference model constructed using LSTM2. In step 712, for example, the wheel speed difference of the vehicle 200 is calculated. Then, it is assumed that when the acceleration difference calculated in step 706 changes after the change in the wheel speed difference, or when the wheel speed difference of the front and rear wheels changes sequentially, the acceleration difference is caused by the road input. In addition, it is assumed that when the wheel speed difference changes after the acceleration difference changes, or when the wheel speed difference of the front and rear wheels or each of the four wheels changes at the same time, the acceleration difference is caused by a collision.

[0124] Fig. 12A This is an example of an image of a situation where the vehicle 200 attempts to cross the height difference 310. Fig. 12B This is an explanatory diagram showing a situation in which, when the vehicle 200 passes over the height difference 310, the left and right front wheels spin and speed up, and then the left and right rear wheels spin and speed up. Fig. 12B In the figure, the solid line is the measured value and the dotted line is the estimated value. Fig. 12B Since the wheel speed difference between the front and rear wheels changes sequentially, it can be inferred that an acceleration difference occurs due to the road surface input.

[0125] Fig.13A is an example of an image of a situation where the vehicle 200 passes through a rough road surface 320, Fig. 13BThis is an explanatory diagram showing a situation where two maximum values ​​of the acceleration difference are observed and the left front wheel and the left rear wheel are accelerated before each maximum value. Fig. 13B In the figure, the solid line is the measured value and the dotted line is the estimated value. Fig. 13B Since the acceleration difference changes after the wheel speed difference changes, and the wheel speed difference between the front and rear wheels changes sequentially, it can be inferred that the acceleration difference occurs due to the road surface input.

[0126] Acceleration not caused by driving operations can also be detected through image analysis. Fig.14A This is an example of an image obtained by the camera device 22 of the vehicle 200 showing a situation in which the right front wheel of the vehicle 200 spins when the accelerator pedal is depressed while turning right at a level difference between a sidewalk and a lane. Fig. 14B This is an explanatory diagram showing a situation where the actual measured value (solid line) and the estimated value (dashed line) of the wheel speed at the right front wheel diverge as a result of the right front wheel of the vehicle 200 spinning and accelerating. Fig.14A In the process, the right front wheel of the vehicle 200 is tilted, forming a so-called three-point contact, so that the right front wheel spins.

[0127] Fig.15A 2 is an example of an image obtained by the camera device 22 of the vehicle 200, showing a situation where the left rear wheel of the vehicle 200 is unloaded (pitched). Fig. 15B This is an explanatory diagram showing a situation where the actual measured value (solid line) and the estimated value (dashed line) of the wheel speed at the left rear wheel diverge as a result of the left rear wheel spinning. Fig.15A , Fig. 15B In the case shown, the driver of vehicle 200 visually recognizes a person 210 riding a bicycle in front of him and applies emergency brakes and turns left. As a result, the left rear wheel slips, the vehicle 200 decelerates due to the braking force of the brake, and a difference between the measured value and the estimated value of the wheel speed of the left rear wheel occurs.

[0128] exist Fig.14A , Fig. 14B , Fig.15A as well as Fig. 15BIn any of the cases, even if the vehicle 200 does not collide, acceleration not caused by driving operation may occur. In step 712, in addition to the road input inference, for example, a violent operation condition other than the collision of the vehicle 200 can be extracted from the time series image data obtained by the camera device 22, and the change in the measured value of the more significant acceleration at the time of causing the operation condition is excluded from the measured value as an acceleration not related to the collision, thereby implementing accident judgment. For extracting a violent operation condition other than the collision of the vehicle 200 from the image data, as an example, judgment is made based on the position change per unit time in the image data of pixels (focus pixels) at specific positions such as the four corners of the vehicle 200. When the position change per unit time of the focus pixel is greater than a predetermined position change threshold, it is judged to be a violent operation condition of the vehicle 200. Furthermore, if the shape and area of ​​the set of pixels representing the vehicle 200 in the image data do not change, it can be judged to be a violent operation condition other than the collision of the vehicle 200.

[0129] In step 714 , if the acceleration difference is caused by the road input, the acceleration involved in the acceleration difference is not used for collision determination and the process ends. In step 714 , if the acceleration difference is not caused by the road input, the process proceeds to step 716 .

[0130] In step 716, the damage direction of the vehicle 200 is estimated using the following formula. fb,t Assume that the vector quantity in the front-back direction is lr,t Assuming it is a vector quantity in the left and right directions, the difference Δa fb,t and the difference Δa lr,t The quotient of becomes the tangent of the angle representing the damage direction. Therefore, the difference Δa fb,t and the difference Δa lr,t The inverse tangent of the quotient d t The damage direction may be estimated based on the displacement of the position information of the vehicle 200 detected by GPS or the change of the speed of each wheel of the vehicle 200 other than the direction of acceleration.

[0131]

[0132] In step 718 , the calculation server 10 notifies the operator terminal 130 and the like of the determination result and ends the process.

[0133] Fig.16 1 is a block diagram showing an example of the operation of the information processing device 100 according to the present exemplary embodiment. Fig.16As shown, the data storage 120 is as shown in Figure 8 The steps 600 and 700 shown collect various driving data of the vehicle 200 in this way.

[0134] The various driving data of the vehicle 200 obtained by the data storage 120 are sent to the computing server and Figure 8 Steps 602 and 702 of data filtering are thus implemented.

[0135] In the computing server 10, the following is implemented: Figure 8 Step 604 is such learning, thereby constructing the acceleration inference model f fb (x t ), f lr (x t ). Then, use the acceleration inference model f fb (x t ), f lr (x t ) to implement Figure 8 The acceleration estimation shown in steps 704 to 710, the difference Δa fb,t , Δa lr,t calculation, collision judgment, and inference of damage direction.

[0136] Then, like Figure 8 In step 712 , the computing server 10 implements notifications such as second-hand car evaluation, vehicle 200 maintenance, vehicle 200 inspection, vehicle 200 operation record, new car sales operation, or safety confirmation call to the operator terminal 130 .

[0137] Although in Fig.16 In the application example shown, data collection and screening, acceleration estimation model f are implemented in the calculation server 10. fb (x t ), f lr (x t ) construction, acceleration estimation, difference Δa fb,t , Δa lr,t However, in order to reduce the load, the calculation server 10 may also implement the acceleration estimation model f based on various driving data of the vehicle 200 obtained from the data storage 120. fb (x t ), f lr (x t ) is constructed, and in other servers, the acceleration inference model f constructed by the computing server 10 is used fb (x t ), f lr (x t), thereby implementing Figure 6 The data collection and screening, acceleration estimation, and difference Δa involved in steps 700 to 710 fb,t , Δa lr,t calculation, collision judgment, and inference of damage direction.

[0138] Fig.17 FIG. 2 is a block diagram showing another example of the operation of the information processing device 100 according to the present exemplary embodiment. Fig.17 As shown, the data storage 120 is as shown in Figure 8 Step 600 shown collects various driving data of vehicle 200 in this way.

[0139] The various driving data of the vehicle 200 obtained by the data storage 120 are sent to the computing server and Figure 8 Step 602 of data filtering is thus implemented.

[0140] In the computing server 10, the following is implemented: Figure 8 Step 604 is such learning, thereby constructing the acceleration inference model f fb (x t ), f lr (x t ). Then, the acceleration inference model f fb (x t ), f lr (x t ) is sent to vehicle 200.

[0141] In the vehicle 200, the computing device 14 performs Figure 8 The data screening, acceleration estimation, and difference Δa shown in steps 702 to 710 are fb,t , Δa lr,t calculation, collision judgment, and inference of damage direction.

[0142] Then, if Figure 8 In step 712 , the vehicle 200 implements notifications such as second-hand car evaluation, maintenance of the vehicle 200 , inspection of the vehicle 200 , operation records of the vehicle 200 , operation of new car sales, or safety confirmation calls to the operator terminal 130 .

[0143] exist Fig.17 In the illustrated application example, since the collision detection is performed in the vehicle 200 , it is not necessary for the data storage 120 to obtain various driving data from the vehicle 200 and forward the obtained various driving data to the calculation server 10 , so the capacity of the data storage 120 can be suppressed.

[0144] As described above, in this exemplary embodiment, when the difference between the actual measured value of the acceleration of the vehicle 200 detected by the IMU 26, etc. and the estimated value of the acceleration of the vehicle 200 calculated based on the observed value related to the change in the acceleration of the vehicle 200 caused by the driving operation is greater than a predetermined threshold, it is determined that the vehicle 200 has collided.

[0145] The estimated value of acceleration calculated based on the observed value related to the change in the acceleration of the vehicle 200 caused by the driving operation is the estimated value of the acceleration of the vehicle 200 caused by the driving operation. In addition, the acceleration of the vehicle 200 generated during the collision is different from the change in the acceleration caused by the driving operation. Therefore, when the difference between the actual value of the acceleration of the vehicle 200 detected by the IMU 26 and the estimated value of the acceleration of the vehicle 200 calculated based on the observed value related to the change in the acceleration of the vehicle 200 caused by the driving operation is greater than a predetermined threshold, it can be determined that the vehicle 200 has collided.

[0146] Since there may be a discrepancy between the actual measured value and the predicted value of acceleration even in a minor collision, the collision of the vehicle 200 can be estimated with high accuracy.

[0147] In this exemplary embodiment, a separate acceleration inference model may be constructed according to each structure of the vehicle 200, such as the vehicle type, product year, grade, and tires used. Furthermore, a model may be constructed according to the country, season, and climate where the vehicle 200 is used. Alternatively, a candidate model for acceleration inference may be constructed by adding the conditions related to the structure of the vehicle 200, such as the vehicle type, product year, grade, and tires used, and the country, season, and climate where the vehicle 200 is used to the various driving data described above. By constructing an acceleration inference model in a manner that subdivides the conditions, the accuracy of collision detection can be improved.

[0148] Furthermore, since the vehicle 200 as a networked vehicle is used for the collection of various driving data, collision detection can be performed using the calculation server 10 or the like that is remote from the vehicle 200 without attaching a device to the vehicle 200 .

[0149] In the present exemplary embodiment, since the acceleration inference model is constructed using machine learning based on so-called big data obtained from a plurality of vehicles 200, it is possible to construct a model that accurately infers the acceleration of vehicle 200 caused by driving operation.

[0150] In this exemplary embodiment, a model f for estimating the longitudinal acceleration of the vehicle 200 can be constructed. fb (x t), and a model f for estimating the left and right acceleration of the vehicle 200 lr (x t ), and the damage direction of the vehicle 200 is estimated based on the predicted value of the acceleration in the front-rear direction and the acceleration in the left-right direction of the vehicle 200. Furthermore, the estimated result of the collision accident and the estimated result of the damage direction of the vehicle 200 can be used in the recording of accident history in used car sales, the maintenance benchmark of commercial vehicles such as rental vehicles, the estimation of the presence or absence of collision when returning a rental car, the notification of warehousing promotion to dealers or repair shops, the confirmation notification of safety from the insurance company when an accident is suspected, and the operation record of the autonomous driving vehicle.

[0151] Furthermore, in this exemplary embodiment, the wheel speed of the vehicle 200 is estimated, and the difference between the measured value and the estimated value of the wheel speed is used to determine whether there is acceleration caused by road input, and the accuracy of collision detection can be improved through the determination involved.

[0152] In addition, the "detection unit" described in the claims corresponds to the "camera 22", "vehicle speed sensor 24", "steering angle sensor 28", "throttle sensor 30" and "brake pedal sensor 32" described in the specific embodiments of the specification. In addition, the "inertial measurement unit" described in the claims corresponds to the "IMU 26" described in the specific embodiments of the specification.

[0153] In addition, the processing that is executed by the CPU reading the software (program) in each of the above-mentioned exemplary embodiments can also be performed by various processors other than the CPU. As the processor in this case, FPGA (Field-Programmable Gate Array) and other PLD (Programmable Logic Device: Programmable Logic Device) whose circuit structure can be changed after manufacturing, and ASIC (Application Specific Integrated Circuit: Application Specific Integrated Circuit) and other processors with a circuit structure specially designed for performing specific processing, that is, a dedicated electrical circuit, etc. can be exemplified. In addition, the processing can be performed by one of these various processors, or by a combination of two or more processors of the same type or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA, etc.). In addition, more specifically, the hardware structure of these various processors is a circuit obtained by combining circuit elements such as semiconductor elements.

[0154] In addition, in each of the above exemplary embodiments, the program is described as being pre-stored (installed) in the disk drive 60, etc., but it is not limited to this. The program can also be provided in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), and a USB (Universal Serial Bus) memory. In addition, the program can also be downloaded from an external device via a network.

[0155] (Supplementary Item 1)

[0156] An information processing device, comprising:

[0157] Memory;

[0158] a processor, which is at least one and is connected to the memory,

[0159] The processor is configured to determine that the vehicle has collided when a difference between an actual value of the acceleration of the vehicle detected by an inertial measurement unit and an estimated value of the acceleration of the vehicle calculated based on an observed value related to a change in the acceleration of the vehicle caused by a driving operation detected by the detection unit is greater than a predetermined threshold.

Claims

1. An information processing device, comprising: a detection unit that detects an observed quantity related to a change in acceleration of the vehicle caused by a driving operation; an inertial measurement unit for detecting an actual value of the acceleration of the vehicle; an inference unit that determines that the vehicle has collided when a difference between an actual value of the acceleration of the vehicle detected by the inertial measurement unit and an estimated value of the acceleration of the vehicle calculated based on the observed value, that is, an acceleration difference value, is greater than a predetermined threshold value, The estimation unit does not use the estimated value in determining a collision of the vehicle when the acceleration related to the acceleration difference is an acceleration input to the vehicle from a road surface via a tire. The detection unit includes a wheel speed detection unit configured to detect the wheel speed of each of four wheels of the vehicle. The inference unit calculates the wheel speed difference, i.e., the difference between the actual measured value of the wheel speed of each of the four wheels, the front and rear wheels of the vehicle, detected by the wheel speed detection unit and the estimated value of the wheel speed of each of the four wheels, the front and rear wheels of the vehicle, calculated based on the observed value using the wheel speed inference model constructed by the long short-term memory network, and in any one of the cases where the acceleration difference changes after the wheel speed difference changes and the wheel speed difference of the front and rear wheels changes sequentially, the acceleration involved in the acceleration difference is judged as the acceleration input from the road surface via the tire.

2. The information processing device according to claim 1, wherein: The detection unit includes an imaging unit that acquires image data of the surrounding area of ​​the vehicle in time series. The estimation unit uses the acceleration related to the acceleration difference at a time when an intense behavior of the vehicle is recorded in the image data acquired by the imaging unit as the acceleration input to the vehicle from a road surface via a tire.

3. The information processing device according to claim 1, wherein: The estimation unit determines that the acceleration related to the acceleration difference when the displacement in the vertical direction of the position information of the vehicle calculated based on the information from the satellite is equal to or larger than a predetermined value is the acceleration input from the road surface via the tire.

4. The information processing device according to claim 1, wherein: The estimation unit refers to a database of locations where there are undulations on the road surface, locations where the slope changes suddenly, and locations where a plurality of vehicles experience acceleration fluctuations, and determines that the acceleration associated with the acceleration difference is the acceleration input from the road surface via the tires.

5. The information processing device according to any one of claims 1 to 4, wherein: The inertial measurement unit is capable of detecting each of the acceleration in the front-rear direction and the acceleration in the left-right direction of the vehicle. The inference unit is capable of calculating each of the inferred value of the acceleration in the front-to-back direction of the vehicle and the inferred value of the acceleration in the left-to-right direction based on the observed value, and determining that the vehicle has collided when any one of the difference between the actual measured value of the acceleration in the front-to-back direction of the vehicle detected by the inertial measurement unit and the inferred value of the acceleration in the front-to-back direction of the vehicle and the difference between the actual measured value of the acceleration in the left-to-right direction of the vehicle detected by the inertial measurement unit and the inferred value of the acceleration in the left-to-right direction of the vehicle is greater than a predetermined threshold.

6. The information processing device according to claim 5, wherein: The inference unit infers the damage direction of the vehicle based on the quotient of the difference between the actual measured value of the acceleration in the front-rear direction of the vehicle detected by the inertial measurement unit and the estimated value of the acceleration in the front-rear direction of the vehicle, and the difference between the actual measured value of the acceleration in the left-right direction of the vehicle detected by the inertial measurement unit and the estimated value of the acceleration in the left-right direction of the vehicle.

7. The information processing device according to any one of claims 1 to 4, wherein: The estimation unit constructs a model for estimating the acceleration of the vehicle based on observed quantities related to changes in acceleration of a plurality of vehicles acquired in advance and actual measurement values ​​of the acceleration of the plurality of vehicles.

8. The information processing device according to any one of claims 1 to 4, wherein: The observed amount includes a measured value of a vehicle motion of the vehicle and a control signal related to a driving operation amount of the vehicle.

9. An information processing method, in which method, The detection unit detects an observed quantity related to a change in the acceleration of the vehicle caused by the driving operation. The inertial measurement unit detects the actual value of the acceleration of the vehicle. When a difference between an actual value of the acceleration of the vehicle and an estimated value of the acceleration of the vehicle calculated based on the observed value, namely an acceleration difference, is greater than a predetermined threshold value, it is determined that the vehicle has collided, When the acceleration related to the acceleration difference is the acceleration input to the vehicle from the road surface via the tires, the estimated value is not used in the collision determination of the vehicle, The wheel speed difference is calculated as the difference between the actual measured value of the wheel speed of each of the four wheels, the front and rear wheels of the vehicle, and the estimated value of the wheel speed of each of the four wheels, the front and rear wheels of the vehicle, calculated based on the observed value using the wheel speed estimation model constructed by the long short-term memory network. In any of the following cases, the acceleration difference changes after the wheel speed difference changes, and the wheel speed difference of the front and rear wheels changes sequentially. The acceleration involved in the acceleration difference is judged as the acceleration input from the road surface via the tire.

10. A computer-readable medium having an information processing program recorded thereon, wherein: The information processing program causes the computer to function as an inference unit that: The estimation unit determines that the vehicle has collided when a difference between an actual value of the acceleration of the vehicle detected by the inertial measurement unit and an estimated value of the acceleration of the vehicle calculated based on an observed value related to a change in the acceleration of the vehicle caused by a driving operation detected by the detection unit, that is, an acceleration difference, is greater than a predetermined threshold value. When the acceleration related to the acceleration difference is the acceleration input to the vehicle from the road surface via the tires, the estimated value is not used in the collision determination of the vehicle, The wheel speed difference is calculated as the difference between the actual measured value of the wheel speed of each of the four wheels, the front and rear wheels of the vehicle, and the estimated value of the wheel speed of each of the four wheels, the front and rear wheels of the vehicle, calculated based on the observed value using the wheel speed estimation model constructed by the long short-term memory network. In any of the following cases, the acceleration difference changes after the wheel speed difference changes, and the wheel speed difference of the front and rear wheels changes sequentially. The acceleration involved in the acceleration difference is judged as the acceleration input from the road surface via the tire.

Images