Calculation unit and program

Abstract

To provide a computing device and program that can estimate road surface conditions with high accuracy for each type of vehicle. [Solution] The computing device according to this disclosure includes: an acquisition unit that acquires behavior information indicating the behavior of a first type of vehicle while it is in motion; a conversion unit that performs a process to convert the behavior information of the first type of vehicle into data corresponding to the behavior information of a second type of vehicle using a second trained model that has learned the relationship between the behavior information of the first type of vehicle and the behavior information of a second type of vehicle which is another specific type of vehicle; a determination unit that determines the road surface conditions on which the first type of vehicle is traveling by inputting the behavior information after the behavior information of the first type of vehicle has been converted into data corresponding to the behavior information of the second type of vehicle into a trained model that has been learned using the behavior information of the second type of vehicle; and an output unit that outputs the determination result of the road surface conditions.

Description

Technical Field

[0006] , ,

[0005] , , ,

[0001] The present disclosure relates to an arithmetic unit and a program.

Background Art

[0002] Based on the obligation of visual inspection of infrastructure such as roads, it has been carried out to improve efficiency by using the measurement data of sensors mounted on vehicles for inspection of the road surface condition. In addition, with the progress of recent machine learning technology, machine learning has been used for analysis of measurement data.

[0003] For example, in Patent Document 1 below, a first threshold value and a second threshold value are set for each type of vehicle, and based on information regarding the load of the vehicle, information regarding the learned vehicle behavior collected for the type of vehicle, and the correct label of the road surface damage, a state estimation device that estimates the presence of road surface damage using a learned model learned by machine learning is disclosed. <000​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

[0007] To solve the above-mentioned problems and achieve the objective, the computing device according to this disclosure includes: an acquisition unit that acquires behavioral information indicating the behavior of a first type of vehicle while it is in motion; a conversion unit that performs a process to convert the behavioral information of the first type of vehicle into data corresponding to the behavioral information of the second type of vehicle using a second trained model that has learned the relationship between the behavioral information of the first type of vehicle and the behavioral information of a second type of vehicle which is another specific type of vehicle; a determination unit that determines the road surface condition on which the first type of vehicle is traveling by inputting the behavioral information after the conversion of the behavioral information of the first type of vehicle into data corresponding to the behavioral information of the second type of vehicle into a first trained model that has learned using the behavioral information of the second type of vehicle; and an output unit that outputs the determination result of the road surface condition.

[0008] To solve the above-mentioned problems and achieve the objective, the program relating to this disclosure causes a computer to perform the following steps: acquire behavioral information indicating the behavior of a first vehicle while it is in motion; convert the behavioral information of the first vehicle into data corresponding to the behavioral information of the second vehicle using a second trained model that has learned the relationship between the behavioral information of the first vehicle and the behavioral information of a second vehicle, which is another specific vehicle; input the behavioral information after converting the behavioral information of the first vehicle into data corresponding to the behavioral information of the second vehicle into a first trained model that has learned using the behavioral information of the second vehicle, thereby determining the road surface conditions on which the first vehicle is traveling; and output the result of determining the road surface conditions. [Effects of the Invention]

[0009] According to this disclosure, it is possible to provide a computing device and a program that can estimate road surface conditions with high accuracy for each type of vehicle. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a diagram illustrating the overview of the computing system related to this disclosure. [Figure 2] Figure 2 shows an example of the configuration of the computing device according to this disclosure. [Figure 3] Figure 3 shows an example of information stored in the measurement data storage unit of the computing device according to this disclosure. [Figure 4] Figure 4 shows an example of information stored in the model storage unit of the computing device according to this disclosure. [Figure 5] Figure 5 is a diagram illustrating the relationships of the models related to this disclosure. [Figure 6] Figure 6 is a diagram illustrating the conversion of measurement data related to this disclosure. [Figure 7] Figure 7 is a flowchart showing the flow of the calculation method related to this disclosure. [Figure 8] Figure 8 shows an example of the configuration of the measuring device related to this disclosure. [Figure 9] Figure 9 shows an example of measurement data obtained by the measuring device relating to this disclosure. [Modes for carrying out the invention]

[0011] Embodiments of this disclosure will be described in detail below with reference to the drawings. However, the embodiments described below will not limit this disclosure.

[0012] (Overview of the computing system) First, an overview of the calculation system 1 related to this disclosure will be explained using Figure 1. Figure 1 is a diagram illustrating the overview of the calculation system related to this disclosure. As shown in Figure 1, the calculation system 1 related to this disclosure is an information processing system that acquires measurement data from a measuring device 200 mounted on a vehicle C, for example, and performs various calculations using a calculation device 100. Specifically, the calculation device 100 calculates the road surface condition of the road traveled by the vehicle C based on the acquired measurement data. In this embodiment, the road surface condition is an index indicating the degree of unevenness of the road surface. More specifically, in this embodiment, the road surface condition is the IRI (International Roughness Index). However, the road surface condition is not limited to the IRI, and may be any index indicating the condition of the road surface. For example, the road surface condition may be at least one of the following: IRI, flatness of the road surface, cracks, rutting, and MCI (Maintenance Control Index).

[0013] Furthermore, the arithmetic unit 100 is not limited to being located outside the vehicle C, as shown in Figure 1, but may also be located inside the vehicle C. The arithmetic unit 100 may also acquire measurement data measured by measurement devices 200 located in multiple vehicles from the measurement devices 200 of multiple vehicles. Moreover, the arithmetic unit 100 may be an integrated device that combines the functions of the measurement device 200 with the functions of the arithmetic unit 100 itself.

[0014] (Configuration of the computing system) Next, the configuration of the computing system relating to this disclosure will be explained using Figure 1. As shown in Figure 1, the computing system 1 relating to this disclosure comprises a computing device 100, a measuring device 200, and a network N. These configurations will be briefly explained in order below.

[0015] The arithmetic unit 100 is an information processing device that executes various arithmetic processes. The arithmetic unit 100 may be realized by, for example, a PC (Personal Computer), a WS (Work Station), etc. Further, the arithmetic unit 100 may be an in-vehicle infotainment system (In-Vehicle Infotainment System) equipped with a car navigation system or the like, that is, an in-vehicle information processing terminal. Note that the arithmetic unit 100 does not necessarily have to be installed in a vehicle and may be installed at a location other than a vehicle.

[0016] The measuring device 200 is a device that is mounted on the vehicle C and detects measurement data (behavior information) indicating the behavior of the vehicle C while the vehicle C is traveling on a road. The measuring device 200 may detect any data indicating the behavior of the vehicle C as measurement data, but it is preferable to at least detect the acceleration of the vehicle C. For example, in the present embodiment, the measuring device 200 may detect at least one (preferably all) of the acceleration of the vehicle C, the suspension displacement amount of the vehicle C, the direction of the vehicle C, the position of the vehicle C, the image data obtained by imaging the surroundings of the vehicle C, and the operation amount of the vehicle C as measurement data. The configuration for detecting these measurement data by the measuring device 200 will be described later. Note that the measuring device 200 may be mounted for each of a plurality of vehicles and connected to the network N. That is, as shown in FIG. 2, the arithmetic system 1 may include a plurality of measuring devices 200.

[0017] The network N connects the arithmetic unit 100 and the measuring device 200 to be communicable with each other by wire or wirelessly. When the network N is wired, it may be realized by Ethernet (registered trademark) defined in IEEE802.3, a USB (Universal Serial Bus) cable, or the like. When the network N is wireless, it may be realized by a wireless LAN (Local Area Network) defined in IEEE802.11 or Bluetooth (registered trademark).

[0018] As shown in FIG. 1, the computing device 100 and the measuring device 200 are communicably connected to each other via the network N. That is, the computing device 100 and the measuring device 200 function as one computing system 1 by exchanging information with each other via the network N.

[0019] (Configuration of the computing device) Next, the configuration of the computing device according to the present disclosure will be described with reference to FIG. 2. FIG. 2 is a diagram showing a configuration example of the computing device according to the present disclosure. As shown in FIG. 2, the computing device 100 according to the present disclosure includes a communication unit 110, a storage unit 120, a control unit 130, an input unit 140, and a display unit 150. These configurations will be described below in order.

[0020] The communication unit 110 is responsible for transmitting and receiving information between the computing device 100 and external devices. The communication unit 110 may be realized by, for example, a CAN (Controller Area Network) communication interface device, a wireless LAN (Local Area Network) card, a serial communication interface device, a Bluetooth (registered trademark) module, a Wi-Fi (registered trademark) module, an antenna, etc.

[0021] The storage unit 120 is a storage device that stores various types of information. The storage unit 120 includes a main storage device and an auxiliary storage device. The main storage device may be realized by a semiconductor memory element such as, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, etc. Also, the auxiliary storage device may be realized by, for example, a hard disk, an SSD (Solid State Drive), an optical disk, etc.

[0022] As shown in FIG. 2, the storage unit 120 includes a measurement data storage unit 121 and a model storage unit 122. An example of the information stored in these configurations will be described below in order.

[0023] The measurement data storage unit 121 stores information related to the measurement data. Here, an example of the information stored in the measurement data storage unit 121 will be explained using Figure 3. Figure 3 is a diagram showing an example of the information stored in the measurement data storage unit of the computing device according to this disclosure.

[0024] As shown in Figure 3, the measurement data storage unit 121 stores information related to the following items: "measurement data ID", "time", "unsprung X-direction acceleration", "unsprung Y-direction acceleration", "unsprung Z-direction acceleration", "sprung Z-direction acceleration", "displacement", and "position information".

[0025] "Measurement Data ID" is an identifier that identifies the measurement data and is represented by a string of characters or a number. "Time" is information indicating the date and time the measurement data was taken. "Unsprung X-direction acceleration" is information representing the measured value of the acceleration in the X direction from an acceleration sensor mounted vertically downward on the suspension spring. "Unsprung Y-direction acceleration" is information representing the measured value of the acceleration in the Y direction from an acceleration sensor mounted vertically downward on the suspension spring. "Unsprung Z-direction acceleration" is information representing the measured value of the acceleration in the Z direction from an acceleration sensor mounted vertically downward on the suspension spring. "Sprung Z-direction acceleration" is information representing the measured value of the acceleration in the Z direction from an acceleration sensor mounted vertically upward on the suspension spring. "Displacement" is information representing the displacement of the suspension shock absorber. "Position Information" is information representing the position (latitude and longitude) measured at the date and time indicated by "Time".

[0026] In other words, Figure 3 shows an example where measurement data identified by the measurement data ID "DTID#1" is stored in association with the following data: unsprung X-direction acceleration "DXACL#1-1", unsprung Y-direction acceleration "DYACL#1-1", unsprung Z-direction acceleration "DZACL#1-1", sprung Z-direction acceleration "UZACL#1-1", displacement "DP#1-1", and position information "LC#1-1", all measured at the date and time indicated by the time "TIME#1-1".

[0027] Furthermore, the information stored in the measurement data storage unit 121 is not limited to information relating to the items "measurement data ID," "time," "unsprung X-direction acceleration," "unsprung Y-direction acceleration," "unsprung Z-direction acceleration," "sprung Z-direction acceleration," "displacement," and "position information," but may also store other arbitrary information related to measurement data.

[0028] The model storage unit 122 stores information related to the machine learning model. Here, an example of the information stored in the model storage unit 122 will be explained using Figure 4. Figure 4 is a diagram showing an example of the information stored in the model storage unit of the computing device according to this disclosure.

[0029] As shown in Figure 4, the model storage unit 122 stores information related to the items "Model ID," "Model Data," "Model Generation Date and Time," "Training Data Directory," and "Accuracy." The model storage unit 122 may also store other items besides "Model ID," "Model Data," "Model Generation Date and Time," "Training Data Directory," and "Accuracy," and in Figure 4, the inclusion of other items is indicated by the "..." column. Also, Figure 4 shows an example where two models, #1 and #2, are stored, but the number of models can be arbitrary; for example, the number of models may increase as the number of target vehicle types increases.

[0030] The "Model ID" is an identifier that identifies a machine learning model and is represented by a string or a number. The "Model Data" is the data of the machine learning model identified by the "Model ID," and may be the data for the first trained model A1 or the second trained model A2 described below. The first trained model A1 and the second trained model A2 may be managed in separate tables, while the second trained models A2, A2', A2'', etc., which correspond to multiple vehicles, may be managed in a single table. The machine learning model may be composed of a neural network such as a Deep Neural Network (DNN) or an LSTM (Long Short Term Memory). The "Model Data" includes various information such as connection information, which describes how the nodes in each of the multiple layers constituting the neural network are connected to each other, and connection coefficients, which are multiplied by the numerical values ​​input and output between connected nodes.

[0031] "Model Generation Date and Time" indicates the date and time when the machine learning model identified by "Model ID" was generated. "Training Data Directory" is the directory where the training data used to train the machine learning model identified by "Model ID" is stored; multiple training data sets are stored in this directory. "Accuracy" indicates the prediction accuracy of the machine learning model identified by "Model ID".

[0032] In other words, Figure 4 shows an example in which the model data "MDDT#1" of a machine learning model identified by the model ID "MDID#1", the model creation date and time "TM#1" of the machine learning model, the training data directory "DR#1" used to train the machine learning model, and the accuracy "ACR#1" of the machine learning model are stored.

[0033] Furthermore, the information stored in the model storage unit 122 is not limited to information relating to the items "model ID," "model data," "model generation date and time," "training data directory," and "accuracy," but may also store any other information related to the machine learning model.

[0034] Next, returning to Figure 2, the control unit 130 will be described. The control unit 130 is a controller that manages and controls the arithmetic unit 100. The control unit 130 is realized by the execution of various programs stored in the memory unit 120 using RAM as the working area by a CPU (Central Processing Unit) or MPU (Micro Processing Unit), etc. Alternatively, the control unit 130 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0035] As shown in Figure 2, the control unit 130 comprises an acquisition unit 131, a preprocessing unit 132, a conversion unit 133, a learning unit 134, a determination unit 135, and an output unit 136. The control unit 130 realizes these functions and performs these processes by reading and executing a program (software) from the storage unit 120. Note that these functions of the control unit 130 may be realized by electronic circuits. Furthermore, the control unit 130 may execute these processes with a single CPU, or it may have multiple CPUs and execute these processes in parallel with the multiple CPUs. These configurations will be described in detail below.

[0036] (Measurement data for each vehicle model) Figure 5 is a diagram illustrating the relationship between the models according to this disclosure. In this embodiment, measurement data from a vehicle traveling on a road with unknown road surface conditions is input into a pre-trained model that has learned the relationship between measurement data and road surface conditions to determine the road surface conditions of the road traveled by the vehicle. However, if the vehicle type from which the measurement data was detected is different, the vehicle's behavior will differ even if it travels on the same road surface, and the relationship between measurement data and road surface conditions may differ for each vehicle type. Therefore, it is conceivable to prepare a pre-trained model for each vehicle type and use the pre-trained model of the vehicle from which the measurement data was acquired to determine the road surface conditions. However, in this case, it is necessary to prepare a pre-trained model for each vehicle type, which increases the workload for estimating the road surface conditions for each vehicle type. In contrast, in this embodiment, as shown in Figure 5, a first pre-trained model A1 is prepared by learning the relationship between the measurement data of vehicle C2 and the road surface conditions from the measurement data of vehicle C2, a second vehicle type. Then, if measurement data is detected using a vehicle C1 of the first vehicle type, which is different from the second vehicle type, the measurement data of the first vehicle type is converted to data equivalent to the measurement data of the second vehicle type, and this converted measurement data (converted measurement data) is input to the first trained model A1 to determine the road surface condition. This eliminates the need to prepare a trained model for each vehicle type, and allows for highly accurate estimation of the road surface condition for each vehicle type without increasing the workload. The specific processing of this embodiment will be described below.

[0037] (Acquisition of measurement data for the first vehicle type) The acquisition unit 131 acquires measurement data related to vehicle C detected by the measuring device 200. Specifically, the acquisition unit 131 acquires measurement data by sending a request for measurement data information to an external device and receiving measurement data from the external device that receives the request. The external device may be, for example, the measuring device 200 equipped with the acceleration sensor unit 240 which will be described later, or it may be a storage medium in which the measurement data of the measuring device 200 is stored. Once the acquisition unit 131 has acquired the measurement data from the external device, it stores the acquired measurement data in the measurement data storage unit 121. Since the measuring device 200 detects measurement data at a predetermined sampling rate, if the period during which the measuring device 200 performs detection is defined as the detection period, then the acquisition unit 131 can be said to acquire measurement data at each timing detected within the detection period.

[0038] In this embodiment, in order to determine the road surface condition from the measurement data of the first vehicle type C1, the acquisition unit 131 acquires measurement data indicating the behavior of vehicle C1, which is detected by the measurement device 200A mounted on vehicle C1.

[0039] (Pre-processing) The preprocessing unit 132 performs predetermined preprocessing on the measurement data. As predetermined preprocessing, the preprocessing unit 132 extracts feature quantities from the measurement data. A specific example of preprocessing is described below.

[0040] The preprocessor 132 divides the detection period into multiple predetermined periods and extracts measurement data from the measurement data detected at each timing within the predetermined period. The preprocessor 132 performs this extraction process for each predetermined period, extracting measurement data detected at each timing within that predetermined period. The predetermined period here can be set arbitrarily and may be, for example, about 3 seconds. Furthermore, adjacent predetermined periods in the time series do not have to overlap in time zones, but they may overlap in some time zones. That is, for example, one predetermined period (for example, 3 seconds) and the next predetermined period (for example, 3 seconds) may overlap in some time zones, and the overlapping time zone may be, for example, about 2 seconds. In other words, in this example, the preprocessor 132 performs a process in the measurement data to divide the time-series measurement data into 3-second intervals every time 1 second has passed.

[0041] The preprocessing unit 132 calculates feature quantities of the measurement data within a predetermined period based on the measurement data within that period. In this embodiment, the preprocessing unit 132 performs a Fourier transform on the measurement data arranged in time series within the predetermined period, converting the time series measurement data into measurement data for each frequency. The preprocessing unit 132 calculates feature quantities of the measurement data based on the measurement data for each frequency. The preprocessing unit 132 divides the measurement data for each frequency into predetermined frequency bands, calculates the POA (Partial Overall) value of the intensity of the measurement data for each frequency band, and uses the POA value as a feature quantity of the measurement data. Since the POA value is calculated for each frequency band, the preprocessing unit 132 calculates the feature quantities (POA values) of the measurement data for each frequency band from the measurement data within the predetermined period. However, the method for calculating the feature quantities of the measurement data is not limited to this and may be any value, and values ​​other than the POA value (for example, the average value of the intensity of the measurement data for each frequency band) may be calculated as feature quantities.

[0042] The preprocessor 132 performs a process to calculate the feature quantities of the measurement data within a predetermined period for each set of measurement data for that predetermined period. Furthermore, if there are multiple types of measurement data, the preprocessor 132 calculates the feature quantities for each set of measurement data. For example, if acceleration and suspension displacement are obtained as measurement data, the preprocessor 132 calculates the feature quantities for acceleration and the feature quantities for both acceleration and suspension displacement.

[0043] (Preparing the first pre-trained model) The learning unit 134 prepares a first trained model A1 that has learned the relationship between the measurement data of the second vehicle type C2 and the road surface condition. The timing of preparing the first trained model A1 is arbitrary, and it may be prepared in advance before acquiring the measurement data of the first vehicle type C1 used to determine the road surface condition. Preferably, the first trained model A1 is a learning model trained by deep learning.

[0044] When preparing the first trained model A1, the second vehicle C2, equipped with the measuring device 200B, is driven on a road surface with known road conditions, allowing the measuring device 200B to detect measurement data. The acquisition unit 30 acquires the measurement data related to vehicle C2 detected by the measuring device 200B, and information on the road surface conditions of the road surface on which vehicle C2 drove (the road surface on which it drove when the measurement data was detected). Then, the preprocessing unit 132 performs preprocessing on the measurement data related to vehicle C2 in the same manner as described above.

[0045] The learning unit 134 uses the pre-processed measurement data of vehicle C2 and the road surface information of the road surface on which vehicle C2 traveled as training data to train the learning model and obtain the first trained model A1. Specifically, the learning unit 134 sets a dataset as training data, in which the pre-processed measurement data (i.e., the features of the measurement data) is the input value (explanatory variable) and the road surface condition at the location where the measurement data was detected is the output value (target variable), and inputs this training data into the learning model. In this case, it is preferable for the learning unit 134 to prepare multiple datasets consisting of the pre-processed measurement data and the road surface condition, and to input each of the multiple datasets into the learning model. As a result, the learning model becomes the first trained model A1, which has already learned the relationship between the measurement data of vehicle C2 and the road surface condition at the location where the measurement data was detected. In other words, the first trained model A1 becomes a model (program) that can calculate the road surface condition at the location where the measurement data was detected when the measurement data of vehicle C2 is input.

[0046] Furthermore, the learning unit 134 may train the learning model with the target variable as a category of road surface conditions. In this case, for example, the learning unit 134 may use information that includes both the type of road surface anomaly and the severity of the road surface condition anomaly, expressed numerically in multiple stages (for example, ten stages where the numerical value increases as the severity of the anomaly increases). Then, the model is trained using training data consisting of a pair of the target variable and explanatory variables. The learning model referred to here may be, for example, an LSTM. An LSTM is a type of RNN (Recurrent Neural Network), which is a recurrent neural network capable of handling time series data. Specifically, by comprising an input gate, a forgetting gate, and an output gate, it recognizes characteristic patterns in time series data and considers older information in the time series data when predicting the future that follows the end of the time series data.

[0047] Road surface abnormalities refer to unusual or abnormal conditions on the road surface, such as potholes (localized depressions, dents, or holes in the road surface), cracks (linear or tortoise-shell-like cracks in the road surface), and rutting (deep depressions only where tires pass). When a vehicle travels over these conditions, characteristic waveforms are generated in the acceleration data, and the relationship between these waveforms and the road surface conditions is learned.

[0048] The first pre-trained model A1, prepared as described above, is used for the converted measurement data obtained by transforming the measurement data of the first vehicle type C1 into measurement data equivalent to the measurement data of the second vehicle type C2, as will be explained later.

[0049] (Conversion of measurement data for the first vehicle type) The conversion unit 133 converts the pre-processed measurement data of the first vehicle type C1 (feature quantities of the measurement data of vehicle C1) into data equivalent to the measurement data of the second vehicle type C2. Hereafter, the data obtained by converting the pre-processed measurement data of vehicle C1 into data equivalent to the measurement data of vehicle C2 will be referred to as the converted measurement data. The converted measurement data can be said to be data equivalent to the pre-processed measurement data of vehicle C1, assuming that vehicle C2 was traveling on the same road surface that vehicle C1 was traveling on when the measurement data of vehicle C1 was detected. The conversion unit 133 converts each of the pre-processed measurement data of vehicle C1 into the converted measurement data.

[0050] Here, the processing of the conversion unit 133 will be explained using Figure 6. Figure 6 is a diagram illustrating the conversion of measurement data according to this disclosure. The conversion unit 133 may convert the pre-processed measurement data of vehicle C1 into converted measurement data by any method, but in this embodiment, as shown in Figure 6, the pre-processed measurement data of vehicle C1 is input to the second trained model A2, and the second trained model A2 converts the pre-processed measurement data of vehicle C1 into converted measurement data.

[0051] The second pre-trained model A2 is a model that has learned the relationship between the pre-processed measurement data of vehicle C1 (features of the measurement data of vehicle C1) and the pre-processed measurement data of vehicle C2 (features of the measurement data of vehicle C2). When the pre-processed measurement data of vehicle C1 is input, it outputs converted measurement data corresponding to the pre-processed measurement data of vehicle C2. The second pre-trained model A2 can be called an autoencoder that converts the pre-processed measurement data of vehicle C1 to converted measurement data. In other words, in this embodiment, the conversion unit 133 converts the measurement data of the first vehicle type to converted measurement data using an autoencoder that has learned the relationship between the pre-processed measurement data of vehicle C1 and the pre-processed measurement data of vehicle C2. An autoencoder is a type of neural network designed to efficiently compress (encode) input data to its essential features and reconstruct (decode) the original input from this compressed representation. In other words, the autoencoder described above encodes the features of the measurement data of the first vehicle type and performs the process of decoding it into the measurement data of the second vehicle type based on the compressed information. Furthermore, the autoencoder used in the processing of the conversion unit 133 may be one that has already learned the relationship between the measurement data of the first vehicle and the measurement data of the second vehicle.

[0052] (Preparing the second pre-trained model) The following describes an example of how to prepare the second pre-trained model A2. Note that the timing of preparing the second pre-trained model A2 is arbitrary; it may be prepared in advance before acquiring measurement data from the first vehicle type C1 used to determine the road surface condition.

[0053] When preparing the second pre-trained model A2, the vehicle C2 equipped with the measuring device 200B is driven on a road surface with known road conditions, allowing the measuring device 200B to detect measurement data. The acquisition unit 30 acquires the measurement data related to vehicle C2 detected by the measuring device 200B, and information on the road surface conditions of the road surface on which vehicle C2 drove (the road surface on which the vehicle drove when the measurement data was detected). Similarly, the vehicle C1 equipped with the measuring device 200A is driven on a road surface with known road conditions, allowing the measuring device 200A to detect measurement data. The acquisition unit 30 acquires the measurement data related to vehicle C1 detected by the measuring device 200A, and information on the road surface conditions of the road surface on which vehicle C1 drove (the road surface on which the vehicle drove when the measurement data was detected). The preprocessing unit 132 performs preprocessing on the measurement data of vehicle C1 and the measurement data of vehicle C2 in the same manner as described above, to obtain the preprocessed measurement data of vehicle C1 and the preprocessed measurement data of vehicle C2.

[0054] In this embodiment, when preparing the second trained model A2, it is preferable to have vehicles C1 and C2 travel on the same road surface and acquire measurement data while they are traveling on the same surface. It is also preferable to have vehicles C1 and C2 travel at the same speed. This allows the second trained model A2 to learn the correspondence between the measurement data of vehicle C1 and the measurement data of vehicle C2 with high accuracy.

[0055] The learning unit 134 uses the pre-processed measurement data of vehicle C1 and the pre-processed measurement data of vehicle C2 as training data to train the learning model and obtain a second trained model A2. Specifically, the learning unit 134 prepares multiple datasets in which the pre-processed measurement data of vehicle C1 is used as input values ​​(explanatory variables) and the pre-processed measurement data of vehicle C2 is used as output values ​​(target variables), uses these datasets as training data, and inputs this training data into the learning model. As a result, the learning model becomes a second trained model A2 that has already learned the relationship between the pre-processed measurement data of vehicle C1 and the pre-processed measurement data of vehicle C2. In other words, the second trained model A2 becomes a model (program) that can calculate the converted measurement data (data corresponding to the pre-processed measurement data of vehicle C2) when the pre-processed measurement data of vehicle C1 is input.

[0056] The second trained model A2 may be a neural network model having an input layer x, an intermediate layer y, and an output layer x', as shown in Figure 6. The second trained model A2 may be the same type of model as the first trained model A1.

[0057] As described above, in this embodiment, when preparing the second trained model A2, vehicles C1 and C2 are driven on the same road surface, but this is not limited to this, and vehicles C1 and C2 may be driven on different road surfaces with known road surface conditions. Similarly, vehicles C1 and C2 may be driven at different speeds. In this case, for example, the road surface conditions on which vehicles C1 and C2 drove, and the speeds of vehicles C1 and C2 can also be incorporated into the training data and trained in the model, thereby allowing the second trained model A2 to learn the relationship between the pre-processed measurement data of vehicles C1 and C2.

[0058] (Determining road surface conditions) The determination unit 135 inputs the converted measurement data into the first trained model A1 to determine the road surface condition on which the first vehicle type C1 is traveling. Since the first trained model A1 has learned the relationship between the measurement data of the second vehicle type and the road surface condition, it can determine the road surface condition on which the first vehicle type C1 is traveling by inputting the converted measurement data, which is the first vehicle type's measurement data equivalent to the second vehicle type's measurement data, into the first trained model A1. In other words, the determination unit 135 determines the road surface condition output by the first trained model A1 as the road surface condition on which vehicle C1 is traveling. The conversion unit 133 inputs each of the converted measurement data into the first trained model A1 to determine the road surface condition for each position traveled by vehicle C1.

[0059] The judgment unit 135 may also output the type of road surface abnormality, such as potholes, cracks, or rutting, and a numerical value indicating the severity of that abnormality. This allows for an appropriate determination of the type and severity of road surface abnormalities encountered by a vehicle without the need for visual inspection.

[0060] (Output of road surface conditions) The output unit 136 outputs various types of information. Specifically, the output unit 136 outputs the road surface condition determination result determined by the determination unit 135. The output unit 136 may transmit the road surface condition determination result to an external device or display it on the display unit 150. For example, the output unit 136 may associate the position of the vehicle, represented by a vehicle icon on a two-dimensional map, with the road surface condition category based on the position information of vehicle C included in the measurement data of the vehicle during driving input to the trained model, and display it on the display unit 150. This makes it easy and accurate to identify the location of the road surface where an abnormality is occurring.

[0061] (Configuration of other elements of the computing unit) The input unit 140 receives various operation information from the user of the arithmetic unit 100. The input unit 140 may be implemented by an input device such as a keyboard, mouse, or touch panel. The user inputs various operation information and operation information for displaying a GUI (Graphical User Interface) that shows various information via the input unit 140.

[0062] The display unit 150 is a display device that displays various types of information. For example, the display unit 150 displays the road surface condition determination result according to the instructions of the output unit 136 described above. The display unit 160 may be implemented by, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, a micro LED (Light Emitting Diode) display, etc.

[0063] As described above, the computing device 100 eliminates the need to prepare a separate road surface determination model for each vehicle type, and allows for the appropriate estimation of road surface conditions for each vehicle type. Therefore, it is possible to provide a computing device 100 that can estimate road surface conditions with high accuracy for each type of vehicle.

[0064] (Regarding calculation methods and programs) Next, the calculation method related to this disclosure will be explained using Figure 7. Figure 7 is a flowchart of the calculation method related to this disclosure. The calculation method related to this disclosure will be explained in accordance with the flow shown in Figure 7.

[0065] First, the arithmetic unit 100 acquires measurement data from the first vehicle type C1 (step S101). Next, the arithmetic unit 100 performs preprocessing on the measurement data from the first vehicle type C1 (step S102). Next, the arithmetic unit 100 inputs the preprocessed measurement data from the first vehicle type C1 into the second trained model A2 and converts it into converted measurement data (step S103). Next, the arithmetic unit 100 inputs the converted measurement data into the first trained model A1 (step S104) and obtains a determination result of the road surface condition (step S105). Next, the arithmetic unit 100 outputs the determination result (step S106).

[0066] This eliminates the need to prepare a separate road surface judgment model for each vehicle type, allowing for accurate estimation of road surface conditions for each vehicle type. Therefore, it is possible to provide a calculation method that can estimate road surface conditions with high accuracy for each type of vehicle.

[0067] In this embodiment, the measurement data of the first vehicle type C1 is converted and input into the first trained model A1 of the second vehicle type C2. However, this process in this embodiment is not limited to being applied to different vehicle types, but may be applied to, for example, different vehicles of the same vehicle type. In this case, although the second vehicle type C2 used to train the first trained model A1 is a different vehicle, the measurement data of the same second vehicle type C2 is treated as the measurement data of the first vehicle type C1 as described above, and the road surface condition is estimated by performing the same processing as described above. In this case, the second trained model A2 only needs to be trained to recognize the relationships between the measurement data of different vehicle C2s.

[0068] (Configuration of the measuring device) Next, the configuration of the measuring device 200 according to this disclosure will be described with reference to Figure 8. Figure 8 is a diagram showing an example of the configuration of the measuring device according to this disclosure. The measuring device 200 according to this disclosure comprises a communication unit 210, a storage unit 220, a control unit 230, and a sensor unit for measuring measurement data. In this embodiment, the sensor unit is provided with an acceleration sensor unit 240, a displacement sensor unit 250, an IMU sensor unit 260, a camera unit 270, and a position information sensor unit 280. These configurations will be described in order below.

[0069] The communication unit 210 is responsible for sending and receiving information with external devices. The communication unit 110 may be implemented by, for example, a CAN communication interface device, a wireless LAN card, a serial communication interface device, a Bluetooth® module, a Wi-Fi® module, an antenna, etc.

[0070] The memory unit 220 is a storage device that stores various types of information. The memory unit 220 comprises a main memory and an auxiliary storage device. The main memory may be implemented using semiconductor memory elements such as RAM, ROM, or flash memory. The auxiliary storage device may be implemented using a hard disk or SSD, for example. The location where the memory unit 220 is installed is not limited, but it may be installed inside the vehicle C, for example.

[0071] As shown in Figure 8, the storage unit 220 includes a measurement data storage unit 221. Note that the items of information stored in the measurement data storage unit 211 of the measuring device 200 are the same as the items of information stored in the measurement data storage unit 121 of the arithmetic unit 100, so their explanation is omitted.

[0072] The control unit 230 is a controller that manages and controls the measuring device 200. The control unit 230 is implemented by a CPU, MPU, etc., which executes various programs stored in the memory unit 220 using RAM as the working area. Alternatively, the control unit 230 may be implemented by an integrated circuit such as an ASIC or FPGA. The location where the control unit 230 is installed is not limited, but for example, it may be installed inside the vehicle C.

[0073] As shown in Figure 8, the control unit 230 includes an acquisition unit 231, a reception unit 232, and a supply unit 233. The control unit 230 realizes these functions and performs these processes by reading and executing a program (software) from the storage unit 220. These functions of the control unit 230 may also be realized by electronic circuits. Furthermore, the control unit 230 may perform these processes with a single CPU, or it may have multiple CPUs and perform these processes in parallel with the multiple CPUs. These configurations will be described in detail below.

[0074] The acquisition unit 231 acquires various types of measurement data. For example, the acquisition unit 231 acquires three-axis acceleration data measured by the acceleration sensor unit 240, which will be described later. Once the acquisition unit 231 has acquired the three-axis acceleration data, it stores the acquired three-axis acceleration data in the measurement data storage unit 221. In addition to the three-axis acceleration data, the acquisition unit 231 may also acquire position information measured by the position information sensor unit 280 and store these together in the measurement data storage unit 221.

[0075] The reception unit 232 receives various information requests from external devices. For example, the reception unit 232 receives a request for measurement data information from the computing unit 100 via the communication unit 210. The information request may include information that identifies the measurement data to be provided, such as the measurement data ID and the measurement date.

[0076] The providing unit 233 provides various types of information to an external device based on an information provision request from the external device received by the receiving unit 232. For example, if the receiving unit 232 receives an information provision request for measurement data from the computing device 100, the providing unit 233 reads the three-axis acceleration data and position information received in the information provision request from the measurement data storage unit 221 and provides the measurement data to the computing device 100 via the communication unit 210.

[0077] The acceleration sensor unit 240 is a sensor that detects the acceleration of vehicle C. The acceleration sensor unit 240 detects acceleration in three directions along three mutually orthogonal detection axes. The three mutually orthogonal detection directions may be named, for example, the X axis, Y axis, and Z axis. The acceleration sensor unit 240 may be a capacitive acceleration sensor that, for example, uses MEMS (Micro Electro Mechanical Systems) to create a movable electrode and a fixed electrode, and measures acceleration using the relationship between the change in capacitance between electrodes caused by the movement of the movable electrode due to acceleration and the acceleration.

[0078] Furthermore, the acceleration sensor unit 240 may be a piezoresistive acceleration sensor that detects the displacement of a weight supported by a spring that fluctuates with acceleration using a piezoresistive element placed on the spring. Alternatively, the acceleration sensor unit 240 may be a thermal-sensing acceleration sensor that detects the airflow of a gas heated inside the housing, which changes with acceleration, by measuring the change in the temperature-measuring resistance value. Alternatively, the acceleration sensor unit 240 may be a piezoelectric acceleration sensor that measures acceleration from the amount of charge generated in a piezoelectric element in proportion to the applied acceleration.

[0079] The acceleration sensor unit 240 may be attached to the unsprung mass (lower arm of each wheel) and sprung mass (near the suspension mounting part) of the suspension provided on the front left wheel, front right wheel, rear left wheel, and rear right wheel of the vehicle C, and the acceleration of these three axes may be measured in a time series.

[0080] The displacement sensor unit 250 is a sensor that detects the amount of suspension displacement of vehicle C. The displacement sensor unit 250 may, for example, connect the upper and lower parts of the suspension with a link and measure the amount of displacement of the link. The displacement sensor unit 250 may measure, for example, the displacement of the shock absorber of the suspension provided on vehicle C as the amount of suspension displacement. The displacement sensor unit 250 may be, for example, a laser displacement meter or a time-measuring laser sensor. For example, a laser displacement meter may be a triangulation type that projects laser light emitted from a light-emitting element (e.g., a semiconductor laser) onto an object to be measured, receives the reflected light reflected from the object to be measured with a light-receiving element (e.g., a linear image sensor, a position-sensitive device, etc.), and detects the displacement of the object to be measured by measuring the displacement of the reflected light caused by the displacement of the object to be measured.

[0081] A time-measuring laser sensor measures the distance to an object by measuring the time it takes for the laser light emitted by the light-emitting element to strike the object and return to the light-receiving element (photodiode). This distance is then converted into the displacement of the object.

[0082] The displacement sensor unit 250 may also be a stroke sensor. The stroke sensor measures the amount of displacement by converting the change in magnetic flux density detected by the Hall IC into an electrical signal according to the stroke position of the movable part. When a stroke sensor is attached to the suspension, it measures the amount of displacement of the rod.

[0083] The IMU sensor unit 260 is a sensor that detects the orientation of vehicle C. The IMU sensor unit 260 may be located in the center of the vehicle body of vehicle C. The IMU sensor unit 260 may be, for example, a 6DoF (Degree of Freedom) IMU (Inertial Measurement Unit) sensor that combines a three-axis gyro sensor and a three-axis accelerometer. The three-axis gyro sensor may be implemented using a capacitive MEMS gyro sensor that detects a change in capacitance, where a primary vibration is generated in a movable electrode that vibrates in one direction, and when rotation is applied to the movable electrode, a Coriolis force acts in a direction 90° from the direction of vibration, causing a secondary vibration and a change in capacitance.

[0084] The camera unit 270 is a camera (sensor) that captures image data of the area around vehicle C. The camera unit 270 may be mounted on the dashboard of vehicle C. The camera unit 270 is, for example, a camera that captures image data of the road surface outside the vehicle. The camera includes optical elements and an image sensor. Optical elements are elements that constitute an optical system, such as lenses, mirrors, prisms, and filters. The image sensor is an element that converts light incident through the optical elements into an image signal, which is an electrical signal. The image sensor may be, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

[0085] The position information sensor unit 280 is a sensor that detects the position of vehicle C (location information of vehicle C). The position information sensor unit 280 may be, for example, a GPS (Global Positioning System) sensor. A GPS sensor is equipped with a GPS receiver and receives radio waves transmitted from GPS satellites. The GPS sensor receives radio waves transmitted from multiple GPS satellites and measures the current position (for example, latitude and longitude) by calculating the distance from multiple GPS satellites using the difference between the time the radio waves were received and the time the GPS satellites transmitted the radio waves. Alternatively, the position information sensor unit 280 may be a module for GNSS (Global Navigation Satellite System). The position information sensor unit 280 may be installed on the dashboard of vehicle C or on the roof of vehicle C.

[0086] Here, an example of measurement data measured by the measuring device 200 will be explained using Figure 9. Figure 9 is a diagram showing an example of measurement data from the measuring device according to this disclosure. As shown in Figure 9, the waveform data (measurement data) of the first vehicle type C1 measured by the measuring device 200 can be input to the second trained model A2 to convert it into virtual waveform data (measurement data) of the second vehicle type C2.

[0087] (Structure and effect) The computing device 100 according to this disclosure includes: an acquisition unit 131 that acquires behavior information indicating the behavior of a first vehicle while it is in motion; a conversion unit 133 that performs a process to convert the behavior information of the first vehicle into data corresponding to the behavior information of the second vehicle using a second trained model that has learned the relationship between the behavior information of the first vehicle and the behavior information of a second vehicle, which is another specific vehicle; a determination unit that determines the road surface conditions on which the first vehicle is traveling by inputting the behavior information (converted measurement data) after the conversion of the behavior information of the first vehicle into data corresponding to the behavior information of the second vehicle into a first trained model A1 that has been learned using measurement data of the second vehicle; and an output unit that outputs the determination result of the road surface conditions.

[0088] This configuration eliminates the need to prepare a separate road surface determination model for each vehicle type, and allows for the appropriate estimation of road surface conditions for each vehicle type. Therefore, it is possible to provide a computing device 100 that can estimate road surface conditions with high accuracy for each type of vehicle.

[0089] The program relating to this disclosure causes a computer to perform the following steps: acquire behavioral information indicating the behavior of a first vehicle while it is in motion; convert the behavioral information of the first vehicle into data corresponding to the behavioral information of a second vehicle, which is a specific vehicle, using a second trained model that has learned the relationship between the behavioral information of the first vehicle and the behavioral information of a second vehicle; input the converted behavioral information (converted measurement data) into a first trained model A1 that has been learned using measurement data of the second vehicle, thereby determining the road surface conditions on which the first vehicle is traveling; and output the road surface condition determination result to a display unit.

[0090] This configuration eliminates the need to prepare a separate road surface judgment model for each vehicle type, allowing for accurate estimation of road surface conditions for each vehicle type. Therefore, it is possible to provide a program that can accurately estimate road surface conditions for each vehicle type.

[0091] Although embodiments of the present disclosure have been described above, the embodiments are not limited to those described herein. Furthermore, the aforementioned components include those that can be easily conceived by those skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the embodiments described above. [Explanation of Symbols]

[0092] 1. Computational System 100 Computing equipment 110 Communications Department 120 Storage section 121 Measurement data storage unit 122 Model Memory Unit 130 Control Unit 131 Acquisition Department 132 Pre-processing section 133 Conversion section 134 Learning Department 135 Judgment section 136 Output section 140 Input section 150 Display section 200 measuring devices 210 Communications Department 220 Storage section 230 Control Unit 231 Acquisition Department 232 Reception Department 233 Provision Department 240 Acceleration sensor section 250 Displacement sensor section 260 IMU Sensor Section 270 Camera Section 280 Location Information Sensor Unit N Network

Claims

1. An acquisition unit that acquires behavioral information indicating the behavior of the first vehicle type while it is in motion, A conversion unit performs a process to convert the behavioral information of the first vehicle into data corresponding to the behavioral information of the second vehicle, using a second trained model that has learned the relationship between the behavioral information of the first vehicle and the behavioral information of a second vehicle, which is another specific vehicle. A determination unit determines the road surface conditions on which the first vehicle is traveling by inputting the behavioral information of the first vehicle, converted into data equivalent to the behavioral information of the second vehicle, into a first trained model that has been trained using the behavioral information of the second vehicle. The system includes an output unit that outputs the result of determining the road surface condition, Computing device.

2. A step of acquiring behavioral information that shows the behavior of the first vehicle type while it is in motion, The process involves converting the behavioral information of the first vehicle into data corresponding to the behavioral information of the second vehicle, using a second trained model that has learned the relationship between the behavioral information of the first vehicle and the behavioral information of a second vehicle, which is another specific vehicle. The steps include determining the road surface conditions on which the first vehicle is traveling by inputting the behavioral information of the first vehicle, converted into data equivalent to the behavioral information of the second vehicle, into a first trained model that has been trained using the behavioral information of the second vehicle, and The steps include: outputting the result of determining the road surface condition, A program that causes a computer to execute something.

Images