Calculation unit and program

Abstract

To provide a computing device and program capable of appropriately determining road surface conditions. [Solution] The computing device 100 according to this disclosure includes: an acquisition unit 131 that acquires behavior information and position information indicating the behavior of a vehicle traveling on a target road surface; a conversion unit 133 that generates converted behavior information indicating the behavior of the vehicle for each frequency based on the behavior information; a determination unit 135 that determines whether there are signs of an abnormality on the target road surface by inputting the converted behavior information into a trained model that has been machine-learned to determine the correspondence between reference behavior information indicating the behavior of a vehicle traveling on a road surface for which position information has already been acquired for each frequency and a label indicating whether there are signs of an abnormality on the road surface; and an output unit 136 that outputs the determination result for the target road surface.

Description

Technical Field

[0001] The present disclosure relates to an arithmetic unit and a program.

Background Art

[0002] In an anomaly detection method using machine learning, generally, data in an abnormal state is rarely obtained, and a slight anomaly is mixed in a countless number of normal data. Therefore, it has been difficult to grasp the actual situation of the anomaly as data.

[0003] For example, in Patent Document 1 below, based on the behavior information of the front and rear wheels, estimating the presence of road surface damage, and using a learned model learned by machine learning based on information related to the behavior of a vehicle for learning and the correct label of road surface damage to estimate the state is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the invention described in Patent Document 1 above, although the presence of road surface damage can be estimated, it has been impossible to detect a sign of the occurrence of an abnormality in the road surface state.

[0006] In view of the above problems, an object of the present disclosure is to provide an arithmetic unit and a program capable of appropriately determining the road surface state.

Means for Solving the Problems

[0007] The computing device according to this disclosure includes: an acquisition unit that acquires behavior information indicating the behavior of a vehicle traveling on a target road surface; a conversion unit that generates converted behavior information indicating the behavior of the vehicle for each frequency based on the behavior information; a determination unit that determines whether there are signs of an abnormality on the target road surface by inputting the converted behavior information into a trained model that has been machine-learned to determine the correspondence between reference behavior information indicating the behavior of a vehicle traveling on a road surface for each frequency and a label indicating whether there are signs of an abnormality on the road surface; and an output unit that outputs the determination result for the target road surface.

[0008] The program relating to this disclosure causes a computer to perform the following steps: acquire behavior information indicating the behavior of a vehicle traveling on a target road surface; generate converted behavior information indicating the behavior of the vehicle for each frequency based on the behavior information; input the converted behavior information into a trained model that has been machine-learned to determine whether there are signs of an abnormality on the target road surface; and output the determination result for the target road surface. [Effects of the Invention]

[0009] This disclosure provides a computing device and a program capable of appropriately determining road surface conditions. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a diagram illustrating the overview of the computing system related to this disclosure. [Figure 2] Figure 2 shows an example of the configuration of the computing device according to this disclosure. [Figure 3] Figure 3 shows an example of information stored in the behavior information storage unit of the computing device according to this disclosure. [Figure 4] Figure 4 shows an example of information stored in the model storage unit of the computing device according to this disclosure. [Figure 5] Figure 5 is a diagram illustrating the processing of the preprocessing unit of the computing device according to this disclosure. [Figure 6] Figure 6 is a diagram illustrating the processing of the conversion unit of the computing device according to this disclosure. [Figure 7] Figure 7 is a diagram illustrating the processing of the conversion unit of the computing device according to this disclosure. [Figure 8] Figure 8 is a diagram illustrating the processing of the conversion unit of the computing device according to this disclosure. [Figure 9] Figure 9 is a diagram illustrating the processing of the learning unit of the computing device relating to this disclosure. [Figure 10] Figure 10 shows an example of training data. [Figure 11] Figure 11 is a schematic diagram showing an example of input data to be fed into a trained model. [Figure 12] Figure 12 is a flowchart showing the flow of the calculation method relating to this disclosure. [Figure 13] Figure 13 is a flowchart showing the flow of the learning method related to this disclosure. [Figure 14] Figure 14 shows an example of the configuration of the measuring device according to this disclosure. [Modes for carrying out the invention]

[0011] Embodiments of this disclosure will be described in detail below with reference to the drawings. However, the embodiments described below will not limit this disclosure.

[0012] (Overview of the computing system) First, the overview of the arithmetic system according to the present disclosure will be described with reference to FIG. 1. FIG. 1 is a diagram for explaining the overview of the arithmetic system according to the present disclosure. As shown in FIG. 1, the arithmetic system 1 according to the present disclosure acquires, for example, behavior information, which is a measured value of the behavior of the vehicle C, from a measuring device 200 mounted on the vehicle C, and performs various calculations by an arithmetic device 100 based on the behavior information. Specifically, the arithmetic device 100 determines whether there is a sign of abnormality in the road surface state of a target road surface, which is the road surface on which the vehicle C travels, based on the behavior information. Here, the road surface state may refer to, for example, the degree of unevenness of the road surface. Also, that the road surface state is abnormal may refer to, for example, that the degree of unevenness is larger than in the normal case, and may refer to a depression, a dent, a pothole meaning a hole, which locally occurs on the paved surface of the road, a crack in the form of a line or a tortoise shell on the paved surface of the road, a crack, or a rut where only the part where the tire passes is depressed, which refers to an abnormal state different from the normal state on the road surface. Further, that there is a sign of abnormality refers to a state where it has not reached an abnormal state but is estimated to reach an abnormal state if a little more time elapses.

[0013] Note that the arithmetic device 100 is not limited to being provided outside the vehicle C, and may be provided inside the vehicle C. Also, the arithmetic device 100 may acquire measurement data measured by a measuring device 200 provided for each of a plurality of vehicles. Further, the arithmetic device 100 may be an integrated device that has the function of the measuring device 200 and also has the function of the arithmetic device 100.

[0014] Thereby, it becomes possible to improve the efficiency of visual inspection in the maintenance obligation of road structures that are defined by the Road Law and that may cause a significant obstacle to the road structure or traffic when an abnormality occurs during the progress of aging. Also, by mounting the arithmetic device 100 on the vehicle C and appropriately determining the road surface state using the measurement data during the travel of the vehicle C, it is possible to travel the vehicle C while predicting an abnormality in the road surface state ahead of the vehicle C.

[0015] (Configuration of the computing system) Next, the configuration of the computing system according to the present disclosure will be described with reference to FIG. 1. As shown in FIG. 1, the computing system 1 according to the present disclosure includes a computing device 100, a measurement device 200, and a network N. These configurations will be briefly described below in order.

[0016] The computing device 100 is an information processing device that executes various computing processes. The computing device 100 may be realized by, for example, a PC (Personal Computer), a WS (Work Station), etc. Further, the computing device 100 may be an in-vehicle infotainment system (In-Vehicle Infotainment System) equipped with a car navigation system or the like, that is, an in-vehicle type information processing terminal. Note that the computing device 100 does not necessarily have to be mounted on a vehicle and may be installed at a location other than a vehicle.

[0017] The measurement device 200 is a device that is mounted on the vehicle C and detects behavior information indicating the behavior of the vehicle C while the vehicle C is traveling on the road. The measurement device 200 may detect any data indicating the behavior of the vehicle C as the behavior information, but in the present embodiment, the acceleration of the vehicle C is detected as the behavior information. The configuration for detecting the behavior information by the measurement device 200 will be described later. Note that one measurement device 200 may be mounted for each of a plurality of vehicles and connected to the network N. That is, the computing system 1 may include a plurality of measurement devices 200.

[0018] The network N connects the computing device 100 and the measurement device 200 to be communicable with each other by wire or wirelessly. When the network N is wired, it may be realized by Ethernet (registered trademark) defined in IEEE802.3, a USB (Universal Serial Bus) cable, or the like. When the network N is wireless, it may be realized by a wireless LAN (Local Area Network) defined in IEEE802.11 or Bluetooth (registered trademark).

[0019] As shown in Figure 1, the arithmetic unit 100 and the measuring device 200 are connected to each other via a network N, enabling them to communicate with one another. In other words, the arithmetic unit 100 and the measuring device 200 function as a single arithmetic system 1 by exchanging information with each other via the network N.

[0020] (Configuration of the computing unit) Next, the configuration of the computing device according to this disclosure will be explained using Figure 2. Figure 2 is a diagram showing an example of the configuration of the computing device according to this disclosure. As shown in Figure 2, the computing device 100 according to this disclosure comprises a communication unit 110, a storage unit 120, a control unit 130, an input unit 140, and a display unit 150. These configurations will be explained in order below.

[0021] The communication unit 110 is responsible for sending and receiving information with external devices. The communication unit 110 may be implemented by, for example, a CAN (Controller Area Network) communication interface device, a wireless LAN (Local Area Network) card, a serial communication interface device, a Bluetooth® module, a Wi-Fi® module, an antenna, etc.

[0022] The memory unit 120 is a storage device that stores various types of information. The memory unit 120 comprises a main memory and an auxiliary storage device. The main memory may be implemented using semiconductor memory elements such as RAM (Random Access Memory), ROM (Read Only Memory), or flash memory. The auxiliary storage device may be implemented using a hard disk, SSD (Solid State Drive), or optical disc, for example.

[0023] As shown in Figure 2, the memory unit 120 comprises a behavior information memory unit 121 and a model memory unit 122. An example of the information stored by these components will be described in detail below.

[0024] The behavior information storage unit 121 stores information related to behavior information. Here, an example of the information stored in the behavior information storage unit 121 will be explained using Figure 3. Figure 3 is a diagram showing an example of the information stored in the behavior information storage unit of the computing device according to this disclosure.

[0025] As shown in Figure 3, the behavior information storage unit 121 stores information related to the following items: "behavior information ID", "time", "unsprung X-direction acceleration", "unsprung Y-direction acceleration", "unsprung Z-direction acceleration", "sprung Z-direction acceleration", "stroke", and "position information".

[0026] "Behavior Information ID" is an identifier that identifies behavior information and is represented by a string or number. "Time" is information indicating the time when the behavior information was measured. "Unsprung X-direction acceleration" is information representing the measured value of the acceleration in the X direction from an acceleration sensor mounted vertically downward on the suspension spring. "Unsprung Y-direction acceleration" is information representing the measured value of the acceleration in the Y direction from an acceleration sensor mounted vertically downward on the suspension spring. "Unsprung Z-direction acceleration" is information representing the measured value of the acceleration in the Z direction from an acceleration sensor mounted vertically downward on the suspension spring. "Sprung Z-direction acceleration" is information representing the measured value of the acceleration in the Z direction from an acceleration sensor mounted vertically upward on the suspension spring. "Displacement" is information representing the displacement of the suspension shock absorber, and may be represented, for example, by the amount of displacement from the reference position of the shock absorber. "Position Information" is information representing the position (latitude and longitude) measured at the time indicated by "Time".

[0027] In other words, Figure 3 shows an example in which the behavior information identified by the behavior information ID "DTID#1" stores the following: unsprung X-direction acceleration "DXACL#1-1", unsprung Y-direction acceleration "DYACL#1-1", unsprung Z-direction acceleration "DZACL#1-1", sprung Z-direction acceleration "UZACL#1-1", displacement "DP#1-1", and position information "LC#1-1", all measured at the time indicated by time "TIME#1".

[0028] Furthermore, the information stored in the behavior information storage unit 121 is not limited to information relating to the items "behavior information ID," "time," "unsprung X-direction acceleration," "unsprung Y-direction acceleration," "unsprung Z-direction acceleration," "sprung Z-direction acceleration," "displacement," and "position information," but may also store other arbitrary behavior information.

[0029] The model storage unit 122 stores information related to the machine learning model. Here, an example of the information stored in the model storage unit 122 will be explained using Figure 4. Figure 4 is a diagram showing an example of the information stored in the model storage unit of the computing device according to this disclosure.

[0030] As shown in Figure 4, the model storage unit 122 stores information related to the items "Model ID" and "Model Data".

[0031] The "Model ID" is an identifier that identifies a machine learning model and is represented by a string or a number. The "Model Data" is the data of the machine learning model identified by the "Model ID". The machine learning model may consist of neural networks such as Deep Neural Networks (DNN), Long Short-Term Memory (LSTM), and Convolutional Neural Networks (CNN). The "Model Data" includes various information such as connection information, which describes how the nodes in each of the multiple layers that make up the neural network are connected to each other, and connection coefficients, which are multiplied by the numerical values ​​input and output between connected nodes.

[0032] In other words, Figure 4 shows an example where the model data "MDDT#1" of a model identified by the model ID "MDID#1" is stored.

[0033] Furthermore, the information stored in the model storage unit 122 is not limited to information related to the items "model ID" and "model data," but may also store any other information related to the machine learning model.

[0034] Next, returning to Figure 2, the control unit 130 will be described. The control unit 130 is a controller that manages and controls the arithmetic unit 100. The control unit 130 is realized by the execution of various programs stored in the memory unit 120 using RAM as the working area by a CPU (Central Processing Unit) or MPU (Micro Processing Unit), etc. Alternatively, the control unit 130 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0035] As shown in Figure 2, the control unit 130 includes an acquisition unit 131, a preprocessing unit 132, a conversion unit 133, a learning unit 134, a determination unit 135, and an output unit 136. The control unit 130 realizes these functions and performs these processes by reading and executing a program (software) from the storage unit 120. These functions of the control unit 130 may also be realized by electronic circuits. Furthermore, the control unit 130 may perform these processes with a single CPU, or it may have multiple CPUs and perform these processes in parallel with multiple CPUs. The processing details of the control unit 130 will be described later.

[0036] The input unit 140 receives various operation information from the user of the arithmetic unit 100. The input unit 140 may be implemented by an input device such as a keyboard, mouse, or touch panel. The user inputs various operation information and operation information for displaying a GUI (Graphical User Interface) that shows various information via the input unit 140.

[0037] The display unit 150 is a display device that displays various types of information. The display unit 150 displays, for example, the result of determining the road surface condition according to the instructions of the output unit 136. The display unit 150 may be implemented by, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, a micro LED (Light Emitting Diode) display, etc.

[0038] (Processing by the arithmetic unit) Next, we will explain the process by which the computing device 100 determines whether there are any signs of abnormality on the target road surface.

[0039] The acquisition unit 131 acquires behavior information detected by the measuring device 200 while the vehicle C is traveling on the target road surface. Specifically, the acquisition unit 131 obtains behavior information from an external device by sending a request for behavior information to an external device and receiving the behavior information from the external device that receives the request. The external device may be, for example, the measuring device 200 equipped with the acceleration sensor unit 240 which will be described later, or it may be a storage medium in which the behavior information of the measuring device 200 is stored. Once the acquisition unit 131 has acquired the behavior information from the external device, it stores the acquired behavior information in the behavior information storage unit 121.

[0040] The target period is defined as the period within the time frame during which the measurement device 200 performs detections that are subject to road surface condition determination. In this case, since the measurement device 200 detects behavior information at a predetermined sampling rate, the acquisition unit 131 acquires behavior information at each timing detected within the target period. Furthermore, if there are multiple types of behavior information, the acquisition unit 131 acquires behavior information for each type of behavior information at each timing detected within the target period. For example, in this embodiment, acceleration in the X, Y, and Z axes is acquired as behavior information, so the acquisition unit 131 acquires the acceleration in each of the three axes at each timing detected within the target period.

[0041] (Pre-processing) Figure 5 is a diagram illustrating the processing of the preprocessing unit of the computing device according to this disclosure. The preprocessing unit 132 performs predetermined preprocessing on the measurement data. Specifically, the preprocessing unit 132 divides the target period into a plurality of predetermined periods and extracts behavior information detected at each timing within the predetermined period from the behavior information detected at each timing within the target period. The preprocessing unit 132 performs this extraction process for each predetermined period and extracts behavior information detected at each timing within that predetermined period. The predetermined period here can be set arbitrarily and may be, for example, about 3 seconds, as shown in Figure 5. In addition, adjacent predetermined periods in the time series do not have to overlap in time zones, but some time zones may overlap. That is, for example, one predetermined period (for example, 3 seconds) and the next predetermined period (for example, 3 seconds) may have some overlap in time zones, and the overlapping time zone may be, for example, about 2 seconds. In this example, the preprocessing unit 132 performs a process in the measurement data to divide the time-series measurement data into 3-second intervals every time 1 second has elapsed.

[0042] Note that the processing performed by the preprocessing unit 132 described above is not mandatory, and the conversion unit 133, which will be described later, may perform the conversion processing described later on behavior information that is not processed by the preprocessing unit 132 (i.e., on the behavior information itself acquired by the acquisition unit 131).

[0043] (Conversion process) Figures 6, 7, and 8 illustrate the processing of the conversion unit of the computing device according to this disclosure. The conversion unit 133 performs a predetermined conversion process on the behavior information after processing by the preprocessing unit 132. Specifically, the conversion unit 133 generates converted behavior information based on the behavior information. Converted behavior information is data that shows the behavior of vehicle C for each frequency. More specifically, in this embodiment, the converted behavior information is image data in which frequency information showing the behavior of vehicle C for each frequency is arranged in time series.

[0044] The conversion unit 133 performs a wavelet transform on the behavior information detected within a predetermined period (behavior information after processing by the preprocessing unit 132). The wavelet transform is a process that simultaneously extracts signal information related to the time and frequency changes of the behavior information by changing the time width of the analysis according to the frequency of the behavior information; that is, shortening the time width for high frequencies and widening the time width for low frequencies. Therefore, it is possible to extract information on local frequency changes in the time series changes of the behavior information. The conversion unit 133 performs a wavelet transform on the behavior information and outputs an image as shown in Figure 6. That is, the conversion unit 133 outputs image data in which the horizontal axis is time, the vertical axis is frequency, and the color is intensity. In this way, by using the wavelet transform, it is possible to visualize behavior information over short periods of time by changing the time width used for analysis according to the frequency of the behavior information. Furthermore, it is possible to easily grasp what frequency components, from low to high frequencies, were included in the behavior information that changes moment by moment. Therefore, it is possible to easily train a machine learning model on the characteristics of behavior information that change according to the road surface condition and to make judgments about the road surface condition using the machine learning model.

[0045] Furthermore, the conversion unit 133 may convert the behavior information detected within a predetermined period (behavior information after processing by the preprocessing unit 132) into frequency information detected within a predetermined period. Frequency information is data that shows the vehicle's behavior for each frequency. For example, the conversion unit 133 performs a Fourier transform on the behavior information detected within a predetermined period. A Fourier transform is a process that converts time-series data showing the amplitude of intensity (in this case, acceleration) in the time domain into data showing the intensity (in this case, acceleration) in the frequency domain. Note that the Fourier transform may also be an FFT (Fast Fourier Transform). In the example in Figure 7, the conversion unit 133 converts the behavior information arranged in time series (time on the horizontal axis and intensity (acceleration) on the vertical axis), shown in the upper graph G1 of Figure 7, into frequency information showing the intensity (amplitude) for each frequency (data with frequency on the horizontal axis and intensity on the vertical axis), shown in the lower graph G2 of Figure 7. The conversion unit 133 performs the same processing on the behavior information for each predetermined period to generate frequency information for each predetermined period.

[0046] In this embodiment, the behavior information is acceleration in the three axes: X, Y, and Z. Therefore, the conversion unit 133 converts each of the time-series data of the three axes' accelerations, which are arranged in a time series, into frequency information indicating the intensity for each frequency using a Fourier transform.

[0047] The conversion unit 133 then arranges frequency information for predetermined periods in a time series to generate conversion behavior information in which frequency information is arranged in a time series. In other words, conversion behavior information is data in which time series data of intensity indicating the behavior of vehicle C is arranged for each frequency, or in other words, it can be said to be three-dimensional data of time, frequency, and intensity. In this embodiment, the conversion unit 133 generates conversion behavior information for each of the three axes of acceleration. Figure 8 shows an example of conversion behavior information, where the horizontal axis is time, the vertical axis is frequency, and the color is intensity, and the time series data of intensity is image data arranged in the vertical axis direction for each frequency. However, the conversion behavior information shown in Figure 8 is just an example, and is not limited to representing intensity with color, for example. Also, conversion behavior information is not limited to image data, but may be any format of data in which frequency information is arranged in a time series.

[0048] (Obtaining a pre-trained model) The learning unit 134 acquires a trained model that has been machine-learned to determine the correspondence between reference behavior information, which shows the behavior of a vehicle traveling on the road surface for each frequency, and labels, which indicate whether there are signs of an anomaly on the road surface. The reference behavior information here is the same data as the converted behavior information, and in this embodiment, it is image data in which frequency information showing the behavior of a vehicle for each frequency is arranged in time series. The labels are labels that indicate the road surface condition on which the vehicle traveling with the behavior shown in the reference behavior information has traveled.

[0049] When preparing a trained model by having the learning model undergo machine learning, the vehicle equipped with the measuring device 200 is driven on a road surface with known road conditions, and the measuring device 200 is made to detect behavioral information. The acquisition unit 131 acquires the behavioral information and information on the road surface condition of the road surface on which the vehicle drove (the road surface on which the vehicle drove when the behavioral information was detected). Then, the pre-processing unit 132 and the conversion unit 133 perform the same processing on this behavioral information as described above, and convert the behavioral information into reference behavioral information in which frequency information showing the vehicle's behavior at each frequency is arranged in time series.

[0050] The learning unit 134 uses reference behavior information and road surface condition labels as training data to train a learning model and obtain a trained model. That is, the learning unit 134 trains the learning model using road surface condition labels as the target variable and reference behavior information as the explanatory variable. Specifically, the learning unit 134 sets a dataset as training data, where reference behavior information is the input data (explanatory variable) and the road surface condition labels at the location where the reference behavior information is detected are the output data (target variable), and inputs this training data into the learning model. It is preferable that the learning unit 134 generates reference behavior information for each of the three acceleration axes and combines this reference behavior information and road surface condition labels into a single dataset. It is also preferable that the learning unit 134 prepares multiple datasets consisting of reference behavior information and labels, and inputs each of the multiple datasets into the learning model. As a result, the learning model becomes a trained model (program) that has learned the relationship between data showing the vehicle's behavior for each frequency and the road surface condition labels at the location where that data is detected.

[0051] By using reference behavior information for machine learning in this way, it becomes possible to accurately represent characteristic fluctuations in behavior information (acceleration data) caused by changes in road surface conditions by converting them into reference behavior information, which is image data. This allows the model to appropriately capture and learn these characteristics. Therefore, it is possible to generate a model that can appropriately judge changes in road surface conditions.

[0052] The learning unit 134 may set two categories as labels: one indicating signs of an anomaly, and another indicating no signs of an anomaly (normal). For example, the learning unit 134 may use reference behavior information from when driving through a location with signs of an anomaly, along with a label indicating signs of an anomaly, as a set of training data. This allows the trained model to determine that there are signs of an anomaly if the features of the input data are close to the features of the reference behavior information from when driving through a location with signs of an anomaly, and to determine that there are no signs of an anomaly (normal) if the features are far apart. In this case, the learning unit 134 may also use data consisting of reference behavior information from when driving through a location without signs of an anomaly (normal), along with a label indicating no signs of an anomaly (normal), as training data.

[0053] The learning unit 134 may use reference behavior information from multiple time periods with different road surface conditions as training data. In this way, reference behavior information from multiple time periods with different road surface conditions during various abnormality progression states may be used as training data. This allows the system to learn the relationship between the characteristics of behavior information before the abnormality occurs and the road surface conditions, and between the characteristics of behavior information after the abnormality occurs and the road surface conditions, thereby enabling the system to appropriately predict the future progression of abnormalities from the behavior information before the abnormality occurs.

[0054] Figure 9 is a diagram illustrating the processing of the learning unit of the computing device according to this disclosure. As shown in the example in Figure 9, the learning unit 134 may input, for example, training data including reference behavior information converted from behavior information measured in 20XX while a vehicle was traveling on a road with normal road surface conditions and the label "normal", and training data including reference behavior information converted from behavior information measured in 20YY while a vehicle was traveling on a road with road surface conditions just before an anomaly occurred (with signs of an anomaly) and the label "just before an anomaly (signs of an anomaly)" into the learning model and perform machine learning.

[0055] For example, the learning unit 134 may represent the severity of road surface abnormalities using numerical values ​​in multiple stages (for example, ten stages where the numerical value increases as the severity of the abnormality increases), and use these as labels for the road surface conditions. Alternatively, the labels may be in three stages: "normal," "signs of abnormality (just before abnormality)," and "abnormal." In this case, it is preferable for the learning unit 134 to perform machine learning using a dataset that includes the abnormality labels and reference behavior information at that time. That is, in the example in Figure 9, the learning model may be trained using training data that includes reference behavior information obtained by converting behavior information measured by a vehicle traveling on a road with an abnormal road surface condition in 20ZZ, and the label "abnormal."

[0056] The behavioral information (reference behavioral information) corresponding to the "anomaly warning" label may be arbitrarily selected. For example, if vehicle C regularly travels on the target road surface, the behavioral information (reference behavioral information) from when the vehicle traveled on the road surface immediately before the anomaly occurred (in the example of Figure 9, 20ZZ) may be associated with the "anomaly warning" label. Alternatively, the behavioral information from when the vehicle traveled on the road surface a predetermined period before the anomaly occurred may be associated with the "anomaly warning" label. In this case, the predetermined period can be selected as appropriate.

[0057] Furthermore, the road surface used when measuring the behavioral information for the training data may be a different road from the target road surface for detecting anomalies in this case, or it may be the same road. Also, the vehicle used when measuring the behavioral information for the training data may be the same vehicle C used for detecting anomalies in this case, or it may be a different vehicle.

[0058] The learning model (pre-trained model) may be a CNN (Convolutional Neural Network). Alternatively, the learning model (pre-trained model) may be, for example, an LSTM. An LSTM is a type of RNN (Recurrent Neural Network), which is a recurrent neural network that can handle time-series data. By having an input gate, a forgetting gate, and an output gate, it recognizes characteristic patterns in time-series data and considers older information in the time-series data when predicting the future that follows the end of the time-series data. In other words, the learning unit 134 analyzes the differences in measurement data from multiple time periods and trains the model to learn the characteristics of signs of abnormal road surface conditions. By doing so, the LSTM learns the trend of the progression of abnormal road surface conditions, which can lead to predictions of future damage.

[0059] Furthermore, in this embodiment, it is preferable that the learning unit 134 trains the learning model to learn the correspondence between multiple reference behavior information detected at different times and a single label. Figure 10 shows an example of training data. Specifically, as shown in Figure 10, the learning unit 134 trains the learning model to learn the correspondence between the reference behavior information at a predetermined first time period, the reference behavior information at a second time period earlier than the first time period, and the label at the first time period, as a set of training data. As a result, the trained model becomes a model that has learned the correspondence between the reference behavior information at the first and second time periods and the label at the first time period. Here, the first time period refers to the time when the reference behavior information corresponding to the label for the training data was detected. The second time period may be any time earlier than the first time period, but it is preferable that it is a time when the vehicle drove on the same road surface and at the same location as the first time period. That is, it is preferable that the reference behavior information at the second time period is data from the same road surface and at the same location where the reference behavior information at the first time period was detected. Furthermore, the third period may be any period prior to the second period, but it is preferable that it is a period when the vehicle traveled on the same road surface and at the same location as the first period. In other words, it is preferable that the reference behavior information for the third period is data from the same road surface and at the same location where the reference behavior information for the first period was detected.

[0060] Furthermore, the learning unit 134 may perform machine learning using the first period and the second period, the second period and the third period, and each combination of the second period and the third period as training data. Regarding labeling, the first period may be labeled "abnormal" and the others "normal," but this is not limited to this. For example, when generating a trained model, if you want to make predictions more than three months in advance, it is necessary to compare the first period with data from more than three months prior. In this case, assuming that the second period was about one month before the first period and the third period was five months before, the second period is not three months away from the first period, but more than three months away from the third period. In this case, the second period may also be treated as equivalent to the first period, and the second and third periods may be labeled as "abnormal."

[0061] In the example in Figure 10, one set of training data consists of reference behavior information from a first period when driving on a normal road surface, reference behavior information from a second period when driving on the same location on a normal road surface prior to the first period, and a label indicating normal behavior. In addition, in the example in Figure 10, one set of training data consists of reference behavior information from a first period when driving on a road surface showing signs of abnormality (just before an abnormality), reference behavior information from a second period when driving on the same location on a normal road surface prior to the first period, and a label indicating signs of abnormality. That is, if the label for the first period indicates signs of abnormality, it is preferable that the reference behavior information for the second period is data from when driving on a normal road surface.

[0062] In this way, by using reference behavior information from the first and second periods as training data, it becomes possible to perform machine learning while considering, for example, the features of past vehicle behavior, thereby enabling highly accurate anomaly prediction.

[0063] As described above, in this embodiment, the computing device 100 constructs a trained model by having the learning unit 134 perform machine learning on the learning model. However, the entity that constructs the trained model is not limited to the computing device 100. For example, the computing device 100 may receive a trained model that has undergone machine learning by another device via the communication unit 110 or the like, or it may read a trained model that has undergone machine learning that has been pre-stored in the storage unit 120.

[0064] (Determination of abnormal signs) The determination unit 135 determines whether there are signs of an abnormality on the target road surface by inputting the conversion behavior information of the target road surface at the target time, generated by the conversion unit 133, into a trained model. Specifically, since the trained model has learned the relationship between reference behavior information and road surface state labels, it outputs a road surface state label for the target road surface by inputting the conversion behavior information measured while the vehicle is driving on the target road surface as input data into the trained model, and the determination unit 135 uses the label output from the trained model as the determination result. In other words, if the trained model outputs a label indicating signs of an abnormality, the determination unit 135 determines that there are signs of an abnormality on the target road surface. Also, if the trained model outputs a label indicating normal, the determination unit 135 determines that there are no signs of an abnormality on the target road surface (it is normal). If categories other than signs of an abnormality and normal are also set as labels, the determination unit 135 can also make determinations regarding signs of an abnormality and other categories for the target road surface.

[0065] In this embodiment, since transformation behavior information is generated for each of the three axes of acceleration, the determination unit 135 inputs the transformation behavior information for each of the three axes of acceleration at the target time period as a set of input data into the trained model, and obtains a label for the road surface condition at the target time period as the determination result from the trained model.

[0066] Figure 11 is a schematic diagram showing an example of input data to be input to the trained model. More specifically, in this embodiment, the trained model has learned the relationship between the reference behavior information of the first and second periods and the label of the first period. Therefore, in this embodiment, the determination unit 135 inputs the transformation behavior information of the target period (the period subject to anomaly determination) and the transformation behavior information of a reference period prior to the target period (the same position as the first period; for example, if 3 seconds of data is acquired, this includes the location where the anomaly occurred, and when converted to distance, it represents the transformation behavior information of the road surface over a distance of approximately 10m) as input data to the trained model, and obtains the label of the road surface condition at the target period as the determination result from the trained model. As a result, the determination unit 135 can determine with high accuracy whether there are signs of an anomaly at the target period, taking into account the features of past vehicle behavior.

[0067] Furthermore, the conversion behavior information at the reference time can be generated in the same way as the conversion behavior information at the target time. That is, the acquisition unit 131 acquires the behavior information at the reference time, and the preprocessing unit 132 and the conversion unit 133 perform the same processing on this behavior information as described above, converting the behavior information into reference behavior information in which frequency information showing the vehicle's behavior for each frequency is arranged in time series, thereby becoming the conversion behavior information at the reference time. The reference time may be any time prior to the target time, but it is preferable that it is a time when the vehicle traveled on the same road surface and at the same location as the target time. That is, it is preferable that the behavior information at the reference time is data from the same road surface and at the same location where the behavior information at the target time was detected. Furthermore, it is preferable that the period from the reference time to the target time is the same length as the period from the second period to the first period in the training data.

[0068] (Output of judgment results) The output unit 136 outputs the determination result (determination result of whether there are signs of abnormality on the target road surface) from the determination unit 135. The output unit 136 may transmit the determination result to an external device or display the determination result on the display unit 150. For example, the output unit 136 may associate the position of the vehicle, which is represented by a vehicle icon on a two-dimensional map based on the position information of vehicle C included in the measurement data of the vehicle while it is running, which is input into the trained model, with the determination result of the road surface condition, and display it on the display unit 150. This makes it easy and accurate to grasp the location of the road surface where an abnormality is occurring.

[0069] (Regarding calculation methods and programs) Next, the calculation method related to this disclosure will be explained using Figure 12. Figure 12 is a flowchart of the calculation method related to this disclosure. The calculation method related to this disclosure will be explained in accordance with the flow shown in Figure 12.

[0070] First, the computing unit 100 acquires behavioral information of the target road surface from an external device (step S101). Next, the computing unit 100 generates transformation behavioral information of the target road surface based on the behavioral information of the target road surface (step S103). Next, the computing unit 100 inputs the transformation behavioral information of the target road surface into a trained model (step S105) and determines the road surface condition of the target road surface (step S107). Finally, the computing unit 100 outputs the determination result (step S109).

[0071] In this embodiment, the behavior information of the target road surface is converted into converted behavior information that shows the vehicle's behavior for each frequency, and this converted behavior information is input into a trained model to determine whether there are signs of an anomaly on the target road surface. By using converted behavior information that shows the vehicle's behavior for each frequency in this way, it is possible to determine with high accuracy whether there are signs of an anomaly by considering the characteristic behavior of the vehicle when there are signs of an anomaly. Therefore, according to this embodiment, the road surface condition can be appropriately determined.

[0072] (Regarding learning methods) Next, the learning method related to this disclosure will be explained using Figure 13. Figure 13 is a flowchart showing the flow of the learning method related to this disclosure. The learning method related to this disclosure will be explained according to the flow shown in Figure 13.

[0073] First, the computing unit 100 acquires behavioral information of the target road surface from an external device (step S201). Next, the computing unit 100 generates transformation behavioral information of the target road surface based on the behavioral information of the target road surface (step S202). Next, the computing unit 100 assigns labels indicating the road surface condition to the transformation behavioral information of the target road surface (step S203). Next, the computing unit 100 uses the transformation behavioral information and the labels indicating the road surface condition as training data to train a learning model (step S204).

[0074] In this way, the behavior information of the target road surface is converted into transformed behavior information that shows the vehicle's behavior for each frequency, and labels indicating the road surface condition are assigned to this transformed behavior information. By using the transformed behavior information and the labels indicating the road surface condition as training data and training a learning model, a trained model can be generated.

[0075] (Configuration of the measuring device) Next, the configuration of the measuring device according to this disclosure will be described using Figure 14. Figure 14 is a diagram showing an example of the configuration of the measuring device according to this disclosure. The measuring device 200 according to this disclosure comprises a communication unit 210, a storage unit 220, a control unit 230, and a sensor unit for measuring behavioral information. In this embodiment, the sensor unit is provided with an acceleration sensor unit 240, a displacement sensor unit 250, an IMU sensor unit 260, a camera unit 270, and a position information sensor unit 280. However, the sensor unit may consist only of the acceleration sensor unit 240 and the position information sensor unit 280. These configurations will be described in order below.

[0076] The communication unit 210 is responsible for sending and receiving information with external devices. The communication unit 110 may be implemented by, for example, a CAN communication interface device, a wireless LAN card, a serial communication interface device, a Bluetooth® module, a Wi-Fi® module, an antenna, etc.

[0077] The memory unit 220 is a storage device that stores various types of information. The memory unit 220 comprises a main memory and an auxiliary storage device. The main memory may be implemented using semiconductor memory elements such as RAM (Random Access Memory), ROM (Read Only Memory), or flash memory. The auxiliary storage device may be implemented using a hard disk or an SSD (Solid State Drive), for example.

[0078] As shown in Figure 14, the storage unit 220 includes a behavior information storage unit 221. The information stored in the behavior information storage unit 211 of the measuring device 200 is the same as the information stored in the behavior information storage unit 121 of the arithmetic unit 100, so its explanation is omitted.

[0079] The control unit 230 is a controller that manages and controls the measuring device 200. The control unit 230 is implemented by a CPU, MPU, etc., which executes various programs stored in the memory unit 220 using RAM as the working area. Alternatively, the control unit 230 may be implemented by an integrated circuit such as an ASIC or FPGA.

[0080] As shown in Figure 14, the control unit 230 includes an acquisition unit 231, a receiving unit 232, and a providing unit 233. The control unit 230 realizes these functions and performs these processes by reading and executing a program (software) from the storage unit 220. These functions of the control unit 230 may also be realized by electronic circuits. Furthermore, the control unit 230 may perform these processes with a single CPU, or it may have multiple CPUs and perform these processes in parallel with the multiple CPUs. These configurations will be described in detail below.

[0081] The acquisition unit 231 acquires the three-axis acceleration data measured by the acceleration sensor unit 240, which will be described later. Once the acquisition unit 231 has acquired the three-axis acceleration data, it stores the acquired three-axis acceleration data in the behavior information storage unit 221. At this time, the acquisition unit 231 may associate an acceleration data ID with the acquired three-axis acceleration data and store it.

[0082] The reception unit 232 receives various information requests from external devices. For example, the reception unit 232 receives a request for behavior information from the computing unit 100 via the communication unit 210. The information request may include information that identifies the behavior information to be provided, such as a behavior information ID and the measurement time.

[0083] The provisioning unit 233 provides various types of information to external devices based on information requests. For example, if the receiving unit 232 receives an information request for behavior information from the computing device 100, the provisioning unit 233 reads the three-axis acceleration data received in the information request from the behavior information storage unit 221 and provides the behavior information to the computing device 100 via the communication unit 210.

[0084] The acceleration sensor unit 240 is a sensor that detects the acceleration of vehicle C. The acceleration sensor unit 240 detects acceleration in three directions along three mutually orthogonal detection axes. The three mutually orthogonal detection directions may be named, for example, the X axis, Y axis, and Z axis. The acceleration sensor unit 240 may be a capacitive acceleration sensor that, for example, uses MEMS (Micro Electro Mechanical Systems) to create a movable electrode and a fixed electrode, and measures acceleration using the relationship between the change in capacitance between electrodes due to the movement of the movable electrode due to acceleration and the acceleration.

[0085] Furthermore, the acceleration sensor unit 240 may be a piezoresistive acceleration sensor that detects the displacement of a weight supported by a spring that fluctuates with acceleration using a piezoresistive element placed on the spring. Alternatively, the acceleration sensor unit 240 may be a thermal-sensing acceleration sensor that detects the airflow of a gas heated inside the housing, which changes with acceleration, by measuring the change in the temperature-measuring resistance value. Alternatively, the acceleration sensor unit 240 may be a piezoelectric acceleration sensor that measures acceleration from the amount of charge generated in a piezoelectric element in proportion to the applied acceleration.

[0086] The acceleration sensor unit 240 may be attached to the unsprung and sprung masses of the suspensions provided on the front left wheel, front right wheel, rear left wheel, and rear right wheel of the vehicle, respectively, and the acceleration of these three axes may be measured in a time series.

[0087] The displacement sensor unit 250 is a sensor that detects the amount of suspension displacement of vehicle C. The displacement sensor unit 250 measures, for example, the displacement of the shock absorber of the suspension provided on vehicle C as the amount of suspension displacement. The displacement sensor unit 250 may be, for example, a laser displacement meter or a time-measuring laser sensor. For example, a laser displacement meter may be a triangulation type that projects laser light emitted from a light-emitting element (e.g., a semiconductor laser) onto an object to be measured, receives the reflected light reflected from the object to be measured with a light-receiving element (e.g., a linear image sensor, a position-sensitive device, etc.), and detects the displacement of the object to be measured by measuring the displacement of the reflected light caused by the displacement of the object to be measured.

[0088] A time-measuring laser sensor measures the distance to an object by measuring the time it takes for the laser light emitted by the light-emitting element to strike the object and return to the light-receiving element (photodiode). This distance is then converted into the displacement of the object.

[0089] The IMU sensor unit 260 is a sensor that detects the orientation of vehicle C. The IMU sensor unit 260 may be, for example, a 6DoF (Degree of Freedom) IMU (Inertial Measurement Unit) sensor that combines a three-axis gyro sensor and a three-axis accelerometer. The three-axis gyro sensor is implemented by generating a primary vibration in one direction on a movable electrode, and when rotation is applied to the movable electrode, a Coriolis force acts in a direction 90° from the direction of vibration, causing a secondary vibration and a change in capacitance, which can be detected by a capacitive MEMS (Micro Electro Mechanical Systems) gyro sensor.

[0090] The camera unit 270 is a camera (sensor) that captures image data of the area around vehicle C. The camera unit 270 is, for example, a camera that captures image data of the road surface outside the vehicle. The camera includes optical elements and an image sensor. Optical elements are elements that constitute an optical system, such as lenses, mirrors, prisms, and filters. The image sensor is an element that converts light incident through the optical elements into an image signal, which is an electrical signal. The image sensor may be, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

[0091] The position information sensor unit 280 is a sensor that detects the position of vehicle C (location information of vehicle C). The position information sensor unit 280 may be, for example, a GPS (Global Positioning System) sensor. A GPS sensor is equipped with a GPS receiver and receives radio waves transmitted from GPS satellites. The GPS sensor receives radio waves transmitted from multiple GPS satellites and measures the current position (for example, latitude and longitude) by calculating the distance from multiple GPS satellites using the difference between the time the radio waves were received and the time the GPS satellites transmitted the radio waves. Alternatively, the position information sensor unit 280 may be a module for GNSS (Global Navigation Satellite System).

[0092] (Structure and effect) The computing device 100 according to this disclosure includes: an acquisition unit 131 that acquires behavior information and location information indicating the behavior of a vehicle C traveling on a target road surface; a conversion unit 133 that generates converted behavior information indicating the behavior of the vehicle C for each frequency based on the behavior information; a determination unit 135 that determines whether there are signs of an abnormality on the target road surface by inputting the converted behavior information into a trained model that has been machine-learned to determine the correspondence between reference behavior information indicating the behavior of a vehicle traveling on a road surface acquired in the past for each frequency and a label indicating whether there are signs of an abnormality on that road surface; and an output unit 136 that outputs the determination result for the target road surface.

[0093] With this configuration, by using converted behavior information that shows the vehicle's behavior for each frequency, it is possible to accurately determine whether there are signs of an anomaly by considering the characteristic behavior of the vehicle when there are signs of an anomaly. Therefore, it is possible to provide a computing device 100 that can appropriately determine the road surface condition.

[0094] The behavioral information preferably includes the vertical acceleration of vehicle C.

[0095] By using this behavioral information, it is possible to appropriately detect characteristic vehicle behavior when there are signs of an anomaly, and to determine with high accuracy whether there are signs of an anomaly.

[0096] The conversion unit 133 preferably uses image data in which frequency information showing the behavior of vehicle C for each frequency is arranged in time series as the conversion behavior information.

[0097] By using this type of data as transformation behavior information, it is possible to appropriately detect characteristic vehicle behavior when there are signs of an anomaly, and to determine with high accuracy whether there are signs of an anomaly.

[0098] Preferably, the trained model has learned the correspondence between reference behavior information at a predetermined first time period, reference behavior information prior to the first time period, and the label at the same location in the first time period. The conversion unit 133 generates conversion behavior information for the target time period and conversion behavior information prior to the target time period. The determination unit 135 inputs the conversion behavior information for the target time period and prior to the target time period into the trained model to determine whether there are signs of an anomaly on the target road surface at the target time period.

[0099] In this way, by using conversion behavior information between the target period and a past period (reference period), and by using the data as conversion behavior information, it is possible to accurately determine whether there are signs of an anomaly by also considering the characteristics of past vehicle behavior.

[0100] The program relating to this disclosure causes a computer to perform the following steps: acquire behavior information and location information indicating the behavior of vehicle C while it is traveling on a target road surface; generate converted behavior information indicating the behavior of vehicle C for each frequency based on the behavior information; and input the converted behavior information into a trained model that has been machine-learned to determine whether there are signs of an anomaly on the target road surface.

[0101] This configuration allows for the provision of a program that can appropriately determine road surface conditions.

[0102] Although embodiments of the present disclosure have been described above, the embodiments are not limited to those described herein. Furthermore, the aforementioned components include those that can be easily conceived by those skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the embodiments described above. [Explanation of Symbols]

[0103] 1. Computational System 100 Computing equipment 110 Communications Department 120 Storage section 121 Behavioral Information Storage Unit 122 Model Memory Unit 130 Control Unit 131 Acquisition Department 132 Pre-processing section 133 Conversion section 134 Learning Department 135 Judgment section 136 Output section 140 Input section 150 Display section 200 measuring devices 210 Communications Department 220 Storage section 230 Control Unit 231 Acquisition Department 232 Reception Department 233 Provision Department 240 Acceleration sensor section 250 Displacement sensor section 260 IMU Sensor Section 270 Camera Section 280 Location Information Sensor Unit N Network

Claims

1. An acquisition unit that acquires behavioral information and positional information indicating the behavior of a vehicle traveling on the target road surface, A conversion unit generates converted behavior information that shows the behavior of the vehicle for each frequency based on the aforementioned behavior information, A determination unit determines whether there are signs of an anomaly on the target road surface by inputting the converted behavior information into a trained model that has been machine-learned to determine the correspondence between reference behavior information, which shows the behavior of a vehicle traveling on a road surface for which the location information has been acquired, for each frequency, and a label that indicates whether there are signs of an anomaly on that road surface. The system includes an output unit that outputs the determination result of the target road surface. Computing device.

2. The behavioral information includes the vertical acceleration of the vehicle. The computing device according to claim 1.

3. The conversion unit uses image data in which frequency information showing the behavior of the vehicle for each frequency is arranged in time series as the conversion behavior information. The computing device according to claim 1 or claim 2.

4. The trained model has learned the correspondence between the reference behavior information at a predetermined first time period, the reference behavior information at the same location prior to the first time period, and the label at the first time period through machine learning. The conversion unit generates the conversion behavior information for the target period and the conversion behavior information for a period prior to the target period. The determination unit inputs the target time period and the conversion behavior information prior to the target time period into the trained model to determine whether there are signs of an abnormality on the target road surface at the target time period. The computing device according to claim 1 or claim 2.

5. The steps include acquiring behavioral information and location information that indicate the behavior of a vehicle traveling on the target road surface, Based on the aforementioned behavior information, a step is to generate converted behavior information that shows the behavior of the vehicle for each frequency, The steps include: determining whether there are signs of an anomaly on the target road surface by inputting the converted behavior information into a trained model that has been machine-learned to determine the correspondence between reference behavior information, which shows the behavior of a vehicle traveling on a road surface for which the location information has been acquired, for each frequency, and a label indicating whether there are signs of an anomaly on that road surface; The steps include outputting the determination result for the target road surface, A program that causes a computer to execute something.

Images