Road surface state estimation device, road surface state estimation method, program, and learning model generation device

Abstract

To estimate road surface conditions from vehicle behavior.SOLUTION: An estimation device (1) is provided with a processor, the processor acquires as driving data the time series data input to the vehicle via tires from the road surface when the vehicle is traveling, and uses as learning data each of the time series data input to the vehicle via tires from each of multiple different road surface conditions related to deformation of the road surface when the vehicle is traveling, the acquired driving data is input to a learning model that is trained to use time series data input to the vehicle as input and to output information indicating the road surface conditions, and at least one road surface condition out of multiple different road surface conditions related to deformation of the road surface is estimated as the road surface condition corresponding to the input driving data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a road surface condition estimation device, a road surface condition estimation method, a program, and a learning model generation device. [Background technology]

[0002] The deterioration state of road surfaces is estimated in order to carry out road repairs, etc. For example, a technique is known in which an abnormal area of ​​the road surface is detected from the road surface on a road image captured by a camera, and whether the acceleration measured when a vehicle travels through the abnormal area is outside a predetermined range is determined (see, for example, Patent Document 1). Also known is a technique in which an index indicating the degree of unevenness of the road surface is measured using vibration data measured by a measuring device mounted on a vehicle (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-139672 A [Patent Document 2] JP 2015-161580 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in terms of detecting abnormal areas on the road surface on a road image, a system including a camera and an image analysis method for detecting abnormal areas on the road surface on a road image using a camera and an image analysis method is large-scale, which is not preferable because it leads to an increase in the size of the device and an increase in cost. In addition, there are various road surface conditions, and in terms of directly obtaining the degree of unevenness of the road surface from vibration data measured on the vehicle side, it is difficult to detect a wide variety of road surface conditions and is not sufficient to specify the road surface condition. Therefore, there is room for improvement in estimating the road surface condition.

[0005] An object of the present disclosure is to provide a road surface condition estimation device, a road surface condition estimation method, a program, and a learning model generation device that are capable of estimating the condition of a road surface from the behavior of a vehicle. [Means for solving the problem]

[0006] In order to achieve the above object, the first aspect is At least one processor; The processor, The time series data input to the vehicle from the road surface through the tires while the vehicle is traveling is acquired as driving data, using, as learning data, each of time series data input to a vehicle via tires from each of a plurality of different road surface conditions related to deformation of a road surface while the vehicle is traveling, and information indicating the road surface condition corresponding to the input time series data, inputting the acquired traveling data into a learning model trained to input the time series data input to the vehicle and outputting information indicating the road surface condition, and estimating at least one road surface condition of the plurality of different road surface conditions related to deformation of the road surface as the road surface condition corresponding to the input traveling data; A road surface condition estimation device.

[0007] A second aspect is the estimation device of the first aspect, The plurality of different road surface conditions includes conditions indicative of at least one of ruts in the road surface, cracks in the road surface, and potholes in the road surface.

[0008] A third aspect is the estimation device according to the first or second aspect, The learning model learns using as input time series data input to the vehicle via the tires from a road surface, which includes the plurality of different road surface conditions, including a condition indicating cracks in the road surface and a condition indicating at least one of ruts in the road surface and potholes in the road surface.

[0009] A fourth aspect is the estimation device according to any one of the first to third aspects, each of the plurality of different road surface conditions indicates a different road surface type; the learning model is a model that is trained using time-series data input within a predetermined time when a vehicle is traveling on the different road surface types as explanatory variables, and a road surface type corresponding to the input time-series data as a response variable, The processor, A process is performed to estimate the road surface type corresponding to the acquired driving data as the road surface condition.

[0010] A fifth aspect is the estimation device according to any one of the first to fourth aspects, the time series data is data indicating a change over time of at least one physical quantity, which is each of a plurality of accelerations in different directions and each of a plurality of rotational velocities in different directions; The road surface condition is a condition indicating a road surface type that is deformed from a predetermined normal road surface.

[0011] The sixth aspect is The processor: The time series data input to the vehicle from the road surface through the tires while the vehicle is traveling is acquired as driving data, using, as learning data, each of time series data input to a vehicle via tires from each of a plurality of different road surface conditions related to deformation of a road surface while the vehicle is traveling, and information indicating the road surface condition corresponding to the input time series data, inputting the acquired traveling data into a learning model trained to input the time series data input to the vehicle and outputting information indicating the road surface condition, and estimating at least one road surface condition of the plurality of different road surface conditions related to deformation of the road surface as the road surface condition corresponding to the input traveling data; This is a road surface condition estimation method that processes the above.

[0012] The seventh aspect is The processor: The time series data input to the vehicle from the road surface through the tires while the vehicle is traveling is acquired as driving data, using, as learning data, each of time series data input to a vehicle via tires from each of a plurality of different road surface conditions related to deformation of a road surface while the vehicle is traveling, and information indicating the road surface condition corresponding to the input time series data, inputting the acquired traveling data into a learning model trained to input the time series data input to the vehicle and outputting information indicating the road surface condition, and estimating at least one road surface condition of the plurality of different road surface conditions related to deformation of the road surface as the road surface condition corresponding to the input traveling data; It is a program that processes this information.

[0013] The eighth aspect is The processor: Acquire, as learning data, a plurality of data sets including time series data input to a vehicle via tires from each of a plurality of different road surface conditions related to deformation of the road surface while the vehicle is traveling, and information indicating the road surface condition while the vehicle is traveling corresponding to the time series data; using the acquired learning data, a process is performed to generate a learning model that receives unknown time-series data as input and outputs information indicating a road surface condition corresponding to the unknown time-series data; A learning model generation device. Effect of the Invention

[0014] According to the present disclosure, there is an effect that the road surface condition can be estimated from the vehicle behavior. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an estimation device according to an embodiment. [Diagram 2] FIG. 1 is a diagram illustrating a conceptual configuration of a learning model generating device. [Diagram 3] FIG. 4 is a diagram showing an example of an acceleration characteristic indicating a behavior of a vehicle. [Figure 4] FIG. 2 is a diagram showing an example of a road surface condition. [Diagram 5] FIG. 4 is a diagram showing an example of an acceleration characteristic indicating a behavior of a vehicle. [Figure 6] FIG. 2 is a diagram showing an example of a road surface condition. [Figure 7] FIG. 2 is a diagram showing an example of a road surface image obtained by photographing a road surface. [Figure 8] FIG. 2 is a diagram illustrating a functional configuration of a learning processing unit. [Figure 9] FIG. 2 is a diagram illustrating an example of a configuration of an output layer. [Figure 10] FIG. 13 is a diagram showing an example of a state in an output layer. [Figure 11] 13 is a flowchart showing an example of the flow of a learning process. [Figure 12] FIG. 2 is a diagram illustrating an electrical configuration of the estimation device. [Figure 13] 13 is a flowchart showing a flow of an estimation process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment for realizing the technology of the present disclosure will be described in detail with reference to the drawings. Note that components and processes that have the same action and function are given the same reference numerals throughout the drawings, and duplicated descriptions may be omitted as appropriate. In addition, the present disclosure is not limited to the following embodiments, and can be implemented by making appropriate modifications within the scope of the purpose of the present disclosure.

[0017] <Estimation device> Next, an example of an estimation device that estimates the road surface condition on which a vehicle is traveling will be described.

[0018] The road surface on which the vehicle 2 travels deteriorates due to friction with the tires caused by the travel of the vehicle 2, etc. For example, a uniformly flat road surface may reach a characteristic road surface state indicating deterioration of the road surface, such as parts being raised or gouged or cracks that create voids appearing as the vehicle 2 travels. Therefore, the estimation device according to this embodiment estimates, as an example of the road surface state of the road surface, a characteristic road surface state indicating deterioration of the road surface from the behavior of the vehicle 2 while traveling on the road surface.

[0019] 1 is a diagram showing an example of the configuration of an estimation device 1 capable of executing an estimation process for estimating a road surface condition on a road surface on which a vehicle is traveling. In this embodiment, a case where the estimation device 1 is provided in a vehicle 2 will be described as an example. Note that the technology disclosed herein is not limited to providing the estimation device 1 in the vehicle 2.

[0020] In the estimation device 1, the road surface condition of the road surface during traveling is estimated. Specifically, the vehicle 2 is equipped with tires 20, and tire behavior such as acceleration changes according to the road surface condition (state shown as state data 3) of the road surface during traveling, and the behavior of the vehicle 2 such as acceleration changes according to the tire behavior. That is, the behavior of the vehicle 2 changes according to the force input from the road surface through the tire to the vehicle during traveling. The estimation device 1 is equipped with an estimation unit 5, and the estimation unit 5 is connected to a detection unit 22 of the vehicle 2 so that a physical quantity indicating the behavior of the vehicle 2 such as acceleration is input as input data 4. In addition, the estimation unit 5 is connected to an output unit 7 so that the estimation result of the estimation unit 5 (i.e., the state estimated as the road surface condition) is output as output data 6. That is, the estimation unit 5 estimates the road surface condition of the road surface corresponding to the input behavior of the vehicle 2, and outputs it to the output unit 7 (details will be described later). The estimation device 1 can be realized by a computer equipped with a CPU as an execution device described later. The estimation device 1 is an example of a road surface condition estimation device of the present disclosure. The detection unit 22 is an example of a detection unit that detects time-series data and driving data of the present disclosure.

[0021] The detection unit 22 detects, as the vehicle behavior data, a physical quantity indicating the behavior of the vehicle that changes according to the road surface condition during driving. In this embodiment, time series data that is a characteristic indicating at least one of the acceleration of each of the orthogonal XYZ axes and the rotation speed of each axis is applied as an example of the physical quantity indicating the behavior of the vehicle. A measurement sensor that measures the acceleration and rotation speed of each of these XYZ axes can be applied to the detection unit 22. In addition, when the acceleration and the rotation speed are applied, the acceleration value and the rotation speed value may be corrected according to the magnitude of the vehicle speed, or may be corrected using frequency characteristics. For example, the acceleration value and the rotation speed value may be corrected so as to become larger or smaller as the vehicle speed increases based on a predetermined vehicle speed, for example, a vehicle speed determined as normal driving. In addition, for example, the acceleration value and the rotation speed value may be corrected so as to become larger or smaller according to frequency characteristics, for example, as the frequency increases.

[0022] The physical quantity indicating the behavior of the vehicle may include tire information such as tire pressure and temperature. The tire information may be corrected according to the magnitude of the tire information with respect to acceleration indicating the behavior of the vehicle. For example, the tire pressure may be corrected so that the acceleration value is smaller or larger as the tire pressure decreases. Another example of the physical quantity indicating the behavior of the vehicle may include vehicle information such as a yaw angle. Therefore, the detection unit 22 may be a detection device that detects vehicle information such as a yaw angle. In the following, for simplicity of explanation, the acceleration characteristic indicating the time series of acceleration vibrating in the vertical direction (Z-axis direction) will be mainly explained as the physical quantity indicating the behavior of the vehicle.

[0023] The detection unit 22 may be arranged so as to be able to estimate the road surface condition, and a measurement sensor such as a G sensor may be applied. The measurement sensor such as a G sensor may be attached to the body of the vehicle 2, for example, a predetermined position in the cabin such as a dashboard, or may be attached to at least one of the tires 20.

[0024] The estimation device 1 can store in memory the vehicle behavior data detected by the detection unit 22, i.e., the data of acceleration characteristics showing the behavior of the vehicle while traveling, in association with the position data of the vehicle 2 at the time the data was detected and the date and time data. The position data can be coordinate data of the vehicle's position, and for example, the coordinates can be acquired by a GPS or the like (not shown) or from an external device (not shown). The date and time data can be acquired from a timer (not shown) mounted on the vehicle 2 or from an external device (not shown).

[0025] The acceleration characteristics, which indicate the behavior of the vehicle while traveling and are detected by the detection unit 22, change in a time series according to the traveling on the road surface conditions due to the road surface shape including unevenness such as bumps, gouges, and cracks that create voids. Therefore, it is possible to estimate the road surface conditions from the acceleration characteristics which change in a time series.

[0026] The estimation device 1 estimates the road surface condition of an unknown road surface by an estimation process described later using a trained learning model 51. This makes it possible to estimate the road surface condition of a road surface without using a special or large device or directly measuring deformation of the road surface. The learning model 51 learns by inputting the road surface condition of the road surface and the acceleration characteristics, which are the behavior of the vehicle. The learning of the learning model 51 will be described later.

[0027] The estimation unit 5 is a functional unit that is connected to the detection unit 22 of the vehicle 2 and estimates the road surface condition of the road surface on which the vehicle is traveling using a learning model 51 based on the acceleration characteristics that change in time series according to the road surface condition. Specifically, the estimation unit 5 receives time-series input data 4 as time-series data representing acceleration characteristics indicating the behavior of the vehicle 2 detected by the detection unit 22, that is, acceleration related to vertical vibration. The input data 4 corresponds to state data 3 indicating the road surface condition of the road surface. The road surface condition of the road surface is a concept that includes a road surface type that indicates the type of road surface including a portion that indicates characteristics related to deformation of the road surface such as roughness, steps, depressions, protuberances, and cracks. The road surface condition of the road surface includes the state of a road surface having at least one of a portion having roughness on the road surface, a portion with different steps from the flat ground, a portion with a protruding part, a portion with a gouged part, and a portion with a crack. For example, a characteristic portion showing at least one of roughness of the road surface, a step, a crack on the road surface, a rut on the road surface, and a pothole on the road surface can be applied to a portion showing a characteristic related to deformation of the road surface, and a state of the road surface including the portion can be applied as a road surface state of the road surface. For example, when the vehicle 2 runs on the road surface, the tire 20 comes into contact with the rut on the road surface or the crack on the road surface, and the detection unit 22 detects acceleration related to the vibration in the vertical direction, and the acceleration characteristic in time series changes. Therefore, the behavior of the vehicle 2 indicated by the acceleration characteristic that changes in time series indicated by the input data 4 corresponds to the road surface state of the road surface. In addition, the estimation unit 5 outputs output data 6 indicating the road surface state of the road surface corresponding to the acceleration characteristic of the vehicle 2 that changes in time series as an estimation result using the trained learning model 51.

[0028] The above-mentioned "crack" in the road surface is a concept including a portion showing a crack (for example, a gap) occurring in the road surface, and is a concept including a crack, a depression, and a groove having a gap occurring regularly or irregularly in the road surface. The "rut" in the road surface is a concept including a depression and a groove occurring due to wear of the road surface when the vehicle 2 is traveling. The "pothole" in the road surface is a concept including a depression and a groove occurring irregularly in the road surface. The "roughness" of the road surface is a concept including a minute unevenness occurring on a flat road surface due to wear of the road surface when the vehicle 2 is traveling. The above-mentioned crack, rut, and pothole in the road surface can be determined by the values ​​of length, width, and depth of the deformation of the road surface occurring on a uniformly continuous road surface, or the ratio of these values. For example, a deformation of the road surface having a gap having a length of a predetermined length or more and less than a predetermined width can be determined as a crack in the road surface. In addition, a deformation of the road surface having a shape of a length of a predetermined length or more, a width of a predetermined width or more, and a depth of a predetermined depth or more can be determined as a rut in the road surface. For the road ruts, at least one of the following deformations can be defined as a pit hole on the road surface: a deformation on the road surface deeper than the road ruts, a deformation on the road surface shorter than the road ruts, and a deformation on the road surface with a depth equal to or greater than a predetermined depth and a predetermined shape. Note that the numerical specifications of the length, width, and depth are not absolute, and can be determined based on empirical or statistical values. For example, a deformation on the road surface with a width of 200 mm or more, a depth of 20 mm or more, and a length of 1000 mm or more can be defined as a road rut. These values ​​can be changed as appropriate.

[0029] The learning model 51 is a model that has completed learning to derive output data 6 indicating the road surface condition from the acceleration characteristics (input data 4) of the detection unit 22 that indicate the behavior of the vehicle 2, which changes depending on the road surface condition. The learning model 51 is, for example, a model that defines a trained neural network, and is expressed as a collection of information on the weights (strengths) of connections between nodes (neurons) that constitute the neural network.

[0030] It is preferable that the learning model 51 learns from time series data input to the vehicle via the tires from a road surface including a plurality of different road surface conditions, such as a condition indicating cracks in the road surface and a condition indicating at least one of ruts in the road surface and potholes in the road surface. In the following, for simplicity, road surface conditions related to deformation of the road surface will be mainly described in terms of conditions indicating "cracks," "ruts," and "potholes."

[0031] The estimation process in the estimation device 1 estimates and outputs the road surface condition of an unknown road surface using a trained learning model 51. The learning model 51 is trained using learning data including data indicating the road surface condition of the road surface and the behavior of the vehicle 2 when traveling on a road surface including the road surface condition, i.e., the acceleration characteristics detected by the detection unit 22.

[0032] <Learning process> Next, a learning process for generating a learning model 51 used in the estimation unit 5 will be described. In the learning process, learning is performed using input data 4 indicating the acceleration characteristics of a vehicle when traveling on a road surface, with state data 3 indicating the road surface state as a label, as learning data. The state data 3 indicating the road surface state can be the data indicating the road surface type (road surface type data). In the learning process, a learning data collection process and a learning model generation process are executed.

[0033] 2 is a diagram showing a conceptual configuration of a learning model generation device that generates a learning model 51. The learning model generation device includes a learning processing unit 52. The learning model generation device can be configured to include a computer equipped with a CPU (not shown), and is executed as the learning processing unit 52 by a learning data collection process and a learning model generation process executed by the CPU to generate the learning model 51. Note that the estimation device 1 can also function as a learning model generation device by having the above functions.

[0034] <Learning data collection process> Next, a learning data collection process for collecting learning data used in generating the learning model 51 will be described. In the learning data collection process, the learning processing unit 52 collects learning data including at least a large amount of vehicle behavior data indicating acceleration characteristics (e.g., vertical acceleration) of the vehicle when traveling on the road surface, using state data 3 indicating road surface conditions such as ruts on the road surface as labels, as input data 4. Therefore, the learning data includes a large amount of sets of the input data 4, which is vehicle behavior data indicating acceleration characteristics, and state data 3 corresponding to the input data 4 and indicating road surface conditions such as ruts on the road surface when traveling on the road surface.

[0035] The above-mentioned learning data can be collected by storing the vehicle behavior data in association with the position data of the vehicle 2 and the date and time data at the time the data was detected.

[0036] In the learning data collection process, acceleration characteristics (e.g., vertical acceleration) that change depending on the road surface condition are acquired in a time series. Next, state data 3 indicating ruts on the road surface or the like is assigned as a label to the acquired time series acceleration characteristics (input data 4), and the process is repeated until a predetermined number of sets of state data 3 and input data 4 are obtained, or until a predetermined time is reached. These sets of state data 3 and time series acceleration characteristics (input data 4) become learning data. Note that the state data 3 in the learning data is stored in a memory (not shown) so that it is treated as output data 6 indicating the road surface condition whose estimated result is correct in the learning process described later.

[0037] The learning data may correspond to time series information by adding information indicating a measurement time to each of the acceleration characteristics (input data 4) such as acceleration. Also, for a driving period determined as a road surface condition such as ruts on the road surface, information indicating a measurement time may be added to a set including a time series of acceleration characteristics to correspond to time series information.

[0038] The acceleration characteristics (time characteristics due to time-series acceleration) detected as the above-mentioned vehicle behavior can be regarded as a characteristic pattern in the road surface condition related to the deformation of the road surface.

[0039] Next, an example of the learning data will be described. 3 to 6 are diagrams showing an example of acceleration characteristics and road surface conditions that indicate the behavior of the vehicle 2 when traveling, which are applied as learning data. FIG. 3 is an example of acceleration characteristics that indicate the behavior of the vehicle 2 when traveling on ruts on the road surface. The acceleration characteristics are characteristics that indicate the acceleration in time series when traveling on the road surface, and a group of acceleration values ​​arranged in time series becomes time series data. FIG. 4 is a conceptual cross-sectional view showing the shape of a rut as an example of the road surface condition of the road surface, and the depth of the rut to the deepest part is shown as a depth Dw. FIG. 5 is an example of acceleration characteristics that indicate the behavior of the vehicle 2 when traveling on a pothole on the road surface. FIG. 6 is a conceptual cross-sectional view showing the shape of a pothole as an example of the road surface condition of the road surface, and the depth of the pothole to the deepest part is shown as a depth Dw. Although not shown in the figure, the acceleration characteristics that indicate the behavior of the vehicle 2 when traveling on a cracked part of the road surface and information that indicates the shape of the cracked part of the road surface can be applied as learning data.

[0040] It is preferable to associate the acceleration characteristics during running with parts of the road surface that show characteristics of the road surface, such as cracks and ruts on the road surface. For example, as shown in FIG. 3, in the acceleration characteristics during running on ruts on the road surface, a part Ar1 of the detected acceleration characteristics can be associated as the acceleration characteristics during running on ruts on the road surface. The acceleration characteristics of the part Ar1 can be applied to the learning data. Also, as shown in FIG. 5, in the acceleration characteristics during running on potholes on the road surface, a part Ar2 of the detected acceleration characteristics can be associated as the acceleration characteristics during running on potholes on the road surface. The acceleration characteristics of the part Ar2 can be applied to the learning data. The association of the parts of the road surface condition with these acceleration characteristics may be performed manually, or may be performed in response to an input of the time (hour) when the vehicle traveled on a characteristic road surface condition (for example, cracks or ruts).

[0041] Therefore, the learning data is a set of vehicle behavior data (data related to vehicle behavior, which is acceleration time-series data) indicating the acceleration, etc., which continues in a time series while the vehicle is traveling on a road surface, and information indicating the road surface condition of the road surface. An example of the learning data is shown in Table 1 below. Table 1 is an example of a data set that associates vehicle behavior data (r) applicable as learning data with condition data (R) indicating the road surface condition.

[0042] [Table 1]

[0043] Table 1 associates the condition data (R) with an index indicating a specific road surface condition. That is, the condition data (R) is expressed as data corresponding to at least one of cracks, ruts, and potholes, which indicate characteristics that appear on the road surface. Therefore, the time-series vehicle behavior data (time-series acceleration characteristics) when traveling on the road surface characteristically indicates the road surface condition, and therefore functions effectively in the learning process.

[0044] In the above-mentioned learning data, in order to improve the learning accuracy of the learning model described below and the estimation accuracy of the road surface conditions using the learning model, it is preferable that the characteristic data and the condition data correspond to the road surface conditions on the actual road surface and the time-series acceleration characteristics when traveling on that road surface. Therefore, in this embodiment, as an example of the learning data, it is possible to determine the learning data using a road surface image obtained by photographing the road surface.

[0045] FIG. 7 is a diagram showing an example of a road surface image obtained by photographing the road surface when the vehicle 2 is traveling. The road surface image shown in FIG. 7 includes an image area Gs including a portion where a foreign object such as a fallen rock exists on the road surface of the road GR, an image area Gw including a portion where a rut is formed, and an image area Ga including a portion where the road surface is rough. These image areas Gs, Gw, and Ga can be set in the road surface image as areas indicating the road surface state determined by the user's visual judgment. The above-mentioned image areas may be set by the user's operation, or may be set by image processing. This determined road surface state can be associated with the road surface state of the road GR. Note that although a foreign object such as a fallen rock on the road surface is an obstacle to vehicle travel, the image area Gs including the portion where a foreign object such as a fallen rock exists can be excluded from the road surface state related to the deformation of the road surface since it is considered that the contribution to the deformation of the road surface is low.

[0046] The information indicating the determined road surface condition may be reflected in the condition data 3. In the example shown in FIG. 7, information indicating that the road surface has ruts and roughness may be set as the condition data 3. Therefore, information indicating that the road surface condition of the road surface includes or does not include a portion showing characteristics related to the deformation of the road surface, such as roughness, steps, depressions, and protuberances, may be set as the condition data 3. For example, information indicating the presence or absence of roughness on the road surface, the presence or absence of different steps from the flat ground, the presence or absence of a protruding portion, and the presence or absence of a hollowed portion may be set as the condition data 3 indicating the road surface condition. Specifically, information (e.g., an index) indicating the presence or absence of roughness on the road surface, the presence or absence of steps, the presence or absence of cracks on the road surface, the presence or absence of ruts on the road surface, and the presence or absence of potholes on the road surface can be applied as the condition data 3 indicating characteristics related to the deformation of the road surface.

[0047] In the technology disclosed herein, it is not essential to use road surface images captured of the road surface when collecting the above-mentioned learning data. The learning data may be vehicle behavior data that indicates the acceleration characteristics of the vehicle while traveling on the road surface, using the state data 3 that indicates the road surface state, such as ruts on the road surface, as a label. Data generated using road surface images captured of the road surface (vehicle behavior data and state data that indicates the road surface state) may be used as additional learning data for the learning data.

[0048] <Learning model generation process> Next, the learning model generation process will be described. The learning model generation device shown in Fig. 2 generates a learning model 51 using the above-mentioned learning data by the learning model generation process in the learning processing unit 52. The process of generating this learning model 51 is an example of an optimization process that optimizes the learning model 51 that outputs a road surface condition indicating ruts on the road surface corresponding to input vehicle behavior data that is a time-series acceleration characteristic during driving on the road surface.

[0049] 8 is a diagram showing the functional configuration of the learning processing unit 52, i.e., the functional configuration of a CPU (not shown) for the learning model generation process executed by the learning processing unit 52. The CPU (not shown) of the learning processing unit 52 operates as the functional units of the generator 54 and the calculator 56.

[0050] The learning processing unit 52 holds a large number of sets of the above-mentioned input data 4 (e.g., time-series acceleration characteristics when driving on a road surface) and output data 6, which is condition data 3 indicating road surface conditions such as ruts on the road surface, as learning data in a memory (not shown).

[0051] The generator 54 includes an input layer 540, an intermediate layer 542, and an output layer 544 to configure a known neural network (NN). Since the neural network itself is a known technology, detailed description will be omitted, but the intermediate layer 542 includes a large number of node groups (neuron groups) having inter-node connections and feedback connections. Data from the input layer 540 is input to the intermediate layer 542, and data of the calculation result of the intermediate layer 542 is output to the output layer 544.

[0052] The generator 54 is a neural network that generates generated output data 6A as data representing the road surface condition or data approximating the road surface condition from the input data 4 (acceleration characteristics). The generated output data 6A is data that estimates the road surface condition from the input data 4. The generator 54 generates generated output data that indicates a condition approximating the road surface condition from the input time-series input data 4. The generator 54 learns using a large amount of input data 4, thereby becoming able to generate generated output data 6A that is approximating the road surface condition.

[0053] The calculator 56 is a calculator that compares the generated output data 6A with the output data 6 of the learning data, and calculates the error of the comparison result. The learning processing unit 52 inputs the generated output data 6A and the output data 6 of the learning data to the calculator 56. In response to this, the calculator 56 calculates the error between the generated output data 6A and the output data 6 of the learning data, and outputs a signal indicating the calculation result.

[0054] The learning processing unit 52 performs learning of the generator 54 by tuning the weight parameters of the connections between the nodes based on the error calculated by the calculator 56. Specifically, the weight parameters of the connections between the nodes of the input layer 540 and the intermediate layer 542 in the generator 54, the weight parameters of the connections between the nodes in the intermediate layer 542, and the weight parameters of the connections between the nodes of the intermediate layer 542 and the output layer 544 are fed back to the generator 54 using a method such as the gradient descent method or the error backpropagation method. That is, with the output data 6 of the learning data as the target, the connections between all the nodes are optimized so as to minimize the error between the generated output data 6A and the output data 6 of the learning data.

[0055] The generator 54 may use a recurrent neural network having a function of generating an output by taking into account the context of the time-series input, or may use another method.

[0056] In this embodiment, the output layer 544 includes a function of estimating a degree of likelihood indicating the likelihood of each of a plurality of road surface conditions that indicate characteristics related to deformation of the road surface. The plurality of road surface conditions can include cracks on the road surface, ruts on the road surface, and potholes on the road surface, and can further include roughness of the road surface and steps. Here, a case where cracks on the road surface, ruts on the road surface, and potholes on the road surface are applied as the plurality of road surface conditions will be described as an example.

[0057] FIG. 9 is a diagram showing an example of the configuration of the output layer 544. The output layer 544 includes a first output layer 544A that calculates the accuracy, and a second output layer 544B that performs a judgment based on the accuracy. The first output layer 544A receives the data of the calculation result of the intermediate layer 542, and the data of the calculation result of the first output layer 544A is output to the second output layer 544B. The first output layer 544A includes nodes corresponding to types of features (road surface types) related to deformation of the road surface, and each node outputs the accuracy of the road surface state indicated by the feature. The accuracy calculated in the first output layer 544A is normalized so that the sum of the nodes is a predetermined value (for example, "1"). The second output layer 544B judges the road surface state of the road surface using the data from the first output layer 544A, and outputs information indicating the road surface state of the judgment result.

[0058] The output layer 544 includes a function of outputting the likelihood calculated in the first output layer 544A as likelihood data 6P, and a function of outputting the determination result of the second output layer 544B as generated output data 6A.

[0059] FIG. 10 is a diagram showing an example of a state in the output layer 544. The first output layer 544A calculates the probabilities P1, P2, and P3 (total 1) of cracks on the road surface, ruts on the road surface, and potholes on the road surface, respectively, and the second output layer 544B outputs information indicating the road surface condition of the road surface with a probability equal to or higher than a predetermined threshold, for example, the maximum probability, as a judgment result. In the example shown in Fig. 10, information indicating a pothole is output as the road surface condition of the road surface corresponding to the input vehicle behavior data.

[0060] By estimating the degree of certainty of each of a plurality of road surface conditions as described above, it is possible to derive the likelihood of each of a plurality of road surface conditions for the vehicle behavior data.

[0061] The learning processing unit 52 generates a learning model 51 by using the above-mentioned learning data through a learning model generation process. The learning model 51 is expressed as a collection of information on weight parameters (weights or strengths) of connections between nodes of the learning result, and is stored in a memory (not shown).

[0062] 11 is a flowchart showing an example of the flow of the learning process executed in the learning processing unit 52. The learning process is performed by a CPU (not shown) in the learning processing unit 52 described above.

[0063] In step S110, a set of road surface conditions associated with vehicle behavior data that is acceleration characteristics when traveling on the road surface is acquired as learning data. Specifically, input data 4 indicating acceleration characteristics of a vehicle when traveling on the road surface, with state data 3 indicating the road surface condition as a label, is acquired as learning data. Next, in step S112, a learning model 51 is generated using the acquired learning data. That is, a set of information on weight parameters (weights or strengths) of connections between nodes of the learning result learned using the above-mentioned large number of learning data is acquired. In this step S112, the learning model 51 is optimized by executing the above-mentioned optimization process. Specifically, in step S112, the learning model 51 is optimized so as to output a road surface condition (ruts on the road surface, etc.) corresponding to an input of a time-series acceleration characteristic (vehicle behavior data) during traveling, with the vehicle behavior data as an explanatory variable and the road surface condition assigned as a label as an objective variable. Next, in step S114, data expressed as a set of information on weight parameters (weights or strengths) of connections between nodes of the learning result is stored as the learning model 51.

[0064] In the estimation device 1, the trained generator 54 (i.e., data expressed as a collection of information on weight parameters of connections between nodes as a result of training) is used as the training model 51. If a sufficiently trained training model 51 is used, it is not impossible to estimate road surface conditions such as ruts on the road surface from time-series acceleration characteristics (vehicle behavior data) during driving.

[0065] In this embodiment, the learning model 51 may be any device configuration capable of machine learning using a plurality of data, and is not limited to the above-mentioned network model. For example, a recurrent neural network model and a network model called reservoir computing may be applied to the learning model 51, or other methods may be used.

[0066] Incidentally, the estimated road surface condition, i.e., the characteristic road surface condition indicating the deterioration of the road surface, may be required as information indicating the road surface condition in numerical values ​​in order to confirm the degree of deterioration of the road surface. In this embodiment, the road surface condition of the road surface may include information indicating the road surface condition in numerical values.

[0067] Specifically, the above-mentioned learning model 51 may learn to output information that numerically expresses the shape, such as the size, of a road surface condition that indicates the characteristics related to the deformation of the road surface. That is, as the learning data of the learning model 51, information indicating the type of road surface condition, such as ruts, is associated with information indicating the size of the deformed portion of the road surface, such as ruts, and the associated state data is applied to a label. The learning model 51 learns using the vehicle behavior data (vehicle acceleration characteristics) to which the label is assigned, as learning data. This makes it possible to construct the learning model 51 so that it receives the vehicle behavior data (vehicle acceleration characteristics) as an input and outputs information indicating the size of the deformed portion of the road surface.

[0068] The shape of the road surface condition, such as the size, which indicates the characteristics related to the deformation of the road surface, can be obtained by using data measured along the extension direction (XY direction) of the road surface and along the depth direction or height direction (Z direction). The shape of the road surface condition, such as the size, can be measured using a radar or the like. It is also possible to estimate the shape of the road surface condition, such as the size, by performing image processing on the road image.

[0069] By using the learning model 51 that outputs information indicating the size of the deformed portions of the road surface described above, the estimation device 1 is able to estimate and output information indicating the size of the deformed portions of the road surface that corresponds to the input vehicle behavior data (vehicle acceleration characteristics).

[0070] <Configuration of the estimation device> Next, an example of a specific configuration of the above-mentioned estimation device 1 will be further described. 12 is a diagram showing an example of the electrical configuration of the estimation device 1. The estimation device 1 is configured to include a computer as an execution device that executes processes to realize the various functions described above. The estimation device 1 described above can be realized by causing the computer to execute a program representing each of the functions described above.

[0071] The computer functioning as the estimation device 1 includes a computer main body 100. The computer main body 100 includes a CPU 102, a RAM 104 such as a volatile memory, a ROM 106, an auxiliary storage device 108 such as a hard disk drive (HDD), and an input / output interface (I / O) 110. The CPU 102, the RAM 104, the ROM 106, the auxiliary storage device 108, and the input / output I / O 110 are connected via a bus 112 so as to be able to exchange data and commands with each other. In addition, a communication unit 114 for communicating with an external device, an operation display unit 116 such as a display and a keyboard, and a detection unit 22 are connected to the input / output I / O 110. The detection unit 22 is a measurement sensor that measures the acceleration in each of the X, Y, and Z axial directions and the rotation speed. The detection unit 22 outputs vehicle behavior data (acceleration characteristics) that is time-series data indicating the behavior of the vehicle 2 during traveling. The detection unit 22 may be connected via the communication unit 114.

[0072] A control program 108P for causing the computer main body 100 to function as the estimation device 1, which is an example of an estimation device of the present disclosure, is stored in the auxiliary storage device 108. The CPU 102 reads out the control program 108P from the auxiliary storage device 108, loads it into the RAM 104, and executes processing. In this way, the computer main body 100 that has executed the control program 108P operates as the estimation device 1.

[0073] The auxiliary storage device 108 stores a learning model 108M including the learning model 51, and data 108D including various data. The control program 108P may be provided by a recording medium such as a CD-ROM.

[0074] <Estimation process> Next, a further description will be given of the estimation process in the computer-implemented estimation device 1. In the estimation process, the above-mentioned learning model 51 is used to estimate the road surface condition of the road surface on which the vehicle is traveling.

[0075] 13 shows an example of the flow of the estimation process by the control program 108P executed by the computer main body 100. The estimation process is executed by the CPU 102 when the computer main body 100 is powered on. The CPU 102 reads the control program 108P from the auxiliary storage device 108, loads it in the RAM 104, and executes the process. The control program 108P is an example of a program of the present disclosure. Moreover, the processing of the control program 108P executed by the CPU 102 is an example of processing of a road surface condition estimation method of the present disclosure.

[0076] First, the CPU 102 reads out the learning model 51 from the learning model 108M in the auxiliary storage device 108, and expands it in the RAM 104 to acquire the learning model 51 (step S200). Specifically, by expanding in the RAM 104 a network model (see Figs. 8 and 9) in which nodes are connected by weight parameters expressed as the learning model 51, the learning model 51 in which connections between nodes are realized by weight parameters is constructed.

[0077] Next, CPU 102 acquires unknown input data 4 (acceleration characteristics, which are vehicle behavior data) in time series, from which the road surface condition on which vehicle 2 is traveling is to be estimated, via detection unit 22 (step S202). Next, CPU 102 estimates output data 6 (unknown road surface condition) corresponding to the acquired input data 4 using learning model 51 in estimation unit 5 (step S204). Then, CPU 102 outputs output data 6 (road surface condition) of the estimation result from output unit 7 (step S206), and ends this processing routine.

[0078] In step S206, control is performed such that information indicating the road surface condition is output to an external device via the communication unit 114 and displayed on the operation display unit 116.

[0079] The information indicating the road surface condition output in step S206 can include information that numerically expresses the shape of the road surface condition, such as the size, which indicates the above-mentioned characteristics related to the deformation of the road surface. This allows the user to quantitatively check the road surface condition.

[0080] When outputting the above-mentioned information indicating the road surface condition, the estimation device 1 can perform control so that the information is superimposed on a map including the road surface on which the vehicle 2 is traveling when displayed on the operation display unit 116. Specifically, the estimation device 1 may obtain map data during traveling and position data of the vehicle 2, and perform control to display screen information on the operation display unit 116 in which the information indicating the road surface condition is superimposed on the position of the vehicle on the obtained map data. When the information indicating the road surface condition is superimposed on the map, a predetermined mark corresponding to the road surface condition may be displayed, or message information such as ruts corresponding to the road surface condition may be displayed.

[0081] The estimation device 1 can store information indicating the estimated road surface condition as history in a memory (auxiliary storage device 108, etc.). This makes it possible to accumulate road surface information for each piece of driving data acquired at different times. By checking the accumulated road surface information, it becomes possible to check the history of deformation of a specific road surface. Furthermore, when outputting the road surface information, it is possible to display information indicating the latest road surface condition from among the accumulated road surface information. This makes it possible to output the latest road surface information when the road surface has been repaired, and allows the user to confirm that the road surface has been repaired.

[0082] Furthermore, the estimation device 1 can output information indicating a road surface condition that is more deteriorated than other road surface conditions among a plurality of road surface conditions estimated within a predetermined period. The degree of deterioration of the road surface can be determined using information that numerically expresses the shape, which is information indicating the size of the deformed portion of the road surface described above. For example, information indicating the road surface condition in which the size of the deformed portion of the road surface is the largest may be extracted and output.

[0083] The information indicating the road surface condition is not limited to the road surface of the road on which the vehicle 2 is traveling. For example, road surface information related to the road surface of the surrounding roads on which the vehicle 2 is traveling may be output. This makes it possible to create a driving plan such as avoiding driving on or investigating road surfaces with a high degree of deterioration according to the degree of deterioration of the road surface while the vehicle 2 is traveling.

[0084] As described above, the estimation device 1 can estimate the road surface condition from the acceleration characteristics, which are vehicle behavior data during traveling. Therefore, it is possible to estimate the road surface condition without using a special device or a large device.

[0085] In this disclosure, the application of areas including rough road surfaces, bumps, cracks in the road surface, ruts in the road surface, and potholes in the road surface has been described as examples of areas that exhibit characteristics related to deformation of the road surface; however, it goes without saying that the areas in question are not limited to these.

[0086] Although the technology of the present disclosure has been described above using the embodiments, the technical scope of the present disclosure is not limited to the scope described in the above embodiments. Various modifications or improvements can be made to the above embodiments without departing from the gist of the disclosure, and the forms with such modifications or improvements are also included in the technical scope of the disclosure.

[0087] In the above embodiment, the process is performed by executing the program stored in the auxiliary storage device, but at least a part of the process of the program may be realized by hardware. The process flow of the program described in the above embodiment is also an example, and unnecessary steps may be deleted, new steps may be added, or the process order may be changed without departing from the spirit of the invention.

[0088] Furthermore, in order to cause a computer to execute the processing in the above-described embodiment, a program describing the above-described processing in code that can be processed by a computer may be stored on a storage medium such as an optical disk and distributed.

[0089] In the above-described embodiment, a CPU has been used as an example of a general-purpose processor. However, in the above-described embodiment, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0090] Furthermore, the operations of the processors in the above-described embodiments may not only be performed by a single processor, but may also be performed by multiple processors working together, or may be performed by multiple processors located at physically separate locations working together.

[0091] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[0092] The SDGs have been proposed to realize a sustainable society. One embodiment of the present invention is thought to be a technology that can contribute to "No.9" and other goals. [Explanation of symbols]

[0093] 1 Estimation device 2 Vehicles 3. State Data 4. Input Data 5 Estimation part 6 Output Data 7 Output section 20 Tires 22 Detection unit 51 Learning Model 52 Learning processing unit 100 Computer main unit 108 Auxiliary storage 108P control program 116 Operation display section 540 Input Layer 542 Middle Class 544 Output Layer

Claims

1. At least one processor; The processor, The time series data input to the vehicle from the road surface through the tires while the vehicle is traveling is acquired as driving data, using, as learning data, each of time series data input to a vehicle via tires from each of a plurality of different road surface conditions related to deformation of a road surface while the vehicle is traveling, and information indicating the road surface condition corresponding to the input time series data, inputting the acquired traveling data into a learning model trained to input the time series data input to the vehicle and outputting information indicating the road surface condition, and estimating at least one road surface condition of the plurality of different road surface conditions related to deformation of the road surface as the road surface condition corresponding to the input traveling data; Road surface condition estimation device.

2. the plurality of different road surface conditions include conditions indicative of at least one of ruts in the road surface, cracks in the road surface, and potholes in the road surface; The road surface condition estimating device according to claim 1 .

3. The learning model is trained using time-series data input from a road surface, including a state showing cracks on the road surface and a state showing at least one of ruts on the road surface and potholes on the road surface, as an input, to a vehicle via tires. The road surface condition estimating device according to claim 1 .

4. each of the plurality of different road surface conditions indicates a different road surface type; the learning model is a model that is trained using time-series data input within a predetermined time when a vehicle is traveling on the different road surface types as explanatory variables, and a road surface type corresponding to the input time-series data as a response variable, The processor, performing a process of estimating a road surface type corresponding to the acquired driving data as the road surface condition; The road surface condition estimating device according to claim 1 .

5. the time series data is data indicating a change over time of at least one physical quantity, which is each of a plurality of accelerations in different directions and each of a plurality of rotational velocities in different directions; The road surface condition is a condition indicating a road surface type that is deformed from a predetermined normal road surface. The road surface condition estimating device according to claim 1 .

6. The processor: The time series data input to the vehicle from the road surface through the tires while the vehicle is traveling is acquired as driving data, using, as learning data, each of time series data input to a vehicle via tires from each of a plurality of different road surface conditions related to deformation of a road surface while the vehicle is traveling, and information indicating the road surface condition corresponding to the input time series data, inputting the acquired traveling data into a learning model trained to input the time series data input to the vehicle and outputting information indicating the road surface condition, and estimating at least one road surface condition of the plurality of different road surface conditions related to deformation of the road surface as the road surface condition corresponding to the input traveling data; The road surface condition estimation method includes the steps of:

7. The processor: The time series data input to the vehicle from the road surface through the tires while the vehicle is traveling is acquired as driving data, using, as learning data, each of time series data input to a vehicle via tires from each of a plurality of different road surface conditions related to deformation of a road surface while the vehicle is traveling, and information indicating the road surface condition corresponding to the input time series data, inputting the acquired traveling data into a learning model trained to input the time series data input to the vehicle and outputting information indicating the road surface condition, and estimating at least one road surface condition of the plurality of different road surface conditions related to deformation of the road surface as the road surface condition corresponding to the input traveling data; A program that processes things.

8. The processor: Acquire, as learning data, a plurality of data sets including time series data input to a vehicle via tires from each of a plurality of different road surface conditions related to deformation of the road surface while the vehicle is traveling, and information indicating the road surface condition while the vehicle is traveling corresponding to the time series data; using the acquired learning data, a process is performed to generate a learning model that receives unknown time-series data as input and outputs information indicating a road surface condition corresponding to the unknown time-series data; Learning model generation device.

Images