Road surface judgment device and road surface judgment method

Abstract

To provide a road surface determination device that can improve the determination accuracy of a road surface.SOLUTION: A road surface determination device 4 includes: a data acquisition unit 410 that acquires road surface data including image data of a determination target in which a road surface of a road has been photographed and additional element data related to a point and time at which the image data has been photographed; and a determination unit 411 that determines, based on the road surface data acquired by the data acquisition unit 410, a state of the road surface at the point and time at which the image data has been photographed.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a road surface judgment device and a road surface judgment method. [Background technology]

[0002] The condition of the road surface is one of the factors that greatly affect the safety of vehicles when traveling. For road operators, the condition of the road surface is an important criterion for determining, for example, whether or not it is necessary to spray antifreeze when it snows and the area to spray it over, and whether or not traffic restrictions and drainage work are necessary when roads are flooded during rainfall. In light of this, a drive recorder that determines the condition of the road surface by performing a predetermined image processing on image data captured by a camera has been developed as a road surface determination device that determines the condition of the road surface (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-002163 A Summary of the Invention [Problem to be solved by the invention]

[0004] The road surface judgment device disclosed in Patent Document 1 judges the condition of the road surface using image data captured by a camera as described above. However, when capturing an image of the road surface with a camera, the actual shooting conditions vary depending on various situations, such as the brightness of the surroundings at the time of shooting, backlight, reflection of the road surface, etc., and therefore there is a limit to improving the judgment accuracy of the road surface only by image processing of the image data.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a road surface judgment device and a road surface judgment method that make it possible to improve the accuracy of road surface judgment. [Means for solving the problem]

[0006] In order to achieve the above object, a road surface judgment device according to one aspect of the present invention comprises: a data acquisition unit that acquires road surface data including image data of a road surface to be determined, the image data being an image of the road surface, and additional element data related to the point and time at which the image data was captured; and a determination unit that determines a condition of the road surface at the location and the time based on the road surface data acquired by the data acquisition unit. Effect of the Invention

[0007] According to a road surface judgment device according to an aspect of the present invention, a judgment unit judges the state of the road surface at a location and a time based on road surface data including image data of a judgment target in which the road surface of a road is photographed and additional element data related to the location and time at which the image data was photographed. Therefore, by taking the additional element data into consideration when the judgment unit judges the state of the road surface, it is possible to supplement situations that are difficult to judge using only image data, and to further reinforce situations in which a certain degree of accuracy can be ensured using only image data. Therefore, it is possible to improve the judgment accuracy when judging the state of the road surface.

[0008] Other objects, configurations and effects will become apparent from the detailed description of the invention described below. [Brief description of the drawings]

[0009] [Figure 1] 1 is an overall view showing an example of a road surface judgment system 1. FIG. [Diagram 2] FIG. 2 is a block diagram showing an example of a road surface judgment device 4A according to the first embodiment. [Diagram 3] FIG. 4 is a data configuration diagram showing an example of a road surface database 401. [Figure 4] 2 is a functional explanatory diagram showing an example of a road surface judgment device 4A according to the first embodiment. FIG. [Diagram 5]FIG. 2 is a schematic diagram showing an example of a learning model 403A according to the first embodiment. [Figure 6] FIG. 2 is a schematic diagram showing an example of a road surface division. [Figure 7] 11 is an explanatory diagram showing a first determination example by a determining unit 411. FIG. [Figure 8] FIG. 13 is an explanatory diagram showing a second determination example by the determining unit 411. [Figure 9] FIG. 13 is an explanatory diagram showing a third determination example by the determining unit 411. [Figure 10] 5 is a flowchart showing an example of the operation of the road surface judgment system 1 according to the first embodiment. [Figure 11] 13 is a flowchart (continuation of FIG. 10) showing an example of the operation of the road surface judgment system 1 according to the first embodiment. [Figure 12] FIG. 2 is a diagram showing an example of a road surface judgment information display screen 11. [Figure 13] FIG. 2 is a diagram showing an example of a road traffic information display screen 12. [Figure 14] FIG. 11 is a functional explanatory diagram showing an example of a road surface judgment device 4B according to a second embodiment. [Figure 15] FIG. 11 is a schematic diagram showing an example of a learning model 403B according to the second embodiment. [Figure 16] FIG. 11 is a functional explanatory diagram showing an example of a road surface judgment device 4C according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. In the following, the scope necessary for the explanation to achieve the object of the present invention will be shown in a schematic manner, and the scope necessary for the explanation of the relevant part of the present invention will be mainly explained, and the parts that are omitted from the explanation will be based on the publicly known technology.

[0011] (First embodiment) 1 is an overall view showing an example of a road surface judgment system 1. The road surface judgment system 1 is a system that judges the condition of the road surface based on sensor data, weather data, etc. acquired by a vehicle 10 traveling on a road, and provides the judgment result to each vehicle 10, a road management company, etc.

[0012] The road surface judgment system 1 comprises, as its main components, a terminal device 2, a weather information providing device 3, a road surface judgment device 4A, and a road traffic information providing device 5. The devices 2 to 5 are connected to a wired or wireless network 6 and configured to be able to transmit and receive various data to and from each other (in FIG. 1, transmission and reception of some data is indicated by dashed arrows). Note that the number of the devices 2 to 5 may be changed as appropriate, and the configuration of the network 6 is not limited to the example in FIG. 1.

[0013] The terminal device 2 is composed of, for example, a mobile terminal device such as a smartphone or tablet terminal that can be carried into the vehicle 10, and an in-vehicle device such as a drive recorder, a car navigation system, a collision prevention system, or a parking assistance system that is installed in the vehicle 10. In this embodiment, the terminal device 2 is composed of a smartphone carried by the driver, and the description will be centered on a case where the terminal device 2 is used while being installed on a fixed stand (not shown) on the dashboard. The vehicle 10 may be of any type or purpose, and may be, for example, a household vehicle, a commercial vehicle such as a taxi, a bus, or a truck, or a vehicle managed by a road management company.

[0014] The terminal device 2 includes a sensor group 20 including a position sensor 21, a camera 22, a microphone 23, an acceleration sensor 24, etc., and transmits sensor data, such as position data, image data, environmental sound data, and acceleration data, respectively acquired by the sensor group 20 while the vehicle 10 is traveling, to the road surface judgment device 4A at a predetermined transmission period, and also receives road surface judgment information, road traffic information, etc. The terminal device 2 also accepts various input operations, and outputs various information (image data, road surface judgment information, road traffic information, etc.) via display screens and voices by various applications, programs such as web browsers, etc.

[0015] The position sensor 21 receives satellite positioning signals and, if necessary, combines with the acceleration sensor 24 to measure the current position (latitude, longitude) and traveling direction of the terminal device 2 (vehicle 10) as position data. When the camera 22 is used while being installed on a fixed stand in a position and orientation that allows it to capture an image of the area in front of the vehicle 10, it captures an image of the road surface located in front of the vehicle 10 and generates image data. When the microphone 23 is used in the vehicle 10, it records the environmental sounds (running sounds) of the vehicle 10 while it is running as environmental sound data. The acceleration sensor 24 is, for example, composed of a three-axis acceleration sensor (which may further be combined with a three-axis angular velocity sensor), and when used in the vehicle 10, it records the acceleration of the vehicle 10 while it is running as acceleration data.

[0016] The weather information providing device 3 is operated by the Japan Meteorological Agency, a weather information provider, or the like, and provides (transmits) weather mesh information to other devices (e.g., the terminal device 2, the road surface judgment device 4A, the road traffic information providing device 5, etc.). The weather information providing device 3 is composed of, for example, a general-purpose or dedicated computer that functions as a server or cloud.

[0017] The weather information providing device 3 provides, as weather mesh information, weather condition data at the current time or past time, and weather forecast data for a future time a predetermined time after the current time (for example, 1 hour, 2 hours, etc.). The weather mesh information includes weather, temperature, humidity, precipitation, snowfall (dry snow, wet snow), solar radiation, snow depth, wind speed, etc. as meteorological elements for each area divided into a mesh shape. The method of providing the weather mesh information may be either pull type or push type, and the date, time period, time interval, area, etc. to be provided may be specified.

[0018] The road surface judgment device 4A receives sensor data (position data, image data, environmental sound data, acceleration data, etc.) from the terminal device 2, and also receives weather mesh information from the weather information providing device 3, and judges the state of the road surface at each point and each time based on these data. The road surface judgment device 4A provides (transmits) road surface judgment information indicating the judgment result of the road surface to other devices (for example, the terminal device 2, the road traffic information providing device 5, etc.). The road surface judgment device 4A is composed of, for example, a general-purpose or dedicated computer that functions as a server or a cloud.

[0019] The road traffic information providing device 5 is operated by a road management company or the like, and provides (transmits) road traffic information to other devices (e.g., the terminal device 2, etc.). The road traffic information providing device 5 is configured, for example, with a general-purpose or dedicated computer that functions as a server or a cloud.

[0020] The road traffic information providing device 5 provides, as road traffic information, road actual condition data at the current time and road forecast data for a predetermined time in the future from the current time (for example, one hour, two hours, etc.). The road traffic information includes, for example, road conditions related to the road surface condition of each road, congestion, accidents, construction, etc., and may be provided with the road conditions superimposed on a map. The road traffic information may be provided in either a pull or push manner, and the date, time, road, and area to be provided may be specified.

[0021] Fig. 2 is a block diagram showing an example of a road surface judgment device 4A according to the first embodiment. Fig. 3 is a data configuration diagram showing an example of the road surface database 401.

[0022] The road surface judgment device 4A includes a storage unit 40 including an HDD, SSD, memory, etc., a control unit 41 including a processor such as a CPU, GPU, MPU, etc., an input unit 42 including a keyboard, mouse, touch panel, etc., a display unit 43 including a display, touch panel, etc., a communication unit 44 which is an interface with the network 6 based on a predetermined communication standard (which may be either wired or wireless), an external device interface (I / F) unit 45 which is an interface with external devices such as a printer, scanner, USB memory, etc., and a media input / output unit 46 which is an interface with storage media such as a CD, DVD, etc. Note that the input unit 42, the display unit 43, the external device I / F unit 45, and the media input / output unit 46 may be omitted as appropriate.

[0023] The storage unit 40 stores an operating system (OS) which is a basic program, a road surface judgment program 400 which controls the operation of the road surface judgment device 4A, and various data used in the road surface judgment program 400, such as a road surface database 401, a weather database 402, and a learning model 403A. The road surface judgment program 400 and the various data used in the road surface judgment program 400 are basically stored in the storage unit 40, but these programs and data may be acquired from an external storage device via the communication unit 44 or the external device I / F unit 45, or may be acquired from a storage medium via the media input / output unit 46, and may be updated as appropriate.

[0024] The road surface database 401 is a database that stores sensor data and the like received from the terminal device 2. As shown in Fig. 3, the road surface database 401 has a plurality of fields for each record, and information related to time, position data, image data, environmental sound data, acceleration data, road surface condition, and accuracy of each road surface division is registered in each field. The road surface condition (see Fig. 6) and the accuracy of each road surface division are the results of road surface judgment by the road surface judgment device 4A, and will be described in detail later.

[0025] The weather database 402 is a database that stores the weather mesh information received from the weather information providing device 3. In the weather database 402, weather mesh information is registered from time to time, and a plurality of weather elements for each region and each time (past time, current time, future time) are accumulated. As described above, the weather mesh information includes weather, temperature, humidity, rainfall, snowfall (dry snow, wet snow), solar radiation, snow depth, wind speed, etc. as weather elements for each region divided into meshes. The size of one mesh may be, for example, 1 km square, 5 km square, 20 km square, etc., and may be different for each weather element. Furthermore, the value of the weather element may be an instantaneous value, or may be an integrated value if the weather element can be integrated over time (for example, rainfall, snowfall, etc.).

[0026] 2, the control unit 41 executes the road surface judgment program 400 to function as a data acquisition unit 410, a judgment unit 411, and an information provision unit 412. In addition, the judgment unit 411 functions as a road surface division classification unit 4110 and a road surface division correction unit 4111.

[0027] Fig. 4 is a functional explanatory diagram showing an example of a road surface judgment device 4A according to the first embodiment. Fig. 5 is a schematic diagram showing an example of a learning model 403A according to the first embodiment. Fig. 6 is a schematic diagram showing an example of a road surface division. The road surface judgment device 4A operates as a subject that implements a road surface judgment method, and the processing contents performed by each part of the road surface judgment device 4A correspond to each step of the road surface judgment method.

[0028] The data acquisition unit 410 acquires road surface data including image data of the road surface to be judged, which is an image of the road surface photographed by a traveling vehicle 10, and additional element data related to the location (hereinafter referred to as the "target location") and time (hereinafter referred to as the "target time") at which the image data was photographed.

[0029] The image data to be determined is data of the road surface photographed by the camera 22 of the terminal device 2 through the windshield of the vehicle 10 while it is moving. The image data may be any of a color image, a grayscale image, and an image photographed by infrared light or the like, and may be any of a two-dimensional image and a three-dimensional image. The image data may be any of a still image and a moving image. When the image data is a still image photographed by the camera 22 at a predetermined photographing period, the data acquisition unit 410 may acquire the image data one by one, or may acquire a plurality of pieces of image data together. When the image data is a moving image photographed by the camera 22, the data acquisition unit 410 may acquire the divided image data one by one, or may acquire a plurality of pieces of image data together. When the image data is a moving image, for example, dynamic information such as the state of splashing water, rainfall, and snowfall is included in the image data, so that the accuracy of determining the road surface can be further improved.

[0030] The additional element data is, for example, weather data, environmental sound data, acceleration data, etc. In this embodiment, the additional element data is described as being of these three types, but the additional element data may be at least one of these types, or may include other additional element data as long as it is related to the determination target location and the determination target time. If the additional element data is data that can be acquired by a sensor built into the terminal device 2 such as a smartphone, as in this embodiment, there is no need to install a new sensor, and therefore the introduction cost of the system can be reduced.

[0031] The weather data includes a determination target point (or an area including the determination target point) and a plurality of meteorological elements (e.g., temperature, amount of precipitation, amount of snowfall, etc.) in a determination target period from a time before the determination target time to the determination target time. The weather data is acquired from the weather database 402, for example, by referring to meteorological mesh information accumulated in the meteorological database 402, and using as extraction conditions the determination target point based on position data acquired in accordance with the time when the image data was captured, and the determination target period based on the time when the image data was captured. Note that the weather data may be acquired directly from the weather information providing device 3.

[0032] The environmental sound data is data recorded by the microphone 23 while the vehicle 10 is traveling at a target location and target time, and is time-series data having a predetermined recording period including before and after the target time. Note that, when the image data is a moving image, it is preferable that the recording period of the environmental sound data is the same time period as the shooting period when the moving image was shot.

[0033] The acceleration data is data recorded by the acceleration sensor 24 while the vehicle 10 is traveling at a target location and target time, and is time-series data having a predetermined recording period including before and after the target time. Note that, if the image data is a moving image, the recording period of the acceleration data is preferably the same time period as the shooting period during which the moving image was captured.

[0034] For example, the data acquisition unit 410 receives image data to be determined and environmental sound data and acceleration data as additional element data from the terminal device 2, and registers them in the road surface database 401. The data acquisition unit 410 also receives weather mesh information from the weather information providing device 3, and registers it in the weather database 402. Then, the data acquisition unit 410 acquires weather data as additional element data by referring to the weather database 402 using, as extraction conditions, a determination target point (=position data) and a determination target period for the image data to be determined received from the terminal device 2. As a result, the data acquisition unit 410 acquires road surface data including the image data to be determined and additional element data (weather data, environmental sound data, and acceleration data).

[0035] The determination unit 411 determines the state of the road surface at a determination target point and a determination target time when the image data to be determined included in the road surface data was captured, based on the road surface data acquired by the data acquisition unit 410. In this embodiment, the determination unit 411 determines the state of the road surface using a learning model 403A (FIG. 5) that has undergone machine learning, and includes a road surface division classification unit 4110 and a road surface division correction unit 4111. The determination unit 411 also registers the determination result of the road surface determined as described above in the road surface database 401, whereby the road surface determination results are accumulated in the road surface database 401.

[0036] The learning model 403A used in the determination unit 411 is a learned learning model that has been subjected to machine learning using a plurality of learning data. The learning data is data used as learning data (training data), verification data, and test data in supervised learning. The learning model 403A is, for example, configured by a convolutional neural network (CNN).

[0037] As shown in Fig. 5, the learning model 403A performs machine learning using learning data that is composed of road surface data including image data of a learning target in which a road surface is photographed, and correct labels to which classification results obtained when the state of the road surface is classified into a plurality of road surface categories are attached. That is, in the learning phase of the machine learning, the image data of the learning target constituting the learning data is input as input data to the learning model, and the classification results (output data) of the road surface category output by the input data are compared with the judgment results (correct labels) of the road surface category constituting the learning data, and the weighting parameters of the convolutional neural network are adjusted based on the comparison results, thereby performing machine learning.

[0038] As shown in FIG. 6, the multiple road surface classifications include at least dry, wet, puddle, flooded, wet snow, frozen, and packed snow. In this embodiment, a case will be described in which the multiple road surface classifications are the above seven types. The learning model 403A functions as a multi-class (seven classes in this embodiment) classifier, and outputs the accuracy when the road surface is classified into each of the multiple road surface classifications for each road surface classification. The accuracy for each road surface classification is output as a numerical value in a predetermined range (for example, 0 to 1). Note that the classification method and number of classifications of road surface classifications are not limited to the seven types shown in FIG. 6 as an example.

[0039] The road surface division classification unit 4110 is a subject that executes the inference phase of machine learning, and as shown in Fig. 4, by inputting image data to be determined, which is included in the road surface data acquired by the data acquisition unit 410, to the trained learning model 403A, the road surface division classification unit 4110 outputs a classification result of the road surface into a road surface division for each road surface division as a classification result of the road surface division. Note that the road surface division classification unit 4110 may perform a predetermined image adjustment process (e.g., image format, image size, image filter, image mask, etc.) on the image data as a preprocessing before inputting the image data to the learning model 403A.

[0040] The road surface division correction unit 4111 determines for each road surface division whether or not there is a possibility that a road surface division will occur based on additional element data included in the road surface data, and corrects the classification results of the road surface division by the road surface division classification unit 4110 based on the presence or absence of the possibility of occurrence for each road surface division. In doing so, the road surface division correction unit 4111 excludes road surface divisions determined to have no possibility of occurrence from the multiple road surface divisions based on the presence or absence of the possibility of occurrence for each road surface division, and determines the road surface condition based on the accuracy of each road surface division remaining after the exclusion.

[0041] For example, when weather data is acquired as additional element data included in the road surface data, the road surface division correction unit 4111 analyzes the weather data to determine whether or not an event may occur for each road surface division. When environmental sound data is acquired as additional element data included in the road surface data, the road surface division correction unit 4111 analyzes the environmental sound data to determine whether or not water splashes are present, and determines whether or not an event may occur for each road surface division depending on whether or not water splashes are present. When acceleration data is acquired as additional element data included in the road surface data, the road surface division correction unit 4111 analyzes the acceleration data to detect whether or not a slip component is present, and determines whether or not an event may occur for each road surface division depending on whether or not the slip component is present.

[0042] Then, the road surface division correction unit 4111 outputs the road surface division with the highest accuracy among the accuracy of each remaining road surface division as the state of the road surface at the judgment target point and judgment target time when the road surface data was acquired. Note that the judgment unit 411 may not include the road surface division correction unit 4111, and in that case, the road surface division with the highest accuracy among the accuracy of each road surface division output by the road surface division classification unit 4110 may be output as the state of the road surface. Furthermore, when the possibility of multiple road surface divisions in descending order of accuracy is required in examining the road management work, the judgment unit 411 may output the accuracy of each road surface division output by the road surface division classification unit 4110 as it is, or may output the accuracy of each remaining road surface division after excluding the road surface division that the road surface division correction unit 4111 judges to have no possibility of occurrence. At this time, the determination unit 411 may output the accuracy rank of the road surface sections when the accuracy values ​​of the road surface sections are arranged in descending order of their values, instead of or in addition to the accuracy values ​​of the road surface sections.

[0043] Three specific examples of the case where the road surface division correction unit 4111 corrects the classification result will be described below with reference to FIGS.

[0044] 7 is an explanatory diagram showing a first determination example by the determination unit 411. In the first determination example, a case will be described in which the accuracy ranking of the road surface sections when the accuracy of each road surface section output as a result of inputting the image data to be determined, which is included in the road surface data, into the learning model 403A is sorted in descending order of value, is a determination result as shown below. 1: Wet > 2: Flooded > 3: Frozen > 4: Puddle > 5: Wet snow > 6: Dry > 7: Packed snow

[0045] First, the road surface division correction unit 4111 analyzes weather data at the determination target point and the determination target period. If the analysis result shows that the temperature at the determination target point is below freezing continuously from a past time (e.g., 6 hours ago) before the time when the image data was captured (the determination target time) to the determination target time, the road surface division correction unit 4111 determines that there is no possibility of the road surface classification of "puddle" or "flooded". Therefore, the road surface division correction unit 4111 excludes the road surface classification of "puddle" or "flooded".

[0046] Next, the road surface division correction unit 4111 analyzes the frequency of the environmental sound data at the determination target point and determination target time. For example, as a result of the analysis, if the sound of splashing water is not detected in the environmental sound data, the road surface division correction unit 4111 determines that there is no possibility of the occurrence of the road surface classification of "puddle", "flooded", or "wet snow". Here, since the road surface classifications of "puddle" and "flooded" have already been excluded, the road surface classification correction unit 4111 excludes the road surface classification of "wet snow".

[0047] Next, the road surface classification correction unit 4111 analyzes the acceleration data at the determination target point and determination target time. As a result of this analysis, for example, if a slip component is detected in the acceleration data, the road surface classification correction unit 4111 determines that there is no possibility of the occurrence of the road surface classifications "dry", "wet", "puddle", and "flooded". Here, since the road surface classifications "puddle" and "flooded" have already been excluded, the road surface classification correction unit 4111 excludes the road surface classifications "dry" and "wet".

[0048] Then, the road surface classification correction unit 4111 determines that the condition of the road surface at the target point is classified as a "frozen" road surface classification based on the ranking of the accuracy of each of the remaining "frozen" and "packed snow" road surface classifications after excluding the road surface classifications that are determined to have no possibility of occurrence as described above from the multiple road surface classifications.

[0049] 8 is an explanatory diagram showing a second determination example by the determining unit 411. In the second determination example, a case will be described in which the accuracy ranking of the road surface division by the learning model 403A is a determination result as shown below. 1: Packed snow > 2: Wet > 3: Wet snow > 4: Frozen > 5: Flooded > 6: Puddle > 7: Dry

[0050] First, as a result of the analysis of the weather data, for example, if the temperature at the determination target point is continuously at or above a predetermined freezing determination temperature (for example, 3°C) from a past time (for example, 6 hours ago) to the determination target time, the road surface division correction unit 4111 determines that there is no possibility of the occurrence of the road surface classification of "frozen". Therefore, the road surface division correction unit 4111 excludes the road surface classification of "frozen". The freezing determination temperature is set to 3°C in consideration of the radiation cooling phenomenon, but it may be set to any temperature between 0°C and 5°C at which road surface freezing is expected.

[0051] Next, as a result of analyzing the frequency of the environmental sound data, for example, if the sound of splashing water is detected in the environmental sound data, the road surface classification correction unit 4111 determines that there is no possibility of the occurrence of the road surface classifications of "dry", "wet", "frozen", and "packed snow". Here, since the road surface classification of "frozen" has already been excluded, the road surface classification correction unit 4111 excludes the road surface classifications of "dry", "wet", and "packed snow".

[0052] Next, as a result of analyzing the acceleration data, for example, if a slip component is detected in the acceleration data, the road surface classification correction unit 4111 determines that there is no possibility of the occurrence of the road surface classifications of "dry", "wet", "puddle", and "flooded". Here, since the road surface classifications of "dry" and "wet" have already been excluded, the road surface classification correction unit 4111 excludes the road surface classifications of "puddle" and "flooded".

[0053] Then, the road surface classification correction unit 4111 determines that the road surface condition at the determination target point is classified as a "wet snow" road surface classification since the remaining road surface classification after excluding the road surface classifications that are determined to have no possibility of occurrence as described above from the multiple road surface classifications is "wet snow."

[0054] 9 is an explanatory diagram showing a third determination example by the determining unit 411. In the third determination example, a case will be described in which the accuracy ranking of the road surface division by the learning model 403A is a determination result as shown below. 1: Frozen > 2: Wet > 3: Flooded > 4: Puddle > 5: Wet snow > 6: Packed snow > 7: Dry

[0055] First, if the analysis result of the weather data shows that the temperature at the determination point is higher than the freezing determination temperature continuously from the past time (e.g., 6 hours ago) to the determination target time, and the amount of rainfall at the determination point is not zero during the period from the past time (e.g., 24 hours ago) to the determination target time, and the amount of snowfall at the determination point is continuously zero from the past time (e.g., 2 weeks ago) to the determination target time, the road surface classification correction unit 4111 determines that there is no possibility of the road surface classification of "dry" or "frozen". Therefore, the road surface classification correction unit 4111 excludes the road surface classifications of "dry" and "frozen".

[0056] Next, as a result of analyzing the frequency of the environmental sound data, for example, if the sound of splashing water is not detected in the environmental sound data, the road surface classification correction unit 4111 determines that there is no possibility of the occurrence of the road surface classifications of "puddle", "flooding", and "wet snow". Therefore, the road surface classification correction unit 4111 excludes the road surface classifications of "puddle", "flooding", and "wet snow".

[0057] Next, as a result of analyzing the acceleration data, for example, if no slippage component is detected in the acceleration data, the road surface classification correction unit 4111 determines that there is no possibility of the road surface classification of "wet snow", "frozen", or "packed snow". Here, since the road surface classifications of "wet snow" and "frozen" have already been excluded, the road surface classification correction unit 4111 excludes the road surface classification of "packed snow".

[0058] Then, the road surface division correction unit 4111 determines that the road surface condition at the target point is classified as a "wet" road surface division since the remaining road surface divisions after excluding the road surface divisions that are determined to have no possibility of occurrence as described above are "wet".

[0059] 7 to 9, the road surface division correction unit 4111 analyzes the additional element data in the order of weather data, environmental sound data, and acceleration data, but the order in which the road surface division correction unit 4111 analyzes the additional element data may be changed as appropriate. Also, in Fig. 7 to 9, the road surface division correction unit 4111 analyzes three types of data, weather data, environmental sound data, and acceleration data, but when the data acquisition unit 410 acquires any one or two types of data among the weather data, environmental sound data, and acceleration data as the additional element data, the road surface division correction unit 4111 may analyze the one or two types of data acquired by the data acquisition unit 410.

[0060] In response to the road surface data being acquired by the data acquisition unit 410, the information provision unit 412 generates road surface judgment information indicating the judgment result of the road surface for the road surface data, and provides (transmits) it to the terminal device 2 that captured the image data of the judgment target included in the road surface data. The road surface judgment information is generated in an arbitrary format that can be handled by an arbitrary program such as various applications or a web browser.

[0061] Furthermore, the information providing unit 412 compiles the road surface conditions by road section based on the road surface determination result by the determining unit 411, generates road section road surface determination information including the road surface conditions by road section in an arbitrary format similar to the road surface providing information, and provides (transmits) it to, for example, the terminal device 2, the road traffic information providing device 5, etc. The road sections are, for example, each section when a road is divided at a predetermined distance (for example, 100 m).

[0062] For example, the information providing unit 412 may compile the road surface conditions for each road section by referring to the road surface database 401, and when road surface judgment results at multiple points for a specific road section are registered, the final road surface judgment result may be determined by majority vote, or the judgment result at the latest time may be given priority. The information providing unit 412 may also compile the road surface judgment results for each traffic direction and each lane. The information providing unit 412 may output road surface judgment information for each road section in response to an output request from the terminal device 2, the road traffic information providing device 5, etc., or may output road surface judgment information for each road section at a predetermined output cycle.

[0063] 10 and 11 are flowcharts showing an example of the operation of the road surface judgment system 1 according to the first embodiment. The following describes the operation when the terminal device 2 installed on a fixed stand of the vehicle 10 transmits sensor data (position data, image data, environmental sound data, and acceleration data) to the road surface judgment device 4A at a predetermined transmission period while the vehicle 10 is traveling on a road.

[0064] First, in step S100, the terminal device 2 transmits sensor data (position data, image data, environmental sound data, and acceleration data) at a given time point (a time point to be determined) to the road surface determination device 4A every time a predetermined transmission period arrives. If the image data capturing period is shorter than the image data transmission period, the terminal device 2 may transmit image data at multiple times (multiple time points to be determined) together.

[0065] Next, in step S101, the data acquisition unit 410 of the road surface judgment device 4A receives the sensor data (position data, image data, environmental sound data, and acceleration data) transmitted in step S100, thereby acquiring the image data of the judgment target and the environmental sound data and acceleration data as additional element data. Then, the data acquisition unit 410 registers these data in the road surface database 401.

[0066] In step S110, the weather information providing device 3 transmits weather mesh information to the road surface judgment device 4A. Next, in step S111, the data acquisition unit 410 receives the weather mesh information transmitted in step S110 and registers it in the weather database 402. Then, in step S112, the data acquisition unit 410 acquires weather data as additional element data by referring to the weather database 402 using as extraction conditions the judgment target point based on the position data received in step S101 and the judgment target period based on the shooting time when the image data received in step S101 was shot. As a result of the above steps S101 and S112, in step S120, the data acquisition unit 410 acquires road surface data including the judgment target image data and the weather data, environmental sound data, and acceleration data as additional element data.

[0067] Next, in step S130 (S131 to S133), the judgment unit 411 judges the condition of the road surface at the location (target location) and time (target time) when the image data to be judged, which is included in the road surface data, was photographed based on the road surface data acquired in step S120, and registers the condition in the road surface database 401.

[0068] Specifically, in step S131, the road surface division classification unit 4110 inputs the image data to be judged, which is included in the road surface data, into the trained learning model 403A, and outputs the classification result of the road surface division for the road surface, i.e., the accuracy of each road surface division.

[0069] Next, in step S132, the road surface division correction unit 4111 judges the presence or absence of the possibility of the road surface division occurring for each road surface division based on the additional element data (weather data, environmental sound data, and acceleration data) included in the road surface data. Then, in step S133, the road surface division correction unit 4111 excludes road surface divisions that are judged to have no possibility of occurrence from the multiple road surface divisions, and judges the state of the road surface based on the accuracy of each road surface division remaining after the exclusion. As a result, the road surface division correction unit 4111 outputs the road surface division with the highest accuracy among the accuracy of each remaining road surface division as the state of the road surface at the judgment target point, for example, as shown in Figs. 7 to 9.

[0070] Next, in step S140, the information providing unit 412 transmits road surface determination information indicating the determination result of the road surface determined in step S130 to the terminal device 2 which is the transmission source of the sensor data received in step S101. Then, in step S141, the terminal device 2 receives the road surface determination information transmitted in step S140, and displays the determination result of the road surface indicated by the road surface determination information on the display screen (FIG. 12).

[0071] 12 is a diagram showing an example of the road surface judgment information display screen 11. The road surface judgment information display screen 11 includes an image display area 110 for displaying an image captured by the camera 22, and a judgment result display area 111 for displaying road surface judgment information.

[0072] 12 shows the road surface judgment result for the image data captured by the camera 22, and the judgment result display area 111 displays the state when the road surface classification is judged to be "flooded" as the most accurate road surface classification among the road surface judgment results for the image data. Note that the road surface judgment information display screen 11 may additionally display information based on the road surface judgment result, and for example, when the road surface judgment result is "frozen", a warning message or icon may be displayed to alert the driver.

[0073] Furthermore, in step S150, the information providing unit 412 repeatedly performs the above steps S110 to S133 in response to receiving image data and the like transmitted at a predetermined transmission cycle from a plurality of terminal devices 2, thereby tallying up the road surface judgment results accumulated in the road surface database 401 by road section at any time interval (for example, by minute or hour), generating road section road surface judgment information including the road surface conditions by road section, and transmitting it to the road traffic information providing device 5. Note that the information providing unit 412 may transmit the road section road surface judgment information to the terminal device 2.

[0074] Next, in step S151, upon receiving the road section road surface judgment information transmitted in step S150, the road traffic information providing device 5 generates road traffic information by adding road conditions such as congestion, accidents, construction, etc. to the road section road surface judgment information, and transmits the road traffic information to the multiple terminal devices 2. Then, in step S152, each of the multiple terminal devices 2 receives the road traffic information transmitted in step S151, and displays on a display screen (FIG. 13) the road surface judgment results for each road section indicated by the road traffic information, and road conditions such as congestion, accidents, construction, etc.

[0075] 13 is a diagram showing an example of the road traffic information display screen 12. The road traffic information display screen 12 includes a map display area 120 that displays various road conditions (road surface classification, congestion, accidents, construction, etc.) on a map, and a legend display area 121 that displays a legend indicating the display mode of the road surface classification.

[0076] Figure 13The map display area 120 shown in FIG. 1 displays road surface classifications for each road section superimposed on a map, and a mark 120a indicating the current position of the terminal device 2 (vehicle 10) is displayed in the judgment result display area 111. The road surface classifications for each road section are generated by the road surface judgment device 4A aggregating a large number of road surface judgment results based on image data collected by the driving of a plurality of vehicles 10. Note that, like the road surface judgment information display screen 11, the map display area 120 may additionally display information based on the road surface judgment results, or may display a recommended driving route to the destination of the terminal device 2 (vehicle 10) based on the road surface judgment results.

[0077] In this manner, the series of operations shown in Fig. 10 and Fig. 11 is completed. The road surface judgment device 4A executes the above-mentioned road surface judgment process in parallel by constantly receiving road surface data from a plurality of terminal devices 2. Note that the order of the above steps may be changed as appropriate.

[0078] Therefore, according to the road surface judgment device 4A and the road surface judgment method of this embodiment, the judgment unit 411 judges the road surface condition at a point and a time based on the road surface data including the image data to be judged and additional element data related to the point and the time when the image data was captured. Therefore, by taking the additional element data into consideration when the judgment unit 411 judges the road surface condition, it is possible to supplement a situation where it is difficult to judge with only the image data, and to further reinforce a situation where a certain degree of accuracy can be ensured with only the image data. Therefore, it is possible to improve the judgment accuracy when judging the road surface condition.

[0079] Second Embodiment Fig. 14 is a functional explanatory diagram showing an example of a road surface judgment device 4B according to the second embodiment. Fig. 15 is a schematic diagram showing an example of a learning model 403B according to the second embodiment.

[0080] In the road surface judgment device 4A according to the first embodiment, the road surface division classification unit 4110 inputs the image data of the judgment target included in the road surface data to the learning model 403A, thereby outputting the classification result of the road surface division, and the road surface division correction unit 4111 corrects the classification result of the road surface division based on the additional element data included in the road surface data. In contrast, in the road surface judgment device 4B according to the second embodiment, the road surface division correction unit 4111 is omitted, and the road surface division classification unit 4110 inputs the image data of the judgment target and the additional element data included in the road surface data to the learning model 403B, thereby outputting the classification result of the road surface division. Note that the basic configuration and operation of the road surface judgment system 1 are the same as those of the first embodiment, and therefore the following description will focus on the differences.

[0081] The determination unit 411 determines the state of the road surface using the machine-learned learning model 403B (FIG. 15), and includes a road surface division classification unit 4110.

[0082] 15, the learning model 403B used in the determination unit 411 performs machine learning using learning data configured of road surface data including image data of a learning target in which a road surface is photographed and additional element data, and correct labels to which classification results obtained when the state of the road surface is classified into a plurality of road surface sections are assigned. That is, in the learning phase of the machine learning, the image data of the learning target and additional element data constituting the learning data are input as input data to the learning model, and the classification results (output data) of the road surface sections outputted by the input data are compared with the judgment results (correct labels) of the road surface sections constituting the learning data, and the weighting parameters of the convolutional neural network are adjusted based on the comparison results, thereby performing machine learning.

[0083] The road surface division classification unit 4110 is a subject that executes the inference phase of machine learning, and inputs the image data and additional element data of the judgment target included in the road surface data acquired by the data acquisition unit 410 to the trained learning model 403B, thereby outputting the classification result of the road surface division for the road surface as the state of the road surface. At that time, when the learning model 403B outputs the accuracy of each road surface division as the classification result of the road surface division, the road surface division classification unit 4110 outputs the road surface division with the highest accuracy among the accuracy of each road surface division as the state of the road surface. Note that the road surface division classification unit 4110 may output the accuracy of each road surface division as the classification result of the road surface division, or may output the road surface division with the highest accuracy among the accuracy of each road surface division.

[0084] Therefore, according to the road surface judgment device 4B and road surface judgment method of this embodiment, the judgment unit 411 judges the road surface condition at the relevant point and time based on the road surface data including the image data of the judgment target and the additional element data, as in the first embodiment. Therefore, by taking the additional element data into consideration when the judgment unit 411 judges the road surface condition, it is possible to supplement a situation where it is difficult to judge with only the image data, and to further reinforce a situation where a certain degree of accuracy can be ensured with only the image data. Therefore, it is possible to improve the judgment accuracy when judging the road surface condition.

[0085] (Third embodiment) FIG. 16 is a functional explanatory diagram showing an example of a road surface judgment device 4C according to the third embodiment.

[0086] A road surface judgment device 4C according to the third embodiment is a combination of a road surface division classification unit 4110 and a learning model 403B according to the second embodiment, and a road surface division correction unit 4111 according to the first embodiment. Note that the basic configuration and operation of a road surface judgment system 1 are similar to those of the first and second embodiments, so the following description will focus on the differences.

[0087] The determination unit 411 determines the state of the road surface using a machine-learned learning model 403B (FIG. 15), and includes a road surface division classification unit 4110 and a road surface division correction unit 4111. The learning model 403B used in the determination unit 411 is the same as that in the second embodiment, and therefore a description thereof will be omitted.

[0088] The road surface division classification unit 4110 inputs the image data of the object to be determined and the additional element data included in the road surface data to the learning model 403B, and outputs the accuracy of each road surface division as the classification result of the road surface division. The road surface division correction unit 4111 corrects the classification result of the road surface division output by the road surface division classification unit 4110 based on the additional element data included in the road surface data. The road surface division correction unit 4111 functions in the same way as in the first embodiment, and therefore a description thereof will be omitted.

[0089] Therefore, according to the road surface judgment device 4C and road surface judgment method of this embodiment, the judgment unit 411 judges the road surface condition at the relevant point and time based on the road surface data including the image data of the judgment target and the additional element data, as in the first and second embodiments. Therefore, by taking the additional element data into consideration when the judgment unit 411 judges the road surface condition, it is possible to supplement a situation where it is difficult to judge with only the image data, and to further reinforce a situation where a certain degree of accuracy can be ensured with only the image data. Therefore, it is possible to improve the judgment accuracy when judging the road surface condition.

[0090] (Other embodiments) The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit and scope of the present invention, all of which are included in the technical concept of the present invention.

[0091] In the above embodiment, the data acquisition unit 410 of the road surface judgment devices 4A to 4C has been described as acquiring image data captured by the camera 22 (terminal device 2) on the vehicle 10, but as long as the image data is of the road surface, image data captured by a device other than the camera 22 on the vehicle 10 may be acquired. For example, the data acquisition unit 410 may acquire image data captured by a road monitoring camera provided in a road monitoring device installed on the road, or may acquire image data captured by a drone camera provided in an unmanned aerial vehicle such as a drone. In this case, the data acquisition unit 410 may further acquire additional element data detected by a group of sensors provided in the road monitoring device or the unmanned aerial vehicle.

[0092] In the above embodiment, the determination unit 411 determines the road surface state using the trained learning models 403A and 403B, but the learning models 403A and 403B may be re-trained by batch learning or online learning. In that case, the learning data may be, for example, image data in which the classification results of the road surface divisions by the learning models 403A and 403B are corrected by the road surface division correction unit 4111. The re-learning process may be performed by the road surface determination devices 4A to 4C, or by another device.

[0093] In the above embodiment, a case where a convolutional neural network is adopted as a specific method of machine learning using the learning models 403A and 403B has been described, but any other machine learning method may be adopted for the learning models 403A and 403B. Examples of other machine learning methods include tree types such as decision trees and regression trees, ensemble learning such as bagging and boosting, other neural network types (including deep learning) such as recurrent neural networks, clustering types such as hierarchical clustering, non-hierarchical clustering, k-nearest neighbors, and k-means, multivariate analysis such as principal component analysis, factor analysis, and logistic regression, and support vector machines.

[0094] In the above embodiment, the judgment unit 411 judges the condition of the road surface at the current time based on the road surface data. However, if the road surface data includes additional element data at a future time, such as weather forecast data for a predetermined time period (e.g., every hour), the judgment unit 411 may be configured to predict the condition of the road surface at a future time.

[0095] For example, the road surface division classification unit 4110 may input the image data of the determination target included in the road surface data and the additional element data at the current time and future time to the learning model by adding the additional element data at the future time to the input data of the learning model and adding the classification result of the road surface division at the future time to the output data of the learning model, and output the classification result of the road surface division at the current time and future time as the state of the road surface. The road surface division correction unit 4111 may determine the presence or absence of occurrence possibility of each road surface division at the current time and future time based on the additional element data at the current time and future time included in the road surface data, and correct the classification result of the road surface division at the current time and future time by the road surface division classification unit 4110 based on the determination result. The control unit 41 of the road surface judgment device 4A to 4C may further include a road surface division prediction unit that predicts the state of the road surface at the future time based on the classification result of the road surface division at the current time by the learning models 403A and 403B and the additional element data at the future time (for example, weather forecast data for each predetermined time).

[0096] In the above embodiment, the road surface division correction unit 4111 corrects the classification result of the road surface division by the road surface division classification unit 4110. However, the road surface division correction unit 4111 may receive the time, the position, and the classification result of the road surface division (the accuracy of each road surface division) from an external device, correct the classification result of the received road surface division, and send the result back to the external device. For example, the road surface division correction unit 4111 may extract weather data based on the time and the position received from the external device, and determine the occurrence possibility for each road surface division based on the weather data, thereby correcting the accuracy of each road surface division. In this case, the correction result may be, for example, a result of correcting the accuracy of the road surface division determined to have no occurrence possibility to 0 with respect to the accuracy of each road surface division received from the external device, or may represent the road surface division with the highest accuracy among the remaining road surface divisions after excluding the road surface divisions determined to have no occurrence possibility.

[0097] In the above embodiment, the information providing unit 412 provides the road surface judgment information and road section road surface judgment information to, for example, the terminal device 2, the road traffic information providing device 5, etc., but the information provided may be combined with other information (for example, maps, traffic, disaster prevention, people flow, logistics, etc.) as long as the information is based on the road surface judgment result. In that case, the device to which the information is provided is not limited to the above example, and the information can be provided to any device.

[0098] In the above embodiment, the control unit 41 of the road surface judgment devices 4A to 4C executes the road surface judgment program 400 to function as the data acquisition unit 410, the judgment unit 411, and the information provision unit 412. However, the control unit of each device, such as the terminal device 2, the road traffic information provision device 5, the vehicle-mounted device, the vehicle (vehicle control device), the road monitoring device, or the unmanned flying object, may execute the road surface judgment program 400 to function in the same manner as the road surface judgment devices 4A to 4C, or may function as a part of the data acquisition unit 410, the judgment unit 411, and the information provision unit 412. In this case, the road surface database 401, the weather database 402, and the learning model 403A may be stored in a storage unit of each device, or may be stored in an external device.

[0099] In the above embodiment, the road surface judgment program 400 has been described as being stored in the storage unit 40. In contrast, the road surface judgment program 400 may be provided by being recorded in a computer-readable storage medium such as a CD-ROM or DVD as a file in an installable or executable format. The road surface judgment program 400 may also be provided by being downloaded from an external device via the network 6. Various learning models used in the judgment unit 411 may be stored in the storage unit 40, or may be stored in an external device and accessed via the network 6. [Explanation of symbols]

[0100] 1...road surface judgment system, 2...terminal device, 3...weather information providing device, 4A~4C...Road surface determination device, 5...Road traffic information providing device, 6...Network, 10...vehicle, 11...road surface judgment information display screen, 12...road traffic information display screen, 20: sensor group, 21: position sensor, 22: camera, 23: microphone, 24...Accelerometer, 40: memory unit, 41: control unit, 42: input unit, 43: display unit, 44: communication unit, 45...external device I / F unit, 46...media input / output unit, 110...Image display area, 111...Judgment result display area, 120: map display area, 121: legend display area, 400...road surface judgment program, 401...road surface database, 402...weather database, 403A, 403B...learning model, 410: data acquisition unit, 411: determination unit, 412: information providing unit, 4110...Road surface classification section, 4111...Road surface classification correction section

Claims

1. a data acquisition unit that acquires road surface data including image data of a road surface to be determined, the image data being an image of the road surface, and additional element data related to the location and time at which the image data was captured; a determination unit that determines a state of the road surface at the location and the time based on the road surface data acquired by the data acquisition unit, The determination unit is a road surface division classification unit that outputs a classification result of the road surface into the road surface division for the road surface as the state of the road surface by inputting the image data of the judgment target included in the road surface data acquired by the data acquisition unit into a learning model that has been machine-learned using learning data composed of road surface data including image data of a learning target in which the road surface is photographed and correct answer labels to which classification results obtained when the state of the road surface is classified into a plurality of road surface divisions are assigned; and a road surface division correction unit that determines whether or not the road surface division is likely to occur for each road surface division based on the additional element data included in the road surface data, and corrects the classification result of the road surface division output by the road surface division classification unit based on the occurrence possibility for each road surface division. Road surface determination device.

2. a data acquisition unit that acquires road surface data including image data of a road surface to be determined, the image data being an image of the road surface, and additional element data related to the location and time at which the image data was captured; a determination unit that determines a state of the road surface at the location and the time based on the road surface data acquired by the data acquisition unit, The determination unit is a road surface division classification unit that outputs a classification result of the road surface into a road surface division for the road surface as the state of the road surface by inputting the image data of the judgment target and the additional element data included in the road surface data acquired by the data acquisition unit into a learning model that has been machine-learned using learning data consisting of road surface data including image data of a learning target in which the road surface was photographed and the additional element data related to the location and the time when the image data was photographed, and a correct answer label to which a classification result when the state of the road surface is classified into a plurality of road surface divisions is assigned; and a road surface division correction unit that determines whether or not the road surface division is likely to occur for each road surface division based on the additional element data included in the road surface data, and corrects the classification result of the road surface division output by the road surface division classification unit based on the occurrence possibility for each road surface division. Road surface determination device.

3. The road surface classification unit outputting, as a classification result of the road surface division, a probability when the road surface is classified into each of the plurality of road surface divisions for each of the road surface divisions; The road surface division correction unit excluding the road surface sections for which it has been determined that the possibility of occurrence is not present from the plurality of road surface sections based on the presence or absence of the possibility of occurrence for each of the road surface sections, and determining the state of the road surface based on the accuracy of each of the remaining road surface sections after the exclusion; The road surface judgment device according to claim 1 or 2.

4. The data acquisition unit is As the additional element data, weather data including a plurality of weather elements for the location and the period from a time before the time to the time is acquired, The road surface division correction unit determining whether or not there is a possibility of occurrence of each of the road surface segments by analyzing the weather data; The road surface judgment device according to any one of claims 1 to 3.

5. The data acquisition unit is As the additional element data, environmental sound data recorded while a vehicle is traveling on the road at the location and the time is acquired, The road surface division correction unit determining whether or not there is water splashing by analyzing the environmental sound data, and determining whether or not there is a possibility of water splashing for each road surface segment according to the presence or absence of water splashing; The road surface judgment device according to any one of claims 1 to 4.

6. The data acquisition unit is As the additional element data, acceleration data recorded while a vehicle is traveling on the road at the location and the time is acquired; The road surface division correction unit detecting the presence or absence of a slip component by analyzing the acceleration data, and determining the presence or absence of the occurrence possibility for each road surface section according to the presence or absence of the slip component; The road surface judgment device according to any one of claims 1 to 5.

7. The plurality of road surface segments include This includes at least dry, wet, puddle, flooding, wet snow, frozen, and packed snow. The road surface judgment device according to any one of claims 1 to 6.

8. an information providing unit that collects the road surface conditions determined by the determining unit for each road section and provides road surface determination information including the road surface conditions for each road section, The road surface judgment device according to any one of claims 1 to 7.

9. The information providing unit is providing road traffic information for displaying the road surface judgment result and the road conditions for each road section on a map by adding the road conditions to the road surface judgment information; The road surface judgment device according to claim 8.

10. The information providing unit is When the road surface condition is compiled for each road section, a final road surface judgment result is determined by majority vote among the road surface judgment results at a plurality of points included in the road section, or a most recent road surface judgment result is given priority. The road surface judgment device according to claim 8 or 9.

11. The image data includes: The photograph was taken by a camera equipped on a mobile terminal device, an in-vehicle device, a road monitoring device, or an unmanned aerial vehicle, The data acquisition unit is Acquiring the image data by receiving the image data from a mobile terminal device, an in-vehicle device, a road monitoring device, or an unmanned aerial vehicle; The road surface judgment device according to any one of claims 1 to 10.

12. Computer, A road surface judgment device according to any one of claims 1 to 11, Road surface assessment program.

13. a data acquisition step of acquiring road surface data including image data of a road surface to be determined, the image data being an image of the road surface, and additional element data relating to the point and time at which the image data was captured; a determination step of determining a state of the road surface at the location and the time based on the road surface data acquired by the data acquisition step, The determining step includes: a road surface classification step of inputting the image data of the object to be determined, which is included in the road surface data acquired in the data acquisition step, into a learning model that has been machine-learned using learning data composed of road surface data including image data of the object to be learned in which the road surface has been photographed, and correct answer labels to which classification results obtained when the state of the road surface is classified into a plurality of road surface classes are assigned, thereby outputting the classification results of the road surface into the road surface classes as the state of the road surface; a road surface division correction step of determining whether or not the road surface division is likely to occur for each road surface division based on the additional element data included in the road surface data, and correcting the classification result of the road surface division outputted by the road surface division classification step based on the occurrence possibility for each road surface division. Road surface judgment method.

14. a data acquisition step of acquiring road surface data including image data of a road surface to be determined, the image data being an image of the road surface, and additional element data relating to the point and time at which the image data was captured; a determination step of determining a state of the road surface at the location and the time based on the road surface data acquired by the data acquisition step, The determining step includes: a road surface classification step of inputting the image data of the object to be determined and the additional element data included in the road surface data acquired in the data acquisition step into a learning model that has been machine-learned using learning data consisting of road surface data including image data of a learning object in which the road surface was photographed and the additional element data related to the location and time at which the image data was photographed, and a correct answer label to which a classification result when the state of the road surface is classified into a plurality of road surface classes is assigned, and outputting a classification result of the road surface class for the road surface as the state of the road surface; a road surface division correction step of determining whether or not the road surface division is likely to occur for each road surface division based on the additional element data included in the road surface data, and correcting the classification result of the road surface division outputted by the road surface division classification step based on the occurrence possibility for each road surface division. Road surface judgment method.

Images