Road condition determination device, road condition determination program, and road condition determination method

Abstract

To provide a road condition determination device capable of determining a snow damage condition in a simple device configuration.SOLUTION: A road condition determination device 4 comprises: a data acquisition section 410 for acquiring road data including image data obtained by imaging a road of a determination target and a periphery thereof at a predetermined point and at a predetermined time; and a determination section 411 for determining a snow damage condition in the road of the determination target and the periphery thereof by inputting to learning models 403A-403D the image data of the determination target included in the road data acquired by the data acquisition section 410. For the learning models 403A-403D, machine learning is performed by a data set for learning consisting of image data, which are obtained by imaging a road of a learning target and a periphery thereof, and a correct answer label indicating a determination result of a snow damage condition in the road of the learning target and the periphery thereof.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a road condition determination device, a road condition determination program, and a road condition determination method. [Background technology]

[0002] The occurrence of snow damage on roads is one of the factors that greatly affect the safety of vehicles when traveling. For road operators, the occurrence of snow damage is an important criterion for determining, for example, whether or not it is necessary to spray antifreeze when it snows and the area to spray it, and whether or not it is necessary to remove snow that has accumulated on the road and the area to remove it. In light of this, a road condition detection device has been developed as a device for determining the occurrence of snow damage, which determines the road surface condition, snow banks, and effective width by detecting the distance to the road surface and snow surface using a light wave type transmitter / receiver attached to a gate-shaped frame installed across the road surface (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 11-174161 Summary of the Invention [Problem to be solved by the invention]

[0004] In the road surface condition detection device disclosed in Patent Document 1, as described above, a light wave transmitter / receiver attached to a gate-shaped frame is used, so the location where the occurrence of snow damage can be detected is limited to the location where the light wave transmitter / receiver is installed. In addition, a gate-shaped frame is required to install the light wave transmitter / receiver, which causes a problem that the equipment costs required for installation and maintenance are high.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a road condition determination device, a road condition determination program, and a road condition determination method that enable the determination of snow damage conditions at any location with a simple device configuration. [Means for solving the problem]

[0006] In order to achieve the above object, a road condition determination device according to one aspect of the present invention comprises: a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model, The learning model is Machine learning was performed using a learning dataset that includes multiple learning data consisting of image data of the target road and its surroundings, and correct answer labels that indicate the judgment results of the snow damage situation on the target road and its surroundings. Effect of the Invention

[0007] According to the road condition judgment device of one aspect of the present invention, the judgment unit judges the snow damage condition of the road to be judged and its surroundings by inputting image data of the road to be judged and its surroundings into a learning model. Therefore, since it is only necessary to obtain image data of the road to be judged and its surroundings, it is possible to judge the snow damage condition at any point with a simple device configuration.

[0008] Other objects, configurations and effects will become apparent from the detailed description of the invention described below. [Brief description of the drawings]

[0009] [Figure 1] 1 is an overall view showing an example of a road condition determination system 1. FIG. [Diagram 2]FIG. 2 is a block diagram showing an example of a terminal device 2. [Diagram 3] FIG. 2 is a block diagram showing an example of a road condition determination device 4. [Figure 4] FIG. 4 is a data configuration diagram showing an example of a road condition database 401. [Diagram 5] FIG. 2 is a diagram showing an example of road surface divisions. [Figure 6] FIG. 11 is a diagram showing an example of a side margin width. [Figure 7] FIG. 13 is a diagram showing an example of snow bank height. [Figure 8] FIG. 2 is a diagram illustrating an example of a visibility distance. [Figure 9] FIG. 2 is a functional explanatory diagram showing an example of a road condition determination device 4. [Figure 10] FIG. 11 is a schematic diagram showing an example of a first learning model 403A. [Figure 11] FIG. 11 is a schematic diagram showing an example of a second learning model 403B. [Figure 12] A schematic diagram showing an example of a third learning model 403C. [Figure 13] A schematic diagram showing an example of a fourth learning model 403D. [Figure 14] FIG. 2 is a block diagram showing an example of an administrator device 5. [Figure 15] 4 is a flowchart showing an example of the operation of the road condition determination system 1. [Figure 16] 15 is a flowchart showing an example of the operation of the road condition determination system 1 (continuation of FIG. 15). [Figure 17] 16 is a flowchart showing an example of the operation of the road condition determination system 1 (continuation of FIG. 16). [Figure 18] FIG. 11 is a diagram showing an example of a judgment result display screen 11. [Figure 19] FIG. 12 is a diagram showing an example of a compilation result display screen 12. [Figure 20] FIG. 2 is a diagram showing an example of an image display screen 13. [Figure 21] FIG. 2 is a diagram showing an example of a detailed display screen 14. [Figure 22]FIG. 13 is a diagram showing an example of the number of passable lanes. [Diagram 23] FIG. 13 is a diagram showing an example of a degree of caution regarding stacking. [Figure 24] FIG. 4 is a diagram showing an example of packed snow thickness. [Diagram 25] FIG. 11 is a diagram showing an example of roof snow depth. [Figure 26] FIG. 13 is a diagram showing an example of a snow cornice caution level. [Figure 27] FIG. 11 is a diagram showing an example of a degree of caution regarding fallen trees. [Figure 28] FIG. 13 is a diagram showing an example of rain / snow classification. [Figure 29] FIG. 13 is a functional explanatory diagram showing a modified example of the road condition determination device 4. [Diagram 30] A schematic diagram showing an example of a fifth learning model 403E. [Diagram 31] A schematic diagram showing an example of a sixth learning model 403F. [Diagram 32] A schematic diagram showing an example of a seventh learning model 403G. [Diagram 33] A schematic diagram showing an example of the eighth learning model 403H. [Diagram 34] A schematic diagram showing an example of the 9th learning model 403I. [Diagram 35] FIG. 13 is a schematic diagram showing an example of the tenth learning model 403J. [Diagram 36] FIG. 16 is a schematic diagram showing an example of an 11th learning model 403K. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. In the following, the scope necessary for the explanation to achieve the object of the present invention will be shown in a schematic manner, and the scope necessary for the explanation of the relevant part of the present invention will be mainly explained, and the parts that are omitted will be explained as being related to the publicly known technology.

[0011] (Configuration of road condition judgment system 1) 1 is an overall view showing an example of a road condition determination system 1. The road condition determination system 1 is a system that determines the road conditions of the road and its surroundings based on sensor data acquired by a vehicle 10 traveling on the road and weather data provided by a weather information providing device 3, and provides the determination results to the driver of each vehicle 10, a road management company, etc.

[0012] The road condition determination system 1 comprises, as its main components, a terminal device 2, a weather information providing device 3, a road condition determination device 4, and an administrator device 5. The devices 2 to 5 are connected to a wired or wireless network 6 and configured to be able to transmit and receive various data to and from each other (in FIG. 1, transmission and reception of some data is indicated by dashed arrows). Note that the number of the devices 2 to 5 may be changed as appropriate, and the configuration of the network 6 is not limited to the example in FIG. 1.

[0013] The terminal device 2 is composed of, for example, a mobile terminal device such as a smartphone or tablet terminal that can be carried into the vehicle 10, and an in-vehicle device such as a drive recorder, a car navigation system, a collision prevention system, or a parking assistance system that is installed in the vehicle 10. In this embodiment, the terminal device 2 is composed of a smartphone carried by the driver, and the description will be centered on a case where the terminal device 2 is used while being installed on a fixed stand (not shown) on the dashboard. The vehicle 10 may be of any type or purpose, and may be, for example, a household vehicle, a commercial vehicle such as a taxi, a bus, or a truck, a vehicle managed by a road management company, or a two-wheeled vehicle such as a motorcycle or an electric motorcycle.

[0014] The terminal device 2 includes a sensor group 26 including a position sensor 260, a camera 261, a microphone 262, an acceleration sensor 263, etc. The terminal device 2 transmits sensor data such as position data, image data, environmental sound data, and acceleration data recorded by the sensor group 26 at a predetermined recording interval (e.g., one-second interval) while the vehicle 10 is traveling to the road condition determination device 4 at a predetermined transmission interval (e.g., one-minute interval). The terminal device 2 receives road condition determination information and road condition summary information from the road condition determination device 4, and displays a display screen based on the received information.

[0015] The position sensor 260 receives a satellite positioning signal and, by combining with the acceleration sensor 263 as necessary, measures the current position (latitude, longitude) and traveling direction of the terminal device 2 (vehicle 10) as position data. When the camera 261 is used while being installed on a fixed stand in a position and orientation that allows it to capture an image of the area in front of the vehicle 10, it captures an image of the road and its surroundings located in front of the vehicle 10 and generates image data. When the microphone 262 is used in the vehicle 10, it records the environmental sounds (running sounds) of the vehicle 10 while it is running as environmental sound data. The acceleration sensor 263 is composed of, for example, a three-axis acceleration sensor (which may further be combined with a three-axis angular velocity sensor), and when used in the vehicle 10, it records the acceleration of the vehicle 10 while it is running as acceleration data.

[0016] The weather information providing device 3 is operated by the Japan Meteorological Agency, a weather information provider, or the like, and provides (transmits) weather mesh information to other devices (e.g., the terminal device 2, the road condition determination device 4, the administrator device 5, etc.). The weather information providing device 3 is composed of, for example, a general-purpose or dedicated computer that functions as a server or cloud.

[0017] The weather information providing device 3 provides, as weather mesh information, weather condition data at the current time or past time, and weather forecast data for a future time a predetermined time after the current time (for example, 1 hour, 2 hours, etc.). The weather mesh information includes weather, temperature, humidity, precipitation, accumulated precipitation, snowfall (dry snow, wet snow), accumulated snowfall (dry snow, wet snow), solar radiation, snow depth, wind speed, etc. as meteorological elements for each area divided into a mesh shape. The method of providing the weather mesh information may be either pull type or push type, and the date, time period, area, etc. to be provided may be specified.

[0018] The road condition determination device 4 receives sensor data (position data, image data, environmental sound data, acceleration data, etc.) from the terminal device 2, and also receives weather mesh information from the weather information providing device 3, and determines the road conditions of the road and its surroundings at each point and each time based on these data. Specifically, the road condition determination device 4 determines the snow damage condition of the road and the snow damage condition around the road as the road conditions of the road and its surroundings. At this time, the snow damage condition of the road includes, for example, those related to the roadway, and the snow damage condition around the road includes, for example, those related to the side strip, road shoulder, median strip, sidewalk, guardrail, boundary block, lane division sign, buildings along the road, fences, trees, etc. In addition, the road condition determination device 4 provides (transmits) road condition determination information indicating the determination result of the road condition and road condition compilation information indicating the compilation result of the road condition determination result to other devices (for example, the terminal device 2, the administrator device 5, etc.). The method by which the road condition determination device 4 provides various types of information may be either a pull method or a push method, and the date, time period, area, etc. to which the information is to be provided may be specified.

[0019] The administrator device 5 is used by road management businesses and the like, receives road condition determination information and road condition summary information from the road condition determination device 4, and displays a display screen based on the received information. The administrator device 5 may be configured, for example, as a stationary computer, or may be configured, like the terminal device 2, as a mobile terminal device or an in-vehicle device.

[0020] The administrator device 5 may function as a road traffic information providing device that provides (transmits) road traffic information based on road condition judgment information and road condition compilation information to other devices (e.g., the terminal device 2, etc.). In this case, the administrator device 5 provides, as road traffic information, road actual condition data at the current time and road forecast data at a future time a predetermined time after the current time (e.g., one hour, two hours, etc.). The road traffic information includes not only the judgment results and compilation results of the road conditions, but also traffic conditions related to congestion, accidents, construction, etc., and may be provided in a state where the road traffic information is superimposed on a map. The method of providing various information by the administrator device 5 may be either a pull method or a push method, and the date, time, period, area, etc. to be provided may be specified.

[0021] (Configuration of Terminal Device 2) 2 is a block diagram showing an example of the terminal device 2. The terminal device 2 includes a storage unit 20 including an HDD, an SSD, a memory, etc., a control unit 21 including a processor such as a CPU, a GPU, an MPU, etc., an input unit 22 including a button, a touch panel, etc., a display unit 23 including a display, a touch panel, etc., a communication unit 24 which is an interface with a network 6 based on a predetermined communication standard (which may be either wired or wireless), an external device interface (I / F) unit 25 which is an interface with an external device such as a printer, a scanner, a USB memory, etc., and a sensor group 26 including a position sensor 260, a camera 261, a microphone 262, an acceleration sensor 263, etc.

[0022] The storage unit 20 stores an operating system (OS) which is a basic program, various application programs such as a sensor data recording program 200, a road condition display program 201, a web browser, and various data used by these programs. Although the various data are basically stored in the storage unit 20, these programs and data may be acquired from an external storage device via the communication unit 24 or the external device I / F unit 25, and may be updated as appropriate.

[0023] The control unit 21 functions as a sensor data recording processing unit 210 by executing the sensor data recording program 200. The control unit 21 functions as a road condition display processing unit 211 by executing the road condition display program 201.

[0024] For example, when the sensor data recording processing unit 210 receives an input operation to start recording via the input unit 22 while the terminal device 2 is installed in the vehicle 10, it records sensor data using the sensor group 26 at a predetermined recording interval (for example, one second interval) and transmits the recorded sensor data to the road condition determination device 4 at a predetermined transmission interval (for example, one minute interval). The sensor data recording processing unit 210 performs the above process (sensor data recording process) until it receives an input operation to end recording via the input unit 22.

[0025] When the road condition display processing unit 211 receives the road condition determination information and the road condition compilation information from the road condition determination device 4, it displays on the display unit 23 a display screen based on the road condition determination information and the road condition compilation information.

[0026] (Configuration of road condition determination device 4) Fig. 3 is a block diagram showing an example of the road condition determination device 4. Fig. 4 is a data configuration diagram showing an example of the road condition database 401. Fig. 5 is a diagram showing an example of road surface division. Fig. 6 is a diagram showing an example of a lateral margin width. Fig. 7 is a diagram showing an example of a snow bank height. Fig. 8 is a diagram showing an example of a visibility distance.

[0027] The road condition determination device 4 includes a storage unit 40 including an HDD, SSD, memory, etc., a control unit 41 including a processor such as a CPU, GPU, MPU, etc., an input unit 42 including a keyboard, mouse, touch panel, etc., a display unit 43 including a display, touch panel, etc., a communication unit 44 which is an interface with the network 6 based on a predetermined communication standard (which may be either wired or wireless), an external device interface (I / F) unit 45 which is an interface with external devices such as a printer, scanner, USB memory, etc., and a media input / output unit 46 which is an interface with storage media such as a CD, DVD, etc. Note that the input unit 42, the display unit 43, the external device I / F unit 45, and the media input / output unit 46 may be omitted as appropriate.

[0028] The storage unit 40 stores an operating system (OS) which is a basic program, and various application programs such as a road condition determination program 400 which controls the operation of the road condition determination device 4. The storage unit 40 also stores a road condition database 401, a weather database 402, learning models 403A to 403D, and the like as various data used in these programs. Note that the various data are basically stored in the storage unit 40, but these programs and data may be acquired from an external storage device via the communication unit 44 or the external device I / F unit 45, or may be acquired from a storage medium via the media input / output unit 46, and may be updated as appropriate.

[0029] The road condition database 401 is a database that stores sensor data received from the terminal device 2, road condition determination results, and the like. As shown in Fig. 4, the road condition database 401 has a plurality of fields for each record, and information related to the driver, time, position data (point), image data, environmental sound data, acceleration data, road surface classification, lateral margin width, snow bank height, and visibility distance is registered in each field. The road surface classification, lateral margin width, snow bank height, and visibility distance are the results of road condition determination by the road condition determination device 4, and are defined as shown in Figs. 5 to 8, respectively.

[0030] The weather database 402 is a database that stores the weather mesh information received from the weather information providing device 3. In the weather database 402, weather mesh information is registered from time to time, and a plurality of weather elements for each region and each time (past time, current time, future time) are accumulated. As described above, the weather mesh information includes weather, temperature, humidity, rainfall, snowfall (dry snow, wet snow), solar radiation, snow depth, wind speed, etc. as weather elements for each region divided into meshes. The size of one mesh may be, for example, 1 km square, 5 km square, 20 km square, etc., and may be different for each weather element. Furthermore, the value of the weather element may be an instantaneous value, or may be an integrated value if the weather element can be integrated over time (for example, rainfall, snowfall, etc.).

[0031] The control unit 41 executes the road condition determination program 400 to function as a data acquisition unit 410, a determination unit 411, and an information provision unit 412, as shown in FIG.

[0032] FIG. 9 is a functional explanatory diagram showing an example of the road condition judgment device 4. FIG. 10 is a schematic diagram showing an example of the first learning model 403A. FIG. 11 is a schematic diagram showing an example of the second learning model 403B. FIG. 12 is a schematic diagram showing an example of the third learning model 403C. FIG. 13 is a schematic diagram showing an example of the fourth learning model 403D. The road condition judgment device 4 operates as a subject that implements a road condition judgment method, and the processing contents performed by each part 410 to 412 of the control unit 41 correspond to each step (data acquisition step, judgment step, information provision step) of the road condition judgment method.

[0033] The data acquisition unit 410 acquires road data including image data of the road to be judged and its surroundings (hereinafter referred to as the "road to be judged") photographed by a traveling vehicle 10, and correction data related to the location (hereinafter referred to as the "location to be judged") and time (hereinafter referred to as the "time to be judged") at which the image data was photographed.

[0034] The image data to be determined is data obtained by photographing the road to be determined (the road to be determined and its surroundings) through the windshield of the vehicle 10 while it is traveling by the camera 261 of the terminal device 2. The photographing range photographed in the image data may be any range that allows the road conditions to be determined based on the image, and is appropriately set to include not only the road (mainly the road surface) but also the surroundings of the road. The image data may be any of a color image, a grayscale image, and an image photographed with infrared light or the like, and may be any of a two-dimensional image and a three-dimensional image. The image data may be any of a still image and a moving image. When the image data is a still image photographed by the camera 261 at a predetermined recording interval, the data acquisition unit 410 may acquire the image data one by one, or may acquire a plurality of pieces of image data collectively. When the image data is a moving image photographed by the camera 261, the data acquisition unit 410 may acquire the divided image data when the moving image is divided at a predetermined shooting period one by one, or may acquire a plurality of pieces of image data collectively. When the image data is a moving image, the image data contains dynamic information such as the state of splashing water, or the state of rain or snow, so that the accuracy of determining the road conditions can be further improved.

[0035] The correction data is, for example, weather data, position data, environmental sound data, acceleration data, etc. In this embodiment, the correction data is described as being of these four types, but the correction data may be at least one of these types, or may include other correction data as long as it is related to the determination target point and the determination target time. If the correction data is data that can be acquired by a sensor built into the terminal device 2 such as a smartphone, as in this embodiment, there is no need to install a new sensor, and therefore the introduction cost of the system can be reduced.

[0036] The weather data includes a determination target point (or an area including the determination target point) and a plurality of meteorological elements (e.g., temperature, amount of precipitation, amount of snowfall, etc.) in a determination target period from a time before the determination target time to the determination target time. The weather data is acquired from the weather database 402, for example, by referring to meteorological mesh information accumulated in the meteorological database 402, and using as extraction conditions the determination target point based on position data acquired in accordance with the time when the image data was captured, and the determination target period based on the time when the image data was captured. Note that the weather data may be acquired directly from the weather information providing device 3.

[0037] The position data is data indicating the current position (target point) of the vehicle 10 at the target time. When the image data is a moving image, the position data records the travel path, and the recording period of the travel path is preferably the same time period as the shooting period when the moving image was shot.

[0038] The environmental sound data is data recorded by the microphone 262 while the vehicle 10 is traveling at a target location and target time, and is time-series data having a predetermined recording period including before and after the target time. Note that, when the image data is a moving image, it is preferable that the recording period of the environmental sound data is the same time period as the shooting period when the moving image was shot.

[0039] The acceleration data is data recorded by the acceleration sensor 263 while the vehicle 10 is traveling at the determination target point and determination target time, and is time-series data having a predetermined recording period including before and after the determination target time. Note that, when the image data is a moving image, the recording period of the acceleration data is preferably the same time period as the shooting period during which the moving image was shot.

[0040] For example, the data acquisition unit 410 receives image data to be judged and position data, environmental sound data, and acceleration data as correction data from the terminal device 2, and registers them in the road condition database 401. The data acquisition unit 410 also receives weather mesh information from the weather information providing device 3, and registers it in the weather database 402. Then, the data acquisition unit 410 acquires weather data as correction data by referring to the weather database 402 using, as an extraction condition, a judgment target point (= position data) and a judgment target period for the image data to be judged received from the terminal device 2. As a result, the data acquisition unit 410 acquires road data including the image data to be judged and correction data (weather data, position data, environmental sound data, and acceleration data).

[0041] The determination unit 411 determines the road conditions of the road to be determined at the determination point and determination time when the image data to be determined included in the road data was captured, based on the road data acquired by the data acquisition unit 410. Then, the determination unit 411 registers the determination result of the road conditions in the road condition database 401, whereby the road condition determination results are accumulated.

[0042] In this embodiment, the determination unit 411 uses the machine-learned learning models 403A to 403D (FIGS. 10 to 13) to determine the snow damage state, which is the occurrence state of snow damage, as the road condition of the road to be determined. Specifically, the determination unit 411 uses the following as the determination items of the snow damage state: (A) The condition of the road surface of the road to be evaluated; (B) The condition of the lateral clearance of the road to be evaluated; (C) The state of snow banks formed on the road to be assessed; (D) The visibility condition of the road subject to the judgment due to snowfall or snowstorm; That is, the determination unit 411 is a subject that executes the inference phase of machine learning, and inputs the image data to be determined to the learning models 403A to 403D, respectively, to determine the snow damage status for the above determination items (A) to (D), respectively.

[0043] The learning models 403A to 403D used in the judgment unit 411 correspond to the above judgment items (A) to (D), respectively, and are learned learning models that have been subjected to machine learning using a learning dataset. The learning dataset includes a plurality of learning data for performing supervised learning. The learning data is data used as training data, verification data, and test data in supervised learning. The learning model 403A is configured, for example, by a convolutional neural network (CNN).

[0044] As shown in Figs. 10 to 13, the learning models 403A to 403D perform machine learning using learning data consisting of image data of a learning target road and its surroundings (hereinafter referred to as "learning target road") photographed and a correct answer label indicating the judgment result of the snow damage situation of the learning target road (the learning target road and its surroundings). That is, in the learning phase of the machine learning, the judgment result of the snow damage situation (output data) output by inputting the learning target image data constituting the learning data as input data into the learning models 403A to 403D is compared with the judgment result of the snow damage situation (correct answer label) constituting the learning data, and the weighting parameters of the convolutional neural network are adjusted based on the comparison result, thereby performing machine learning. Note that the learning target image data may be subjected to a predetermined image adjustment process (for example, image format, image size, image filter, image mask, etc.). In addition, data expansion process (rotation, shift, inversion, shear transformation, etc.) for generating multiple image data from the learning target image data may be performed. This makes it possible to uniformize the learning data set.

[0045] The snow damage situation output as output data by the first learning model 403A is a classification result of the road surface classification when the "road surface condition" which is the judgment item (A) is classified into a plurality of road surface classifications. The plurality of road surface classifications include at least a plurality of stages of wet snow depths which classify the depth of wet snow in stages, and a plurality of stages of dry snow depths which classify the depth of dry snow in stages. In this embodiment, the road surface classification is defined by 17 classes consisting of dry, wet, puddle, flooded, 5 stages of wet snow depth, frozen, 5 stages of dry snow depth, compacted snow, and light snow, as shown in FIG. 5 and FIG. 10. The first learning model 403A functions as a multi-class (17 classes in this embodiment) classifier, and outputs the accuracy when the road surface condition is classified into each of the classes of the plurality of road surface classifications for each class. The accuracy for each class is output as a numerical value in a predetermined range (for example, 0 to 1). The classification method and number of classifications of road surface sections are not limited to the 17 types shown in the examples of FIGS.

[0046] The snow damage situation output as output data by the second learning model 403B is the classification result of the side margin width when the "side margin condition" which is the judgment item (B) is classified into multiple stages of side margin width. In this embodiment, the side margin condition is defined by four stages of side margin width and five classes consisting of non-judgment targets, as shown in Figs. 6 and 11. The second learning model 403B functions as a multi-class (five classes in this embodiment) classifier like the first learning model 403A, and outputs the accuracy when the side margin condition is classified into each class for each class. Note that the classification method and number of classifications of the side margin are not limited to the five-class example shown in Figs. 6 and 11.

[0047] The snow damage status output as output data by the third learning model 403C is the classification result of snow bank height when the judgment item (C) "snow bank status" is classified into multiple stages of snow bank height. In this embodiment, the snow bank status is defined by four stages of snow bank height and five classes consisting of non-judgment targets, as shown in Figs. 7 and 12. The third learning model 403C, like the first learning model 403A, functions as a multi-class (five classes in this embodiment) classifier, and outputs the accuracy for each class when the snow bank status is classified into each class. Note that the classification method and number of classes of snow banks are not limited to the five-class example shown in Figs. 7 and 12.

[0048] The snow damage situation output as output data by the fourth learning model 403D is the classification result of the visibility distance when the judgment item (D) "visibility condition" is classified into multiple stages of visibility distance. In this embodiment, the visibility condition is defined as five classes consisting of five stages of visibility distance, as shown in Figures 8 and 13. The fourth learning model 403D functions as a multi-class (five classes in this embodiment) classifier, similar to the first learning model 403A, and outputs the accuracy when the visibility condition is classified into each class for each class. Note that the visibility classification method and the number of classes are not limited to the five-class example shown in Figures 8 and 13.

[0049] The determination unit 411 is an entity that executes the inference phase of machine learning, and as shown in FIG. 9, inputs image data of the road to be determined, in which the road to be determined is photographed, into the trained learning models 403A-403D, respectively, to determine the snow damage status for the above determination items (A)-(D). Then, the determination unit 411 outputs the classification results of the road surface classification, the side margin width, the snow bank height, and the visibility distance for the road to be determined as the determination results. Specifically, the determination unit 411 outputs the class with the highest accuracy among the accuracy for each class, for example, for each determination item (A)-(D), such as the road surface classification being "wet snow 1 cm-3 cm" (see FIG. 10), the side margin width being "narrow" (see FIG. 11), the snow bank height being "slightly high" (see FIG. 12), and the visibility distance being "slightly poor" (see FIG. 13). In this case, the judgment unit 411 may output the accuracy for each class as is, or instead of or in addition to the accuracy for each class, it may output the accuracy ranking when the accuracy for each class is sorted in descending order of value.

[0050] In addition, the judgment unit 411 may perform a predetermined image adjustment process (e.g., image format, image size, image filter, image mask, etc.) on the image data to be judged as a preprocessing when inputting the image data to be judged to the learning models 403A to 403D.

[0051] Furthermore, when determining the side margin width and snow bank height, for example, the determination unit 411 may determine the snow damage condition on the left side of the road to be determined and the snow damage condition on the right side of the road to be determined. On a road with a median strip, the snow damage condition on the right side of the road to be determined is determined by the side margin width and snow bank height on the median strip side. On a road without a median strip, the snow damage condition on the right side of the road to be determined is determined by the side margin width and snow bank height on the side beyond the oncoming lane.

[0052] In this case, the determination unit 411 cuts out the left image area and the right image area from the image data to be determined to generate the left image data and the right image data. The determination unit 411 inputs one side of the left image data and the right image data to the learning models 403B and 403C to determine the snow damage situation (side margin width and snow bank height) on one side of the road to be determined, and inputs the other side of the left image data and the right image data in a state of being inverted in the left-right direction to the learning models 403B and 403C to determine the snow damage situation (side margin width and snow bank height) on the other side of the road to be determined. The learning models 403B and 403C used here are machine-learned by a learning dataset including a plurality of learning data consisting of image data on one side of the road to be determined and a correct answer label indicating the determination result of the snow damage situation on one side of the road to be determined. As a result, for example, the left side image data is input as is to the learning models 403B and 403C for the left side, which have been machine-learned using learning data composed of image data including the image area of ​​the left side of the road, and the right side image data is input after being inverted in the left-right direction, so that the snow damage situation can be determined not only on the left side of the road but also on the right side of the road using the learning models 403B and 403C for the left side. In other words, the snow damage situation can be determined separately for the left and right sides of the road using the learning models 403B and 403C common to the left and right sides of the road. Note that the snow damage situation may also be determined separately for the left and right sides of the road for determination items other than the lateral margin width and snow bank height. In addition, the image data of the learning target constituting the learning data may be image data in which the image area on the other side of the learning target road is inverted in the left-right direction.

[0053] Furthermore, as a correction process of the snow damage condition, the determination unit 411 may determine the possibility of occurrence of the snow damage condition based on the correction data included in the road data, and correct the determination result of the snow damage condition (road condition) by the learning models 403A to 403D based on the possibility of occurrence. The following describes the case where the determination unit 411 corrects the classification result of the road surface division.

[0054] In a first example, when weather data is acquired as correction data included in the road data, the determination unit 411 analyzes the weather data to determine the occurrence possibility of each road surface classification. For example, when the temperature at the determination target point and determination target time is 5°C or higher, it determines that there is no possibility of the occurrence of the "frozen" class of road surface classification. Then, the determination unit 411 outputs the road surface classification with the highest probability among the probabilities of each remaining road surface classification excluding "frozen" as the road surface condition at the determination target point and determination target time.

[0055] In the second example, when position data is acquired as correction data included in road data, the determination unit 411 analyzes whether the position data is in a tunnel to determine the presence or absence of rainfall and snowfall, and determines the occurrence possibility of each road surface classification according to the presence or absence of rainfall and snowfall. For example, when the position data is in a tunnel, the determination unit 411 determines that the occurrence possibility of the road surface classifications of "wet snow 0-1 cm", "wet snow 1-3 cm", "wet snow 3-5 cm", "wet snow 5-10 cm", "wet snow 10 cm or more", "dry snow 0-1 cm", "dry snow 1-3 cm", "dry snow 3-5 cm", "dry snow 5-10 cm", "dry snow 10 cm or more", and "packed snow" is not possible. Then, the determination unit 411 outputs the road surface classification with the highest probability among the remaining road surface classifications as the road surface condition at the determination target point and determination target time.

[0056] In a third example, when environmental sound data is acquired as correction data included in road data, the determination unit 411 analyzes the environmental sound data to determine the presence or absence of splashing water, and determines the possibility of occurrence of each road surface classification according to the presence or absence of splashing water. For example, when the sound of splashing water is detected, the determination unit 411 determines that there is no possibility of occurrence of the road surface classifications of "dry", "wet", "frozen", and "packed snow". Then, the determination unit 411 outputs the road surface classification with the highest probability among the probabilities of each of the remaining road surface classifications as the road surface condition at the determination target point and determination target time.

[0057] In a fourth example, when acceleration data is acquired as correction data included in road data, the determination unit 411 detects the presence or absence of a slip component by analyzing the acceleration data, and determines the presence or absence of the possibility of occurrence of each road surface classification according to the presence or absence of the slip component. For example, when a slip component is detected in the acceleration data, the determination unit 411 determines that the occurrence of the road surface classifications "dry", "wet", "puddle", and "flooded" is not possible. Then, the determination unit 411 outputs the road surface classification with the highest probability among the remaining road surface classifications as the road surface condition at the determination target point and determination target time.

[0058] In addition, when the data acquisition unit 410 acquires some of the weather data, the position data, the environmental sound data, and the acceleration data as the correction data, the determination unit 411 may perform the correction process based on the part of the data acquired by the data acquisition unit 410. In addition, the determination unit 411 may not perform the above correction process, and in that case, the road data acquired by the data acquisition unit 410 may not include the correction data.

[0059] The information providing unit 412 generates road condition determination information indicating the result of the road condition (snow damage condition) determined by the determination unit 411 for the image data to be determined, and provides (transmits) the information to the terminal device 2 that captured the image data to be determined.

[0060] Furthermore, the information providing unit 412 tally up the road condition (snow damage condition) determination results determined by the determining unit 411 for the image data to be determined for each road section or time section. A road section is, for example, each road section when a road is divided into a predetermined distance (e.g., 50 m). A time section is, for example, each time section when time is divided into a predetermined division interval (e.g., 1 minute or 1 hour). The information providing unit 412 then generates road condition tally information for superimposing and displaying the tally results of the road condition (snow damage condition) on the road map on the display screen for each road section or time section, and provides the information to the terminal device 2, the administrator device 5, etc.

[0061] For example, the information providing unit 412 may tally up the determination results of the road conditions (snow damage conditions) for each road section or each time section by referring to the road condition database 401. Therefore, when image data collected by the running of multiple vehicles 10 is registered in the road condition database 401, the road condition tally information is generated by tallying up the determination results of the road conditions for the collected image data. At that time, when multiple road condition determination results are registered for a specific road section, the final road condition determination result may be determined by majority vote, or the determination result at the latest time may be prioritized. Also, when multiple road condition determination results are registered for a specific time section, the final road condition determination result may be determined by majority vote, or the determination result at the latest time may be prioritized. Furthermore, the information providing unit 412 may tally up the road condition determination results separately for the left and right sides of the road, or may tally up by traffic direction or by lane. The information providing unit 412 may provide the road condition tally information in response to an output request from the terminal device 2, the administrator device 5, or the like, or may provide the road condition tally information at a predetermined cycle.

[0062] Furthermore, when the information providing unit 412 receives the designation of a specific point or specific time on the road where the tally result is superimposed on the road map on the display screen, the information providing unit 412 generates image display information for displaying on the display screen image data of the determination target photographed at the determination target point or determination time corresponding to the specific point or specific time, and provides the information to the terminal device 2, the administrator device 5, etc. At that time, the information providing unit 412 may provide image display information for displaying on the display screen image data of the determination targets in chronological order or in reverse chronological order based on the image data of the determination target photographed at the determination target point corresponding to the specific point or at the determination target time corresponding to the specific time.

[0063] The information providing unit 412 also provides additional display information for superimposing additional information related to the determination point and the determination time on the road map. The additional information is, for example, weather mesh information including meteorological elements for each area divided into a mesh, or disaster mesh information including disaster risk for each area divided into a mesh. The additional information may be any information that can be superimposed and displayed on the road map, and is not limited to the weather mesh information and disaster mesh information described above.

[0064] The various information provided by the information providing unit 412 is information that can be displayed by the road condition display program 201 executed on the terminal device 2 and the road condition display program 500 executed on the administrator device 5, but may also be generated in a format that can be displayed by any program such as a web browser.

[0065] (Configuration of Administrator Device 5) 14 is a block diagram showing an example of the administrator device 5. The administrator device 5 includes a storage unit 50 including an HDD, SSD, memory, etc., a control unit 51 including a processor such as a CPU, GPU, MPU, etc., an input unit 52 including a keyboard, mouse, touch panel, etc., a display unit 53 including a display, touch panel, etc., a communication unit 54 which is an interface with the network 6 based on a predetermined communication standard (which may be either wired or wireless), an external device interface (I / F) unit 55 which is an interface with external devices such as a printer, scanner, USB memory, etc., and a media input / output unit 56 which is an interface with storage media such as a CD, DVD, etc. The external device I / F unit 55 and the media input / output unit 56 may be omitted as appropriate.

[0066] The storage unit 50 stores an operating system (OS) which is a basic program, a road condition display program 500, various application programs such as a web browser, and various data used by these programs. Although the various data are basically stored in the storage unit 50, these programs and data may be obtained from an external storage device via the communication unit 54 or the external device I / F unit 55, or may be obtained from a storage medium via the media input / output unit 56, and may be updated as appropriate.

[0067] The control unit 51 executes the road condition display program 500 to function as a road condition display processing unit 510 .

[0068] When the road condition display processing unit 510 receives the road condition judgment information and the road condition summary information from the road condition judgment device 4, it displays on the display unit 53 a display screen based on the road condition judgment information and the road condition summary information.

[0069] (Operation of road condition judgment system) 15 to 17 are flowcharts showing an example of the operation of the road condition determination system 1. The following describes the operation when the terminal device 2 installed on a fixed stand of the vehicle 10 records sensor data (position data, image data, environmental sound data, and acceleration data) at a predetermined recording interval while the vehicle 10 is traveling on a road, and transmits the sensor data to the road condition determination device 4 at a predetermined transmission interval.

[0070] First, in step S100, each time a predetermined transmission interval arrives, the sensor data recording processing unit 210 of the terminal device 2 transmits sensor data (position data, image data, environmental sound data, and acceleration data) at that time (target determination time) to the road condition determination device 4. When the recording interval of the sensor data is shorter than the transmission interval of the sensor data, the terminal device 2 may transmit the sensor data for a plurality of times (a plurality of target determination times) together.

[0071] Next, in step S101, the data acquisition unit 410 of the road condition determination device 4 receives the sensor data (position data, image data, environmental sound data, and acceleration data) transmitted in step S100, thereby acquiring the image data to be determined, and the position data, environmental sound data, and acceleration data as correction data. Then, the data acquisition unit 410 registers these data in the road condition database 401.

[0072] In step S110, the weather information providing device 3 transmits weather mesh information to the road condition determining device 4. Next, in step S111, the data acquiring unit 410 receives the weather mesh information transmitted in step S110 and registers it in the weather database 402. Then, in step S112, the data acquiring unit 410 acquires weather data as correction data by referring to the weather database 402 using as extraction conditions the determination target point based on the position data received in step S101 and the determination target period based on the shooting time when the image data received in step S101 was shot. As a result of the above steps S101 and S112, in step S120, the data acquiring unit 410 acquires road data including the image data to be determined and the weather data, position data, environmental sound data, and acceleration data as correction data.

[0073] Next, in step S130 (S131 to S135), the judgment unit 411 judges the snow damage situation of the road to be judged at the location (target location) and time (target time) when the image data to be judged, which is included in the road data acquired in step S120, was photographed, based on the road data acquired in step S120, and registers the snow damage situation in the road condition database 401.

[0074] Specifically, in step S131, the determination unit 411 determines the road surface condition by inputting the image data of the determination target into the first learning model 403A, and outputs a classification result of the road surface division. In step S132, the determination unit 411 determines the side margin condition by inputting the image data of the determination target into the second learning model 403B, and outputs a classification result of the side margin width. In step S133, the determination unit 411 determines the snow bank condition by inputting the image data of the determination target into the third learning model 403C, and outputs a classification result of the snow bank height. In step S134, the determination unit 411 determines the visibility condition by inputting the image data of the determination target into the fourth learning model 403D, and outputs a classification result of the visibility distance.

[0075] Next, in step S135, the determination unit 411 determines whether or not a snow damage situation is likely to occur based on the correction data (weather data, position data, environmental sound data, and acceleration data), and corrects the determination results of the snow damage situation determined in steps S131 to S134 based on the presence or absence of the possibility of occurrence. Note that step S135 may be omitted, and in that case, the process of acquiring the correction data (steps S101, S111, S112, etc.) may also be omitted.

[0076] Next, in step S140, the information providing unit 412 transmits road condition determination information indicating the determination result of the snow damage condition determined in step S130 to the terminal device 2 which is the transmission source of the sensor data received in step S101. Then, in step S141, the road condition display processing unit 211 of the terminal device 2 receives the road condition determination information transmitted in step S140, and displays the determination result display screen 11 (FIG. 18) based on the road condition determination information.

[0077] 18 is a diagram showing an example of the judgment result display screen 11. The judgment result display screen 11 includes an image display area 110 for displaying image data of the judgment target, and a judgment result display area 111 for displaying the judgment result of the snow damage condition for the image data of the judgment target based on the road condition judgment information.

[0078] 18, the judgment result display area 111 displays the judgment results when the road surface classification is judged as "wet snow 0 cm to 1 cm", the side margin width is "slightly narrow", the snow bank height is "low", and the visibility distance is "good". Note that the judgment result display screen 11 may additionally display information based on the judgment result of the snow damage situation, and for example, a warning message or icon that calls the driver's attention may be displayed.

[0079] In step S150, the information providing unit 412 repeatedly performs steps S110 to S130 in response to sequentially receiving image data and the like transmitted at a predetermined transmission interval from the terminal device 2 (or multiple terminal devices 2), thereby tallying up the snow damage situation judgment results accumulated in the road condition database 401 for each road section and each time section. At this time, the road data extracted from the road condition database 401 as the target of tallying is extracted based on, for example, a driver condition for the driver, a region condition for the location, and a date and time condition for the time. Then, based on the tallying results, road condition tallying information is generated and transmitted to the terminal device 2 and the administrator device 5.

[0080] Next, in step S151, the road condition display processing unit 211 of the terminal device 2 receives the road condition tally information transmitted in step S150, and displays the tally result display screen 12 based on the road condition tally information. Also, in step S152, the road condition display processing unit 510 of the administrator device 5 receives the road condition tally information transmitted in step S150, and displays the tally result display screen 12 (FIG. 19) based on the road condition tally information. The following mainly describes a case where the tally result display screen 12 is displayed by the administrator device 5 and various input operations are performed by the road management company, but the operation is similar when the tally result display screen 12 is displayed on the terminal device 2.

[0081] 19 is a diagram showing an example of the tally result display screen 12. The tally result display screen 12 includes a tally condition specification area 120 for specifying extraction conditions for road data to be tallyed, a map display area 121 for displaying the tally result of snow damage conditions superimposed on a road map, a judgment item specification area 122 for specifying judgment items for the snow damage conditions to be displayed in the map display area 121, and a legend display area 123 for displaying a legend showing the display mode of the snow damage conditions to be displayed in the map display area 121. The tally result display screen 12 may include an input interface (e.g., a slider) for displaying the change over time in the tally result of the snow damage conditions.

[0082] In the tally condition specification area 120, an input interface is arranged that allows specification of, for example, a driver condition for specifying all drivers or an individual driver, and a date and time condition for specifying a date and time range. The tally result display screen 12 shown in FIG. 19 is displayed when "all drivers" is specified as the driver condition, and "February 3, 2023, 10:00" to "February 3, 2023, 11:00" is specified as the date and time condition. Note that the date and time condition may be specified based on the current time.

[0083] In the map display area 121, a road map is displayed, and the results of snow damage status compilation for each road displayed on the road map with respect to the judgment item specified in the judgment item specification area 122 (side margin width in FIG. 19) are superimposed and displayed according to the legend displayed in the legend display area 123. Note that in the map display area 121, the results of snow damage status compilation for multiple judgment items, such as road surface classification and snow bank height, may be displayed simultaneously.

[0084] The map display area 121 is also provided with an input interface 121a for moving and enlarging / reducing the road map, and is configured to be able to change the display range of the road map displayed in the map display area 121. When the display range of the map display area 121 is changed, it is treated as if a region condition for the point has been specified as an extraction condition for road data to be counted. Therefore, when the driver condition or the date and time condition in the counting condition specification area 120 is changed, or when the display range (region condition) of the map display area 121 is changed, the information providing unit 412 retransmits road condition counting information according to the changed extraction condition, thereby updating the counting result display screen 12. When the information providing unit 412 transmits additional display information, the road condition display processing unit 211 may display weather mesh information and disaster mesh information superimposed on the road map displayed in the map display area 121 based on the additional display information.

[0085] Then, in step S160, when the road condition display processing unit 510 of the administrator device 5 receives an input operation to designate a specific point on the road on which the tabulation result of the snow damage situation is superimposed on the tabulation result display screen 12 displayed in step S152, the road condition display processing unit 510 transmits specific point information indicating the specific point to the road condition determination device 4. On the tabulation result display screen 12, for example, the designation of the specific point is received by placing the cursor 121b on an arbitrary point on the road and clicking it.

[0086] Next, in step S161, the information providing unit 412 receives the specific location information transmitted in step S160, and when it accepts the designation of a specific location based on the specific reception information, transmits to the administrator device 5 image display information for displaying on a display screen image data of the determination target taken at a location corresponding to the specific location. In this embodiment, the information providing unit 412 transmits to the administrator device 5 image display information for displaying on a display screen image data of multiple determination targets in chronological order or reverse chronological order based on the image data of the determination target taken at a location corresponding to the specific location.

[0087] Next, in step S162, the road condition display processing unit 510 of the administrator device 5 receives the image display information transmitted in step S161, and displays the image display screen 13 (FIG. 20) based on the image display information.

[0088] 20 is a diagram showing an example of the image display screen 13. The image display screen 13 includes an image display area 130 that displays image data of the object to be judged, a compilation list area 131 that displays in list form a list of road sections or time sections when snow damage conditions are compiled, an image designation area 132 that designates changing the order of the image data of the object to be judged displayed in the image display area 130 to chronological order or reverse chronological order, a compilation result display area 133 that displays the compilation result of the snow damage conditions in the road section or time section including the image data of the object to be judged displayed in the image display area 130, and a details confirmation button 134 for confirming the details of the judgment result of the snow damage conditions for the image data of the object to be judged.

[0089] In the initial state, image data to be judged as a reference is displayed in the image display area 130. The image display screen 13 shown in Fig. 20 is displayed when image data captured at one-second intervals is collected every minute (every time period), that is, when 60 pieces of image data are collected every minute, and the image data captured at 10:03:10 is displayed in the image display area 130.

[0090] When an input operation for selecting a specific time by using the selection frame 131a is received in the tally list area 131, image data taken at the specific time is displayed in the image display area 130. When an input operation for the play button, reverse play button, or slider 132a is received in the image designation area 132, a plurality of image data are displayed in chronological order or reverse chronological order in response to the input operation in the image display area 130. Therefore, when an input operation for the tally list area 131 or the image designation area 132 is received, the information providing unit 412 retransmits image display information in response to the input operation, thereby updating the image display screen 13. This allows a road management company (or a driver) to simply designate a specific point on a road map to check the snow damage situation at the specific point by image, and also check the snow damage situation on the roads before and after the point by image.

[0091] The counting result display area 133 displays the counting result of the snow damage situation in a specific road section or time section (1 minute in FIG. 20). The image display screen 13 shown in FIG. 20 is displayed when, as a classification result of the side margin width for 60 pieces of image data, "52 pieces" of image data are determined to have a side margin width of "slightly narrow", "5 pieces" of image data are determined to have a side margin width of "narrow", and "3 pieces" of image data are determined to have a side margin width of "wide". In this case, when a majority vote is used as the counting result for the road section or time section, the counting result of the side margin width for the road section or time section in which 60 pieces of image data are captured is determined to be "slightly narrow" as shown by the thick frame line in FIG. 20. Such counting results are obtained for each road section or time section, and the counting result of the snow damage situation is displayed for each road section or time section on the counting result display screen 12 shown in FIG. 19.

[0092] Then, in step S170, when the road condition display processing unit 510 of the administrator device 5 accepts an input operation of pressing the details confirmation button 134 on the image display screen 13 displayed in step S162, it transmits specific time information indicating the specific time selected in the selection box 131a at that time to the road condition determination device 4.

[0093] Next, in step S171, the information providing unit 412 receives the specific time information transmitted in step S170, and when it accepts the designation of a specific time based on the specific time information, transmits to the administrator device 5 image display information for displaying on a display screen the image data of the determination target photographed at the time corresponding to the specific time. In this embodiment, the information providing unit 412 transmits to the administrator device 5 image display information for displaying on a display screen the image data of the determination target in chronological order or reverse chronological order based on the image data of the determination target photographed at the time corresponding to the specific time.

[0094] Next, in step S172, the road condition display processing unit 510 of the administrator device 5 receives the image display information transmitted in step S171, and displays the detail display screen 14 (FIG. 20) based on the image display information.

[0095] 21 is a diagram showing an example of the detailed display screen 14. The detailed display screen 14 includes a detailed result list area 140 in which the image data to be judged and the judgment results of the snow damage situation for the image data are arranged in chronological order or reverse chronological order in a list format.

[0096] The detailed display screen 14 shown in FIG. 21 is displayed when "10:03:00" is designated as the specific time using the selection frame 131a as shown in FIG.

[0097] The series of operations described above is carried out mainly by the road condition determination device 4, whereby the road conditions of the roads at each location and each time are determined, and the road condition determination results and compilation results are provided to the terminal device 2 and the administrator device 5.

[0098] Therefore, according to the road condition determination device 4 and the road condition determination method of this embodiment, the determination unit 411 determines the snow damage condition of the road to be determined by inputting image data of the road to be determined to the learning models 403A to 403D. Therefore, since it is only necessary to obtain image data of the road to be determined, it is possible to determine the snow damage condition at any point with a simple device configuration.

[0099] (Other embodiments) The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit and scope of the present invention. All such modifications are included in the technical concept of the present invention.

[0100] (Modification of judgment items) In the above embodiment, the determination unit 411 uses the trained learning models 403A to 403D to determine the snow damage status of the road to be determined for the four determination items (A) to (D). (E) The number of lanes available on the road to be assessed; (F) The likelihood of stacks occurring on the road being assessed; (G) The condition of packed snow on the road to be evaluated; (H) The condition of snow accumulated on the roofs of buildings along the road to be assessed; (I) The condition of snow cornices formed on the roofs of buildings along the road to be assessed; (J) The status of fallen trees on the road to be assessed; (K) The rainfall or snowfall conditions on the road to be evaluated; In this case, the determining unit 411 may determine the snow damage state of the road to be determined for at least one of the determination items (A) to (K).

[0101] Fig. 22 is a diagram showing an example of the number of passable lanes. Fig. 23 is a diagram showing an example of the degree of caution for getting stuck. Fig. 24 is a diagram showing an example of packed snow thickness. Fig. 25 is a diagram showing an example of roof snow depth. Fig. 26 is a diagram showing an example of the degree of caution for snow cornices. Fig. 27 is a diagram showing an example of the degree of caution for fallen trees. Fig. 28 is a diagram showing an example of rain / snow classification.

[0102] FIG. 29 is a functional explanatory diagram showing a modified example of the road condition determination device 4. FIG. 30 is a schematic diagram showing an example of the fifth learning model 403E. FIG. 31 is a schematic diagram showing an example of the sixth learning model 403F. FIG. 32 is a schematic diagram showing an example of the seventh learning model 403G. FIG. 33 is a schematic diagram showing an example of the eighth learning model 403H. FIG. 34 is a schematic diagram showing an example of the ninth learning model 403I. FIG. 35 is a schematic diagram showing an example of the tenth learning model 403J. FIG. 36 is a schematic diagram showing an example of the eleventh learning model 403K.

[0103] In the road condition judgment device 4 shown in FIG. 29, the judgment unit 411 judges the snow damage conditions for the judgment items (E) to (K) respectively by inputting the image data to be judged to the learning models 403E to 403K, respectively, in the same manner as in the above embodiment. The judgment results of the snow damage conditions for the judgment items (E) to (K) are defined respectively as shown in FIG. 22 to FIG. 28, and are registered in each field of the road condition database 401 (FIG. 4) in the same manner as the judgment items (A) to (D). Furthermore, the judgment results of the snow damage conditions for the judgment items (E) to (K) are displayed on the judgment result display screen 11 (FIG. 18), the counting result display screen 12 (FIG. 19), the image display screen 13 (FIG. 20), and the details display screen 14 (FIG. 21) in the same manner as the judgment items (A) to (D). At that time, for example, The configurations and processing contents of the data acquisition unit 410 and the information provision unit 412 are the same as those in the above embodiment, so the following description will mainly focus on the learning models 403E to 403 and the determination unit 411.

[0104] The learning models 403E to 403K used in the judgment unit 411 correspond to the judgment items (E) to (K) described above, respectively, and, like the learning models 403A to 403D, are stored in the storage unit 40 as trained learning models that have undergone machine learning using the learning data sets shown in Figures 30 to 36. Note that the configurations and machine learning techniques of the learning models 403E to 403K are similar to those of the learning models 403A to 403D according to the above embodiment, and therefore will not be described.

[0105] The snow damage situation output as output data by the fifth learning model 403E is a classification result of the number of passable lanes when the judgment item (E) "the situation of the number of passable lanes of the road" is classified into multiple stages of the number of passable lanes. In the example of FIG. 22 and FIG. 30, the situation of the number of passable lanes is defined by 6 stages of the number of passable lanes and 7 classes consisting of non-judgment targets. The fifth learning model 403E functions as a multi-class (7 classes in this embodiment) classifier and outputs the accuracy when the situation of the number of passable lanes of the road is classified into each class for each class. Note that the classification method and the number of classifications of the number of passable lanes are not limited to the 7-class example shown in FIG. 22 and FIG. 30.

[0106] The snow damage situation output as output data by the sixth learning model 403F is a classification result of the stack attention degree when the judgment item (F) "situation of likelihood of getting stuck" is classified into multiple stages of stack attention degree. The situation of likelihood of getting stuck varies depending on, for example, the type and weight of the vehicle. In addition, the situation of likelihood of getting stuck varies depending on whether or not depressions are formed, whether or not the surface is crunchy, and the degree of depression or crunchy, when snow or ice on the road melts or is crushed by the passage of vehicles. In the example of FIG. 23 and FIG. 31, the situation of likelihood of getting stuck is defined by four stages of stack attention degree and five classes consisting of non-judgment targets. The sixth learning model 403F functions as a multi-class (five classes in this embodiment) classifier, and outputs the accuracy when the situation of likelihood of getting stuck is classified into each class for each class. Note that the classification method and number of classifications of stack attention degree are not limited to the five-class example shown in FIG. 23 and FIG. 31.

[0107] The snow damage condition output as output data by the seventh learning model 403G is the classification result of the packed snow thickness when the judgment item (G) "packed snow condition" is classified into multiple levels of packed snow thickness. In the example of Figures 24 and 32, the packed snow condition is defined by four levels of packed snow thickness and five classes consisting of non-judgment targets. The seventh learning model 403G functions as a multi-class (five classes in this embodiment) classifier, and outputs the accuracy for each class when the packed snow condition is classified into each class. Note that the classification method and number of classes of packed snow thickness are not limited to the five-class example shown in Figures 24 and 32.

[0108] The snow damage situation output as output data by the eighth learning model 403H is the classification result of roof snow depth when the judgment item (H) "roof snow situation" is classified into multiple levels of roof snow depth. In the example of Fig. 25 and Fig. 33, the roof snow situation is defined by 6 classes consisting of 5 levels of roof snow depth and non-judgment target. The eighth learning model 403H functions as a multi-class (6 classes in this embodiment) classifier, and outputs the accuracy when the roof snow situation is classified into each class for each class. Note that the classification method and number of classifications of roof snow depth are not limited to the 6-class example shown in Fig. 25 and Fig. 33.

[0109] The snow damage situation output as output data by the ninth learning model 403I is the classification result of the snow cornice warning level when the judgment item (I) "snow cornice situation" is classified into multiple levels of snow cornice warning level. In the example of Fig. 26 and Fig. 34, the snow cornice situation is defined by four levels of snow cornice warning level and five classes consisting of non-judgment target. The ninth learning model 403I functions as a multi-class (five classes in this embodiment) classifier, and outputs the probability for each class when the snow cornice situation is classified into each class. Note that the classification method and number of classes of snow cornice warning level are not limited to the five-class example shown in Fig. 26 and Fig. 34.

[0110] The snow damage situation output as output data by the tenth learning model 403J is the classification result of the degree of caution for fallen trees when the judgment item (J) "fallen tree situation" is classified into multiple levels of fallen tree caution. In the example of Fig. 27 and Fig. 35, the fallen tree situation is defined by four levels of fallen tree caution and five classes consisting of non-judgment targets. The tenth learning model 403J functions as a multi-class (five classes in this embodiment) classifier, and outputs the accuracy for each class when the fallen tree situation is classified into each class. Note that the classification method and number of classes for the degree of caution for fallen trees are not limited to the five-class example shown in Fig. 27 and Fig. 35.

[0111] The snow damage situation output as output data by the eleventh learning model 403K is a classification result of rain / snow classification when the judgment item (K) "rainfall or snowfall situation" is classified into multiple rain / snow classifications. In the example of FIG. 28 and FIG. 36, the rain / snow situation is defined by 7 rain / snow classifications and 8 classes that are not judged. The eleventh learning model 403K functions as a multi-class (8 classes in this embodiment) classifier, and outputs the accuracy when the rain / snow situation is classified into each class for each class. Note that the classification method and number of classifications of rain / snow classifications are not limited to the example of 8 classes shown in FIG. 28 and FIG. 36.

[0112] In addition, when the determination unit 411 determines, for example, the roof snow depth and the snow cornice attention level, as in the above embodiment, the left side image data and the right side image data are generated from the image data to be determined, and one side of the left side image data and the right side image data are input to the learning models 403H and 403I to determine the snow damage situation (roof snow depth and snow cornice attention level) on one side of the road to be determined, and the other side of the left side image data and the right side image data are input to the learning models 403H and 403I in a state of being inverted in the left-right direction to determine the snow damage situation (roof snow depth and snow cornice attention level) on the other side of the road to be determined. The learning models 403H and 403I used here are machine-learned by a learning dataset including a plurality of learning data consisting of image data on one side of the road to be learned and a correct answer label indicating the determination result of the snow damage situation on one side of the road to be learned. Furthermore, the determination unit 411 may determine the possibility of occurrence of snow damage conditions based on correction data included in the road data as a correction process for snow damage conditions, and correct the determination result of the snow damage conditions (road conditions) by the learning models 403E to 403K based on the possibility of occurrence. The other configurations and processing contents of the determination unit 411 are the same as those of the above embodiment, and therefore the description will be omitted.

[0113] Furthermore, the determination unit 411 may determine the snow damage status of the road to be determined for other determination items of the snow damage status, such as stuck vehicles, stranded vehicles, accident vehicles, snowdrifts, and avalanches. The determination unit 411 may determine the road status of the road to be determined, and may determine road status other than the snow damage status. Examples of the road status of the road to be determined include, but are not limited to, damage to the pavement caused by melting or freezing (e.g., potholes, etc.), flooding caused by flooding or tsunami (which may be flooding itself or traces of flooding), occurrence of earthquake disasters such as fissures and cracks, occurrence of landslides such as cliff collapses, mudslides, and landslides.

[0114] In the above embodiment, the data acquisition unit 410 of the road condition determination device 4 has been described as acquiring image data captured by the camera 261 (terminal device 2) on the vehicle 10. In contrast, the data acquisition unit 410 may acquire image data of the road to be determined, and may acquire image data captured by a device other than the camera 261 on the vehicle 10. For example, the data acquisition unit 410 may acquire image data captured by a road monitoring camera provided in a road monitoring device installed on the road, or may acquire image data captured by a drone camera provided in an unmanned aerial vehicle such as a drone. In this case, the data acquisition unit 410 may further acquire correction data detected by a sensor group provided in the road monitoring device or the unmanned aerial vehicle. The data acquisition unit 410 may also acquire image data captured by a bicycle or on foot using the terminal device 2 such as a smartphone.

[0115] In the above embodiment and the modified example of the judgment item, the case where a convolutional neural network is adopted as a specific method of machine learning using the learning models 403A to 403K has been described. In contrast, the learning models 403A to 403K may adopt any other machine learning method. Examples of other machine learning methods include tree types such as decision trees and regression trees, ensemble learning such as bagging and boosting, other neural net types (including deep learning) such as recurrent neural networks, clustering types such as hierarchical clustering, non-hierarchical clustering, k-nearest neighbors and k-means, multivariate analysis such as principal component analysis, factor analysis, and logistic regression, and support vector machines. In addition, the learning models 403A to 403K may be re-learned by batch learning or online learning. In that case, the learning data may be, for example, image data in which the judgment result of the road condition (snow damage condition) is corrected based on the correction data. The re-learning process may be performed by the road condition judgment device 4 or by another device.

[0116] In the above embodiment and the modified examples of the judgment items, the first to eleventh learning models 403A to 403K have been described as judging each of the judgment items (A) to (K) of the snow damage situation. In contrast, one learning model may judge multiple judgment items of the snow damage situation. Also, multiple learning models may be used for one judgment item, for example, two learning models that have been machine-learned separately for daytime and nighttime may be used, or two learning models that have been machine-learned separately for general roads and expressways may be used.

[0117] In the above embodiment and the modified example of the judgment item, the judgment unit 411 judges the snow damage condition of the judgment target road for eleven judgment items (A) to (K) using the trained learning models 403A to 403K. In contrast, the judgment unit 411 may judge the road condition of the judgment target road by performing image processing for detecting feature points (e.g., edges and corners) contained in the image data or image processing for detecting differences from image data captured under normal circumstances, instead of using the trained learning models 403A to 403K. In this case, the information providing unit 412 may provide various information based on the judgment result of the road condition judged by the judgment unit 411.

[0118] In the above embodiment and the modified example of the judgment item, the judgment unit 411 corrects the judgment result of the snow damage situation (road situation) by the learning models 403A to 403K with the correction data. In contrast, when there is a judgment item among the judgment items (A) to (K) of the snow damage situation for which the snow damage situation can be judged only by the correction data, the judgment unit 411 may judge the snow damage situation based on the correction data without inputting the image data of the judgment target to the learning model corresponding to the judgment item. For example, when the position data is acquired as the correction data included in the road data, the judgment unit 411 judges whether the position data is inside a tunnel. Then, when it is inside a tunnel, the judgment unit 411 may judge the side margin width as "wide" and the snow bank height as "none" for the judgment items (B) and (C) of the snow damage situation, respectively, without inputting the image data of the judgment target to the second and third learning models 403B and 403C. In addition, when inside a tunnel, the judgment unit 411 may judge the roof snow depth to be "none", the snow cornice warning level to be "none", and the fallen tree warning level to be "none" for the snow damage situation judgment items (H) to (J) without inputting the image data to be judged to the eighth, ninth, and tenth learning models 403H to 403J.

[0119] In the above embodiment, the control unit 41 of the road condition determination device 4 executes the road condition determination program 400 to function as the data acquisition unit 410, the determination unit 411, and the information provision unit 412. In contrast, the control unit of each device, such as the terminal device 2, the administrator device 5, the vehicle-mounted device, the vehicle (vehicle control device), the road monitoring device, or the unmanned aerial vehicle, may execute the road condition determination program 400 to function in the same manner as the road condition determination device 4, or may function as a part of the data acquisition unit 410, the determination unit 411, and the information provision unit 412. In this case, the road condition database 401, the weather database 402, and the learning model 403A may be stored in the storage unit of each device, or may be stored in an external device. In addition, the road condition determination program 400 may include all or a part of the functions realized by the sensor data recording program 200, the road condition display program 201, and the road condition display program 500.

[0120] In the above embodiment, each program (sensor data recording program 200, road condition display program 201, road condition judgment program 400, road condition display program 500) has been described as being stored in a storage unit of each device. However, each program may be provided by being recorded in a computer-readable storage medium such as a CD-ROM or DVD as a file in an installable or executable format. Furthermore, each program may be provided by being downloaded from an external device via the network 6. [Explanation of symbols]

[0121] 1...road condition determination system, 2...terminal device, 3...weather information providing device, 4...road condition determination device, 5...administrator device, 6...network, 10...vehicle, 11...judgment result display screen, 12...collection result display screen, 13...Image display screen, 14...Details display screen, 40: memory unit, 41: control unit, 42: input unit, 43: display unit, 44: communication unit, 45...external device I / F unit, 46...media input / output unit, 200: sensor data recording program; 201: road condition display program; 210: sensor data recording processing unit; 211: road condition display processing unit; 400...road condition judgment program, 401...road condition database, 403A...first learning model, 403B...second learning model, 403C…Third learning model, 403D…Fourth learning model, 403E…5th learning model, 403F…6th learning model, 403G: 7th learning model, 403H: 8th learning model, 403I…9th learning model, 403J…10th learning model, 403K…11th learning model, 410: data acquisition unit, 411: determination unit, 412: information providing unit, 500...road condition display program, 510...road condition display processing unit

Claims

1. a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model, The learning model is Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings, The snow damage situation is as follows: A classification result of the number of passable lanes when the situation of the number of passable lanes of the road is classified into a plurality of stages of the number of passable lanes. Road condition determination device.

2. a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model, The learning model is Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings, The snow damage situation is as follows: a classification result of the stack caution level when the situation of the road where the vehicle is likely to become stuck is classified into a plurality of stages of stack caution levels according to the type or weight of the vehicle; Road condition determination device.

3. a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model, The learning model is Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings, The snow damage situation is as follows: A classification result of the snow cornice warning level when the state of the snow cornice formed on the roof of the building along the road is classified into a plurality of stages of the snow cornice warning level. Road condition determination device.

4. a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model, The learning model is Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings, The snow damage situation is as follows: a classification result of the degree of caution for fallen trees when the state of fallen trees with respect to the road and its road boundary is classified into a plurality of levels of the degree of caution for fallen trees according to the extent to which the fallen trees exist with respect to the road and its road boundary; Road condition determination device.

5. a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model, The image data to be determined is an image of a front view of a vehicle traveling on a road to be determined is taken from the vehicle, The determination unit is A left image region and a right image region are cut out from the image data to be determined to generate left image data and right image data; The left side image data and the right side image data are input to the learning model to determine the snow damage situation on the one side of the road to be determined; The other side of the left side image data and the right side image data is input to the learning model in a state where the other side is inverted in the left-right direction, thereby determining the snow damage situation on the other side of the road to be determined; The learning model is Machine learning is performed using a learning dataset including a plurality of learning data sets each including image data of a learning target road and its surroundings, the image data being of the one side of the learning target road and a correct answer label indicating a determination result of the snow damage situation on the one side of the learning target road. Road condition determination device.

6. The snow damage situation is: a classification result of the road surface classification when the road surface conditions are classified into a plurality of road surface classifications; a classification result of the side margin width when the side margin condition of the road is classified into a plurality of stages of side margin width; a classification result of the snow bank height when the state of the snow bank formed on the road is classified into a plurality of stages of snow bank height; a classification result of the visibility distance when the visibility condition of the road due to snowfall or snowstorm is classified into a plurality of visibility distance stages; A classification result of the number of passable lanes when the situation of the number of passable lanes of the road is classified into a plurality of stages of the number of passable lanes; a classification result of the stack caution level when the state of the road where stacks are likely to occur is classified into a plurality of stack caution levels; a classification result of the packed snow thickness when the packed snow condition on the road is classified into a plurality of stages of packed snow thickness; a classification result of the roof snow depth when the state of roof snow accumulated on the roof of the building along the road is classified into a plurality of levels of roof snow depth; a classification result of the snow cornice warning level when the state of the snow cornice formed on the roof of the building along the road is classified into a plurality of stages of the snow cornice warning level; A classification result of the degree of caution for fallen trees when the situation of fallen trees on the road is classified into a plurality of levels of the degree of caution for fallen trees, or A classification result of the rain / snow classification when the rainfall or snowfall condition on the road is classified into a plurality of stages of rain / snow classifications. The road condition determining device according to claim 5.

7. an information providing unit that provides road condition summary information for tallying up the snow damage status judgment results determined by the judging unit for each road section or each time section, and superimposing and displaying the snow damage status summary results on a road map on a display screen; The information providing unit is When a designation of a specific point or a specific time on a road on which the tabulation results are superimposed on the road map is accepted, image display information is provided for displaying, on the display screen, in chronological order or in reverse chronological order, image data of the object to be determined that was taken while the object was traveling on the road before and after the point at which the image data of the object to be determined was taken, based on image data of the object to be determined that was taken at the point corresponding to the specific point or at the time corresponding to the specific time. The road condition determination device according to any one of claims 1 to 6.

8. A method for causing a computer to function as each unit included in the road condition determination device according to any one of claims 1 to 6, Road condition judgment program.

9. A data acquisition step of acquiring road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; A determination step of determining a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined, which is included in the road data acquired by the data acquisition step, into a learning model, The learning model is Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings, The snow damage situation is as follows: A classification result of the number of passable lanes when the situation of the number of passable lanes of the road is classified into a plurality of stages of the number of passable lanes. Method for determining road conditions.

10. A data acquisition step of acquiring road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; A determination step of determining a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined, which is included in the road data acquired by the data acquisition step, into a learning model, The learning model is Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings, The snow damage situation is as follows: a classification result of the stack caution level when the situation of the road where the vehicle is likely to become stuck is classified into a plurality of stages of stack caution levels according to the type or weight of the vehicle; Method for determining road conditions.

11. A data acquisition step of acquiring road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; A determination step of determining a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined, which is included in the road data acquired by the data acquisition step, into a learning model, The learning model is Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings, The snow damage situation is as follows: A classification result of the snow cornice warning level when the state of the snow cornice formed on the roof of the building along the road is classified into a plurality of stages of the snow cornice warning level. Method for determining road conditions.

12. A data acquisition step of acquiring road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; A determination step of determining a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined, which is included in the road data acquired by the data acquisition step, into a learning model, The learning model is Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings, The snow damage situation is as follows: a classification result of the degree of caution for fallen trees when the state of fallen trees with respect to the road and its road boundary is classified into a plurality of levels of the degree of caution for fallen trees according to the extent to which the fallen trees exist with respect to the road and its road boundary; Method for determining road conditions.

13. A data acquisition step of acquiring road data including image data of a road to be determined and its surroundings captured at a predetermined point and time; A determination step of determining a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined, which is included in the road data acquired by the data acquisition step, into a learning model, The image data to be determined is an image of a front view of a vehicle traveling on a road to be determined is taken from the vehicle, The determining step includes: A left image region and a right image region are cut out from the image data to be determined to generate left image data and right image data; The left side image data and the right side image data are input to the learning model to determine the snow damage situation on the one side of the road to be determined; The other side of the left side image data and the right side image data is input to the learning model in a state where the other side is inverted in the left-right direction, thereby determining the snow damage situation on the other side of the road to be determined; The learning model is Machine learning is performed using a learning dataset including a plurality of learning data sets each including image data of a learning target road and its surroundings, the image data being of the one side of the learning target road and a correct answer label indicating a determination result of the snow damage situation on the one side of the learning target road. Method for determining road conditions.

14. The snow damage situation is: a classification result of the road surface classification when the road surface conditions are classified into a plurality of road surface classifications; a classification result of the side margin width when the side margin condition of the road is classified into a plurality of stages of side margin width; a classification result of the snow bank height when the state of the snow bank formed on the road is classified into a plurality of stages of snow bank height; a classification result of the visibility distance when the visibility condition of the road due to snowfall or snowstorm is classified into a plurality of visibility distance stages; A classification result of the number of passable lanes when the situation of the number of passable lanes of the road is classified into a plurality of stages of the number of passable lanes; a classification result of the stack caution level when the state of the road where stacks are likely to occur is classified into a plurality of stack caution levels; a classification result of the packed snow thickness when the packed snow condition on the road is classified into a plurality of stages of packed snow thickness; a classification result of the roof snow depth when the state of roof snow accumulated on the roof of the building along the road is classified into a plurality of levels of roof snow depth; a classification result of the snow cornice warning level when the state of the snow cornice formed on the roof of the building along the road is classified into a plurality of stages of the snow cornice warning level; A classification result of the degree of caution for fallen trees when the situation of fallen trees on the road is classified into a plurality of levels of the degree of caution for fallen trees, or A classification result of the rain / snow classification when the rainfall or snowfall condition on the road is classified into a plurality of stages of rain / snow classifications. The method for determining road conditions according to claim 13.

15. an information providing step of providing road condition summary information for tallying up the snow damage status judgment results determined by the judgment step for each road section or each time section, and superimposing and displaying the snow damage status summary results on a road map on a display screen; The information providing step includes: When a designation of a specific point or a specific time on a road on which the tabulation results are superimposed on the road map is accepted, image display information is provided for displaying, on the display screen, in chronological order or in reverse chronological order, image data of the object to be determined that was taken while the object was traveling on the road before and after the point at which the image data of the object to be determined was taken, based on image data of the object to be determined that was taken at the point corresponding to the specific point or at the time corresponding to the specific time. The method for determining road conditions according to any one of claims 9 to 14.

Images