Information processing device, information processing method and information processing program

Abstract

To provide an occurrence risk degree calculation device, an occurrence risk degree display system, an occurrence risk degree calculation method and an occurrence risk degree calculation program which can improve service for managing a road.SOLUTION: An occurrence risk degree calculation device (100) comprises: an image analysis unit (110) which analyzes an image of a road; an environment information acquisition unit (120) which acquires environment information in the surroundings of the road; an occurrence risk degree calculation unit (130) which calculates an occurrence risk degree showing possibility that a damage factor which can be hindrance of passage of the road occurs on the basis of an analysis result obtained by being analyzed by the image analysis unit (110) and the environment information acquired by the environment information acquisition unit (120); and a display control unit (140) which associates the occurrence risk degree with map information showing position information of the road.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an occurrence risk calculation device, an occurrence risk display system, an occurrence risk calculation method, and an occurrence risk calculation program. [Background technology]

[0002] Patent Document 1 describes a server that detects road damage and assigns routes based on environmental information such as temperature, air pressure, humidity, rainfall, snowfall, wind direction, wind pressure, and images of the road surface.

[0003] Patent Document 2 describes a navigation device that provides a driver with a drawing that outputs the results of detection of road surface information, thereby allowing the driver to select an optimal route that avoids icy roads and snow-covered roads.

[0004] Patent Document 3 describes a method of preparing separate cameras for taking ultraviolet photos, infrared photos, and temperature distribution photos, and calculating the characteristic quantities of road surface conditions.

[0005] Patent Document 4 describes the use of a panoramic fisheye camera and a temperature sensor to calculate the likelihood of a road surface freezing.

[0006] Patent Document 5 describes a method in which wet and snowy images corresponding to the road surface condition to be detected are captured in advance for each weather condition that is a disturbance factor, such as sunny, cloudy, or nighttime, and these are stored as reference images. The road surface condition is determined by comparing the difference with a newly captured inspection image.

[0007] Non-Patent Document 1 describes displaying information on currently frozen roads.

[0008] Non-Patent Document 2 describes the conditions under which potholes are formed. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] International Publication No. 2017 / 111126 [Patent Document 2] Japanese Patent Application Publication No. 11-051682 [Patent Document 3] Patent No. 4814855 [Patent Document 4] JP 2009-042115 A [Patent Document 5] JP 2003-240869 A [Non-patent literature]

[0010] [Non-Patent Document 1] Ministry of Land, Infrastructure, Transport and Tourism, "Winter Road Information", [online], [searched February 15, 2021], Internet,<https: / / www.mlit.go.jp / road / fuyumichi / fuyumichi.html> [Non-Patent Document 2] Kimio Maruyama, Ryuji Abe, Takashi Kimura, “Actual state and mechanism of damage to pavement during snowmelt period”, [online], [Retrieved February 15, 2021], Internet,<https: / / www.mlit.go.jp / chosahokoku / giken / program / kadai / pdf / jusyo / H26 / anzen2_02.pdf> Summary of the Invention [Problem to be solved by the invention]

[0011] For example, fixed cameras that detect frozen roads only provide information about frozen areas at one location, such as mountain roads or near bridges. Information about areas not captured by fixed cameras is unknown until you go to the site. Therefore, in order to detect frozen areas not captured by such fixed cameras, dedicated cameras, dedicated sensors, dedicated vehicles, etc. are required, which are costly.

[0012] For example, holes in roads (hereafter referred to as potholes) are dangerous for vehicle traffic and therefore require repairs by the local government that manages the road. However, in reality, it is difficult to determine where potholes have occurred until complaints are received from residents, so they are managed as a post-event maintenance style. Ideally, management would be preventive, where potholes could be predicted before they occur and repairs could be made to the pavement before the road deteriorates due to potholes. This corresponds to the Ministry of Land, Infrastructure, Transport and Tourism's road management guidelines calling for a shift to preventive maintenance.

[0013] As in the examples above, there are expectations for the provision of a variety of services regarding road management.

[0014] The object of the present disclosure has been made to solve such problems, and is to provide an occurrence risk calculation device, an occurrence risk display system, an occurrence risk calculation method, and an occurrence risk calculation program that can improve the services required for managing roads. [Means for solving the problem]

[0015] The occurrence risk calculation device of the present disclosure comprises an image analysis means for analyzing an image of a road, an environmental information acquisition means for acquiring environmental information around the road, an occurrence risk calculation means for calculating an occurrence risk indicating the possibility of an obstacle factor occurring that may hinder passage along the road based on the analysis results of the image analysis means and the environmental information acquired by the environmental information acquisition means, and a display control means for correlating the occurrence risk with map information indicating the position information of the road.

[0016] In addition, the occurrence risk display system of the present disclosure includes a mobile imaging device that captures images of a road while moving and acquires location information of the road, an occurrence risk calculation device that calculates an occurrence risk indicating the possibility of an obstacle factor occurring that may hinder passage along the road based on the analysis results of the images output from the mobile imaging device and environmental information around the road, and associates the calculated occurrence risk with map information indicating the location information of the road, and a display device that displays the map information to which the occurrence risk is associated by the occurrence risk calculation device.

[0017] Furthermore, the risk of occurrence calculation method disclosed herein analyzes an image of a road, acquires environmental information around the road, calculates a risk of occurrence indicating the possibility of an obstacle factor occurring that may hinder passage along the road based on the results of analyzing the image and the acquired environmental information, and associates the risk of occurrence with map information indicating the location information of the road.

[0018] In addition, the risk of occurrence calculation program of the present disclosure causes a computer to analyze an image of a road, acquire environmental information around the road, calculate a risk of occurrence indicating the possibility of an obstacle factor occurring that may hinder passage along the road based on the analysis results of the image and the acquired environmental information, and associate the risk of occurrence with map information indicating the location information of the road. Effect of the Invention

[0019] According to the present disclosure, it is possible to provide an occurrence risk calculation device, an occurrence risk display system, an occurrence risk calculation method, and an occurrence risk calculation program that can improve the services required for managing roads. [Brief description of the drawings]

[0020] [Figure 1] 1 is a configuration diagram illustrating an example of an occurrence risk display system according to an embodiment. [Diagram 2]1 is a block diagram illustrating an occurrence risk calculation device according to an embodiment; [Diagram 3] FIG. 1 is a flowchart illustrating an occurrence risk calculation method according to an embodiment. [Figure 4] 1 is a configuration diagram illustrating an occurrence risk display system according to a first embodiment. [Diagram 5] 1 is a block diagram illustrating a moving imaging device according to a first embodiment. [Figure 6] 1 is a sequence diagram illustrating a method for displaying an occurrence risk according to the first embodiment. FIG. [Figure 7] 4 is a flowchart illustrating a mobile imaging method according to the first embodiment. [Figure 8] FIG. 2 is a flow chart illustrating an occurrence risk calculation method according to the first embodiment. [Figure 9] 4 is a flow chart illustrating a method for calculating a risk of road surface freezing according to the first embodiment. FIG. [Figure 10] FIG. 2 is a flow chart illustrating a method for calculating a risk of pothole occurrence according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, the embodiments will be described with reference to the drawings. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. In addition, in each drawing, the same elements are given the same reference numerals, and duplicate explanations are omitted as necessary.

[0022] (Overview of the embodiment) An overview of an embodiment will now be described. An occurrence risk display system according to an embodiment estimates the possibility that an obstacle factor that impedes road traffic will occur in the future as an occurrence risk, and displays it on a map.

[0023] Fig. 1 is a configuration diagram illustrating an occurrence risk display system according to an embodiment. As shown in Fig. 1, the occurrence risk display system 10 includes an occurrence risk calculation device 100, a mobile image capture device 200, and a display device 300. The occurrence risk calculation device 100, the mobile image capture device 200, and the display device 300 each have the functions of occurrence risk calculation means for calculating an occurrence risk, mobile image capture means for capturing images while moving, and display means.

[0024] The mobile imaging device 200 captures images of the road while moving and acquires the position information of the road. The occurrence risk calculation device 100 calculates the occurrence risk indicating the possibility of an obstacle factor that will impede passage along the road occurring based on the analysis result of the image of the road output from the mobile imaging device 200 and the environmental information around the road. The occurrence risk calculation device 100 then associates the calculated occurrence risk with map information indicating the position information of the road. The display device 300 displays the map information associated with the occurrence risk by the occurrence risk calculation device 100.

[0025] Fig. 2 is a block diagram illustrating an occurrence risk calculation device 100 according to an outline of an embodiment. As shown in Fig. 2, the occurrence risk calculation device 100 includes an image analysis unit 110, an environmental information acquisition unit 120, an occurrence risk calculation unit 130, and a display control unit 140. The image analysis unit 110, the environmental information acquisition unit 120, the occurrence risk calculation unit 130, and the display control unit 140 each have functions as an image analysis means, an environmental information acquisition means, an occurrence risk calculation means, and a display control means.

[0026] The image analysis unit 110 analyzes an image of a road. The environmental information acquisition unit 120 acquires environmental information about the surroundings of the road. The occurrence risk calculation unit 130 calculates an occurrence risk indicating the possibility of an obstacle factor that will impede passage along the road occurring, based on the analysis result of the image analysis unit 110 and the environmental information acquired by the environmental information acquisition unit 120. The display control unit 140 associates the occurrence risk with map information indicating the position information of the road.

[0027] 3 is a flow chart illustrating an example of a method for calculating a risk of occurrence according to an embodiment. As shown in step S110 of FIG. 3, an image of a road is analyzed. Also, as shown in step S120, environmental information about the periphery of the road is acquired. Note that the order of steps S110 and S120 is not limited to this, and may be reversed or may be performed simultaneously.

[0028] Next, as shown in step S130, an occurrence risk indicating the possibility of an obstacle factor that will impede passage along a road occurring is calculated based on the analysis result of the image and the acquired environmental information. Next, as shown in step S140, the occurrence risk is associated with map information indicating the position information of the road.

[0029] The method for calculating the risk of occurrence of this embodiment may be programmed to execute a risk of occurrence calculation program on a computer.

[0030] According to the occurrence risk display system 10 of the present embodiment, it is possible to estimate and display the occurrence risk indicating the possibility that an obstacle factor that will impede road traffic will occur in the future. Therefore, it is possible to improve services related to road management.

[0031] (Embodiment 1) Next, a description will be given of the occurrence risk display system according to embodiment 1. The occurrence risk display system according to this embodiment estimates the possibility of road surface freezing and potholes occurring in the future in advance as an occurrence risk by taking into consideration, for example, conditions that make road surface freezing and potholes likely to occur in a composite manner, and displays the estimated risk on a map.

[0032] Fig. 4 is a configuration diagram illustrating an occurrence risk display system according to the first embodiment. As shown in Fig. 4, the occurrence risk display system 11 of this embodiment includes an occurrence risk calculation device 100, a mobile imaging device 200, and a display device 300. The occurrence risk display system 11 may include one mobile imaging device 200 or multiple mobile imaging devices 200. The occurrence risk display system 11 may include one display device 300 or multiple display devices 300. In the following, the <mobile imaging device>, the <occurrence risk calculation device>, and the <display device> will be described, and then the <occurrence risk display method>, the <mobile imaging method>, and the <occurrence risk calculation method> will be described.

[0033] <Mobile imaging device> The mobile imaging device 200 has an imaging function such as a camera, and captures an image of the road while moving. Therefore, the mobile imaging device 200 is mounted on a moving body 210 for use. The moving body 210 may be, for example, a vehicle, a drone, or a person. The mobile imaging device 200 may share a drive recorder arranged in the vehicle, a smartphone carried by a vehicle occupant, a camera attached to a drone, or a camera carried by a pedestrian. The mobile imaging device 200 may use an omnidirectional camera as a camera, and there is no limit to the type or number of cameras. In this way, the mobile imaging device 200 uses the moving body 210 to capture an image of the road while moving while activating an imaging function such as a camera.

[0034] As the mobile object 210, in addition to a vehicle such as a general automobile, for example, a patrol car, a delivery vehicle, a garbage truck, etc. of a local government may be used. In this case, the patrol car, the delivery vehicle, the garbage truck, etc. run through the city as usual and collect images including information on the pavement surface. Note that the mobile imaging device 200 may use a mobile object 210 specially manufactured for the occurrence risk display system 11 as the mobile object 210, instead of a general vehicle, etc.

[0035] Fig. 5 is a block diagram illustrating the mobile imaging device 200 according to the embodiment 1. As shown in Fig. 5, the mobile imaging device 200 has an imaging section 220, a time information acquisition section 230, a position information acquisition section 240, a sensor section 250, a communication section 260, and a storage section 270. The imaging section 220, the time information acquisition section 230, the position information acquisition section 240, the sensor section 250, the communication section 260, and the storage section 270 have functions as an imaging means, a time information acquisition means, a position information acquisition means, a sensor means, a communication means, and a storage means, respectively.

[0036] The imaging unit 220 is, for example, a camera. The imaging unit 220 may be shared with a camera of a drive recorder or a smartphone. The imaging unit 220 may capture an image in the running direction of the moving body 210 or in the lateral direction of the moving body 210, or may capture an image in all directions. The imaging unit 220 may capture one or more locations at the same time. The image captured by the imaging unit 220 may be a still image or a video.

[0037] The imaging unit 220 captures an image of the road. The imaging unit 220 may capture the shadow of the road, or may capture the ruts on the road. The imaging unit 220 may also capture the image of vehicles passing on the road, or may capture the image of snow accumulated on the road. The imaging unit 220 may also capture the image of road markings provided on the road, or may capture the image of the road shoulder and roadside strip of the road. Furthermore, the imaging unit 220 may capture not only the image of the road, but also the image of the surrounding area of ​​the road. For example, in a heavy snowfall area, the imaging unit 220 may capture the height of snow on roofs, utility poles and / or electric wires, or the occurrence of icicles.

[0038] The time information acquisition unit 230 acquires time information. The time information acquisition unit 230 is, for example, a clock. The time information acquisition unit 230 may acquire the time information from a clock built into the mobile imaging device 200, or may acquire the time information from the Internet or the like via the communication unit 260. The time information acquisition unit 230 outputs the acquired time information to the imaging unit 220. The imaging unit 220 associates the time information with the captured image.

[0039] The position information acquisition unit 240 acquires position information of the mobile imaging device 200 and the mobile object 210. The position information acquisition unit 240 is, for example, a GPS (Global Positioning System) receiver. The position information acquisition unit 240 outputs the acquired position information to the imaging unit 220. The imaging unit 220 associates the captured image with the position information.

[0040] The sensor unit 250 is, for example, an acceleration sensor. The sensor unit 250 is a sensor that detects a change in acceleration when the mobile object 210 is traveling. The sensor unit 250 also detects a vibration state when the mobile object 210 is traveling based on cracks and ruts on the road surface. This makes it possible to estimate an IRI index (International Roughness Index) of the road surface. The sensor unit 250 may include a sensor that detects meteorological data such as temperature, snowfall, and wind speed.

[0041] The communication unit 260 transmits the captured image linked with the time information and the position information to the occurrence risk calculation device 100. In addition to the captured image, the communication unit 260 transmits time information acquired by the time information acquisition unit 230, position information acquired by the position information acquisition unit 240, and sensor information acquired by the sensor unit 250 to the occurrence risk calculation device 100. The image linked with the time information and position information may be uploaded to a predetermined receiving device via the communication unit 260.

[0042] The storage unit 270 stores the captured image captured by the imaging unit 220 in association with time information and position information. The captured image stored in the storage unit 270 may be uploaded to the occurrence risk calculation device 100 via a storage.

[0043] Each of the components of the mobile imaging device 200 described above may be shared with components constituting a drive recorder, a smartphone, or the like. In addition to the above-mentioned components, any other components such as a temperature measuring device and a sunshine measuring device may be added for the purpose of detecting the road surface condition and the driving situation. Furthermore, the installation position of each component is arbitrary, and may be inside or outside the mobile imaging device 200 and the moving body 210. For example, the storage unit 270 may be an external storage device or a storage device on the cloud.

[0044] <Occurrence risk calculation device> The occurrence risk calculation device 100 is, for example, an information processing device such as a server device, a personal computer, etc. The occurrence risk calculation device 100 is not limited to being fixed to a predetermined location, and may be provided on the cloud.

[0045] 4, the occurrence risk calculation device 100 may include a storage unit 150 in addition to the above-mentioned image analysis unit 110, environmental information acquisition unit 120, occurrence risk calculation unit 130, and display control unit 140. The storage unit 150 functions as a storage unit.

[0046] The image analysis unit 110 receives an image transmitted from the mobile imaging device 200. Then, the image analysis unit 110 analyzes at least any of information such as a shadow on the road at the time of image capture, traffic volume (vehicle volume), and cracks from the received image. The image analysis unit 110 may have a shadow detection unit 111, a traffic volume detection unit 112, and a crack detection unit 113. The shadow detection unit 111, the traffic volume detection unit 112, and the crack detection unit 113 function as a shadow detection means, a traffic volume detection means, and a crack detection means, respectively.

[0047] The shadow detection unit 111 detects shadows from the image of the road. The traffic volume detection unit 112 detects traffic volume such as vehicle volume from the image of the road. The crack detection unit 113 detects cracks from the image of the road. The analysis result of the image analysis unit 110 is output to the occurrence risk calculation device 100.

[0048] The environmental information acquisition unit 120 acquires environmental information including at least one of weather, temperature, snowfall, traffic volume, the presence or absence of ruts, and pavement material (asphalt, concrete, drainage pavement, etc.). The environmental information acquisition unit 120 acquires the environmental information in association with time information and position information at which the environmental information is detected. The acquired environmental information may be acquired from the mobile imaging device 200, may be acquired from information generally available on the Internet, etc., or may be information independently held by the user. For example, if the user is an administrator who manages roads, the user holds information on pavement materials. Thus, the environmental information including the pavement materials is acquired from the user. The environmental information acquired by the environmental information acquisition unit 120 is output to the occurrence risk degree calculation device 100.

[0049] The occurrence risk calculation unit 130 calculates the occurrence risk based on the analysis result output from the image analysis unit 110 and the environmental information output from the environmental information acquisition unit 120. The occurrence risk indicates the possibility of an obstacle factor occurring that may hinder traffic on the road. For example, the occurrence risk indicates the possibility of road surface freezing. Also, for example, the occurrence risk indicates the possibility of potholes occurring on the road. In this way, the occurrence risk differs from the current state of road surface freezing and pothole occurrence. The occurrence risk may indicate the possibility of an obstacle factor occurring that may hinder traffic on the road.

[0050] The risk of occurrence calculation unit 130 may include a road surface freezing calculation unit 131 and a pothole calculation unit 132. The road surface freezing calculation unit 131 and the pothole calculation unit 132 function as a calculation means for calculating the risk of occurrence of road surface freezing and a calculation means for calculating the risk of occurrence of potholes, respectively.

[0051] The road surface freezing calculation unit 131 calculates an occurrence risk level indicating the possibility of road surface freezing occurring in the future, based on the analysis result output from the image analysis unit 110 and the environmental information output from the environmental information acquisition unit 120. The pothole calculation unit 132 calculates an occurrence risk level indicating the possibility of potholes occurring in the future, based on the analysis result output from the image analysis unit 110 and the environmental information output from the environmental information acquisition unit 120. The occurrence risk level calculated by the occurrence risk level calculation unit 130 is linked to time information and position information, and is output to the display control unit 140 and the storage unit 150.

[0052] The display control unit 140 associates the occurrence risk level output from the occurrence risk level calculation unit 130 or the occurrence risk level output from the storage unit 150 with map information indicating the position information of the road. Specifically, the display control unit 140 controls display on the map information according to the level of the occurrence risk level. At this time, the occurrence risk level may be divided into a plurality of stages, and the information may be presented by explicitly dividing the stages in a visually easy-to-understand manner using colors, icons, or the like. For example, a point and / or a section where the occurrence risk level is significantly high may be displayed with a red arrow. In addition, the user may be allowed to set a threshold value of the risk level for dividing the stages. Furthermore, the display control unit 140 may control so that the user can confirm the information on which the occurrence risk level calculation is based and the image of the specific point as evidence from the map information. In addition, the map information may be associated with time information.

[0053] The storage unit 150 stores the occurrence risk output from the occurrence risk calculation unit 130 in association with time information and location information. The storage unit 150 may be provided in a storage device external to the occurrence risk calculation device 100, or may be provided in a storage device on the cloud.

[0054] <Display device> The display device 300 displays map information associated with the occurrence risk level by the occurrence risk level calculation device 100. For example, the display device 300 receives map information controlled by the display control unit 140 from the occurrence risk level calculation device 100 and displays the received map information to the user. The display device 300 is, for example, a terminal device with a display held by a user, such as a personal computer, a tablet, or a smartphone. For example, the display device 300 may receive the map information output by the occurrence risk level calculation device 100 via the Internet. The display device 300 provides the user with a user interface with map information that allows the user to check the risk of road surface freezing and the occurrence of potholes in an overall city from a bird's-eye view.

[0055] <How to display risk of occurrence> Next, a description will be given of an occurred risk display method by the occurred risk display system 11. Fig. 6 is a sequence diagram illustrating an occurred risk display method according to the first embodiment.

[0056] 6, the mobile imaging device 200 captures an image of a road while moving, and acquires the captured image. The mobile imaging device 200 also acquires time information, position information, and sensor information when the image of the road is captured.

[0057] Next, as shown in step S220, the mobile imaging device 200 transmits the captured image, time information, position information, and sensor information to the occurrence risk calculation device 100. In response to this, the occurrence risk calculation device 100 receives the captured image, time information, position information, and sensor information.

[0058] Next, as shown in step S230, the occurrence risk calculation device 100 analyzes the captured image output from the mobile imaging device 200. For example, the occurrence risk calculation device 100 detects shadows, traffic volume, and cracks from the image of the road.

[0059] Moreover, as shown in step S240, the occurrence risk calculation device 100 acquires environmental information. The occurrence risk calculation device 100 may acquire the environmental information from the mobile imaging device 200, may acquire it from information generally available on the Internet, or may acquire information held by the user. The order of steps S230 and S240 may be reversed, or steps S230 and S240 may be performed simultaneously.

[0060] Then, as shown in step S250, the risk of road accident occurrence calculation device 100 calculates the risk of road accident occurrence based on the analysis result of the road image and the environmental information. For example, the risk of road accident occurrence calculation device 100 calculates the risk of road surface freezing and the risk of pothole occurrence.

[0061] Next, as shown in step S260, the occurrence risk degree calculation device 100 associates the occurrence risk degree with the map information. The occurrence risk degree calculation device 100 may store in the storage unit 150 the map information associated with the occurrence risk degree.

[0062] Next, as shown in step S270, the occurred risk degree calculation device 100 transmits the map information associated with the occurred risk degree to the display device 300. In response to this, the display device 300 receives the map information associated with the occurred risk degree.

[0063] Next, as shown in step S280, the display device 300 displays the map information associated with the occurrence risk level.

[0064] <Moving imaging method> Next, a description will be given of a mobile imaging method using the mobile imaging device 200. Fig. 7 is a flow chart illustrating the mobile imaging method according to the first embodiment.

[0065] As shown in step S310 of Fig. 7, the mobile imaging device 200 is moved. For example, the mobile imaging device 200 is moved by moving the moving body 210 on which the mobile imaging device 200 is mounted.

[0066] Next, as shown in step S320, an image of the road is captured. For example, the image capturing unit 220 is caused to capture an image of the road.

[0067] Next, as shown in step S330, time information is acquired. For example, the time information acquisition unit 230 is caused to acquire time information at which the image of the road was captured by the imaging unit 220. Then, the time information acquisition unit 230 outputs the acquired time information to the imaging unit 220 and associates it with the image.

[0068] Also, as shown in step S340, position information is acquired. For example, position information acquisition unit 240 is caused to acquire position information at which image capturing unit 220 captured an image of a road. Then, position information acquisition unit 240 outputs the acquired position information to image capturing unit 220 and causes it to be associated with the image.

[0069] Also, as shown in step S350, sensor information is acquired. For example, the sensor unit 250 is caused to acquire the sensor information. Then, the sensor unit 250 outputs the acquired sensor information to the imaging unit 220. Note that the order of steps S330 to S350 is not limited to this. Also, steps S330 to S350 may be performed simultaneously.

[0070] Next, as shown in step S360, the captured image, the time information, the position information, and the sensor information are transmitted to the occurrence risk calculation device 100. For example, the communication unit 260 transmits to the occurrence risk calculation device 100 the captured image linked with the time information and the position information.

[0071] Next, as shown in step S370, the captured image, time information, position information, and sensor information may be stored in the storage unit 270.

[0072] <How to calculate risk of occurrence> Next, an explanation will be given of an occurred risk calculation method by the occurred risk calculation device 100. Fig. 8 is a flow chart illustrating an occurred risk calculation method according to the embodiment 1. As shown in Fig. 8, the occurred risk calculation method according to the embodiment includes steps S110 to S140, similar to the occurred risk calculation method according to the outline of the above-mentioned embodiment.

[0073] 8, first, an image of a road is input as shown in step S410. Specifically, the image analysis unit 110 receives an image of a road captured by the mobile imaging device 200.

[0074] Next, as shown in step S411, the image analysis unit 110 analyzes the image. Specifically, as shown in step S412, the shadow detection unit 111 detects a shadow from the image of the road. Furthermore, as shown in step S413, the traffic volume detection unit 112 detects traffic volume such as the number of vehicles from the image of the road. Furthermore, as shown in step S414, the crack detection unit 113 detects cracks from the image of the road. The analysis result of the image analysis unit 110 is output to the occurrence risk calculation unit 130.

[0075] Also, in step S120, as shown in step S415, the environmental information acquisition unit 120 acquires environmental information such as temperature, weather, pavement material information (asphalt, concrete, drainage pavement, etc.), snowfall amount, etc., in association with time information and position information at which the information is detected. Next, as shown in step S416, the environmental information acquisition unit 120 may acquire environmental information such as temperature difference between day and night and hours of sunlight by performing calculations using the environmental information. The environmental information acquired by the environmental information acquisition unit 120 is output to the occurrence risk calculation unit 130.

[0076] Next, in step S130, the occurrence risk calculation unit 130 calculates the occurrence risk. Here, a method for calculating the occurrence risk will be described using the occurrence risk of road surface freezing and the occurrence risk of potholes as examples of the occurrence risk.

[0077] As shown in step S417, the road surface freezing calculation unit 131 calculates a risk of road surface freezing, which indicates the possibility of road surface freezing occurring in the future, based on the analysis results output from the image analysis unit 110 and the environmental information output from the environmental information acquisition unit 120.

[0078] Fig. 9 is a flow chart illustrating a method for calculating the road surface freezing risk according to embodiment 1. As shown in step S510 in Fig. 9, the image analysis unit 110 acquires an image of a road from the mobile imaging device 200 together with information such as time information and position information.

[0079] Next, as shown in step S511, the shadow detection unit 111 detects a shadow reflected on the road surface from the image of the road. Also, as shown in step S512, the traffic volume detection unit 112 detects the traffic volume such as the vehicle volume from the number of vehicles reflected in the image of the road. Note that the order of steps S511 and S512 is not limited to this, and may be reversed, or may be performed simultaneously.

[0080] Meanwhile, as shown in step S520, the environmental information acquisition unit 120 acquires environmental information such as temperature. For example, the environmental information acquisition unit 120 may acquire environmental information such as the weather, temperature, information on pavement material (asphalt, concrete, drainage pavement, etc.), and snowfall at the time of capturing an image, based on time information and position information output together with the captured image from the mobile imaging device 200. Next, as shown in step S521, the environmental information acquisition unit 120 acquires the temperature difference between day and night by performing calculations using the environmental information such as temperature.

[0081] Next, as shown in step S530, the road surface freezing calculation unit 131 takes into consideration the analysis results output from the image analysis unit 110 and the environmental information output from the environmental information acquisition unit 120 in a combined manner, and calculates the risk of road surface freezing for each point, which indicates the possibility of road surface freezing occurring.

[0082] A specific example of a method for calculating the risk of road icing will be described below. For example, the risk of road icing K1 [%] is calculated using equation (1).

[0083] K1 = ((α1 + α2 + α3) / 3) × 100 (1)

[0084] Here, α1 indicates the degree of influence of the shadow, α2 indicates the degree of influence of the temperature difference between day and night, and α3 indicates the degree of influence of the traffic volume.

[0085] The degree of shadow influence α1 may be defined as, for example, α1 = 0.0 when shadows cover less than 30% of the road surface area, α1 = 0.6 when shadows cover 30% or more but less than 70% of the road surface area, and α1 = 1.0 when shadows cover 70% or more of the road surface area.

[0086] The influence of the temperature difference between day and night α2 may be defined as α2 = 0.2 when the temperature difference between day and night is less than 5°C, α2 = 0.5 when the temperature difference between day and night is 5°C or more and less than 10°C, and α2 = 1.0 when the temperature difference between day and night is 10°C or more.

[0087] The influence of traffic volume α3 may be defined as α3 = 0.0 when the traffic volume of the road is less than 1000 vehicles / directions per day in one direction, and α3 = 0.8 when the traffic volume of the road is 1000 vehicles / directions or more per day in one direction.

[0088] In this manner, the road surface freezing occurrence risk K1 [%] may be calculated. Note that these methods of calculating the road surface freezing occurrence risk are merely examples, and the method of calculating the influence, the items used in the method of calculating the road surface freezing occurrence risk, and the threshold and influence for each item may be changed. For example, the influence of the shadow α1 may be defined as (area of ​​the shadow area) × (importance (weight) of the shadow in road surface freezing), etc. In addition, parameters α4 and α5 indicating the presence or absence of bridges and mountain passes that are considered to be prone to road surface freezing may be added, and weights may be assigned to items that are considered to be important. In addition, for example, a dedicated model for calculating the occurrence risk may be created by constructing a machine learning model.

[0089] Furthermore, the more the following conditions exist, the higher the risk of snow formation may be set. That is, roads with a lot of shade on sunny days are inevitably less likely to melt, so the risk of snow formation is set to be high. Also, even if the road is exposed to sunlight during the day, the temperature drops toward the evening, and if there is a large difference in temperature between day and night, ice will form, so the risk of snow formation is set to be high. Also, the more traffic there is, the more likely the road is to be compressed and turn into ice, so the priority for snow removal is to be given to the road, so the risk of snow formation is set to be high.

[0090] Returning to FIG. 8 , as shown in step S418, the pothole calculation unit 132 calculates a pothole occurrence risk indicating the possibility of a pothole occurring in the road in the future, based on the analysis results output from the image analysis unit 110 and the environmental information output from the environmental information acquisition unit 120.

[0091] 10 is a flow chart illustrating a method for calculating the risk of pothole occurrence according to embodiment 1. Steps S610 to S612 and steps S620 to S621 in FIG. 10 are similar to steps S510 to S512 and steps S520 to S521 in FIG.

[0092] 10, the crack detection unit 113 detects the crack rate from the image of the road. Also, as shown in step S622, the environmental information acquisition unit 120 detects the occurrence of a zero crossing, that is, a state in which the temperature falls below 0° C.

[0093] Next, as shown in step S630, the pothole calculation unit 132 takes into consideration the analysis results output from the image analysis unit 110 and the environmental information output from the environmental information acquisition unit 120 in a composite manner, and calculates a pothole occurrence risk, which indicates the possibility of a pothole occurring, for each point.

[0094] A specific example of a method for calculating the pothole occurrence risk will be described below. For example, the pothole occurrence risk K2 [%] is calculated using equation (2).

[0095] K2=((β1+β2+β3+β4) / 4)×100 (2)

[0096] Here, β1 indicates the influence of season, β2 indicates the influence of crack rate, β3 indicates the influence of traffic volume, and β4 indicates the influence of zero crossing.

[0097] The seasonal influence degree β1 may be defined as, for example, β1=1.0 when it is the snowmelt season, and β1=0.0 when it is not the snowmelt season.

[0098] The influence of the crack rate β2 may be defined as β2 = 0.2 when the crack rate is less than 20%, β2 = 0.5 when the crack rate is 20% or more and less than 40%, and β2 = 1.0 when the crack rate is 40% or more.

[0099] The influence of traffic volume β3 may be defined as β3 = 0.0 when the traffic volume of the road is less than 1000 vehicles / directions per day in one direction, and β3 = 0.8 when the traffic volume of the road is 1000 vehicles / directions or more per day in one direction.

[0100] The degree of influence β4 of the zero crossing may be defined as β4=1.0 if there has been a day within the last one or two days when the temperature has fallen below 0° C., and β4=0.0 if the above condition does not apply.

[0101] In this manner, the pothole occurrence risk K2 [%] may be calculated. Note that these methods of calculating the pothole occurrence risk are merely examples, and the method of calculating the impact, the items used in the method of calculating the pothole occurrence risk, the threshold value and impact for each item may be changed, or weighting may be applied to items that are emphasized. Also, for example, a machine learning model may be constructed to create a dedicated model for calculating the occurrence risk.

[0102] Furthermore, the risk may be set to be higher as the number of the following conditions increases. That is, the risk of occurrence is increased during periods when potholes are more likely to occur, such as during the snowmelt season. The risk of occurrence is also increased in locations where potholes are likely to occur, such as locations with weak points such as cracks or joints. The risk of occurrence is also increased in locations where meltwater is likely to flow in and accumulate. The risk of occurrence is also increased in sections and routes with a high crack rate, particularly in locations where tortoiseshell cracks are likely to develop into potholes. The weather conditions when potholes occur are the day the zero crossing occurs and one or two days later, so the risk of occurrence is increased in such cases.

[0103] 8, the display control unit 140 associates the risk of freezing road surface and the risk of pothole occurrence with map information indicating the position information of the road as shown in step S419. Then, as shown in step S420, the display control unit 140 outputs the map information in which the risk of freezing road surface and the risk of pothole occurrence are associated with each other to the display device 300.

[0104] Next, the effects of this embodiment will be described. According to this embodiment, the risk of road surface freezing, potholes, etc. can be calculated by collecting and analyzing data during driving, including image information of the road surface, from the mobile imaging device 200. Then, the information can be displayed as map information on the user interface. In this way, instead of simply presenting the occurrence status of road surface freezing, potholes, etc., it is possible to estimate and present the occurrence risk indicating the possibility of road surface freezing, potholes, etc. occurring in the future by taking into account the conditions that are likely to cause them. Therefore, it is possible to improve services related to road management.

[0105] Compared to related icy road information display systems, the device that acquires information can be moved, so it can calculate the risk of road surface icy roads and potholes in smaller areas other than fixed locations. This makes it possible to provide information overlooking the entire town. It can also indicate the risk of potholes, which are very difficult to manage because no one knows when they will occur. This makes it possible to provide road managers with a priority order for patrol points.

[0106] In addition, the occurrence risk display system 11 of this embodiment can collect road data using inexpensive, general-purpose cameras (such as drive recorders and smartphones) without using expensive dedicated equipment, sensors, etc., thereby reducing costs.

[0107] Furthermore, by using a mobile camera attached to a mobile object 210 (such as a local government patrol car, delivery vehicle, or garbage truck) instead of a fixed camera, it is possible to grasp the detailed road surface condition of each road, even on small residential roads. In addition, it is possible to grasp the road surface condition frequently. Therefore, it is possible to manage and monitor roads in smaller detail and on an hourly basis than before.

[0108] In addition to showing the current occurrence status, the system also shows the degree of risk of occurrence on a map, making it possible to visualize the overall situation in the town and use this information to prioritize routes that need to be patrolled.

[0109] The occurrence risk display system 11 of this embodiment may store the daily analysis results, the acquired information, and the calculation results of the occurrence risk in the storage unit 150. The occurrence risk display system 11 may statistically process the results for each day at the same location in the time direction. For example, when the variance of the occurrence risk calculated over a week at a certain location is confirmed, if the occurrence risk is clearly higher or lower only on one day compared to the others, there is a possibility of a misjudgment. For this reason, a method may be taken in which the occurrence risk is estimated and displayed more accurately by removing noise, such as by presenting an average occurrence risk as a misjudgment from a statistical viewpoint. An algorithm may be set that determines an arbitrary statistical value (for example, a mode value, etc.) other than the average value. In this case, the data in the database accumulated in the occurrence risk calculation device 100 may be statistically recalculated, and the information may be provided to the display control unit 140.

[0110] (Embodiment 2) Next, an occurrence risk display system according to the second embodiment will be described. The occurrence risk display system according to the present embodiment may calculate the possibility of freezing on road marking lines provided on a road as the occurrence risk. The road marking lines include a crosswalk, a center line, an overtaking line, a speed display, and the like provided on the road surface. Therefore, in addition to the items related to the occurrence risk calculation mentioned in the above-mentioned first embodiment, the following environmental information may be added as γ1 to γ4, and the occurrence risk calculation device 100 may perform analysis and calculation. Note that the occurrence risk is not limited to the items α1 to α3 and / or β1 to β4 related to the occurrence risk calculation mentioned in the first embodiment, plus γ1 to γ4, and the like. The calculated occurrence risk is displayed on a user interface in association with map information.

[0111] Recent rain Zero Crossing Increased or decreased traffic volume - Materials of road markings (newer ones may take drainage into consideration) and shapes (including area)

[0112] For example, if the user is a road manager, the user holds information about the material and shape of the road marking lines, and thus the environmental information including the material and shape of the road marking lines is acquired from the user.

[0113] The occurrence risk display system may also calculate the possibility of snow mountains accumulating on the road shoulders and side strips as the occurrence risk. Snow mountains accumulating on the road shoulders and side strips are snow that has fallen on the road surface and has been collected on the road shoulders and side strips. Therefore, in addition to the items related to the occurrence risk calculation mentioned in the above-mentioned embodiment 1, the following elements may be added as δ1 to δ4, etc., and the occurrence risk calculation device 100 may perform analysis and calculation. Note that the occurrence risk is not limited to the items α1 to α3 and / or β1 to β4 related to the occurrence risk calculation mentioned in embodiment 1, plus δ1 to δ4, etc. The calculated occurrence risk is displayed on the user interface in association with map information.

[0114] - Check whether there are any piles of snow that have been removed from the road shoulders and sidewalks (this reduces visibility and makes it difficult to see people jumping out into the street, which is dangerous). ·Snow mountain height parameters Lane width parameters (the driving area tends to become narrower when there are snowy mountains) Snow removal schedule

[0115] For example, if the user is a road manager, the user holds a snow removal schedule, and therefore environmental information including the snow removal schedule is acquired from the user.

[0116] According to this embodiment, it is possible to calculate the occurrence probability of obstacle factors that are likely to be overlooked at first glance but have a high risk, such as frozen road markings and snowy mountains. In addition, since it uses environmental information that only the road administrator can grasp, it is possible to calculate the occurrence risk with high accuracy.

[0117] The present invention has been described above with reference to the first and second embodiments, but the present invention is not limited to the above first and second embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. For example, an embodiment that combines the configurations of the first and second embodiments is also included in the scope of the technical idea.

[0118] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0119] (Appendix 1) Image analysis means for analyzing an image of a road; An environmental information acquisition means for acquiring environmental information about the surroundings of the road; an occurrence risk calculation means for calculating an occurrence risk indicating a possibility of an obstacle factor occurring that may hinder passage along the road, based on an analysis result obtained by the image analysis means and the environmental information obtained by the environmental information acquisition means; a display control means for associating the occurrence risk level with map information indicating location information of the road; An occurrence risk calculation device comprising: (Appendix 2) The risk of occurrence indicates the possibility of road surface freezing occurring on the road, 2. An occurrence risk calculation device according to claim 1. (Appendix 3) The risk of freezing indicates the possibility of freezing on road markings provided on the road. 2. An occurrence risk calculation device according to claim 1. (Appendix 4) The occurrence risk indicates the possibility of snow piles accumulating on the shoulders and side strips of the road. 2. An occurrence risk calculation device according to claim 1. (Appendix 5) The occurrence risk indicates the possibility of a pothole occurring on the road. 2. An occurrence risk calculation device according to claim 1. (Appendix 6) The image analysis means analyzes at least one of a shadow, a traffic volume, and a crack from the image. 6. The risk calculation device according to any one of claims 1 to 5. (Appendix 7) The environmental information includes at least one of weather, temperature, snowfall, traffic volume, the presence or absence of ruts, and pavement material. 7. The risk calculation device according to any one of claims 1 to 6. (Appendix 8) a moving image capturing device that captures images of a road while moving and acquires position information of the road; an occurrence risk calculation device that calculates an occurrence risk indicating a possibility of an obstacle factor occurring that may hinder passage along the road based on an analysis result of the image output from the mobile imaging device and environmental information around the road, and associates the calculated occurrence risk with map information indicating position information of the road; a display device that displays the map information to which the occurrence risk level is associated by the occurrence risk level calculation device; and An occurrence risk indication system equipped with (Appendix 9) The risk of occurrence indicates the possibility of road surface freezing occurring on the road, Attachment 8 describes an occurrence risk indication system. (Appendix 10) The risk of freezing indicates the possibility of freezing on road markings provided on the road. Attachment 8 describes an occurrence risk indication system. (Appendix 11) The occurrence risk indicates the possibility of snow piles accumulating on the shoulders and side strips of the road. Attachment 8 describes an occurrence risk indication system. (Appendix 12) The occurrence risk indicates the possibility of a pothole occurring on the road. Attachment 8 describes an occurrence risk indication system. (Appendix 13) The risk occurrence calculation device analyzes at least one of a shadow, a traffic volume, and a crack from the image. 13. An occurrence risk indication system according to any one of appendixes 8 to 12. (Appendix 14) The environmental information includes at least one of weather, temperature, snowfall, traffic volume, the presence or absence of ruts, and pavement material. 14. An occurrence risk indication system according to any one of claims 8 to 13. (Appendix 15) Analyzing images of roads, acquiring environmental information about the vicinity of the road; Calculating an occurrence risk indicating the possibility of an occurrence of an obstacle factor that may hinder passage along the road based on an analysis result of the image and the acquired environmental information; The occurrence risk level is associated with map information indicating location information of the road. Method for calculating risk of occurrence. (Appendix 16) The risk of occurrence indicates the possibility of road surface freezing occurring on the road, The method for calculating the occurrence risk described in Appendix 15. (Appendix 17) The risk of freezing indicates the possibility of freezing on road markings provided on the road. The method for calculating the occurrence risk described in Appendix 15. (Appendix 18) The occurrence risk indicates the possibility of snow piles accumulating on the shoulders and side strips of the road. The method for calculating the occurrence risk described in Appendix 15. (Appendix 19) The occurrence risk indicates the possibility of a pothole occurring on the road. The method for calculating the occurrence risk described in Appendix 15. (Appendix 20) When analyzing the image of the road, Analyzing at least one of shadows, traffic volume, and cracks from the image; 20. A method for calculating an occurrence risk according to any one of appendixes 15 to 19. (Appendix 21) The environmental information includes at least one of weather, temperature, snowfall, traffic volume, the presence or absence of ruts, and pavement material. 21. A method for calculating an occurrence risk according to any one of appendixes 15 to 20. (Appendix 22) Analyzing images of roads, acquiring environmental information about the vicinity of the road; Calculating an occurrence risk indicating the possibility of an occurrence of an obstacle factor that may hinder passage along the road based on an analysis result of the image and the acquired environmental information; The occurrence risk level is associated with map information indicating location information of the road. A non-transitory computer-readable medium storing an occurrence risk calculation program for causing a computer to execute the above. (Appendix 23) The risk of occurrence indicates the possibility of road surface freezing occurring on the road, A non-transitory computer-readable medium storing the occurrence risk calculation program described in Appendix 22. (Appendix 24) The risk of freezing indicates the possibility of freezing on road markings provided on the road. A non-transitory computer-readable medium storing the occurrence risk calculation program described in Appendix 22. (Appendix 25) The occurrence risk indicates the possibility of snow piles accumulating on the shoulders and side strips of the road. A non-transitory computer-readable medium storing the occurrence risk calculation program described in Appendix 22. (Appendix 26) The occurrence risk indicates the possibility of a pothole occurring on the road. A non-transitory computer-readable medium storing the occurrence risk calculation program described in Appendix 22. (Appendix 27) When analyzing the image of the road, Analyzing at least one of shadows, traffic volume, and cracks from the image; A non-transitory computer-readable medium storing an occurrence risk calculation program according to any one of appendices 22 to 26, which causes a computer to execute the above. (Appendix 28) The environmental information includes at least one of weather, temperature, snowfall, traffic volume, the presence or absence of ruts, and pavement material. A non-transitory computer-readable medium storing an occurrence risk calculation program according to any one of appendices 22 to 27.

[0120] In the above example, the program can be stored and provided to the computer using various types of non-transitory computer readable media. The non-transitory computer readable medium includes various types of tangible storage media. Examples of the non-transitory computer readable medium include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be provided to the computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include electric signals, optical signals, and electromagnetic waves. The transitory computer readable medium can provide the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path. [Explanation of symbols]

[0121] 10, 11 Risk Indication System 100 Risk of occurrence calculation device 110 Image Analysis Unit 111 Shadow detection unit 112 Traffic volume detection unit 113 Crack detection unit 120 Environmental Information Acquisition Department 130 Risk of Occurrence Calculation Unit 131 Road surface freezing calculation section 132 Pothole Calculation Section 140 Display control unit 150 Storage section 200 Mobile imaging device 210 Mobile 220 Imaging unit 230 Time information acquisition section 240 Location information acquisition unit 250 Sensor section 260 Communications Department 270 Storage section 300 display device

Claims

1. an image analysis means for analyzing a road surface condition based on an image of the road; An environmental information acquisition means for acquiring environmental information about the surroundings of the road; A prediction means for predicting a road surface freezing state based on an analysis result obtained by the image analysis means and the environmental information obtained by the environmental information acquisition means; A display control means for displaying the predicted road surface freezing condition in association with time information; An information processing device comprising:

2. The predicted road surface freezing condition includes a condition on a road marking line provided on the road. The information processing device according to claim 1 .

3. The image analysis means analyzes at least one of a shadow, a traffic volume, and a crack from the image of the road.

3. The information processing device according to claim 1 or 2.

4. The environmental information may include at least one of the following: weather, temperature, amount of snow, traffic volume, presence or absence of ruts, and pavement material. At least one of the following: The information processing device according to claim 1 .

5. The environmental information includes temperature differences between day and night, The information processing device according to claim 1 .

6. The environmental information includes sunshine hours. The information processing device according to claim 1 .

7. The display control means displays map information that allows the predicted road surface icy condition of the entire city to be viewed from above. The information processing device according to claim 1 .

8. Analyzes road surface conditions based on road images, Acquire environmental information around the road; A road surface freezing condition is predicted based on an analysis result of the image and the acquired environmental information; Linking the predicted road surface freezing condition with time information; Information processing methods.

9. Analyzes road surface conditions based on road images, acquiring environmental information about the vicinity of the road; A road surface freezing condition is predicted based on an analysis result of the image and the acquired environmental information; Linking the predicted road surface freezing condition with time information; An information processing program that causes a computer to execute

Images