Image generation system, image generation method, and program

Abstract

To provide an image generation system, an image generation method, and a program capable of expressing a future deterioration degree of a road in an easy-to-understand manner.SOLUTION: An image generation system includes an acquisition unit that acquires a captured image of a road captured by a camera mounted on a moving body, a deterioration degree determination unit that determines a future deterioration degree of an imaging target of the camera, and a generation unit that generates a prediction image in which a state of deterioration according to the future deterioration degree is represented in the captured image on the basis of the captured image, in which the generation unit inputs the acquired captured image to a learning model that receives an input of an image and outputs a prediction image in which a state of deterioration according to the future deterioration degree is represented in the image. The prediction image is generated.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a road image generation system and the like. [Background technology]

[0002] Roads deteriorate over time. The degree of road deterioration is expressed in terms of values ​​such as crack rate, flatness, and the Maintenance Control Index (MCI). Road repairs are planned based on these values ​​and the road's past and present appearance.

[0003] Patent Document 1 discloses a structure inspection support system that analyzes the deterioration state of a structure and generates data on predicted values ​​of the future degree of deterioration. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-057192 Summary of the Invention [Problem to be solved by the invention]

[0005] When the future deterioration of a road is expressed numerically, it can be difficult to imagine what the road will be like at that point.

[0006] The present disclosure aims to provide an image generation system and the like that can clearly represent the future degree of deterioration of a road. [Means for solving the problem]

[0007] The image generation system according to the present disclosure includes an acquisition means for acquiring a road image obtained by capturing an image of a road, a deterioration level determination means for determining the future deterioration level of the road, and a generation means for generating a predicted image of the road based on the road image, the predicted image showing road deterioration corresponding to the deterioration level.

[0008] The image generation method according to the present disclosure acquires a road image of a road, determines the future degree of deterioration of the road, and generates, based on the road image, a predicted image showing road deterioration on the road according to the degree of deterioration.

[0009] The program disclosed herein causes a computer to perform the following process: acquire a road image of a road, determine the future degree of deterioration of the road, and, based on the road image, generate a predicted image of the road that shows road deterioration corresponding to the degree of deterioration. [Effects of the Invention]

[0010] According to the present disclosure, the future degree of deterioration of a road can be expressed in an easy-to-understand manner. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of the configuration of an image generation system 100. FIG. [Figure 2] 1 is a flowchart showing an example of the operation of the image generation system 100. [Figure 3] FIG. 1 is a diagram illustrating an example of a road image. [Figure 4] FIG. 10 is a diagram illustrating an example of a screen for receiving input of a deterioration level. [Figure 5] FIG. 10 is a diagram illustrating an example of a predicted image. [Figure 6] 10 is an image showing another example of a display screen for a predicted image. [Figure 7] 10 is an image showing an example of a screen that accepts selection of a predicted image to be displayed. [Figure 8] FIG. 5 is a block diagram showing an example of the hardware configuration of a computer 500. DETAILED DESCRIPTION OF THE INVENTION

[0012] Before describing the embodiments, an example of road deterioration etc. in the present disclosure will be described.

[0013] Road deterioration is deterioration that occurs on paved roads due to factors such as vehicle traffic and rainfall. There are multiple types of road deterioration. Road deterioration is classified into multiple types, including, for example, cracks, potholes, ruts, and road irregularities. Cracks may be classified into different types, such as linear cracks and hexagonal cracks, depending on their shape. A linear crack is a single linear crack. A hexagonal crack is a hexagonal crack that occurs, for example, when vertical and horizontal linear cracks are connected. Road cracks progress from linear cracks to hexagonal cracks to potholes.

[0014] Various indices are used to represent the degree of road deterioration. In the present disclosure, the degree of road deterioration is represented by a deterioration degree. The deterioration degree may be any of indices including the degree of cracks, the number of potholes, the size of potholes, the amount of rutting, or flatness. The deterioration degree may also be determined based on a combination of multiple indices representing the degree of road deterioration.

[0015] The crack degree is expressed by any one of the shape, length, area, and number of cracks, or a combination of these. The crack rate is an example of the crack degree. The crack rate is expressed, for example, by 100 x (crack area / road section area). In this case, the deterioration degree value ranges from 0% to 100%. The crack area is calculated by any method. Note that the method for calculating the crack rate is not particularly limited, and other known calculation methods can be applied in addition to those mentioned above.

[0016] The size of a pothole can be expressed by, for example, the area, width, length, or depth of the pothole, or a combination of these. The amount of rutting is the depth of the rut where the vehicle's track is lower than the rest of the road surface due to the load of the vehicle and friction with the tires.

[0017] The degree of cracking, the number and size of potholes, and the amount of rutting may be calculated based on measurement data obtained by measuring the road surface with a sensor, or these indices may be calculated based on the recognition results of road deterioration recognized from images of the road.

[0018] The flatness may be expressed by the International Roughness Index (IRI). The IRI is an index that relates the road surface to the driver's ride comfort, and is a numerical representation of the degree of unevenness. The IRI may be calculated based on measurement data obtained by measuring the road surface with a sensor. Alternatively, the IRI may be calculated based on the value of an acceleration sensor attached to the vehicle while it is traveling. Specifically, for example, the IRI is calculated based on the value of the vertical acceleration included in the acceleration obtained at the detection position. Note that the method of calculating the IRI is not limited to the above, and any known calculation method may be adopted.

[0019] The deterioration level is not limited to the above-mentioned indexes, and any index representing road deterioration, including, for example, the Maintenance Control Index (MCI), may be used. The MCI value is the minimum value obtained by calculating four definition formulas using the crack rate, rutting depth, and flatness. The MCI decreases as the road deteriorates.

[0020] In the following description, the degree of deterioration will be mainly explained using the crack rate. Therefore, the value of the degree of deterioration increases as the deterioration increases. However, the way of expressing the degree of deterioration is not limited to this, and for example, the value of the degree of deterioration may decrease as the deterioration increases.

[0021] [Embodiment] An image generation system according to one embodiment generates a predicted image representing road deterioration predicted to occur on a road in the future. The predicted image generated by the road image generation system represents the state of the road at a certain deterioration level.

[0022] 1 is a block diagram showing an example of the configuration of an image generation system 100 according to an embodiment. The image generation system 100 includes an acquisition unit 101, a degradation level determination unit 102, and a generation unit 103.

[0023] The acquisition unit 101 acquires road images of roads. The road images may be captured by an in-vehicle camera such as a drive recorder. However, the type of camera is not limited to this, and various types of cameras may be used. For example, the road images may be captured by a camera mounted on another moving object such as a bicycle or a drone, a camera carried by a person, or a fixed camera installed on the road. The road images may be captured by a person or automatically.

[0024] The road images captured by the camera may be stored in a database (not shown). In this case, the acquisition unit 101 may acquire the road images from the database. Alternatively, if the image generation system 100 is connected to an arbitrary camera via a wired or wireless connection, the acquisition unit 101 may acquire the road images from the camera.

[0025] The acquisition unit 101 may acquire the date and time when the road image was captured together with the road image. The acquisition unit 101 may also acquire information about the location where the road image was captured together with the road image. The location information includes, for example, a location on a map, latitude and longitude, and location information by a Global Navigation Satellite System (GNSS) or a Global Positioning System (GPS).

[0026] The current or past state of the road can be determined based on the road image acquired by the acquisition unit 101. The road image may or may not include road deterioration such as cracks. The imaging range of the road image is not particularly limited, and the road image may also include objects other than the road. If the image includes objects other than the road, the location or size of the road deterioration may be easier to understand.

[0027] Furthermore, the acquisition unit 101 may acquire the pavement material of the photographed road or the current deterioration level of the road. For example, the acquisition unit 101 may recognize the pavement material and the current deterioration level from the road image. The acquisition unit 101 may transmit the recognized material and deterioration level to the deterioration level determination unit 102.

[0028] The deterioration degree determination unit 102 determines the future deterioration degree of the road from which the road image is acquired. Specifically, the deterioration degree determination unit 102 determines the deterioration degree in the image generated by the generation unit 103. The deterioration degree determination unit 102 determines, for example, the crack rate as the deterioration degree.

[0029] The deterioration degree determination unit 102 may determine the deterioration degree input from the user as the future deterioration degree. The user inputs the deterioration degree using, for example, an input device connected to the image generation system 100. The input device is, for example, a mouse, a keyboard, or a touch panel display.

[0030] Alternatively, the deterioration degree determination unit 102 may determine the predicted deterioration degree as the future deterioration degree. The deterioration degree determination unit 102 predicts the deterioration degree on, for example, a target prediction date. An arbitrary time point after the time point at which the road image is captured is set as the target prediction date. The target prediction date is set, for example, after the time point at which the deterioration degree determination unit 102 makes the prediction, but is not limited to this. The deterioration degree determination unit 102 may predict the deterioration degree on a target prediction date specified by the user.

[0031] The deterioration degree prediction may be performed by another device. The other device predicts the deterioration degree on the target prediction date and transmits the deterioration degree to the image generation system 100. The deterioration degree determination unit 102 acquires the deterioration degree from the other device and determines the predicted deterioration degree as the future deterioration degree.

[0032] The deterioration degree can be predicted by any method, including known methods, examples of which will be described later.

[0033] Based on the road image, the generation unit 103 generates a predicted image that shows road deterioration on the photographed road according to the deterioration level determined by the deterioration level determination unit 102. When a predicted image showing future road deterioration on the photographed road is presented, it is easier to imagine the future state of the road compared to when the deterioration level is presented as a value. The method of generating the image will be described later.

[0034] The deterioration level determination unit 102 may determine the deterioration level at each of a plurality of future time points. The generation unit 103 may generate a plurality of predicted images that show road deterioration on the road according to the deterioration levels at each of the plurality of time points. Specifically, for example, the deterioration level determination unit 102 determines the deterioration levels of the road 6 months, 12 months, and 18 months after the time point at which the road image is captured. In this case, the generation unit 103 generates three predicted images that show each deterioration level.

[0035] FIG. 2 is a flowchart showing an example of the operation of the image generation system 100 according to one embodiment.

[0036] The acquisition unit 101 acquires a road image (step S01). The acquisition unit 101 may acquire the road image based on a user's specification. For example, the acquisition unit 101 may acquire a road image of a location specified by the user. FIG. 3 is a diagram showing an example of a road image. FIG. 3 is an image captured of a road on which a car is traveling. For example, FIG. 3 is captured by a camera mounted on a vehicle traveling in the left lane. In this example, the camera captures the area ahead in the direction of travel. The road in FIG. 3 has a crack on the left edge of the road. The road image in FIG. 3 includes sidewalks on both sides of the road. However, the imaging range of the road image is not limited to the range shown in FIG. 3. The imaging range may be narrow or wide, for example, in the vertical or horizontal direction.

[0037] The deterioration degree determination unit 102 determines the future deterioration degree (step S02). FIG. 4 is a diagram showing an example of a screen that accepts input of the deterioration degree from the user. The screen in FIG. 4 includes the acquired road image and the deterioration degree of the road image. The user inputs a numerical value, for example, into an input field for the crack rate. The method of inputting the deterioration degree is not limited to inputting it into an input field. The deterioration degree may also be input using a pull-down list, a check box, or a radio button. When the user presses the "Execute image generation" button, the deterioration degree determination unit 102 determines the accepted deterioration degree as the deterioration degree in the predicted image.

[0038] The generation unit 103 acquires the degree of deterioration from the deterioration degree determination unit 102. Based on the acquired road image, the generation unit 103 generates a predicted image in which road deterioration corresponding to the determined degree of deterioration is displayed on the captured road (step S03). FIG. 5 is a diagram showing an example of the generated predicted image. The predicted image in FIG. 5 has more cracks than the road image in FIG. 3. In addition, some cracks are longer and thicker than those in the road image.

[0039] (Deterioration prediction) An example of a method for predicting the future deterioration level will be described. The deterioration level determination unit 102 may predict the deterioration level using any of the following methods. However, the prediction method is not limited to the following examples. The prediction unit may also predict the deterioration level by combining multiple methods.

[0040] The deterioration level determination unit 102 may predict the deterioration level by applying road-related parameters to the prediction formula. The road-related parameters are parameters that affect the deterioration level. The road-related parameters represent characteristics of the road environment or characteristics of the road.

[0041] The deterioration level determination unit 102 acquires road-related parameters for prediction. For example, the deterioration level determination unit 102 may acquire parameters input by a user. Alternatively, the deterioration level determination unit 102 may acquire parameters from the acquisition unit 101.

[0042] The road-related parameters may be stored in a database for each road. The deterioration level determination unit 102, for example, acquires identification information of the road for which the deterioration level is to be predicted. Next, the deterioration level determination unit 102 acquires parameters corresponding to the road identification information from the database. Alternatively, the parameters may be stored in association with a road image. In this case, the deterioration level determination unit 102 acquires parameters corresponding to the road image.

[0043] Examples of road-related parameters are given below. The road-related parameters include parameters that represent the characteristics of the road environment and parameters that represent the road characteristics. The parameters that represent the characteristics of the road environment include traffic volume, weather information, and area information. The parameters that represent the road characteristics include road construction information. Note that the parameters are not limited to these, and other parameters may be included.

[0044] Traffic volume is the number of vehicles traveling on a road. Roads with heavy traffic deteriorate quickly. Traffic volume may include the number of heavy vehicles traveling on a road. Roads with many heavy vehicles traveling on them deteriorate quickly. Weather information is, for example, precipitation, snowfall, or temperature. Regional information includes whether the area is experiencing rapid road deterioration. For example, road deterioration progresses quickly in areas with heavy rainfall, areas near the coast, or cold regions.

[0045] The construction information includes, for example, the pavement material, the condition of the roadbed, the thickness of the layer, or the construction history. The pavement material includes, for example, asphalt and concrete. Generally, asphalt deteriorates faster than concrete. The construction history includes, for example, when the road was paved or whether or not there has been any construction work on the road. Deterioration progresses as time passes since the road was paved. Furthermore, deterioration progresses when construction work on the road is carried out.

[0046] The deterioration level determination unit 102 may predict the future deterioration level based on, for example, the rate at which road deterioration progresses. The rate at which road deterioration progresses indicates a change in the deterioration level per unit time. The deterioration level determination unit 102 calculates the future deterioration level by applying the deterioration level in the road image and the rate at which road deterioration progresses to a prediction formula. Note that the deterioration level determination unit 102 may predict the deterioration level taking into account changes in the rate at which road deterioration progresses per unit time.

[0047] The deterioration level determination unit 102 may predict the deterioration level taking into consideration that the rate at which road deterioration progresses differs depending on the road. The rate at which road deterioration progresses may be determined based on the above-mentioned road-related parameters. The rate at which road deterioration progresses for each road may be stored in a database. In this case, the deterioration level determination unit 102 may acquire the rate at which road deterioration is predicted from the database. Note that the same rate may be set for all roads as the rate at which road deterioration progresses. The rate at which road deterioration progresses is included in one example of road-related parameters.

[0048] The deterioration level determination unit 102 may calculate the travel speed based on the deterioration levels at multiple past points in time for the road whose deterioration level is to be predicted. The deterioration level determination unit 102 may acquire past deterioration levels from a database. Alternatively, the deterioration level determination unit 102 may acquire images of the road captured in the past and detect the past deterioration level from the acquired images. Specifically, the deterioration level determination unit 102 detects, for example, cracks, potholes, or ruts in the road from past images. The deterioration level determination unit 102 detects the position of the crack, the shape of the crack (line, hexagonal), the length or area of ​​the crack, the number of cracks, the position of the pothole, the area of ​​the pothole, the number of potholes, or the amount of rutting. The deterioration level determination unit 102 calculates the amount of change in the deterioration level from the detected past deterioration level.

[0049] (Image generation) A method for generating a predicted image by the generation unit 103 will be described below, although the generation method is not limited to the following.

[0050] The generating unit 103 may generate a predicted image by adding a graphic representing road deterioration to the road image based on the determined deterioration level. For example, the generating unit 103 first recognizes the road portion of the original road image. Then, the generating unit 103 superimposes a graphic on the road portion of the road image. For example, a line is superimposed as a graphic representing a crack.

[0051] The generating unit 103 may generate a predicted image taking into account the location of road deterioration in the original image. Specifically, for example, the generating unit 103 recognizes cracks. Next, the generating unit 103 increases the number of cracks near the cracks or extends the recognized cracks. In this way, the generating unit 103 can generate a predicted image in which the road deterioration shown in the original road image has progressed.

[0052] The generation unit 103 may generate a predicted image using an image of another road in addition to the road image. Here, the other image is called a stored image because it is stored in an arbitrary database. The stored image represents road deterioration of a road with a predetermined deterioration level. The database stores the stored image in association with the deterioration level of the road.

[0053] The generating unit 103 generates a predicted image by, for example, combining the road image acquired by the acquiring unit 101 with a stored image corresponding to the determined degradation level. The stored image corresponding to the determined degradation level includes a stored image that represents the same degradation level as the determined degradation level. However, the stored image corresponding to the determined degradation level may also include a stored image that represents a degradation level within a predetermined range from the determined degradation level.

[0054] The stored image may be an image of another road. The image range of the stored image may be the same as the road image, or may be wider or narrower than the image range of the road image. For example, the image range of the stored image may be a deteriorated portion of the road.

[0055] The method for combining the road image and the stored image is not particularly limited. For example, the generation unit 103 may superimpose the stored image on the road image. Alternatively, the generation unit 103 may superimpose a deteriorated portion of the road in the stored image on the road in the road image.

[0056] The generation unit 103 may generate a predicted image by inputting a road image into a learning model generated by machine learning. As the learning model, for example, a Generative Adversarial Network (GAN) may be used. For example, among GANs, Cycle-GAN or Pix2Pix may be used. The aforementioned stored image may be used to generate the learning model.

[0057] According to the above embodiment, the acquisition unit 101 acquires a road image obtained by capturing an image of a road. Furthermore, the deterioration level determination unit 102 determines the future deterioration level of the road from which the road image was captured. Furthermore, the generation unit 103 generates a predicted image based on the road image. The predicted image shows road deterioration of the captured road according to the deterioration level determined by the deterioration level determination unit 102. Therefore, according to the embodiment, the future deterioration level of the road can be expressed in an easy-to-understand manner.

[0058] The generated predicted image can be used, for example, to determine the budget for road repairs. Even if the future deterioration level is presented numerically, some people find it difficult to understand the need for repairs. Therefore, by presenting the predicted image, the need for repairs can be understood and the repair budget can be appropriately set.

[0059] [Variations] (Predicted image display) The image generation system 100 may further include a display control unit that displays the predicted image on a display (not shown). The display may be, for example, a display connected to a computer or a tablet. An example of a method for displaying the predicted image will be described below. However, the method for displaying the predicted image is not limited to the following.

[0060] The display control unit may display the predicted image and the road image side by side. The display control unit may display the determined deterioration level together with the predicted image. The display control unit may also display the time related to the predicted image together with the predicted image. The time related to the predicted image indicates the predicted time required for road deterioration to progress from the deterioration level of the road image to the deterioration level of the predicted image. The time related to the predicted image may be displayed in any format, such as the date of the target prediction date and the number of days, months, or years from the date the road image was captured until the target prediction date. The display control unit may also display a map indicating points corresponding to the displayed predicted image together with the predicted image.

[0061] The display control unit may display the actual road deterioration and the predicted road deterioration included in the displayed predicted image in a distinguishable manner, for example, by applying different colors to the actual road deterioration and the predicted road deterioration.

[0062] The display control unit may display multiple predicted images for one location. The display control unit may display multiple predicted images on one screen so that the user can view them all at once. Alternatively, the display control unit may switch between predicted images and display each predicted image one by one. Such a switching display shows the progression of road deterioration like a fast-forward video of one location.

[0063] The display control unit may display a graph showing the change in the deterioration level over time on the captured road together with the predicted image. The generation unit 103 may generate the graph. For example, the generation unit 103 obtains the deterioration level for each time period from the deterioration level determination unit 102 and plots it. The generation unit 103 passes the generated graph to the display control unit. Figure 6 is an image showing another example of the display screen for the predicted image. Figure 6 includes a graph showing the relationship between the crack rate and time.

[0064] The display control unit may display a correspondence relationship between the predicted image to be displayed and its position in the graph, for example, by highlighting a plot on the graph or a value on an axis corresponding to the predicted image to be displayed more than other plots or values.

[0065] In addition, by pressing the "MCI" or "IRI" button on the screen of FIG. 6, a graph relating to MCI and time and a graph relating to IRI and time may be displayed.

[0066] The display control unit may switch between and display predicted images for a plurality of points. The generation unit 103 may generate a predicted image for each of a plurality of road images obtained by capturing a plurality of consecutive points. In this case, the display control unit may, for example, switch between and display predicted images for a plurality of consecutive points. Such a switching display shows how the vehicle will travel along a future road.

[0067] (User Interface) The display control unit may display a screen that accepts selection of a predicted image to be displayed from among the plurality of generated predicted images. The display control unit may switch the predicted image to be displayed in response to an input from a user.

[0068] The display control unit may, for example, display a screen that accepts input of the prediction target date from the user. The prediction target date can be input in any format, such as the date of the prediction target date, or the number of days, months, or years from the date the road image was captured until the prediction target date. The display control unit displays a predicted image corresponding to the input prediction target date.

[0069] Fig. 7 is an image showing an example of a screen that accepts selection of a predicted image to be displayed. The screen of Fig. 7 allows the user to select a prediction target date. The display control unit may display a slider bar and a slider, and switch the predicted image to be displayed based on the sliding of a position on the slider bar.

[0070] Alternatively, similar to the selection of the prediction date, the display control unit may display a screen for accepting the selection of the deterioration level from the user, and the display control unit displays the predicted image of the selected deterioration level.

[0071] The display control unit may switch the predicted image based on a click on a point on the graph in FIG. 6 or a value on an axis.

[0072] The display control unit may display a screen for accepting designation of which road a predicted image is to be displayed for. For example, the display control unit displays a map and displays a predicted image for the selected road.

[0073] (Determining the type of road deterioration) The deterioration level determination unit 102 may determine the type of road deterioration to be represented in the predicted image. The deterioration level determination unit 102 may determine the type of road deterioration to be represented in the predicted image from among linear cracks, tortoiseshell cracks, and potholes.

[0074] The deterioration level determination unit 102 may determine the type of road deterioration input by the user as the type of road deterioration to be displayed in the predicted image. Alternatively, the deterioration level determination unit 102 may determine the predicted type of road deterioration as the type to be displayed in the predicted image. The deterioration level determination unit 102 may predict the type of road deterioration that will occur on the road in the future. Furthermore, the deterioration level determination unit 102 may determine the type of road deterioration predicted by another device as the type to be displayed in the predicted image.

[0075] The type of road deterioration may be predicted by any method. For example, the deterioration level determination unit 102 may predict the type of road deterioration based on the determined deterioration level. Basically, the higher the deterioration level, the more the types of road deterioration occurring on the road progress from linear cracks to hexagonal cracks to potholes. Therefore, the deterioration level determination unit 102 may predict that potholes and hexagonal cracks will occur if it determines that the crack rate is 70% or higher, and may predict that hexagonal cracks will occur if it determines that the crack rate is 50% or higher.

[0076] Alternatively, the deterioration level determination unit 102 may calculate the probability of occurrence of each type of road deterioration based on parameters related to the road. For example, a road with heavy traffic and heavy precipitation has a high probability of pothole occurrence. If the probability of pothole occurrence is greater than a predetermined standard, the deterioration level determination unit 102 predicts that a pothole will occur.

[0077] The generating unit 103 may generate a predicted image showing the determined type of road deterioration. For example, the generating unit 103 may draw the determined type of road deterioration on a road image. Alternatively, the generating unit 103 may combine a stored image corresponding to the determined type of road deterioration with the road image. In this case, the stored image may be stored in a database in association with the type of road deterioration included in the image. Furthermore, the generating unit 103 may use a learning model that generates a predicted image including road deterioration of the determined type of road deterioration from the road image.

[0078] (Generated according to pavement material) The generating unit 103 may generate a predicted image based on the pavement material of the road included in the road image. The appearance of the road varies depending on the pavement material. Furthermore, the progression of road deterioration may vary depending on the pavement material. Therefore, the stored image may be stored in the database in association with the road deterioration level and the pavement material. The generating unit 103 receives the pavement material of the road included in the road image from the acquiring unit 101, for example. Note that the generating unit 103 may also receive input of the pavement material from the user. The generating unit 103 changes the stored image used to generate the predicted image depending on the determined pavement material.

[0079] (Weather-dependent generation) The generation unit 103 may generate a predicted image according to the weather when the road image was captured. The weather includes, for example, sunny, cloudy, and rainy. The appearance of the road differs depending on the weather. For example, the color of the road is darker on a cloudy day than on a sunny day. On a rainy day, the road surface may become wet and puddles may form. Therefore, for example, the generation unit 103 may generate a predicted image based on a stored image captured in the same weather as when the road image was captured.

[0080] The stored image may be stored in a database in association with the degree of road deterioration and the weather at the time the stored image was captured. The generation unit 103, for example, acquires the weather at the time the road image was captured from the acquisition unit 101. The acquisition unit 101 may acquire the results of analyzing the weather at the time the road image was captured by analyzing the sky condition or the road surface condition shown in the road image. Alternatively, the acquisition unit 101 may acquire the weather from a weather database based on the location and time at which the road image was captured. The generation unit 103 may also accept input of the weather at the time the road image was captured from the user.

[0081] [Hardware configuration] In the above-described embodiment, each component of the image generation system 100 is represented by a functional block. Some or all of the components of each device may be realized by any combination of a computer 500 and a program.

[0082] Fig. 8 is a block diagram showing an example of the hardware configuration of a computer 500. Referring to Fig. 8, the computer 500 includes, for example, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a program 504, a storage device 505, a drive device 507, a communication interface 508, an input device 509, an input / output interface 511, and a bus 512.

[0083] The program 504 includes instructions for realizing each function of each device. The program 504 is stored in advance in the ROM 502, RAM 503, or storage device 505. The CPU 501 executes the instructions included in the program 504 to realize each function of each device. For example, the CPU 501 of the image generation system 100 executes the instructions included in the program 504 to realize the functions of the image generation system 100. The RAM 503 may also store data to be processed in each function of each device. For example, the RAM 503 of the computer 500 may store a road image, a stored image, or a predicted image.

[0084] The drive device 507 reads and writes data from and to the recording medium 506. The communication interface 508 provides an interface with a communication network. The input device 509 is, for example, a mouse or keyboard, and accepts information input from a user. The output device 510 is, for example, a display, and outputs (displays) information to the user. The input / output interface 511 provides an interface with peripheral devices. The bus 512 connects these hardware components. The program 504 may be supplied to the CPU 501 via a communication network, or may be stored in advance on the recording medium 506, read by the drive device 507, and supplied to the CPU 501.

[0085] It should be noted that the hardware configuration shown in FIG. 8 is an example, and other components may be added, or some components may not be included.

[0086] There are various variations in the method of realizing each device. For example, each device may be realized by any combination of a different computer and a program for each component. Furthermore, multiple components of each device may be realized by any combination of a single computer and a program.

[0087] Furthermore, some or all of the components of each device may be realized by general-purpose or dedicated circuits including a processor, etc., or a combination of these. These circuits may be configured by a single chip, or may be configured by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and a program.

[0088] Furthermore, when some or all of the components of each device are realized by a plurality of computers, circuits, etc., the plurality of computers, circuits, etc. may be centrally located or distributed.

[0089] Furthermore, at least a part of the image generation system 100 may be provided in a SaaS (Software as a Service) format. That is, at least a part of the functions for realizing the image generation system 100 may be executed by software executed via a network.

[0090] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, the configurations in the embodiments can be combined with each other without departing from the scope of the present disclosure.

[0091] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes.

[0092] [Appendix 1] an acquisition means for acquiring a road image obtained by capturing an image of a road; deterioration level determining means for determining a future deterioration level of said road; a generating means for generating a predicted image based on the road image, the predicted image showing road deterioration corresponding to the deterioration level on the road; An image generation system comprising:

[0093] [Appendix 2] The predicted image is an image showing at least one of road deteriorations such as cracks, potholes, ruts, and irregularities in flatness. 2. The image generation system of claim 1.

[0094] [Appendix 3] The generating means generates the predicted image using a learning model. 3. The image generation system of claim 1 or 2.

[0095] [Appendix 4] The generating means generates the predicted image by superimposing a graphic representing road deterioration on the road image. 4. An image generation system according to any one of claims 1 to 3.

[0096] [Appendix 5] The generating means acquiring a stored image representing road deterioration of another road corresponding to the determined deterioration level; The predicted image is generated based on the acquired stored image and the road image. 5. An image generation system according to any one of claims 1 to 4.

[0097] [Appendix 6] The deterioration degree is an index that represents the degree of at least one of cracks, potholes, ruts, and flatness abnormalities. 6. An image generation system according to any one of claims 1 to 5.

[0098] [Appendix 7] the deterioration level determining means determines a type of road deterioration to be represented in the predicted image; The generating means generates the predicted image in which the determined type of road deterioration is represented in the predicted image. 7. An image generation system according to any one of claims 1 to 6.

[0099] [Appendix 8] 8. The image generation system according to claim 7, wherein the deterioration level determination means determines the type of road deterioration to be represented in the predicted image from among linear cracks, tortoiseshell cracks, and potholes.

[0100] [Appendix 9] The deterioration level determining means determines the deterioration level predicted based on the parameters related to the road as the future deterioration level of the road. 9. An image generation system according to any one of claims 1 to 8.

[0101] [Appendix 10] The deterioration level determining means determines the deterioration level predicted based on the rate of progress of road deterioration as the future deterioration level of the road. 10. The image generation system of claim 9.

[0102] [Appendix 11] The deterioration level determining means determines the deterioration level at a time designated by a user as the future deterioration level of the road. 11. An image generation system according to any one of claims 1 to 10.

[0103] [Appendix 12] The deterioration level determining means determines the deterioration level input by the user as the future deterioration level of the road. 9. An image generation system according to any one of claims 1 to 8.

[0104] [Appendix 13] The predictive image may further include a display control unit for displaying the predictive image. 13. An image generation system according to any one of claims 1 to 12.

[0105] [Appendix 14] The display control means displays, together with the predicted image, any of the deterioration level, the predicted time required for road deterioration to progress from the deterioration level of the road image to the deterioration level of the predicted image, and the road image. 14. The image generation system of claim 13.

[0106] [Appendix 15] the deterioration level determining means determines the deterioration levels at a plurality of future time points; the generating means generates a plurality of predicted images in which road deterioration corresponding to each of the deterioration levels at the plurality of time points is displayed on the road; The display control means displays a plurality of the predicted images. 15. The image generation system of claim 13 or 14.

[0107] [Appendix 16] the generating means further generates a graph representing the relationship between time and the deterioration level on the road; the display control means displays the graph and causes a correspondence between the displayed predicted image and the graph to be displayed. 16. An image generation system according to any one of appendices 13 to 15.

[0108] [Appendix 17] The display control means displays a screen for accepting selection of a predicted image to be displayed from among the plurality of predicted images. 17. An image generation system according to any one of appendices 13 to 16.

[0109] [Appendix 18] The display control means displays a map that accepts designation of the road on which the predicted image is to be displayed. 18. An image generation system according to any one of appendices 13 to 17.

[0110] [Appendix 19] A road image is acquired by capturing a road. determining a future deterioration of the road; generating a predicted image showing road deterioration on the road according to the deterioration level based on the road image; Image generation method.

[0111] [Appendix 20] A road image is acquired by capturing a road. determining a future deterioration of the road; generating a predicted image showing road deterioration on the road according to the deterioration level based on the road image; A recording medium that non-temporarily records a program that causes a computer to execute a process. [Explanation of symbols]

[0112] 100 Image Generation System 101 Acquisition Department 102 Deterioration degree determination section 103 Generation part

Claims

1. an acquisition means for acquiring an image captured by a camera mounted on the moving object; a deterioration degree determining means for determining a future deterioration degree of an object photographed by said camera; a generation means for generating, based on the photographed image, a predicted image in which a state of deterioration according to the future deterioration degree is expressed in the photographed image; Equipped with The generating means generating a predicted image by inputting the acquired captured image into a learning model that receives an input of an image and outputs a predicted image that represents a state of deterioration according to the future degree of deterioration in the image; Image generation system.

2. a display control means for distinguishably displaying a state of deterioration that actually exists in the captured image and a state of deterioration according to the future degree of deterioration, the state of deterioration being included in the predicted image; The image generation system of claim 1 .

3. The generated predicted image is an image that visually represents the state of the crack.

3. The image generation system according to claim 1 or 2.

4. the degradation level determining means determines a type of degradation to be represented in the predicted image; The generating means generates the predicted image in which the determined type of degradation is expressed in the captured image. The image generation system according to any one of claims 1 to 3.

5. the deterioration level determining means determines the deterioration levels at a plurality of future time points; The generating means generates a plurality of predicted images representing a state of deterioration corresponding to each of the degrees of deterioration at the plurality of time points.

5. The image generation system according to claim 1.

6. Acquire images taken by a camera mounted on a moving object, determining future degradation of the object being imaged by the camera; generating a predicted image based on the photographed image, the predicted image representing a state of deterioration according to the future degree of deterioration in the photographed image; The generation of the predicted image includes: The acquired captured image is input to a learning model that receives an input of an image and outputs a predicted image that represents a state of deterioration according to the future degree of deterioration. Image generation method.

7. Acquire images taken by a camera mounted on a moving object, determining future degradation of the object being imaged by the camera; generating a predicted image based on the photographed image, the predicted image representing a state of deterioration according to the future degree of deterioration in the photographed image; The generation of the predicted image includes: The acquired captured image is input to a learning model that receives an input of an image and outputs a predicted image that represents a state of deterioration according to the future degree of deterioration. A program that causes a computer to perform a process.

8. an acquisition means for acquiring an image captured by a camera mounted on the moving object; a deterioration degree determining means for determining a future deterioration degree of an object photographed by said camera; a generation means for generating, based on the photographed image, a predicted image in which a state of deterioration according to the future deterioration degree is expressed in the photographed image; Equipped with The generating means generating the predicted image by superimposing a graphic representing a state of deterioration corresponding to the future degree of deterioration or a stored image in a database storing stored images representing the state of deterioration corresponding to the future degree of deterioration on the captured image; Image generation system.

9. Acquire images taken by a camera mounted on a moving object, determining future degradation of the object being imaged by the camera; generating a predicted image based on the photographed image, the predicted image representing a state of deterioration according to the future degree of deterioration in the photographed image; The generation of the predicted image includes: This is carried out by superimposing a graphic representing the state of deterioration corresponding to the future degree of deterioration, or a stored image from a database that stores stored images representing the state of deterioration corresponding to the future degree of deterioration, on the photographed image. Image generation method.

10. Acquire images taken by a camera mounted on a moving object, determining future degradation of the object being imaged by the camera; generating a predicted image based on the photographed image, the predicted image representing a state of deterioration according to the future degree of deterioration in the photographed image; The generation of the predicted image includes: This is carried out by superimposing a graphic representing the state of deterioration corresponding to the future degree of deterioration, or a stored image from a database that stores stored images representing the state of deterioration corresponding to the future degree of deterioration, on the photographed image. A program that causes a computer to perform a process.

Images