Determination program, determination method, and information processing device

Abstract

To improve accuracy of abnormality detection using an image.SOLUTION: An information processing device 10 specifies a slope face region 5a occupied by a slope face and a road face region 5b occupied by a road face when a prescribed region is observed from a prescribed position on the basis of measurement data 3a, 3b, ... obtained by measuring the positions of the slope face and road face in the prescribed region. Then, the information processing device 10 detects a partial region in which an object having a possibility of an abnormality is captured on the basis of fixed point observation data 4 including the image obtained by imaging the prescribed region from the prescribed position. The information processing device 10 determines the presence / absence of the occurrence of an abnormality in the prescribed region on the basis of whether the partial region is included in the slope face region 5a or the road face region 5b.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a determination program, a determination method, and an information processing device. [Background technology]

[0002] Public roads are inspected regularly to ensure safe passage. Road inspections include not only the condition of the road surface, but also, for example, the condition of the slopes on the side of the road. If any abnormalities or signs of problems are discovered during the inspection, repair work is carried out in the affected areas.

[0003] Inspections of roads and their surrounding areas are typically carried out visually by humans. However, it is difficult to conduct visual inspections frequently for each of the vast number of roads. Furthermore, visual inspections are prone to inconsistencies in judgment between inspectors, resulting in inconsistent inspection quality.

[0004] Therefore, it has been considered to use images captured by a camera for inspection. For example, a technology has been proposed that detects changes that have occurred in the travel path of a mobile object or in structures around the travel path based on images captured by a camera that captures the surroundings of the mobile object. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2018 / 155149 Summary of the Invention [Problem to be solved by the invention]

[0006] For example, images taken with a camera can be subjected to segmentation, a type of machine learning technology, to classify the types of objects captured in the image. For an image taken around a road, segmentation can be used to determine whether the object in the image is a road or a rock. By performing segmentation, it is possible to detect fallen rocks on the road.

[0007] However, it can be difficult to determine whether an image is abnormal or not using image segmentation alone. For example, if the slope beside the road is a stone wall, image segmentation will classify each stone in the stone wall as a stone. As a result, even if there are stones in the area adjacent to the road, it is not possible to determine whether the stone is a fallen rock or a stone that is part of the stone wall.

[0008] In one aspect, the present invention aims to improve the accuracy of anomaly detection using images. [Means for solving the problem]

[0009] In one proposal, a determination program is provided that causes a computer to perform the following processing. The computer identifies the slope area occupied by the slope and the road area occupied by the road when observing the predetermined area from a predetermined position based on measurement data that measures the positions of the slope and road surface in the predetermined area.The computer detects a partial area in which an object that may be abnormal is captured based on fixed-point observation data that includes images of the predetermined area captured from the predetermined position.The computer then determines whether an abnormality has occurred in the predetermined area based on whether the partial area is included in the slope area or the road surface area. [Effects of the Invention]

[0010] According to one aspect, the accuracy of anomaly detection using an image is improved. [Brief explanation of the drawings]

[0011] [Figure 1]FIG. 3 is a diagram illustrating an example of a determination method according to the first embodiment. [Figure 2] FIG. 1 is a diagram illustrating an example of a system configuration for inspecting roads. [Figure 3] FIG. 2 is a diagram illustrating an example of hardware of a road inspection server. [Figure 4] FIG. 2 is a block diagram showing the functions of a road inspection server. [Figure 5] FIG. 2 illustrates an example of an image data storage unit. [Figure 6] FIG. 2 illustrates an example of a point cloud data storage unit. [Figure 7] 10 is a flowchart showing an example of a procedure for road inspection processing. [Figure 8] FIG. 10 is a diagram illustrating an example of an image data acquisition method. [Figure 9] FIG. 10 is a diagram illustrating an example of a method for generating fixed point observation data. [Figure 10] FIG. 10 is a diagram illustrating an example of a category classification result. [Figure 11] FIG. 10 is a diagram illustrating an example of a procedure for category classification processing. [Figure 12] FIG. 10 is a diagram showing an example of segmentation of image data showing a stone wall. [Figure 13] FIG. 10 is a diagram illustrating an example of segmentation of point cloud data around a stone wall. [Figure 14] FIG. 10 is a diagram showing an example of rockfall detection using the segmentation results. [Figure 15] 10A and 10B are diagrams illustrating an example of processing for indicating a rockfall portion in image data. [Figure 16] 10A and 10B are diagrams illustrating an example of correction processing for a front-facing image. [Figure 17] FIG. 10 is a diagram illustrating an example of a depth map creation process. [Figure 18] FIG. 10 is a diagram illustrating an example of abnormality detection by differential extraction. [Figure 19] FIG. 10 is a diagram showing an example of abnormality detection based on the speed of crack progression. [Figure 20] FIG. 10 is a diagram showing an example of an abnormality display screen. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present embodiment will be described below with reference to the drawings. Note that each embodiment can be implemented in combination with a plurality of other embodiments within a range that does not contradict each other. [First embodiment] First, a first embodiment will be described. The first embodiment is an abnormality determination method that can detect abnormalities around a road with high accuracy based on images of a slope and a road surface and the positions of the slope and the road surface measured by a LiDAR (Light Detection and Ranging) sensor.

[0013] Fig. 1 is a diagram illustrating an example of a determination method according to a first embodiment. Fig. 1 shows an information processing device 10 that realizes the determination method. The information processing device 10 can implement the determination method described below by, for example, executing a determination program.

[0014] The information processing device 10 includes a storage unit 11 and a processing unit 12. The storage unit 11 is, for example, a memory or a storage device included in the information processing device 10. The processing unit 12 is, for example, a processor or an arithmetic circuit included in the information processing device 10.

[0015] The information processing device 10 acquires image data 2a, 2b,... and measurement data 3a, 3b,... from, for example, a camera 1a and a LiDAR sensor 1b mounted on a vehicle 1. The vehicle 1 is, for example, a route bus that periodically passes through a road including a predetermined area to be investigated. The camera 1a captures images of the outside of the vehicle while the vehicle 1 is traveling and generates image data 2a, 2b,... The image data 2a, 2b,... are assigned date and time information indicating the date and time of capture and location information indicating the location where the image was captured. The location information is, for example, latitude and longitude information measured by a GPS (Global Positioning System). Note that by using GPS in combination with a quasi-zenith satellite system, the location where the image was captured can be measured with high accuracy.

[0016] The measurement data 3a, 3b,... are data obtained by measuring the positions of slopes and road surfaces in a specified area. For example, the LiDAR sensor 1b irradiates a measurement object with laser light and measures the distance and direction to the measurement object based on the reflected light. Based on the measured distance and direction, the three-dimensional coordinates of the point irradiated with the laser light are obtained. These coordinates are relative to the LiDAR sensor 1b. The LiDAR sensor 1b obtains the coordinates of multiple points on the measurement object by irradiating the laser light in different directions around it. The LiDAR sensor 1b then generates point cloud data indicating the coordinates of each measured point as the measurement data 3a, 3b,.... The measurement data 3a, 3b,... are assigned date and time information indicating the date and time of measurement and location information indicating the measurement location. The absolute position (latitude and longitude) of the measured point can be calculated based on the location information indicating the measurement location and the coordinates indicated in the point cloud data.

[0017] The information processing device 10 stores the acquired image data 2a, 2b,... and measurement data 3a, 3b,... in a memory unit 11. A processing unit 12 determines the presence or absence of abnormalities around the road based on the image data 2a, 2b,... and measurement data 3a, 3b,... stored in the memory unit 11.

[0018] For example, based on the measurement data 3a, 3b, ..., the processing unit 12 identifies a slope area 5a occupied by a slope and a road surface area 5b occupied by a road surface when a predetermined area is observed from a predetermined position. The predetermined position is, for example, the position of the camera 1a when an image (an image included in the fixed-point observation data 4) used to determine the presence or absence of an abnormality is captured. The predetermined area is, for example, an area within the range captured in the image used to determine the presence or absence of an abnormality. In other words, the processing unit 12 identifies the slope area 5a and the road surface area 5b from the area that can be observed within the angle of view of the camera 1a in the optical axis direction of the camera 1a, from the position of the camera 1a when the image used to determine the presence or absence of an abnormality was captured.

[0019] The slope area 5a and the road surface area 5b can be identified by, for example, segmenting the point cloud data. For example, the processing unit 12 has a trained model for segmentation based on the point cloud data, which is generated by machine learning. The processing unit 12 inputs the latest point cloud data to the trained model to identify the slope area 5a and the road surface area 5b.

[0020] Furthermore, the processing unit 12 detects a partial area in which a potentially abnormal object is captured based on the fixed point observation data 4, which includes images of a predetermined area captured from a predetermined position. The fixed point observation data 4 is created based on multiple image data 2a, 2b, .... For example, the processing unit 12 extracts multiple pieces of time-series image data (second image data) captured of a predetermined area at different dates and times from the image data 2a, 2b, ... (first image data) stored in the storage unit 11, based on the location information. The processing unit 12 then generates the fixed point observation data 4, which includes the extracted multiple pieces of second image data.

[0021] By detecting partial regions based on the fixed point observation data 4, regions containing, for example, stones 6a, soil 6b, cracks 6c, etc. are detected as partial regions. The detection of partial regions can be achieved by segmenting the image shown in the fixed point observation data 4. For example, the processing unit 12 has a trained model for segmentation based on image data, which is generated by machine learning. The processing unit 12 inputs the latest image data to the trained model to identify the category of objects contained in the image. In this case, the processing unit 12 detects regions containing objects belonging to a predetermined category (stones 6a, soil 6b, cracks 6c, etc.) as partial regions.

[0022] The processing unit 12 can also detect, as a partial region, a region where there is a difference between multiple images of a specific region in the fixed-point observation data 4 taken at different dates and times. For example, the processing unit 12 extracts the difference between the most recent image data of the specific region and image data taken one month ago. If a rockfall occurred within one month, the region in which the stone 6a appears is extracted as the difference. Similarly, if soil 6b has accumulated on a slope within one month, the region in which the soil 6b appears is extracted as the difference.

[0023] Then, the processing unit 12 determines whether or not an abnormality has occurred in a predetermined region based on whether the partial region is included in the slope region 5a or the road surface region 5b. For example, assume that an area containing a stone 6a is detected as a partial region. In this case, if the partial region is within the road surface region 5b, the processing unit 12 determines that an abnormality (type: "falling rock") exists in the partial region. Also assume that an area containing a crack 6c is detected as a partial region. In this case, if the partial region is within the slope region 5a, the processing unit 12 determines that an abnormality (type: "crack in slope") exists in the partial region.

[0024] The processing unit 12 may detect an area in each of the multiple images where a crack appears as a partial area, and when it determines that a crack exists on the slope, it may determine the risk level according to the size and progression speed of the crack along with the determination. For example, the processing unit 12 may increase the risk level as the progression speed of the crack increases.

[0025] In this way, it is possible to accurately determine whether an abnormality has occurred based on whether a partial area in which a potentially abnormal object is captured is within the slope area 5a or the road surface area 5b. For example, even if a stone is captured in the image, if the partial area in which the stone is captured is within the slope area 5a, the stone is considered to be part of a stone wall and is not abnormal. On the other hand, if the partial area in which the stone is captured is within the road surface area 5b, it can be correctly determined that an abnormality such as a falling rock has occurred.

[0026] Furthermore, by determining the slope area 5a and the road surface area 5b by segmenting the point cloud data acquired as the measurement data 3a, 3b, ..., it is possible to determine the slope area 5a and the road surface area 5b with high accuracy. By improving the accuracy of determining the slope area 5a and the road surface area 5b, the accuracy of determining abnormalities also improves.

[0027] Furthermore, image segmentation identifies the category of objects in an image, and by detecting areas containing objects belonging to a specific category as partial regions, it is possible to extract areas containing objects of any category, making it easier to detect falling rocks, cracks, etc.

[0028] Furthermore, by detecting areas where there are differences between multiple images taken at different dates and times as partial regions, it is possible to prevent oversight of abnormalities. For example, if something that should be there is missing, such as a fallen guardrail, it cannot be detected using only the most recent image. By detecting differences with past images, it is also possible to detect abnormalities such as a fallen guardrail. Furthermore, by treating areas where there are differences between multiple images as partial regions, it is possible to extract as partial regions areas that are difficult to distinguish using segmentation, such as cracks in slopes. As a result, it is possible to prevent oversight of abnormalities.

[0029] Areas where there are differences between multiple images taken at different dates and times are detected as partial areas, and if it is determined that there is a crack on the slope, the risk level can be determined according to the size and rate of progression of the crack, making it possible to accurately determine the risk level of the crack on the slope.

[0030] Furthermore, by detecting abnormalities based on images captured by a camera 1a mounted on a vehicle 1 such as a route bus, it is possible to quickly detect any abnormalities that may occur. For example, by transmitting image data representing images captured by the camera 1a to the information processing device 10 in real time via wireless communication, it becomes possible to quickly discover and deal with abnormalities on the road surface or slope. As a result, safety around the road can be improved.

[0031] Second Embodiment Next, we will explain the second embodiment. The second embodiment is a system that captures images of road conditions using a camera mounted on a vehicle that periodically patrols a road to be monitored, and detects abnormalities on the road at an early stage based on the captured images.

[0032] 2 is a diagram showing an example of a system configuration for inspecting roads. A road inspection server 100, a GIS (Geographic Information System) server 200, and a weather information server 300 are connected to a network 20. The network 20 also includes a wireless communication base station 20a.

[0033] The road inspection server 100 is a computer that detects abnormalities in roads, slopes, etc. based on information such as images showing the road condition. The GIS server 200 is a computer that uses geographical locations as clues to manage and process data containing information about those locations. The weather information server 300 is a computer that distributes weather information.

[0034] Vehicles 30, 40 are used to collect information indicating road conditions. Vehicle 30 is a vehicle that patrols the monitored roads at a high frequency and on a regular basis, such as a route bus, school bus, or postal vehicle. Vehicle 30 is equipped with a camera 31 that can wirelessly transmit data to road inspection server 100 via base station 20a. Camera 31 captures images of the road and its surroundings while vehicle 30 is traveling, and transmits the captured images to road inspection server 100. When inspecting mainly road slopes, camera 31 is installed facing sideways relative to the traveling direction of vehicle 30 (for example, 90 degrees off from the traveling direction).

[0035] The vehicle 40 is a vehicle used by road inspection workers for inspections. The vehicle 40 is equipped with a LiDAR sensor 41 that can wirelessly transmit data to the road inspection server 100 via the base station 20a. The vehicle 40 travels on a road to be inspected. While the vehicle 40 is traveling, the LiDAR sensor 41 measures the distance from the vehicle 40 to points on the road and surrounding objects, as well as the direction of those points. For each measured point on the object, the LiDAR sensor 41 calculates coordinates indicating the position of that point, and generates point cloud data indicating the coordinates of each of the measured points. The LiDAR sensor 41 transmits the generated point cloud data to the road inspection server 100. When mainly inspecting road slopes, the LiDAR sensor 41 is installed sideways to the traveling direction of the vehicle 40 (for example, in a direction offset by 90 degrees from the traveling direction).

[0036] FIG. 3 is a diagram showing an example of hardware of a road inspection server. The road inspection server 100 is entirely controlled by a processor 101. A memory 102 and multiple peripheral devices are connected to the processor 101 via a bus 109. The processor 101 may be a multiprocessor. The processor 101 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). At least some of the functions realized by the processor 101 executing a program may be realized by an electronic circuit such as an ASIC (Application Specific Integrated Circuit) or a PLD (Programmable Logic Device).

[0037] The memory 102 is used as a main storage device of the road inspection server 100. The memory 102 temporarily stores at least a part of the OS (Operating System) programs and application programs to be executed by the processor 101. The memory 102 also stores various data used in processing by the processor 101. As the memory 102, for example, a volatile semiconductor storage device such as a RAM (Random Access Memory) is used.

[0038] The peripheral devices connected to the bus 109 include a storage device 103, a GPU (Graphics Processing Unit) 104, an input interface 105, an optical drive device 106, a device connection interface 107, and a network interface 108.

[0039] The storage device 103 writes and reads data electrically or magnetically to and from a built-in recording medium. The storage device 103 is used as an auxiliary storage device for the road inspection server 100. The storage device 103 stores the OS program, application programs, and various data. Note that the storage device 103 may be, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0040] The GPU 104 is an arithmetic unit that performs image processing and is also called a graphics controller. The GPU 104 is connected to a monitor 21. The GPU 104 displays an image on the screen of the monitor 21 in accordance with an instruction from the processor 101. The monitor 21 may be a display device using organic EL (Electro Luminescence) or a liquid crystal display device.

[0041] The input interface 105 is connected to a keyboard 22 and a mouse 23. The input interface 105 transmits signals sent from the keyboard 22 and the mouse 23 to the processor 101. The mouse 23 is an example of a pointing device, and other pointing devices can also be used. Examples of other pointing devices include a touch panel, a tablet, a touch pad, and a trackball.

[0042] The optical drive device 106 uses a laser beam or the like to read data recorded on an optical disc 24 or write data to the optical disc 24. The optical disc 24 is a portable recording medium on which data is recorded so that it can be read by reflected light. The optical disc 24 includes a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable) / RW (Rewritable), and the like.

[0043] The device connection interface 107 is a communication interface for connecting peripheral devices to the road inspection server 100. For example, a memory device 25 or a memory reader / writer 26 can be connected to the device connection interface 107. The memory device 25 is a recording medium equipped with a function for communicating with the device connection interface 107. The memory reader / writer 26 is a device for writing data to the memory card 27 or reading data from the memory card 27. The memory card 27 is a card-type recording medium.

[0044] The network interface 108 is connected to the network 20. The network interface 108 transmits and receives data to and from other computers or communication devices via the network 20. The network interface 108 is a wired communication interface connected by a cable to a wired communication device such as a switch or a router. The network interface 108 may also be a wireless communication interface connected by radio waves to a wireless communication device such as a base station or an access point.

[0045] The road inspection server 100 can be realized by the hardware described above. The GIS server 200 and the weather information server 300 can also be realized by the same hardware as the road inspection server 100. The information processing device 10 shown in the first embodiment can also be realized by the same hardware as the road inspection server 100 shown in FIG. 3.

[0046] The road inspection server 100 realizes the processing functions of the second embodiment by executing a program recorded on, for example, a computer-readable recording medium. The program describing the processing to be executed by the road inspection server 100 can be recorded on various recording media. For example, the program to be executed by the road inspection server 100 can be stored in a storage device 103. The processor 101 loads at least a part of the program in the storage device 103 into the memory 102 and executes the program. The program to be executed by the road inspection server 100 can also be recorded on a portable recording medium such as the optical disc 24, the memory device 25, or the memory card 27. The program stored on the portable recording medium becomes executable after being installed on the storage device 103 under the control of, for example, the processor 101. The processor 101 can also read and execute the program directly from the portable recording medium.

[0047] In the system shown in FIG. 2, vehicle 30 sends road images to road inspection server 100 at high frequency (for example, several times a day). The sent images are accompanied by location information acquired by a positioning system such as GPS. Vehicle 40 also sends point cloud data to road inspection server 100 at regular intervals. The sent point cloud data is accompanied by location information acquired by a positioning system such as GPS. Road inspection server 100 uses images and point cloud data of the same location to determine whether or not there are any abnormalities on the road and its surroundings.

[0048] 4 is a block diagram showing the functions of the road inspection server 100. The road inspection server 100 has an image data receiving unit 110, an image data storage unit 120, a point cloud data receiving unit 130, a point cloud data storage unit 140, an image preprocessing unit 150, a segmentation unit 160, a difference detection unit 170, and a visualization unit 180.

[0049] The image data receiving unit 110 receives images sent from the vehicle 30. For example, the vehicle 30 has an image data transmitting unit 32 that transmits image data showing images captured by the camera 31. For example, the image data transmitting unit 32 adds, to the image data, location information of the vehicle 30 at the time of capturing the image and date and time information showing the date and time of the image capture, and transmits the image data including the location information and date and time information to the road inspection server 100. The image data receiving unit 110 receives the image data including the location information and date and time information from the image data transmitting unit 32. The image data receiving unit 110 stores the received image data in the image data storage unit 120.

[0050] The image data storage unit 120 stores image data. For example, a part of the memory 102 of the road inspection server 100 or the storage area of ​​the storage device 103 is used as the image data storage unit 120.

[0051] The point cloud data receiving unit 130 receives point cloud data sent from the vehicle 40. For example, the vehicle 40 has a point cloud data transmitting unit 42 that transmits point cloud data indicating the position of each point measured by the LiDAR sensor 41. For example, the point cloud data transmitting unit 42 adds position information of the vehicle 40 at the time of measurement and date and time information indicating the date and time of photography to the point cloud data, and transmits the point cloud data including the position information and date and time information to the road inspection server 100. The point cloud data receiving unit 130 receives the point cloud data including the position information and date and time information from the point cloud data transmitting unit 42. The point cloud data receiving unit 130 stores the received point cloud data in the point cloud data storage unit 140.

[0052] The point cloud data storage unit 140 stores the point cloud data. For example, a part of the storage area of ​​the memory 102 or the storage device 103 of the road inspection server 100 is used as the point cloud data storage unit 140.

[0053] The image preprocessing unit 150 preprocesses the image to make it easier to analyze. For example, the image preprocessing unit 150 performs processes such as normalizing the color information of the image and creating a depth map. The image preprocessing unit 150 can also perform image binarization and contour extraction. The image preprocessing unit 150 sends the preprocessed image to the segmentation unit 160.

[0054] The segmentation unit 160 performs segmentation using AI (Artificial Intelligence) technology. For example, the segmentation unit 160 has a trained model for determining the type of object appearing in an image. The model is represented by, for example, a neural network. The segmentation unit 160 inputs an image into the model, performs calculations according to the model, and outputs, as an output result, for each region of the image, a category to which the object appearing in the corresponding region belongs.

[0055] The segmentation unit 160 also performs segmentation based on the point cloud data of the range captured in the image. For example, the segmentation unit 160 has a trained model for determining the type of object represented by the point cloud data. The model is represented by, for example, a neural network. The segmentation unit 160 inputs the point cloud data into the model, performs calculations according to the model, and outputs as an output result whether the object represented by the point cloud data is a slope or a road surface.

[0056] The segmentation unit 160 detects problem areas such as falling rocks based on the results of segmenting the image and point cloud data. For example, if a stone of a predetermined size or larger is found on the road, the segmentation unit 160 determines that the stone is a fallen rock.

[0057] The difference detection unit 170 detects the difference between multiple images taken at the same location on different dates and times. When detecting the difference between images, the difference detection unit 170 uses images that have been preprocessed by, for example, the image preprocessing unit 150. The difference detection unit 170 determines whether or not there is an abnormality based on the difference between the images taken on different dates and times. For example, the difference detection unit 170 determines that there is an abnormality when a crack in a slope is rapidly expanding.

[0058] When the segmentation unit 160 or the difference detection unit 170 detects an abnormality, the visualization unit 180 outputs information about the abnormal location. For example, the visualization unit 180 outputs an image of the location where the abnormality was detected, along with details of the detected abnormality. The visualization unit 180 also determines the degree of danger for the location where the abnormality was detected, and outputs information indicating the degree of danger. The degree of danger is determined based on, for example, the amount of accumulated earth and sand, the size of the falling rocks, the size and progression speed of the crack, etc.

[0059] The visualization unit 180 can also determine the risk level using information obtainable from the GIS server 200. For example, the GIS server 200 has a slope information DB 210 and a risk assessment DB 220. The slope information DB 210 stores information such as the type of slope for each road location. The risk assessment DB 220 stores information on the risk level of the road or slope for each road location. The GIS server 200 can also obtain weather information from the weather information server 300, calculate the risk level of the slope based on the most recent weather information, and store the calculated risk level in the risk assessment DB 220. For example, the weather information server 300 has a weather information DB 310 that stores information on past weather (such as rainfall). The GIS server 200 obtains weather information around the road to be inspected from the weather information DB 310 of the weather information server 300 and reflects the information in the risk level of the corresponding location. For example, the GIS server 200 sets a high risk level when an abnormality is detected on the road in a location where the total rainfall over the past few days is above a predetermined value.

[0060] Note that the lines connecting the elements shown in Fig. 4 indicate part of the communication paths, and communication paths other than those shown in the figure can also be set. Furthermore, the function of each element shown in Fig. 4 can be realized, for example, by having a computer execute a program module corresponding to that element.

[0061] For example, each time the road inspection server 100 receives new image data, it determines whether or not there are any abnormalities in the road, slopes, etc. at the location shown in the image data. In doing so, the road inspection server 100 determines whether or not there are any abnormalities not only from the image shown in the image data to be determined, but also from point cloud data around the location where the image was taken. The road inspection server 100 also compares the image data with image data showing past images of the same location to determine whether or not there are any abnormalities. The point cloud data of the location where the image was taken can be obtained from the point cloud data storage unit 140, and image data showing past images can be obtained from the image data storage unit 120.

[0062] 5 is a diagram showing an example of an image data storage unit. A plurality of image data 121, 122, ... are stored in the image data storage unit 120. Each of the image data 121, 122, ... is assigned the date and time when the image was captured and the position coordinates (latitude, longitude) of the capture location.

[0063] 6 is a diagram showing an example of a point cloud data storage unit. The point cloud data storage unit 140 stores a plurality of point cloud data 141, 142,... Each point cloud data 141, 142,... is assigned the date and time when the position of the point was measured and the position coordinates (latitude, longitude) of the LiDAR sensor 41 at the time of measurement. The point cloud data 141, 142,... indicates the three-dimensional relative coordinates of the plurality of points whose positions have been measured, as viewed from the LiDAR sensor 41.

[0064] 5 and 6, the image data 121, 122,... and the point cloud data 141, 142,... include date and time (date and time information) and position coordinates (position information). Based on these position coordinates, it is possible to identify point cloud data around the location where the image data to be determined for the presence or absence of an abnormality was taken. Furthermore, based on the position coordinates, it is possible to identify past image data that was taken in the same location as the image data to be determined for the presence or absence of an abnormality.

[0065] Next, the procedure of the road inspection process in the road inspection server 100 will be described with reference to a flowchart. Figure 7 is a flowchart showing an example of the procedure for road inspection processing. The processing shown in Figure 7 will be explained below in order of step number. The road inspection server 100, for example, performs road inspection processing periodically (every hour, every day, etc.). The road inspection server 100 can also perform road inspection processing in real time every time new image data is received.

[0066] [Step S101] The image preprocessing unit 150 selects the latest uninspected image data. For example, the image preprocessing unit 150 selects one of the uninspected image data from the image data received since the previous road inspection process. In addition, when performing road inspection in real time, the image preprocessing unit 150 selects new image data stored in the image data storage unit 120 as the image data to be inspected.

[0067] [Step S102] The image preprocessing unit 150 acquires past image data of the same location as the selected image data and the latest point cloud data of the same location. For example, the image preprocessing unit 150 acquires, from the image data storage unit 120, past image data to which photographing location coordinates that can be considered the same as the photographing location coordinates of the selected image data within a predetermined error range are assigned. The image preprocessing unit 150 also acquires, from the point cloud data storage unit 140, the latest point cloud data to which measurement location coordinates that can be considered the same as the photographing location coordinates of the selected image data within a predetermined error range are assigned.

[0068] [Step S103] The image preprocessing unit 150 creates fixed-point observation data for the shooting location of the selected image data. For example, the image preprocessing unit 150 arranges the image data shot at the corresponding location in chronological order to create the fixed-point observation data.

[0069] [Step S104] The image preprocessing unit 150 and the segmentation unit 160 work together to perform a categorization process on the selected image data. The categorization process will be described in detail later (see FIG. 11).

[0070] [Step S105] The segmentation unit 160 determines whether an abnormality has been detected by categorization. For example, if a stone of a predetermined size or larger is present on the road, the segmentation unit 160 determines that an abnormality, "falling rock," has occurred. If an abnormality is detected, the segmentation unit 160 proceeds to step S109. If no abnormality is detected, the segmentation unit 160 proceeds to step S106.

[0071] [Step S106] The image preprocessing unit 150 performs preprocessing on the time-series image data to emphasize feature points. For example, the image preprocessing unit 150 aligns the time-series image data based on feature points. The image preprocessing unit 150 also performs one or more of the following image processing processes on each piece of time-series image data: angle correction, depth map creation, color information normalization, binarization, and contour extraction, to generate one or more processed image data sets per image. This preprocessing reduces the effects of sunlight conditions, shade, vegetation, and the like, and emphasizes changes.

[0072] [Step S107] The difference detection unit 170 detects differences in images over time based on the fixed-point observation data. For example, the difference detection unit 170 extracts image data at monthly intervals and detects the degree of progress of cracks in the slope each month.

[0073] [Step S108] The difference detection unit 170 determines whether or not there is an abnormality based on the image difference. If there is an abnormality, the difference detection unit 170 proceeds to step S109. If there is no abnormality, the difference detection unit 170 proceeds to step S110.

[0074] [Step S109] The visualization unit 180 outputs anomaly information indicating the details of the detected anomaly. For example, the visualization unit 180 displays on the monitor 21 information such as the amount of change in the crack over time, the amount of precipitation at the relevant point, and the alert zone level at the relevant point.

[0075] [Step S110] The image pre-processing unit 150 determines whether to end the road inspection process. For example, the image pre-processing unit 150 determines to end the road inspection process when an operation is performed to instruct the end of the road inspection process. The image pre-processing unit 150 also determines to end the road inspection process when there is no uninspected image data remaining in the periodic road inspection process. If the image pre-processing unit 150 determines to end the road inspection process, it ends the process. If the image pre-processing unit 150 determines to continue the road inspection process, it proceeds to step S101.

[0076] Based on the image data acquired in this way, abnormalities in the road and its surroundings can be automatically detected. For example, if the vehicle 30 is a route bus, the vehicle 30 will travel on the same road at approximately the same time every day. If the image data of the images taken by the camera 31 mounted on the vehicle 30 is transmitted in real time to the road inspection server 100 and road inspection processing is performed, road abnormalities can be found early.

[0077] Fig. 8 is a diagram showing an example of an image data acquisition method. In Fig. 8, a route bus route 51a along which a route bus vehicle 30 travels is shown on a map 51. The route bus vehicle 30 travels along the route bus route 51a at a fixed time every day. A camera 31 mounted on the vehicle 30 captures images of the outside of the vehicle while traveling. Then, image data representing the captured images is transmitted to the road inspection server 100 by an image data transmission unit 32.

[0078] For example, image data 52 of an image taken at inspection point 51b is assigned location information of inspection point 51b and the date and time of the image taking. On receiving the image data, the road inspection server 100 can recognize that the image data 52 includes an image of the area around inspection point 51b based on the location information.

[0079] Such image data is stored in the image data storage unit 120. The image preprocessing unit 150 can generate fixed-point observation data at a specific survey point based on the image data stored in the image data storage unit 120.

[0080] 9 is a diagram showing an example of a method for generating fixed-point observation data. For example, suppose that the road inspection server 100 received image data 53 showing an image of a certain location from the vehicle 30 in July. In this case, the image preprocessing unit 150 acquires, for example, past image data 54, 55, etc., for the same location as the image data 53, in monthly increments, from the image data storage unit 120. For example, the image data 54 is the image data of an image taken one month before (in June) the image data 53 was taken, and has location information closest to that of the image data 53. Furthermore, the image data 55 is the image data of an image taken two months before (in May) the image data 53 was taken, and has location information closest to that of the image data 53.

[0081] Even if images were taken on a bus traveling the same route, the areas captured on the screen will differ between images taken on different days. Therefore, the image preprocessing unit 150 roughly aligns the other image data 54, 55, etc. based on the coordinates of the shooting locations, using, for example, image data 53 as a reference. Furthermore, the image preprocessing unit 150 performs feature matching between, for example, image data 53 and each of the other image data 54, 55, etc. Furthermore, the image preprocessing unit 150 performs processes such as enlarging and reducing, shifting the position up and down and left and right, and rotating each of the image data 54, 55, etc. so that characteristic parts of the images overlap. The image preprocessing unit 150 then creates fixed-point observation data 60 by stitching together the processed multiple images 61, 62, 63, etc. in chronological order.

[0082] By correcting images using feature matching in this way, it is possible to absorb deviations in the route taken while driving, differences in weather conditions, differences in the angle of view, etc. The image data to be inspected is then subjected to a segmentation model trained through machine learning to categorize the image regions, determining the category to which objects moving into each region belong.

[0083] 10 is a diagram showing an example of a category classification result. For example, the segmentation unit 160 performs segmentation of the image data 52 by inputting the image data 52 to a trained model for segmentation. The segmentation unit 160 outputs category information 71 indicating the category of each region of the image as a result of the segmentation.

[0084] The category information 71 indicates categories such as slopes 71a and 71b, plants 71c, road surfaces 71d, and stones 71e. Each category indicates the category to which objects appearing in the area of ​​image data 52 that overlaps with the area of ​​that category belong when category information 71 is overlaid on image data 52.

[0085] The segmentation unit 160 determines whether or not there is an abnormality by analyzing the category information 71. For example, if there is a large stone 71e on the road surface 71d, the segmentation unit 160 can determine that a rockfall has occurred.

[0086] 10, if a large stone 71e is clearly located in the middle of the road surface 71d, it is possible to determine that this is a fallen rock based solely on the category information generated from the image data 52. However, if a stone is located at the edge of the road surface 71d, it is difficult to distinguish whether the stone is a fallen rock or a stone that is part of a stone wall. Therefore, the segmentation unit 160 improves the accuracy of anomaly detection by categorization by also using segmentation based on point cloud data.

[0087] 11 is a diagram showing an example of a procedure for category classification processing. The processing shown in FIG. 11 will be explained below in order of step number. [Step S201] The segmentation unit 160 performs segmentation based on image data using a trained model for segmenting image data.

[0088] [Step S202] The segmentation unit 160 performs segmentation based on the point cloud data obtained from the LiDAR sensor 41 using a trained model for segmenting point cloud data. Segmentation based on point cloud data can clearly distinguish between, for example, a road surface and a slope. For example, the segmentation unit 160 generates a plane that includes multiple points indicated in the point cloud data, and if the generated plane is nearly horizontal (the tilt angle is equal to or less than a predetermined value), the segmentation unit 160 categorizes the plane as a "road surface." Furthermore, if the segmentation unit 160 generates a plane that includes multiple points indicated in the point cloud data, and if the generated plane is nearly vertical (the tilt angle is equal to or greater than a predetermined value), the segmentation unit 160 categorizes the plane as a "slope."

[0089] [Step S203] The segmentation unit 160 combines the category information obtained by the segmentation based on the image data with the category information obtained by the segmentation based on the point cloud data. For example, the segmentation unit 160 calculates the logical product of the area in the category "stone" in the category information obtained by the segmentation based on the image data and the area in the category "road surface" in the category information obtained by the segmentation based on the point cloud data. This extracts the area containing the image of the falling rock.

[0090] In this way, by performing segmentation based on point cloud data in addition to image data, the accuracy of category classification can be improved. For example, even if a slope is a stone wall, it is possible to clearly distinguish between the stones that make up the wall and fallen rocks.

[0091] Fig. 12 is a diagram showing an example of segmentation of image data showing a stone wall. When image data 72 showing a stone wall is segmented using a model trained by AI, the categories of objects shown in the image data 72 are indicated in category information 73. In the example of Fig. 12, category information 73 indicates that the image contains a plant 73a, numerous stones 73b that make up the stone wall, a road surface 73c, and stones 73d and 73e on the road surface 73c.

[0092] In this case, the segmentation unit 160 can clearly determine that stones 73d and 73e that are on road surface 73c are fallen rocks. However, the segmentation unit 160 cannot determine whether many stones 73b that make up the stone wall are fallen rocks on the road surface or part of the stone wall, based only on category information 73 based on image data 72.

[0093] Therefore, the segmentation unit 160 performs segmentation based on the point cloud data acquired by the LiDAR sensor 41. FIG. 13 is a diagram showing an example of segmentation of point cloud data around a stone wall. The point cloud shown in point cloud data 74 is a point cloud that exists within a view volume (a space indicating the range of the display object) when the same optical axis and angle of view as those of image data 72 are used. In other words, the point cloud is a large number of points on the surface of an object captured in image data 72. The point cloud data 74 indicates the position of each point in the point cloud. In the point cloud data 74, the point cloud on the stone wall exists on a plane that is slightly tilted from the vertical. The point cloud on the road surface exists on a nearly horizontal plane. When segmentation is performed on this point cloud data 74 using a model trained by AI, category information 75 is generated. The category information 75 includes a slope 75a and a road surface 75b.

[0094] The segmentation unit 160 detects falling rocks based on category information 73 obtained from the image data 72 and category information 75 obtained from the point cloud data 74 . FIG. 14 is a diagram showing an example of rockfall detection using the results of segmentation. For example, the segmentation unit 160 binarizes category information 73 obtained from image data 72 into stones and other items. In the example of FIG. 14, binary data 76 is generated in which stones are represented as "1 (white)" and other items are represented as "0 (black)." The segmentation unit 160 also binarizes category information 75 obtained from point cloud data 74 acquired by the LiDAR sensor 41 into road surfaces and other items. In the example of FIG. 14, binary data 77 is generated in which road surfaces are represented as "1 (white)" and other items are represented as "0 (black)."

[0095] The segmentation unit 160 performs a pixel-by-pixel logical AND operation on the two binary data 76, 77. That is, the segmentation unit 160 compares pixels at the same position in the binary data 76, 77, and generates binary data 78 in which only pixels that are both "1 (white)" are left as "1 (white)" and all other pixels are "0 (black)." In the binary data 78, only the fallen rocks are shown in white.

[0096] The segmentation unit 160 can clearly show falling rocks in the image data 72 by superimposing the binary data 78 generated by the logical product on the image data 72. FIG. 15 is a diagram showing an example of processing for highlighting rockfall portions in image data. For example, the segmentation unit 160 converts the "1 (white)" portion of the binary data 78 into a semi-transparent highlight color (e.g., red) and combines it with the image data 72. In the combined image data 79, fallen rocks 79a and 79b are highlighted. The image data 72 and the point cloud data 74 have the same optical axis and angle of view. This makes it easy to perform the combination shown in FIG. 15.

[0097] If the category classification process determines that there is no abnormality, the difference detection unit 170 determines whether or not there is an abnormality based on the change over time in the feature points using the image in which the feature points have been emphasized by processing the image data by the image pre-processing unit 150. For example, the image pre-processing unit 150 calculates the inclination of the slope based on the point cloud data acquired from the LiDAR sensor 41, and corrects the image data so that it is aligned with the optical axis.

[0098] FIG. 16 is a diagram showing an example of correction processing for a front-facing image. For example, if an inclined slope is captured in image data 81, the slope before correction appears farther away at the top. Therefore, the rectangular shape of the slope captured in image data 81 is narrower at the top. The image pre-processing unit 150 calculates the slope's inclination angle with respect to the optical axis based on point cloud data acquired by, for example, the LiDAR sensor 41. The image pre-processing unit 150 then performs image angle correction on the image data 82 to create a front-facing image. This generates image data 82 in which the slope is directly facing the camera's viewpoint. Correcting the image to a front-facing image eliminates image distortion due to differences in perspective of the objects captured in the image, thereby improving the accuracy of difference detection by the difference detection unit 170.

[0099] The image preprocessing unit 150 can also generate a depth map based on the image data. A depth map is data that represents the differences in depth of objects in an image. 17 is a diagram showing an example of a depth map creation process. For example, the image preprocessing unit 150 creates depth maps 84a, 84b, etc. based on multiple pieces of time-series image data 83a, 83b, etc. taken at the same location. By comparing the created depth maps 84a, 84b, etc., areas at different depths become clear. Falling rocks, earth and sand, etc. may exist in areas at different depths.

[0100] 18 is a diagram showing an example of anomaly detection by difference extraction. For example, the difference detection unit 170 extracts the difference between image data 85 taken in June and image data 86 taken at the same location in July. In the example of FIG. 18, part of the image data 86 shows accumulated earth and sand and stones. Difference data 87 generated by difference extraction shows earth and sand 87a, stones 87b, and cracks 87c.

[0101] Comparing the differential data 87 with the category information 75 shown in FIG. 13 reveals that a stone 87b is present on the road surface 75b. Therefore, the stone 87b is a fallen rock and is detected as an abnormality. On the other hand, the earth and sand 87a is located within the slope 75a. If the amount of earth and sand 87a deposited in a depression or the like in the slope 75a exceeds a predetermined amount, it is detected as an abnormality. The crack 87c is located within the slope 75a. If the size of the crack 87c in the slope 75a exceeds a predetermined amount, it is detected as an abnormality.

[0102] In this way, by extracting the difference between the time-series image data 85 and 86, it is possible to easily detect abnormalities such as sediment accumulation, falling rocks, cracks in the slope, etc. Furthermore, the difference detection unit 170 can determine, for example, the speed at which a crack in the slope progresses by comparing the time-series image data.

[0103] 19 is a diagram showing an example of anomaly detection based on the speed of crack growth. For example, image data 91 from May shows a small crack 91a. Image data 92 from June shows a grown crack 92a. Image data 93 from July shows a further grown crack 93a.

[0104] The difference detection unit 170 recognizes the degree of progress of the crack by comparing the time-series image data 91 to 93. The difference detection unit 170 then determines that there is an abnormality if the amount of change in the length of the crack over, for example, one month exceeds a predetermined value. The difference detection unit 170 also determines that there is an abnormality if the length of the crack in the latest image data 93 exceeds a predetermined value.

[0105] When an abnormality is detected, the visualization unit 180 works in conjunction with the GIS to calculate the degree of risk based on information such as the location information and detection results attached to the image data in which the abnormality was detected. The visualization unit 180 then adds an icon indicating the abnormal location on the map obtained from the GIS server 200. When the abnormal location is clicked, the visualization unit 180 displays various information such as confirmation of the time-series data of the abnormal location, changes in differences, and past construction records.

[0106] 20 is a diagram showing an example of an anomaly display screen. When the visualization unit 180 detects an anomaly, it displays, for example, an anomaly detection screen 94 on the monitor 21. The anomaly detection screen 94 is provided with a map display section 94a. The map display section 94a displays a map of the area around the road to be inspected, acquired from the GIS server 200. Furthermore, on the map displayed on the map display section 94a, an icon 94b indicating the detection of an anomaly is displayed at the location where an anomaly has been detected.

[0107] When the icon 94b is selected, the visualization unit 180 displays a detail display section 94c showing details of the abnormality. The detail display section 94c displays, for example, the risk level of the abnormality location. The visualization unit 180 determines the risk level based on the abnormality content and information acquired from the GIS server 200.

[0108] For example, the visualization unit 180 acquires information on the most recent precipitation amount and soil rainfall index around the abnormal location and the degree of risk due to topographical factors from the GIS server 200. The precipitation amount and soil rainfall index are information provided by the weather information server 300. The degree of risk due to topographical factors can be acquired from a hazard map managed by the GIS server 200, for example.

[0109] The visualization unit 180 then determines the degree of danger by taking into account the degree of abnormality (for example, the progression speed of the depth and width of the crack) acquired from image data or the like, the most recent amount of precipitation, and the alert zone level of the relevant area. For example, the visualization unit 180 increases the danger level the deeper the crack. Also, the visualization unit 180 increases the danger level the faster the progression speed of the crack. Also, the visualization unit 180 increases the danger level the more precipitation there is or the higher the soil rainfall index is. Furthermore, the visualization unit 180 increases the danger level the higher the alert zone level is.

[0110] An image of the location where the abnormality was detected is displayed in the detail display section 94c. If the location is a crack in a slope, an enlarged image of the crack location is displayed in the detail display section 94c. The crack depth can be measured by the LiDAR sensor 41. By orienting the LiDAR sensor 41 90 degrees to the side of the traveling direction of the vehicle 40, the LiDAR sensor 41 can measure the distance to the slope from almost the front. By measuring from the front of the slope, if there is a crack in the slope, the laser light emitted by the LiDAR sensor 41 reaches the depth of the crack. As a result, the distance to the depth of the crack can be measured. For example, the difference detection unit 170 determines the depth of the crack to be the difference between the distance to the deepest part of the part determined to be a crack by analyzing the image data and the distance to its periphery.

[0111] The visualization unit 180 may visually display on the map areas that should be repaired with priority, such as areas at high risk of collapse or areas with a wide collapse range, in a distinctive color or size that is easy to understand. For example, the visualization unit 180 can show the difference in risk level with the color of the icon by displaying in red the icon of an abnormality detection area where the risk level is "high" and by displaying in yellow the icon of an abnormality detection area where the risk level is "medium."

[0112] In this way, it becomes possible to quickly detect abnormalities around the road based on image data and point cloud data acquired by the camera 31 and LiDAR sensor 41 mounted on vehicles 30, 40 such as route buses. That is, when a vehicle 30 traveling a fixed route, such as a route bus, passes an abnormal location, image data of the abnormal location is sent to the road inspection server 100, and the abnormality is immediately detected. As a result, falling rocks, accumulation of landslides due to heavy rain, etc. can be quickly detected, and measures such as removing the obstacle or evacuating residents can be quickly taken.

[0113] Other Embodiments In the second embodiment, the camera 31 and the LiDAR sensor 41 are mounted on separate vehicles 30, 40, but the camera 31 and the LiDAR sensor 41 may be mounted on a single vehicle.

[0114] Although the embodiments have been described above, the configuration of each part shown in the embodiments can be replaced with other parts having similar functions. Also, any other components or processes may be added. Furthermore, any two or more configurations (features) of the above-described embodiments may be combined. [Explanation of symbols]

[0115] 1 vehicle 1a Camera 1b LiDAR sensor 2a, 2b,... Image data 3a, 3b, Measurement data 4 Fixed-point observation data 5a Slope area 5b Road area 6a stone 6b Soil and sand 6c Crack 10. Information processing equipment 11 Storage section 12 Processing section

Claims

1. based on measurement data obtained by measuring the positions of a slope and a road surface in a predetermined area, a slope area occupied by the slope and a road surface area occupied by the road surface when the predetermined area is observed from a predetermined position; detecting a partial area in which an object that may be abnormal is captured based on fixed-point observation data including an image of the predetermined area captured from the predetermined position; determining whether or not an abnormality has occurred in the predetermined area based on whether the partial area is included in the slope area or the road surface area; A judgment program that causes a computer to execute a process.

2. the measurement data is point cloud data indicating the positions of a plurality of measurement target points on the slope and the road surface within the predetermined area, In identifying the slope area and the road surface area, the slope area and the road surface area are identified by performing segmentation on the point cloud data using a model that has been trained by machine learning. The determination program according to claim 1.

3. In detecting the partial region, the image shown in the fixed point observation data is segmented using a model trained by machine learning to identify a category of objects appearing in the image, and an area in which an object belonging to a predetermined category is shown is detected as the partial region. The determination program according to claim 1 or 2.

4. In the detection of the partial region, a region in which a stone is photographed is detected as the partial region; In determining whether or not an abnormality has occurred in the predetermined region, if the partial region is within the road surface region, it is determined that a rockfall has occurred in the partial region. The determination program according to claim 3.

5. In the detection of the partial region, a region in which a crack is captured is detected as the partial region; In determining whether or not an abnormality has occurred in the predetermined region, if the partial region is within the slope region, it is determined that a crack exists in the slope. The determination program according to any one of claims 1 to 4.

6. The fixed point observation data shows a plurality of images of the predetermined area taken at different dates and times, In the detection of the partial region, a region where there is a difference between the plurality of images is detected as the partial region. The determination program according to any one of claims 1 to 5.

7. In the detection of the partial region, a region in which a crack appears in each of the plurality of images is detected as the partial region; In determining whether or not an abnormality has occurred in the predetermined region, if the partial region is within the slope region, it is determined that there is a crack in the slope, and a risk level is determined according to the propagation speed of the crack. The determination program according to claim 6.

8. a plurality of first image data pieces showing images taken at different dates and times by a camera mounted on a vehicle that periodically passes through the predetermined area, the plurality of first image data pieces having date and time information indicating the date and time of the image taking and location information indicating the location of the image taking, extracting from the plurality of first image data pieces a plurality of second image data pieces in time series taken of the predetermined area at different dates and times based on the location information, and generating the fixed-point observation data including the extracted plurality of second image data pieces; 8. The determination program according to claim 6, further causing the computer to execute a process.

9. based on measurement data obtained by measuring the positions of a slope and a road surface in a predetermined area, a slope area occupied by the slope and a road surface area occupied by the road surface when the predetermined area is observed from a predetermined position; detecting a partial area in which an object that may be abnormal is captured based on fixed-point observation data including an image of the predetermined area captured from the predetermined position; determining whether or not an abnormality has occurred in the predetermined area based on whether the partial area is included in the slope area or the road surface area; A method for determining whether a process is executed by a computer.

10. a processing unit that identifies a slope area occupied by the slope and a road area occupied by the road when the predetermined area is observed from a predetermined position based on measurement data that measures the positions of a slope and a road surface in the predetermined area, detects a partial area in which an object that may be abnormal is captured based on fixed-point observation data that includes an image of the predetermined area photographed from the predetermined position, and determines whether or not an abnormality has occurred in the predetermined area based on whether the partial area is included in the slope area or the road surface area; An information processing device having the above.

Images