Road management support system

Abstract

To provide a technique for automatically identifying a position of an abnormal place on a road surface.SOLUTION: A road management support system includes: an image acquisition section for acquiring an image of a road surface captured by a camera mounted on a vehicle; a vehicle position acquisition section for acquiring a position of the vehicle; an abnormality detection section for detecting an abnormal place on a road surface in the image; and a position identification section for identifying a point separated by an off-set amount based on a position of the abnormal place in the image from a capture point being the position of the vehicle when the image with the detected abnormal place has been captured as an abnormal point where the abnormal place exists.SELECTED DRAWING: Figure 1

Description

【Technical Field】 【0001】 The present invention relates to a road management support system. 【Background Art】 【0002】 Patent Document 1 describes a method for assisting in creating an inspection report of a highway. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 Japanese Patent Application Laid-Open No. 2018-147314 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 Patent Document 1 has a configuration in which a user visually checks an image of a road taken and determines whether there is an abnormality in the road in the image. Patent Document 1 describes displaying a kilometer post value reflecting a KP offset value for correcting a deviation between the position of the road taken as road image data and the actual position recorded as shooting position data (paragraph 0056), but a specific method for specifying the amount of deviation is not disclosed. The present invention has been made in view of the above problems, and an object thereof is to provide a technique for automatically specifying the position of an abnormal portion of a road surface. 【Means for Solving the Problems】 【0005】 To achieve the above object, a road management support system includes an image acquisition unit that acquires an image of a road surface taken by a camera mounted on a vehicle, a vehicle position acquisition unit that acquires the position of the vehicle, an abnormality detection unit that detects an abnormal portion of the road surface in the image, and a position specifying unit that specifies, as an abnormal point where an abnormal portion exists, a point separated by an offset amount based on the position of the abnormal portion in the image from a shooting point that is the position of the vehicle when an image in which the abnormal portion is detected is taken. 【0006】 In other words, the road management support system detects abnormal areas from images captured by cameras, and identifies an abnormal location as a point that is offset by an amount based on the position of the abnormal area in the image from the point where the image in question was captured. Therefore, the location of an abnormal location where road surface abnormalities exist can be automatically identified. [Brief explanation of the drawing] 【0007】 [Figure 1] A block diagram showing the configuration of the road management support system. [Figure 2] Figures 2A and 2B show the relationship between the camera's field of view and the vehicle. [Figure 3] A diagram showing an example of a captured image. [Figure 4] Flowchart for identifying anomaly locations. [Modes for carrying out the invention] 【0008】 Here, embodiments of the present invention will be described in the following order. (1) Configuration of the road management support system: (2) Anomaly location identification process: (3) Other embodiments: 【0009】 (1) Configuration of the road management support system: Figure 1 is a block diagram showing the configuration of the road management support system 10 according to the present invention. In this embodiment, the road management support system 10 is mounted on a vehicle C. The road management support system 10 includes a control unit 20 equipped with a CPU, RAM, ROM, etc., a recording medium 30, a camera 40, a GNSS receiver 41, a vehicle speed sensor 42, and a communication unit 43. 【0010】 Camera 40 is mounted on vehicle C so as to include the area in front of vehicle C in the direction of travel within its field of view. The optical axis of camera 40 is fixed relative to vehicle C, and it is sufficient that the direction of the optical axis is known to the road management support system 10. In this embodiment, as shown in Figures 2A and 2B, camera 40 is mounted on vehicle C in a manner such that the optical axis center is perpendicular to the vehicle width direction of vehicle C, and the area in front of vehicle C in the direction of travel is included in its field of view. The control unit 20 can detect abnormal areas (potholes, etc.) on the road surface by acquiring the image output by camera 40 and analyzing the image by extracting features, etc. Camera 40 takes images at predetermined intervals, and the control unit 20 acquires the image output by camera 40. 【0011】 The GNSS receiver 41 is a device that receives signals from the Global Navigation Satellite System. The GNSS receiver 41 receives radio waves from navigation satellites and outputs a signal to calculate the current position of vehicle C. The control unit 20 acquires this signal and obtains the position of vehicle C in the map coordinate system. The vehicle speed sensor 42 outputs a signal corresponding to the rotation speed of the wheels of vehicle C. The control unit 20 acquires this signal and obtains the speed of vehicle C. The communication unit 43 is a device for wireless communication with external devices, and the road management support system 10 can communicate with the server 100 via the communication unit 43. 【0012】 The road management support system 10 can execute programs stored in the recording medium 30 or ROM using the control unit 20. In this embodiment, the road management support program 21 can be executed as this program. The communication unit 43 is equipped with a circuit for communicating with the server 100, and the control unit 20 can communicate with the server 100 by processing the road management support program 21. 【0013】 The road management support program 21 enables the control unit 20 to perform a function to identify the location of abnormal areas obtained by analyzing images captured by the camera 40. By executing the road management support program 21, the control unit 20 functions as an image acquisition unit 21a, a vehicle position acquisition unit 21b, an abnormality detection unit 21c, and a position identification unit 21d. 【0014】 The control unit 20 acquires images of the road surface captured by the camera 40 mounted on the vehicle C through the function of the image acquisition unit 21a. That is, the control unit 20 periodically acquires images captured by the camera 40 while the vehicle C is traveling. The control unit 20 acquires the position of the vehicle C through the function of the vehicle position acquisition unit 21b. In this embodiment, the control unit 20 periodically acquires the current position of the vehicle C from the output signal of the GNSS receiver unit 41. 【0015】 The control unit 20 detects abnormal locations on the road surface in the image using the function of the abnormality detection unit 21c. The control unit 20 acquires images continuously captured by the camera 40 and performs corrections for distortions caused by the lens. The control unit 20 determines whether the image contains features such as holes by performing image recognition processing, for example, using YOLO (You Only Look Once) or pattern matching, and detects the coordinates B of the abnormal location on the road surface in front of the vehicle in the image. The control unit 20 also stores the shooting location, which is the position of the vehicle C at the time the image in which the abnormal location was detected was taken. 【0016】 Based on the function of the position identification unit 21d, the control unit 20 identifies a point where an abnormality exists as a point located at an offset amount based on the position of the abnormality in the image, from the shooting point, which is the position of the vehicle when the image in which the abnormality was detected was taken. In this embodiment, the offset amount is the road surface distance corresponding to the distance between the reference position in the image and the abnormality in the image. Figure 3 shows an example of an image I taken by the camera 40. In this embodiment, the horizontal axis of image I is the x-axis and the vertical axis is the y-axis. The x-axis and y-axis are orthogonal. In image I, the x-axis is parallel to the vehicle width direction. The control unit 20 uses the bottom line L0 of image I as the reference position in the image, and divides image I into three blocks B1, B2, and B3 in the y-direction according to the distance from the bottom line L0 in the y-direction, as shown in Figure 3. The control unit 20 then defines the approximate distance to the vehicle C in each block. Specifically, for example, the control unit 20 assumes that the bottommost block B3 of image I is 5 meters from vehicle C, the central block B2 is 10 meters from vehicle C, and the block B1 containing the most information in image I is 15 meters from vehicle C. The control unit 20 identifies the block containing the coordinates indicating the anomaly and estimates the distance from the shooting location where the image containing the anomaly was taken to the anomaly location where the anomaly exists. For example, as shown in Figure 3, if the anomaly is contained in block B2, the control unit 20 estimates that the distance from the shooting location to the anomaly location is 10 meters (the offset amount is 10 meters). 【0017】 The control unit 20 then identifies the position of the vehicle when it has traveled an offset amount from the shooting point as the position of the abnormality point. That is, the control unit 20 acquires the distance traveled from the shooting point, and when it determines that the distance traveled has reached the offset amount, it acquires the position of vehicle C at the time the distance traveled has reached the offset amount using the function of the vehicle position acquisition unit 21b. 【0018】 When the position of the abnormal location is specified in this way, the control unit 20 transmits the position of the abnormal location and the image of the abnormal location (the image in which the abnormal portion is detected) to the server 100 via the communication unit 43. When the server 100 acquires the position of the abnormal location and the image of the abnormal location from the road management support system 10, it stores the position of the abnormal location in association with the image of the abnormal location in a recording medium (not shown). 【0019】 The server 100 provides a map browsing service showing the road surface condition. When receiving a display request for the map from a terminal operated by a person in charge of road maintenance management operations, the server 100 transmits the map display data to the terminal. The display data includes a map, the position of abnormal locations on the map, and icons indicating the abnormal locations. A map is displayed on the terminal using the display data. In this map, when an abnormal location is included within the display range of the map, an icon indicating the abnormal location is displayed at the position of the abnormal location. The person in charge can recognize the position of the abnormal location from the map. 【0020】 As described above, according to the present embodiment, the images periodically captured by the camera mounted on the vehicle are analyzed for each image, and when an abnormal portion is included, the position where the abnormal portion appears in the image is specified. Then, a point separated from the shooting location of the image in which the abnormal portion is detected by an offset amount based on the position of the abnormal portion in the image is specified as the abnormal location where the abnormal portion exists. With such a configuration, according to the road management support system 10, the position of the abnormal location where the abnormal portion of the road surface exists can be automatically specified. If the image in which the abnormal portion is detected and the position of the shooting location of the image are associated and transmitted to the server 100, when the person in charge goes to the site for on-site confirmation or repair work, it is difficult to quickly specify the actual abnormal portion because they will try to specify the abnormal portion around the shooting location separated from the actual abnormal portion by the offset amount. According to the present embodiment, by setting the point separated from the shooting location by the offset amount as the abnormal location, the possibility that the person in charge can quickly specify the actual abnormal portion at the site can be increased. 【0021】 (2) Abnormal location specifying process: Next, the abnormal point identification process executed by the control unit 20 will be described while referring to FIG. 4. The abnormal point identification process is repeatedly executed every time the control unit 20 acquires an image captured by the camera 40 by the function of the image acquisition unit 21a. When the abnormal point identification process is started, the control unit 20 performs an image recognition process by the function of the abnormality detection unit 21c (step S100). That is, the control unit 20 acquires an image captured by the camera 40 by the function of the image acquisition unit 21a, and performs an image recognition process on the image. If the image contains an image indicating an abnormal portion of the road surface, the control unit 20 acquires the coordinates of the abnormal portion in the image. 【0022】 Subsequently, the control unit 20 determines whether an abnormal portion is detected in the image by the function of the abnormality detection unit 21c (step S105). That is, the determination is made using the processing result of step S100. If it is not determined in step S105 that an abnormal portion is detected, the control unit 20 ends the abnormal point identification process for the image targeted this time. 【0023】 If it is determined in step S105 that an abnormal portion is detected, the control unit 20 determines the block in which the abnormal portion exists by the function of the position identification unit 21d (step S110). That is, based on the result of step S100, it is determined whether the coordinates of the detected abnormal portion are included in any of the blocks B1, B2, and B3. If it is determined in step S110 that the abnormal portion is included in block B1, the control unit 20 estimates that the abnormal portion is 15 meters away from the shooting point by the function of the position identification unit 21d (step S115). That is, the control unit 20 regards the offset amount X as 15 meters. 【0024】 In step S110, if it is determined that the area is included in block B2, the control unit 20 estimates that the abnormal area is 10 meters away from the shooting point using the function of the position identification unit 21d (step S120). That is, the control unit 20 considers the offset amount X to be 10 meters. In step S110, if it is determined that the area is included in block B3, the control unit 20 estimates that the abnormal area is 5 meters away from the shooting point using the function of the position identification unit 21d (step S125). That is, the control unit 20 considers the offset amount X to be 5 meters. 【0025】 After step S115, step S120, or step S125 is executed, the control unit 20 uses the function of the position identification unit 21d to calculate the distance traveled from the vehicle speed and elapsed time (step S130). Here, the elapsed time is the time elapsed from the time when the image in which the abnormality was detected was taken. The control unit 20 acquires the output signal of the vehicle speed sensor 42 at a period T and obtains the speed of the vehicle C from the output signal at a period T. The control unit 20 calculates the distance traveled from the vehicle speed at the time of the period T each time a period T has elapsed since the time of the image was taken, and then calculates and calculates the distance traveled. 【0026】 Next, the control unit 20 determines, using the function of the position identification unit 21d, whether or not the vehicle has advanced by an offset amount X (step S135). That is, it is determined whether or not the travel distance accumulated in step S130 (i.e., the travel distance from the shooting point) has reached the offset amount. If it is not determined that the offset amount has been reached, the process returns to step S130. If it is determined that the offset amount has been reached, the control unit 20 uploads the current vehicle position to the server 100, associating it with the image, using the function of the position identification unit 21d (step S140). That is, the control unit 20 acquires the current position of vehicle C based on the output signal of the GNSS receiver 41 using the function of the vehicle position acquisition unit 21b, and identifies the acquired current position as an abnormal location. The control unit 20 then transmits the location of the abnormal location, along with the image in which the abnormal location was detected, to the server 100. The server 100 records the received image and the location of the abnormal location in association with each other on a recording medium (not shown). 【0027】 If no abnormality is detected in the image in step S105, or after step S140 is executed, the control unit 20 terminates the abnormality location identification process. The abnormality location identification process is executed again depending on when the next image is captured by the camera 40. 【0028】 (3) Other embodiments: The embodiments described above are merely examples for carrying out the present invention, and various other embodiments can be adopted. For example, the road management support system 10 may be a device mounted on a vehicle, a device implemented by a portable terminal, or a device implemented by multiple devices (e.g., a client and a server). At least a portion of the image acquisition unit 21a, vehicle position acquisition unit 21b, anomaly detection unit 21c, and position identification unit 21d that constitute the road management support system may be divided among multiple devices. Some configurations of the embodiments described above may be omitted, and the order of processing may be changed or omitted. 【0029】 The road management support system 10 may be implemented by a server. That is, in the above embodiment, the location of the anomaly was determined by an on-board system, but the server 100 may be configured to determine the location of the anomaly. In this case, the server acquires a history of images taken periodically and the location of the location where the images were taken. The history of the location of the location where the images were taken corresponds to the vehicle's movement history. The movement history includes the vehicle's location and the date and time the vehicle passed through that location. The server performs image recognition processing on each image. If an anomaly is detected as a result of the image recognition processing, the server estimates the offset amount between the location of the image where the anomaly was detected and the anomaly based on the image. The control unit 20 identifies a location in the movement history that is offset by the amount from the location of the image where the anomaly was detected as the anomaly, based on the vehicle's movement history. 【0030】 In the vehicle position acquisition unit, the vehicle's current position may be acquired by autonomous navigation using a vehicle speed sensor and a gyro sensor. This makes it possible to acquire the vehicle's position even in sections where GNSS signals cannot be received. 【0031】 In the vehicle position acquisition unit, the acquired vehicle position may be configured to be a position matched using map information. The map information includes node data indicating the location of nodes, shape interpolation point data indicating the location of shape interpolation points for identifying the shape of the section between nodes, and link data indicating the connections between nodes. The vehicle position acquisition unit estimates the vehicle position based on the output signals of the vehicle speed sensor, gyro sensor, and GNSS receiver, and identifies the link the vehicle is traveling on by performing known map matching based on the vehicle position trajectory and map information. The vehicle position may be corrected so that the vehicle position is on the identified link. As a result, the possibility of a decrease in the position accuracy of abnormal areas on the road surface can be reduced. 【0032】 In the positioning unit, the offset amount can be estimated in various ways other than those described in the above embodiment. For example, each coordinate in the image may be associated with the distance between the object captured at that coordinate and the vehicle C, and the unit may be configured to obtain the distance between the abnormal location captured at a specific coordinate and the vehicle C based on this association. 【0033】 Furthermore, the image acquisition unit only needs to be able to acquire images of the road surface taken by a camera mounted on the vehicle, and is not limited to a configuration where images are taken periodically. For example, the system may be configured to have the camera take images of the road surface at a timing instructed by the driver, and to acquire the images taken in this way. 【0034】 In addition to potholes, other road surface abnormalities may include peeling of lane markings and road surface markings, asphalt cracks and bulges, etc. 【0035】 Furthermore, the offset amount may be defined by the time required to travel the distance on the road surface. That is, the system may be configured to estimate the distance on the road surface between the anomaly location and the shooting location, calculate the time required to travel that distance from the vehicle speed of vehicle C, and identify the position of the vehicle at the time that time has elapsed from the time the image in which the anomaly image was detected as the position of the anomaly location. 【0036】 Furthermore, the methods of the present invention can also be applied as programs and methods. Moreover, such systems, programs, and methods may be implemented as standalone devices, or they may be implemented using parts shared with various components of a vehicle, encompassing a variety of embodiments. They can also be modified as appropriate, such as being partly software and partly hardware. Furthermore, the invention also functions as a recording medium for a program that controls the system. Of course, the recording medium for the program may be a magnetic recording medium, a semiconductor memory, or any recording medium developed in the future can be considered in exactly the same way. [Explanation of symbols] 【0037】 10...Road management support system, 20...Control unit, 21...Road management support program, 21a...Image acquisition unit, 21b...Vehicle position acquisition unit, 21c...Anomaly detection unit, 21d...Position identification unit, 30...Recording medium, 40...Camera, 41...GNSS receiver, 42...Vehicle speed sensor, 43...Communication unit, 100...Server, C...Vehicle

Claims

[Claim 1] An image acquisition unit that acquires an image of the road surface captured by a camera mounted on the vehicle, wherein the horizontal direction is parallel to the vehicle's width direction, A vehicle position acquisition unit that acquires the position of the aforementioned vehicle, An abnormality detection unit for detecting abnormal locations on the road surface in the aforementioned image, A position identification unit associates the distance from the shooting point, which is the position of the vehicle when the image was taken, with each of the multiple blocks obtained by dividing the image vertically, identifies the block containing the abnormal area among the multiple blocks, and identifies the position of the vehicle obtained when the vehicle has moved from the shooting point by an offset amount, which is the distance associated with the identified block or the time required to travel that distance, as the abnormal area where the abnormal area exists. A road management support system equipped with the following features. [Claim 2] The position of the vehicle is a position determined by map matching using map information. The road management support system according to claim 1.

Images