Management support system, management support method, and program

Abstract

A management assistance system according to the present disclosure comprises: an acquisition means for acquiring an image captured by a camera mounted on a mobile body and position information pertaining to an imaging point in the image; a detection means for detecting one or more prescribed facilities on a road from the image; a processing object identification means for identifying a facility to be processed among the one or more prescribed facilities on the basis of at least one of the position or the size of each of the facilities in the image; a position calculation means for calculating facility position information, which indicates the geographical position of the facility identified as a processing object, on the basis of the position information and the at least one of the position or the size of the facility identified as the processing object in the image; a management object identification means for identifying, as a management object and on the basis of the facility position information, a facility corresponding to the facility identified as the processing object, from among facilities registered in a database for managing the facilities on the road; and an updating means for updating data pertaining to the facility identified as the management target.

Description

Management support system, management support method, and recording medium

[0001] The present disclosure relates to a management support system and the like.

[0002] Various facilities are installed along or on roads. The managers who manage these facilities perform tasks such as designing and installing new facilities, or maintaining installed facilities. Before starting work, the managers often visit the site to check the facilities and surrounding environment. If the managers could understand the facilities and surrounding environment using images captured by cameras mounted on mobile vehicles, they could save time and effort by visiting the site for advance inspections.

[0003] Patent Document 1 discloses an in-vehicle device that includes a camera, detects road facilities from images captured by the camera, and transmits the images including the road facilities and location information at which the images were captured.

[0004] International Publication No. 2014 / 013545

[0005] When a system detects facilities on a road from an image captured by a camera mounted on a mobile object, facilities far from the mobile object may be detected from the image. The accuracy of identifying the geographical position of distant facilities from the image may be low. Therefore, when updating a database managing facilities on a road based on the geographical position identified from the image for distant facilities, the data may be inaccurate.

[0006] One of the objects of the present disclosure is to provide a management support system etc. that enables more accurate management of data on road facilities.

[0007] The management support system according to the present disclosure includes an acquisition means for acquiring an image captured by a camera mounted on a mobile body and location information of the location where the image was captured; a detection means for detecting one or more specified facilities on a road from the image; a processing target identification means for identifying a facility to be processed from the one or more specified facilities based on at least one of the position or size of each of the facilities in the image; a position calculation means for calculating facility position information indicating the geographical position of the facility identified as the processing target based on the location information and at least one of the position or size of the facility identified as the processing target in the image; a management target identification means for identifying a facility to be managed that corresponds to the facility identified as the processing target from among the facilities registered in a database that manages facilities on the road based on the facility position information; and an update means for updating the data of the facility identified as the management target.

[0008] The management support method according to the present disclosure acquires an image taken by a camera mounted on a mobile body and location information of the location where the image was taken, detects one or more specified facilities on a road from the image, identifies a facility to be processed from the one or more specified facilities based on at least one of the position or size of each of the facilities in the image, calculates facility location information indicating the geographical position of the facility identified as the facility to be processed based on the location information and at least one of the position or size of the facility identified as the facility to be processed, identifies a facility to be managed from among the facilities registered in a database that manages facilities on the road, corresponding to the facility identified as the facility to be processed, based on the facility location information, and updates the data of the facility identified as the facility to be managed.

[0009] A program according to the present disclosure causes a computer to execute a process of acquiring an image captured by a camera mounted on a mobile object and location information of a location where the image was captured, detecting one or more predetermined facilities on a road from the image, identifying a facility to be processed from the one or more predetermined facilities based on at least one of the position or size of each of the facilities in the image, calculating facility location information indicating the geographical location of the facility identified as the processing target based on the location information and at least one of the position or size of the facility identified as the processing target in the image, identifying a facility to be managed that corresponds to the facility identified as the processing target from among facilities registered in a database that manages facilities on the road based on the facility location information, and updating data of the facility identified as the management target. The program may be stored in a computer-readable non-transitory recording medium.

[0010] According to the present disclosure, it is possible to more accurately manage data on road facilities.

[0011] 1 is an explanatory diagram showing an example of connection between the management support system and other devices. FIG. 2 is a block diagram showing an example of the configuration of the management support system. FIG. 3 is a diagram showing an example of driving data. FIG. 4 is a diagram showing an example of detection results for road facilities and roads. FIG. 5 is a diagram showing an example of a direction for correcting position information. FIG. 6 is a diagram showing an example of a ledger for managing road facilities. FIG. 7 is a diagram showing an example of a map divided into a mesh. FIG. 8 is a diagram explaining a method for updating a geographical position. FIG. 9 is a diagram showing an example of a map screen displayed on an administrator terminal. A flowchart showing an example of the operation of the management support system. A block diagram showing another example of the configuration of the management support system. A flowchart showing another example of the operation of the management support system. A block diagram showing an example of the hardware configuration of a computer.

[0012] An embodiment of a management support system that supports the management of road facilities along or on a road is described below. The road facilities covered by the present disclosure are not particularly limited and include, for example, utility poles, electric wires, substations, manholes, signs, traffic signals, convex mirrors, guardrails, and the like. Utility poles, electric wires, substations, manholes, and the like are managed by, for example, electric power companies. Some road facilities are also managed by telecommunications companies or local governments. The road facilities covered by the present disclosure may also be referred to simply as facilities.

[0013] [First Embodiment] An example of a connection between a management support system 100 according to a first embodiment and other devices will be described using Figure 1. The management support system 100 is connected to other devices via a communication network 30, either wired or wirelessly. The management support system 100 is connected to, for example, a camera 10, an administrator terminal 20, a storage 40, and a database 50. Note that the management support system 100 does not necessarily need to communicate with the camera 10 and the administrator terminal 20. Therefore, the management support system 100 may be connected to the camera 10 and the administrator terminal 20 as needed.

[0014] The camera 10 is mounted on a moving body 11. The camera 10 captures images of a road or the environment around the road. The camera 10 is realized, for example, by a drive recorder mounted on a vehicle. The drive recorder continuously captures images of at least one of the front and rear of the vehicle while the vehicle is traveling on a road. However, the types of the camera 10 and the moving body 11 are not limited to this, and the camera 10 can be mounted on various types of moving bodies 11. For example, the camera 10 may be mounted on other moving bodies such as a bicycle or a drone. The camera 10 may be carried by a person traveling on the road. Furthermore, the images captured by the camera 10 are not limited to moving images, but may also be still images. The camera 10 is not limited to a camera that continuously captures images.

[0015] The travel data including the images captured by the camera 10 is stored in the storage 40. The camera 10 transmits the travel data including the images to the storage 40 or the management support system 100, for example.

[0016] The travel data further includes location information of the location where the image was captured. The location information is obtained using, for example, the Global Navigation Satellite System (GNSS), the Global Positioning System (GPS), etc. The location information is expressed, for example, by latitude and longitude or a position on a map.

[0017] The driving data may include information on the shooting direction in which the image was captured. The information on the shooting direction includes the direction in which the camera 10 is facing. The shooting direction can be acquired, for example, by a geomagnetic sensor provided in the camera 10. Furthermore, when the installation direction of the camera 10 with respect to the mobile body 11 is determined, the shooting direction can be identified based on the traveling direction of the mobile body 11. When the camera 10 is installed to capture images ahead of the mobile body 11 and the mobile body 11 is moving forward, the traveling direction is the shooting direction. The traveling direction is identified using time-series position information of the mobile body 11 around the time the image was captured. The fact that the mobile body 11 is moving forward may be identified by driving information such as the operation of the accelerator pedal of the automobile. Furthermore, the fact that the mobile body 11 is moving forward may be identified by performing image recognition on two or more images captured while traveling and recognizing that an object is approaching.

[0018] The travel data may include information about the route the mobile object 11 was traveling on at the time the image was captured. The route is identified by tracking the location information of the mobile object 11. The route the mobile object 11 was traveling on is represented, for example, by the route name and whether the mobile object 11 was traveling on an inbound or outbound lane of the route.

[0019] The administrator terminal 20 presents information to the administrator. The type of the administrator terminal 20 is not particularly limited, and may be a smartphone, a tablet terminal, a PC (Personal Computer), etc. The administrator terminal 20 accesses the database 50, for example, and displays the information stored in the database 50.

[0020] The storage 40 stores driving data including images captured by the camera 10. Fig. 3 is a diagram showing an example of driving data stored in the storage 40. In Fig. 3, the driving data includes a data identifier (ID), a shooting direction, location information of the shooting location, the captured image, and the shooting date and time.

[0021] The database 50 is a database that stores data for managing road facilities. The database 50 stores at least the geographical position of each road facility. The geographical position is represented by, for example, latitude and longitude. Alternatively, the geographical position may be represented by the position of the area to which the road facility belongs, among areas obtained by dividing a map into a mesh. The database 50 may further include information identifying the road on which the road facility is installed, information on the type of road facility, or photographed images of the road facility. Examples of the format of the database 50 include a relational database (RDB), NoSQL, a spreadsheet file, etc. However, the format of the database 50 is not limited to these, and any format can be used.

[0022] In the database 50 for electric power companies, information on the type of road facility is stored, for example, whether the road facility is a utility pole or a substation facility.

[0023] In the database 50 for local governments, when the road facilities managed are road signs, classifications such as guide signs, no parking, pedestrian crossings, and maximum speeds are stored as the types of road facilities. The installation position of the sign relative to the road or the sign installation method may also be stored as the type of road facility. The installation position relative to the road is represented by top, left, right, and the like. The sign installation method includes, for example, a gate-type method in which a sign board is installed above the roadway using a gate-type support straddling the roadway. Other sign installation methods include roadside, cantilever, and attached types.

[0024] An example of the configuration of the management support system 100 according to the first embodiment will be described with reference to Fig. 2. The management support system 100 includes an acquisition unit 101, a detection unit 102, a processing target identification unit 103, a position calculation unit 104, a management target identification unit 105, and an update unit 106.

[0025] The acquisition unit 101 acquires an image captured by the camera 10 mounted on the mobile object 11 and location information of the location where the image was captured. For example, the acquisition unit 101 acquires the image and location information by referring to the driving data in the storage 40. Alternatively, the acquisition unit 101 may receive the image and location information from the camera 10.

[0026] The detection unit 102 detects one or more predetermined facilities on the road from the image. The type of road facility to be detected is determined in advance. The type of road facility to be detected may be one or more. The detection unit 102 detects the road facility using image recognition technology. The detection unit 102 may detect the road facility using a learned model. The detection unit 102 may determine for each pixel in the image whether or not the pixel represents road facility. The detection unit 102 recognizes the position and size of the road facility in the image. The position of the road facility in the image is represented, for example, by coordinates based on a predetermined point in the image.

[0027] The detection unit 102 may detect roads from the image in addition to road facilities. Roads are detected using the same technology as that used to detect road facilities. The detection unit 102 may detect roads using the same model as that used to detect road facilities. Alternatively, the detection unit 102 may detect roads using a model different from that used to detect road facilities. The detection unit 102 may detect the lane in which the mobile object 11 is traveling by distinguishing it from other lanes.

[0028] Fig. 4 is a diagram showing an example of the detection results of road facilities and roads by the detection unit 102. In the example shown in Fig. 4, it is assumed that the road facilities are utility poles installed at the edge of the road. In Fig. 4, the area of ​​the road facilities detected by the detection unit 102 in the image is shown by the shaded area, and the area of ​​the road is shown by the dotted line. In other words, in Fig. 4, utility poles P1 to P4 and roads have been detected from the image.

[0029] The detection unit 102 may further detect the depth of the road facility from the image. Depth detection may use a depth estimation technique that estimates the distance between the camera and the object from a single image. Furthermore, when multiple cameras 10 are mounted on one mobile body 11, the detection unit 102 may estimate the distance from the camera to the object using the results of restoring three-dimensional information from images captured by each camera 10. The multiple cameras 10 mounted on the mobile body 11 may be a stereo camera or two synchronized cameras.

[0030] The processing target identification unit 103 identifies road facilities to be processed from one or more predetermined road facilities based on the position or size of each detected road facility in the image. For example, the processing target identification unit 103 identifies road facilities from among the road facilities shown in the image to be the target of processing for calculating the geographical position of the road facility. The processing target identification unit 103 identifies road facilities photographed as close as possible to the mobile body 11. For example, the processing target identification unit 103 identifies road facilities photographed within several meters of the mobile body 11. Identifying nearby road facilities allows the position calculation unit 104, described later, to more accurately calculate the geographical position.

[0031] The position and size of an object vary depending on the distance from the camera 10. For example, in FIG. 4, utility pole P1 appears to the left of the image and is larger than utility pole P2, which is farther from the mobile object 11. The processing target identification unit 103 can identify road equipment photographed at a predetermined distance from the mobile object 11 by identifying road equipment at a predetermined position or a predetermined size in the image. The position or size of road equipment to be identified can be set appropriately depending on the type of road equipment to be identified.

[0032] The processing target specifying unit 103 may specify road facilities based on both their position and size, or may specify road facilities based on either their position or size.

[0033] A method for identifying road facilities based on their positions in an image is described below. In the following description, it is assumed that a roadside utility pole is identified, but the following method can also be applied to road facilities other than utility poles, such as roadside road signs.

[0034] The processing target identification unit 103 can identify road equipment that is closer to the left or right edge of the image and lower than a predetermined position. This is because roadside road equipment appears closer to the edge of the image and lower as the mobile object 11 approaches. For example, the processing target identification unit 103 identifies road equipment that is closer to the edge of the driving lane of the mobile object 11 than the predetermined position. The driving lane of the mobile object 11 is set to the left side in countries that drive on the left. This allows the processing target identification unit 103 to identify the road equipment that is closer to the driving lane when road equipment is installed on both the driving lane side and the opposite lane side. Road equipment on the opposite lane side can be identified if it appears on the driving lane side in an image captured while the mobile object 11 is traveling in the opposite lane.

[0035] In one example, the processing target identification unit 103 identifies the road facility closest to the bottom corner of the lane on which the mobile object 11 is traveling in the image. In countries where traffic drives on the left side of the road, the processing target identification unit 103 identifies the road facility closest to the bottom left corner in the image. In Figure 4, the processing target identification unit 103 compares the bottom ends of utility poles P1 to P4, for example, and identifies utility pole P1, whose bottom end is closest to the bottom left corner of the image. Road facilities other than the closest road facility may be identified in an image captured after the mobile object 11 has approached the road facility.

[0036] The processing target identification unit 103 may further identify road equipment that is located lower than a predetermined position in the image, closer to the edge of the image opposite the driving lane of the mobile body 11. This allows the processing target identification unit 103 to identify road equipment installed on the opposite side of the driving lane. This is because even if the road equipment is on the opposite side of the driving lane, the installation location of the road equipment may be sufficiently close to the mobile body 11, for example, when a narrow one-lane road or a one-way road is photographed. Therefore, it may be possible to calculate the geographical position of road equipment identified in this way with sufficient accuracy.

[0037] In one example, the processing target identification unit 103 may further identify the road facility closest to the bottom corner on the opposite side to the driving lane side. That is, in a country where people drive on the left side, the processing target identification unit 103 may further identify the road facility closest to the bottom right corner in the image.

[0038] In the above-described manner, the processing target identification unit 103 can identify roadside road facilities. When identifying road facilities such as road signs installed above a road, the processing target identification unit 103 may identify road facilities that are on the driving lane and closest to the top edge of the image. When identifying road facilities such as manholes installed on a road, the processing target identification unit 103 may identify road facilities that are on the driving lane and closest to the bottom edge of the image.

[0039] Next, a method for identifying road facilities based on their size in an image will be described.

[0040] The processing target identification unit 103, for example, identifies road facilities whose size in the image is equal to or exceeds a predetermined standard. This is because the closer the road facilities are to the camera 10, the larger they appear. There are cases where the actual sizes of the detected road facilities are known and common. In such cases, the processing target identification unit 103 may identify road facilities based on their size in the image, without using their positions in the image. For example, there are cases where utility poles of the same size are installed at predetermined intervals. Utility poles that are farther away appear small, and utility poles that are close appear large. Therefore, the processing target identification unit 103 may identify utility poles that are equal to or larger than a predetermined size.

[0041] In one example, the processing target identification unit 103 may compare the sizes of road facilities in the image and identify the largest road facility. For example, the processing target identification unit 103 compares the sizes of detected utility poles and identifies the largest utility pole. In Figure 4, the processing target identification unit 103 compares the sizes of utility poles P1 to P4 in the image and identifies utility pole P1, for example.

[0042] As described above, the method in which the processing target identification unit 103 identifies road facilities based on their positions or sizes in an image has been described. The processing target identification unit 103 may further identify road facilities that are a predetermined distance from the moving object 11 based on the depth estimated by the detection unit 102. By using the depth, the processing target identification unit 103 can identify the closer road facilities among the road facilities that are captured at the same size.

[0043] In one example, the processing target identification unit 103 identifies the road facility closest to the mobile object 11. When combining the position and depth in the image, the processing target identification unit 103 may identify the closest road facility from among the road side road facilities that appear on the driving lane side of the image from a predetermined position. Similarly, the processing target identification unit 103 may further identify the closest road facility from among the road side road facilities that appear on the opposite side of the driving lane side from the predetermined position.

[0044] The processing target specifying unit 103 may specify, as a processing target, only road facilities for which the position calculation unit 104, which will be described next, sets the correction of the position information of the shooting location to "0." In other words, in one example, the processing target specifying unit 103 does not specify, as a processing target, road facilities for which correction of the position information of the shooting location is required.

[0045] The position calculation unit 104 calculates road facility position information indicating the geographical position of the road facility identified as the processing target based on the position information of the image capture location and at least one of the position or size in the image of the road facility identified as the processing target. The road facility position information may also be simply referred to as facility position information. The road facility position information indicates, for example, the latitude and longitude of the location where the road facility is installed. By using the position or size in the image of the road facility identified as the processing target, the position calculation unit 104 can calculate the road facility position information taking into account the distance of the road facility from the mobile object 11.

[0046] The position calculation unit 104 may correct the position information of the shooting location of the image and use the corrected value as the road facility position information. If the road facility to be processed is assumed to be sufficiently close to the mobile body 11 based on the position or size in the image, the position calculation unit 104 may use the position information of the shooting location of the image as the road facility position information. The position calculation unit 104 may determine whether the road facility identified as the processing target is located at a predetermined position or has a predetermined size. In this case, the distance corrected by the position calculation unit 104 may be treated as "0". If the distance from the mobile body 11 to the road facility to be processed is assumed to be greater than a predetermined distance based on the position or size in the image, the position calculation unit 104 corrects the position information of the shooting location of the image by a predetermined distance and uses the corrected value as the road facility position information. The correction method is not particularly limited and can be selected as appropriate.

[0047] In one example, if the road facility to be processed is located on the driving lane side of the mobile object 11 or if its size in the image is larger than a predetermined value, the position calculation unit 104 sets the location information of the image capture location as road facility location information. Whether the road facility is located on the driving lane side or the opposite lane side can be determined by whether the road facility is captured on the left or right side of the image, once it is determined whether the road on which the road facility is located is a left-hand or right-hand traffic system. If road facility on the driving lane side is identified as the processing target, the distance from the mobile object 11 to the road facility can be considered sufficiently close. Furthermore, in an image captured of a narrow road or a one-way road, road facility located on the opposite side of the driving lane side may be identified. Even if the road facility is located on the opposite side of the driving lane side, if its size in the image is larger than a predetermined value, the distance to the road facility can be considered sufficiently close.

[0048] In one example, when the road facility to be processed is located on the opposite lane to the lane on which the mobile object 11 is traveling, the position calculation unit 104 corrects the position information of the image capture point by a predetermined distance in a direction perpendicular to the traveling direction of the mobile object 11. FIG. 5 is a diagram showing an example of the direction in which the position information is corrected. In FIG. 5, the top of the page is set to north. In FIG. 5, the acquisition unit 101 may acquire the latitude and longitude of the mobile object 11 traveling on the left side of the road. The position calculation unit 104 may determine the traveling direction of the mobile object 11 at the time the image was captured by referring to the traveling data in the storage 40. For example, the position calculation unit 104 may determine the traveling direction using time-series position information of the mobile object 11. Alternatively, the position calculation unit 104 may determine the shooting direction included in the traveling data as the traveling direction. In FIG. 5, the traveling direction of the mobile object 11 is northwest. The position calculation unit corrects the position information of the shooting point by a predetermined distance in the northeast direction. The degree of the correction distance is determined as appropriate. For example, when two lanes, one for the road and one for the opposite lane, are recognized, the position calculation unit 104 corrects the position information by a predetermined distance in the direction perpendicular to the direction of travel, which is the width of one lane of the road. In this case, the predetermined distance may be determined appropriately depending on the number of lanes, etc. Then, the position calculation unit 104 sets the corrected result as road facility position information.

[0049] In another example, when the location of the road facility to be processed is assumed to be far from the mobile object 11 in the traveling direction, the position calculation unit 104 may calculate road facility location information by correcting the location information of the shooting location by a predetermined distance in the traveling direction. For example, when the location of the road facility to be processed in the image is farther than a predetermined distance from the edge of the traveling lane side of the image, or when the size of the road facility to be processed in the image is smaller than a predetermined distance, the road facility is assumed to be far. In this case, the position calculation unit 104 calculates road facility location information by correcting the location information of the shooting location by a predetermined distance in the traveling direction. The degree of the correction distance is determined as appropriate. The position calculation unit 104 calculates the road facility location information by correcting the distance determined according to the size of the road facility to be processed in the image. The position calculation unit 104 may use the result of depth estimation by the detection unit 102 to correct the location information of the shooting location in the traveling direction by the estimated distance.

[0050] The management target identification unit 105 identifies, based on the road facility location information, road facilities registered in a database 50 that manages road facilities, which correspond to the road facility identified as the processing target, as the management target. Figure 6 shows an example of data registered in the database 50. As shown in Figure 6, the data in the database 50 can be displayed in the form of a ledger, with one road facility represented by one line. The management target identification unit 105 identifies the management target by, for example, identifying the ledger line by referring to the geographical location information registered in the ledger. Note that the format of the database 50 and the content of the data stored in the database 50 are not limited to the example shown in Figure 6, and may be determined as appropriate.

[0051] The management target identification unit 105, for example, compares the road facility location information with the information on the geographical location of the road facility registered in the database 50. If the geographical location indicated by the road facility location information matches the geographical location of the road facility registered in the database 50, the management target identification unit 105 identifies the registered road facility as a management target. If the geographical location indicated by the road facility location information is within a predetermined range from the geographical location of the road facility registered in the database 50, the management target identification unit 105 may identify the registered road facility as a management target.

[0052] The geographical position indicated by the road facility position information may be inaccurate. For example, even if the processing target identification unit 103 identifies a road facility somewhat far from the mobile body 11 as the processing target, the position calculation unit 104 may use the position information of the shooting location as the road facility position information. It is also possible that the accuracy of the correction performed by the position calculation unit 104 is not high. Furthermore, since the mobile body 11 is moving, the reception conditions of radio waves received from satellites such as GPS may change. Depending on the radio wave reception conditions, an error may occur in the measured position of the shooting location. Therefore, the geographical position indicated by the road facility position information calculated using erroneous position information may be inaccurate. By identifying road facilities within a predetermined range as the managed object, the managed object identification unit 105 can identify the managed object even when the road facility position information is inaccurate.

[0053] An example of a method in which the management target specifying unit 105 specifies road facilities within a predetermined range as management targets will be described.

[0054] The management target identification unit 105 may identify road facilities to be managed that correspond to the road facilities identified as the processing target, using mesh-like areas obtained by dividing the map as units. The map may be divided, for example, at intervals smaller than the installation intervals of the road facilities. For example, because utility poles may be installed at 20-meter intervals, the map may be divided into 7- or 8-meter meshes. FIG. 7 shows an example of a map divided into meshes. In FIG. 7, the map is divided into meshes smaller than the road width. Assume that the locations of the meshes to which road facilities 1 and 2 in FIG. 7 belong are registered in the database 50. If the location indicated by the road facility location information belongs to the same or adjacent area as the road facilities registered in the database 50, the management target identification unit 105 may identify the registered road facilities as corresponding to the road facilities identified as the processing target. If the location indicated by the road facility location information belongs to a mesh location adjacent to road facility 1, the road facilities to be processed are identified as corresponding to road facility 1. Therefore, the registered road facility 1 is identified as the management target.

[0055] Furthermore, the management target identification unit 105 may identify a registered road facility as corresponding to the road facility to be processed if the location indicated by the road facility location information is included within a circular area of ​​a predetermined radius from the geographical location of the road facility registered in the database 50. The radius is appropriately determined so as to distinguish between different road facilities. For example, the radius is set to be smaller than the installation interval of the road facilities and smaller than the road width. As an example, the radius may be set to 2 or 3 meters. The radius can be set arbitrarily to facilitate management, depending on the installation interval and characteristics of the facilities on the road, management standards, etc.

[0056] According to the above-described method of identifying road facilities within a specified range as management targets, the management target identification unit 105 can identify road facilities that correspond to the road facilities to be processed as management targets from among the road facilities registered in the database in advance.

[0057] The management target specifying unit 105 may use a different range depending on the type of road facility as the predetermined range for specifying the management target. For example, when identifying traffic lights, a predetermined range larger than that of utility poles may be used because traffic lights are often installed at wider intervals than utility poles.

[0058] The update unit 106 updates the data of road facilities identified as management targets. For example, the update unit 106 adds, to the database 50, an image of the road facilities to be processed as data of the road facilities to be managed. For example, the update unit 106 may add an image used to identify the road facilities to be processed. Furthermore, the update unit 106 may add an image of a frame before or after the image used to identify the road facilities to add an image of the road facilities photographed from a distant position. The update unit 106 may further add an image of the surrounding environment of the road facilities in addition to the image of the road facilities. The image of the surrounding environment is, for example, a frame before or after the image used to identify the processing target. The image of the surrounding environment does not have to include the road facilities to be managed.

[0059] The update unit 106 may add information about road facilities that can be recognized from images or information about the surrounding environment as data about road facilities to be managed. Information about road facilities includes the type of road facilities, etc. Information about road facilities may also include the state of deterioration of the facilities. When road signs are managed, the sign installation method is recognized and information about the installation method may be added. Information about the surrounding environment includes, for example, the presence or absence of road facilities of types other than the type of road facilities to be managed, the width of the road, or the material of the road. For example, when the object for managing the location of road facilities in the database 50 is utility poles, the presence or absence of road signs around the utility poles may be added as information about the surrounding environment.

[0060] The update unit 106 may further add road facility position information as data on road facilities to be managed, in addition to the geographical positions of the road facilities that have been registered in advance. For example, the database 50 may register multiple pieces of road facility position information calculated from multiple images of the same road facility taken during different travel times, such as on different days or at different times.

[0061] The geographical positions of road facilities registered in the database 50 may not be accurate. For example, the database 50 may be an electronic version of a paper ledger that has been managed in a digital format. In addition, the data in the database 50 may be outdated, and the road facility may have been relocated but the database 50 may not have been updated. In such cases, the update unit 106 may update the geographical positions of the road facilities registered in the database 50 based on multiple pieces of road facility position information registered for the same road facility. This allows the update unit 106 to register more accurate geographical positions in the database 50.

[0062] An example of a method for updating the geographical positions registered in the database 50 will be described. The update unit 106 determines which of the mesh-like regions into which the map is divided belongs to the position indicated by each of the multiple pieces of road facility position information. The update unit 106 may update the data by determining the mesh-like region that has been determined to belong to the position most frequently as the geographical position of the road facility.

[0063] Another example of a method for updating a geographical position will be described with reference to FIG. 8 . The update unit 106 may identify multiple areas of a predetermined radius, each centered on a position indicated by a plurality of pieces of road facility position information. In FIG. 8 , three points are shown, each of which is indicated by road facility position information calculated from images captured during three trips along a road. The overlapping area of ​​these areas may be updated as the geographical position of the road facility. In FIG. 8 , the area indicated by the arrows may be registered as the geographical position of the road facility. Furthermore, the update unit 106 may, for example, determine the midpoint between the positions indicated by the plurality of pieces of road facility position information as the location of the road facility. Alternatively, the center of a circle passing through the three points in FIG. 8 may be determined as the geographical position of the road facility.

[0064] The update unit 106 may determine whether a predetermined number of pieces of road facility position information have been registered. When a predetermined number of pieces of road facility position information have been registered, the update unit 106 may update the geographical positions registered in the database 50.

[0065] By the update unit 106 updating the geographical position registered in the database 50 based on multiple pieces of road facility position information, a more accurate position can be registered in the database 50 even if the road facility position information is inaccurate.

[0066] The administrator terminal 20 accesses the database 50 and presents the stored road facility information to the road facility administrator. The administrator terminal 20 may display the road facility information in the form of a ledger as shown in FIG. 6 . The administrator terminal 20 may also display the geographical location of the road facility as an icon on a map. A user of the administrator terminal 20 may select an icon. When an icon is selected on the administrator terminal 20, the administrator terminal 20 may display more detailed information about the road facility.

[0067] Fig. 9 is a diagram showing an example of a map screen displayed by the administrator terminal 20. The location of each icon in Fig. 9 indicates, for example, the installation location of a road sign registered in the database 50. In Fig. 9, one icon is selected. On the right side of the map, more detailed information about the road sign is displayed, such as the sign ID, the type of sign, the latitude and longitude of the sign, and an image of the sign.

[0068] The administrator terminal 20 may select the type of road facility to be displayed. For example, a user of the administrator terminal 20 inputs that road signs are to be displayed. The administrator terminal 20 filters the road sign information from the information stored in the database 50 and displays the ledger or map.

[0069] An example of the operation of the management support system 100 will be described below with reference to Fig. 10. The management support system 100 may start the process of Fig. 10 when images are collected in the storage 40 or at a predetermined timing such as once a month. The management support system 100 may execute the following process for each point within a predetermined range.

[0070] The acquisition unit 101 acquires an image captured by the camera 10 mounted on the mobile object 11 and location information of the location where the image was captured (step S11). For example, the acquisition unit 101 acquires the image and location information from the storage 40. The detection unit 102 detects one or more predetermined road facilities from the image acquired by the acquisition unit 101 (step S12). For example, if the road facility to be detected is a utility pole, the detection unit 102 detects the utility pole from the image.

[0071] The processing target identification unit 103 identifies a road facility to be processed from among one or more detected predetermined road facilities based on the position or size of each detected road facility in the image (step S13). For example, the detection unit 102 identifies, among the detected utility poles, a utility pole that is captured at a predetermined proximity as the processing target. The position calculation unit 104 calculates road facility position information indicating the geographical position of the identified road facility based on the position information acquired by the acquisition unit 101 and the position or size of the identified road facility in the image (step S14).

[0072] The management target specifying unit 105 specifies, based on the calculated road facility location information, road facilities registered in the database 50 that correspond to the road facility specified as the processing target, as management targets (step S15). The update unit 106 updates the data of the road facility specified as the management target (step S16).

[0073] With the above, the management support system 100 ends the processing of Fig. 10. However, the management support system 100 may execute step S13 after the detection unit 102 has finished detecting road facilities for all images acquired by the acquisition unit 101 in step S12. Steps S13 to S16 may be repeated for all images in which road facilities are detected until the identification of the processing target in step S13 is completed. Note that there may be cases in which the processing target cannot be identified from some images, such as images that do not show road facilities or images that only show distant road facilities.

[0074] According to the first embodiment, the processing target identification unit 103 identifies road facilities to be processed from the detected road facilities based on the position or size of each detected road facility in the image. Then, the position calculation unit 104 calculates the geographical position of the road facilities identified as the processing target. The calculated geographical position is used to update the data in the database 50 that manages the road facilities. Therefore, according to the first embodiment, it is possible to more accurately reflect the data of the photographed road facilities in the database 50.

[0075] The road facility manager can view the information in the database 50 displayed on the manager terminal 20 and check the road facility and the surrounding environment before carrying out work such as designing, installing, and maintaining the road facility. According to the first embodiment, the data is reflected more accurately, allowing the manager to more efficiently check using images. For example, it is assumed that an image of a different road facility is registered as an image of the road facility whose location is registered in the database 50. If such a situation occurs, it will be less convenient for the manager, but this embodiment makes it easier to avoid such a situation.

[0076] Second Embodiment An example of the configuration of a management support system 110 according to the second embodiment will be described with reference to Fig. 11. Regarding the configuration of the management support system 110, the description of the configuration similar to that of the management support system 100 according to the first embodiment will be omitted.

[0077] The management support system 110 differs from the management support system 100 according to the first embodiment in that it includes an image selection unit 107 and a notification unit 108. The image selection unit 107 and the notification unit 108 may be provided as needed.

[0078] The image selection unit 107 selects images to be registered in the database 50 from among images of the same road facility as the processing target. The update unit 106 registers the images selected by the image selection unit 107 as data on the road facility to be managed.

[0079] For example, the image selection unit 107 extracts multiple images of the same road facility as the road facility identified by the processing target identification unit 103. Then, the image selection unit 107 selects an image that shows the road facility in a good light from the multiple extracted images. The image selected by the image selection unit 107 may be the same as the image used to calculate the road facility position information, or may be a different image. By the image selection unit 107 selecting an image that shows the road facility in a good light, it becomes possible to register an image in the database 50 that is easy for road managers to use for confirmation.

[0080] The image selection unit 107 may use the detection results of the road facility to select an image that shows the entire road facility. In Figure 4, the utility pole P1 is not shown in its entirety. Therefore, the image selection unit 107 may select an image that shows the entire utility pole from an image taken at a position farther away from the utility pole P1 than the image in Figure 4. The image selection unit 107 may also select an image that shows a required portion of the road facility at a required size.

[0081] The image selection unit 107 may further select one or more images including at least one frame before or after an image that clearly shows the road facility to be processed. By selecting the previous and next frames, the image selection unit 107 enables road managers to register in the database 50 images that are easy to use for checking the surrounding environment of the road facility. For example, the image selection unit 107 selects several images. The number of images selected by the image selection unit 107 may be changeable. For example, the image selection unit 107 selects the number of images selected by the user. The previous and next frames selected do not need to show the road facility. In other words, the image selection unit 107 may select frames before and after the image in which the processing target is captured.

[0082] The notification unit 108 outputs a notification related to the update of the database 50 to the administrator terminal 20. The content and timing of the notification are not particularly limited. An example of a notification related to the update of the database 50 will be described below.

[0083] In one example, the notification unit 108 notifies that the database 50 has been updated. For example, if a road facility identified as the processing target and a road facility corresponding to the road facility are not registered in the database 50, the notification unit 108 may output a notification regarding the road facility identified as the processing target.

[0084] In one example, the notification unit 108 may output a notification regarding whether or not to update the database 50. The notification may include an image of the processing target and a request to select whether or not to register the road facility identified as the processing target in the database 50. After checking the notification, the user may use the administrator terminal 20 to select to register the road facility in the database 50. The administrator terminal 20 may transmit an instruction to register the road facility in the database 50 to the management support system 110 in accordance with the user's selection.

[0085] In one example, the notification unit 108 may output a notification including a request to select an image to be registered in the database 50. The notification includes multiple images of the processing target. After checking the notification, the user uses the administrator terminal 20 to select an image that shows the road facility in a good light. The image selection unit 107 selects the image selected by the user as the image to be registered in the database 50. The selected image may be registered as the thumbnail image that is displayed first for a certain road facility.

[0086] If a road facility corresponding to the road facility identified as the processing target is not registered in the database 50, the update unit 106 newly registers the road facility identified as the processing target in the database 50. For example, the update unit 106 registers the road facility position information calculated by the position calculation unit 104 as the installation position of the road facility in the database 50. The update unit 106 may register the road facility after receiving an instruction to register the road facility in the database 50 from the administrator terminal 20.

[0087] An example of the operation of the management support system 110 will be described with reference to Fig. 12. The operation of the management support system 110 is basically the same as the operation of the management support system 100 according to the first embodiment.

[0088] The management support system 110 performs the same processes as steps S11 to S13 in the first embodiment. After step S13, the position calculation unit 104 determines whether the road facility identified as the processing target is located at a predetermined position (step S14-1). The flowchart in FIG. 12 illustrates an example in which the determination of whether the road facility is located at a predetermined position is performed by determining whether the road facility is located on the driving lane side or the opposite lane side. For example, if the position calculation unit 104 determines that the road facility is located on the driving lane side of the mobile object 11, the position calculation unit 104 sets the position information of the shooting location as the road facility position information (step S14-2). Furthermore, if the position calculation unit 104 determines that the road facility is located on the opposite lane side of the mobile object 11, the position calculation unit 104 corrects the position information of the shooting location to set the road facility position information (step S14-3). Note that if the processing target identification unit 103 identifies only road facilities whose road facility position information is the position information of the image shooting location in step S13, steps S14-1 and S14-3 may be omitted. The position calculation unit 104 may execute step S14-2, for example, in response to the processing target specification unit 103 specifying road equipment on the driving lane side based on the position or size in the image.

[0089] The management target identification unit 105 determines, based on the calculated road facility location information, whether or not there is any road facility registered in the database 50 that corresponds to the road facility identified as the processing target (step S15-1). If there is a corresponding road facility (step S15-1: Yes), the management target identification unit 105 identifies the corresponding road facility as the management target (step S15-2). If there is no corresponding road facility (step S15-1: No), the notification unit 108 notifies the administrator terminal 20 of the detection of new road facility (step S17). Note that step S17 may be omitted.

[0090] After step S15-2, the update unit 106 updates the data of the road facility identified as the management target (step S16). After step S17, the update unit 106 adds the data of the road facility to the database 50 (step S18).

[0091] With the above, the management support system 110 completes the operation of Fig. 12. The image selection unit 107 may select an image at any timing after step S13.

[0092] According to the second embodiment, the same effects as those of the first embodiment can be obtained. Furthermore, according to the second embodiment, the update unit 106 newly registers data of road facilities that are not registered in the database 50. Therefore, according to the second embodiment, it is possible to manage road facilities even when the locations of the road facilities have not been managed in advance.

[0093] [Hardware Configuration] In each of the above-described embodiments, each component of the management support system 100, 110 represents a functional block. Some or all of the components of the management support system 100, 110 may be realized by any combination of a computer 500 and a program.

[0094] Fig. 13 is a block diagram showing an example of the hardware configuration of a computer 500. Referring to Fig. 13, the computer 500 includes, for example, a processor 501, a read only memory (ROM) 502, a random access memory (RAM) 503, a program 504, a storage device 505, a drive device 507, a communication interface 508, an input device 509, an input / output interface 511, and a bus 512.

[0095] The processor 501 controls the entire computer 500. The processor 501 may be, for example, a CPU (Central Processing Unit). The number of processors 501 is not particularly limited, and there may be one or more processors 501.

[0096] The program 504 includes instructions for implementing each function of the management support systems 100, 110. The program 504 is stored in advance in the ROM 502, RAM 503, or storage device 505. The processor 501 implements each function of the management support systems 100, 110 by executing the instructions included in the program 504. The RAM 503 may also store data to be processed in each function of the management support systems 100, 110.

[0097] The drive device 507 reads and writes data from and to the recording medium 506. The communication interface 508 provides an interface with a communication network. The input device 509 is, for example, a mouse or keyboard, and accepts information input from an administrator or the like. The output device 510 is, for example, a display, and outputs (displays) information to an administrator or the like. The input / output interface 511 provides an interface with peripheral devices. The bus 512 connects these hardware components. The program 504 may be supplied to the processor 501 via a communication network, or may be stored in advance on the recording medium 506, read by the drive device 507, and supplied to the processor 501.

[0098] It should be noted that the hardware configuration shown in FIG. 13 is an example, and other components may be added, or some components may not be included.

[0099] There are various variations in the method of realizing the management support systems 100 and 110. For example, the management support systems 100 and 110 may be realized by any combination of different computers and programs for each component. Furthermore, multiple components included in the management support systems 100 and 110 may be realized by any combination of a single computer and program.

[0100] Furthermore, at least a part of the management support systems 100 and 110 may be provided in a software as a service (SaaS) format. That is, at least a part of the functions for realizing the management support systems 100 and 110 may be executed by software executed via a network.

[0101] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, the configurations in the respective embodiments can be combined with each other without departing from the scope of the present disclosure.

[0102] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes.

[0103] [Supplementary Note 1] A management support system comprising: an acquisition means for acquiring an image taken by a camera mounted on a mobile body and position information of the location where the image was taken; a detection means for detecting one or more predetermined facilities on a road from the image; a processing target identification means for identifying a facility to be processed from the one or more predetermined facilities based on at least one of the position or size of each of the facilities in the image; a position calculation means for calculating facility position information indicating the geographical position of the facility identified as the processing target based on the position information and at least one of the position or size of the facility identified as the processing target in the image; a management target identification means for identifying a facility to be managed that corresponds to the facility identified as the processing target from among facilities registered in a database that manages facilities on the road based on the facility position information; and an update means for updating data of the facility identified as the management target.

[0104] [Supplementary Note 2] The management support system according to Supplementary Note 1, wherein the processing target specifying means specifies the largest piece of equipment in the image as the processing target.

[0105] [Supplementary Note 3] The management support system according to Supplementary Note 1 or 2, wherein the processing target specifying means specifies, as the processing target, a facility that is closest to a corner of a lower corner of the image that is on a side of a lane on which the moving object is traveling.

[0106] [Supplementary Note 4] The management support system described in any one of Supplementary Notes 1 to 3, wherein the detection means detects the depth of the facility from the image, and the processing target identification means identifies the facility closest to the moving body as the processing target based on the depth.

[0107] [Supplementary Note 5] The management support system described in any one of Supplementary Notes 1 to 4, wherein the position calculation means sets the position information as the equipment position information when the equipment to be processed is located on the driving lane side of the mobile body, or when the size of the equipment in the image is larger than a predetermined value.

[0108] [Supplementary Note 6] The management support system described in any one of Supplementary Notes 1 to 5, wherein the position calculation means calculates the equipment position information by correcting the position information by a predetermined distance in a direction perpendicular to the traveling direction of the mobile body when the equipment to be processed is located on the opposite lane to the traveling lane of the mobile body.

[0109] [Appendix 7] The management support system described in any one of Appendices 1 to 6, wherein the position calculation means calculates the equipment position information by correcting the position information by a predetermined distance with respect to the traveling direction of the mobile body when the position of the equipment to be processed in the image is away from the edge of the lane side of the mobile body in the image by a predetermined distance or more, or when the size of the equipment in the image is less than a predetermined value.

[0110] [Appendix 8] The management support system described in any one of Appendices 1 to 7, wherein the management target identification means identifies the registered equipment as a management target when the geographical location indicated by the equipment location information is within a predetermined range from the geographical location of the equipment registered in the database.

[0111] [Appendix 9] The management support system described in any one of Appendices 1 to 8, wherein the management target identification means identifies a piece of equipment as corresponding to the equipment to be processed when the location indicated by the equipment location information and the location of the equipment registered in the database belong to the same or adjacent areas among mesh-like areas obtained by dividing a map at intervals smaller than the installation intervals of the equipment.

[0112] [Supplementary Note 10] The management support system according to any one of Supplementary Notes 1 to 9, wherein the updating means updates the geographical position of the equipment registered in the database based on the equipment position information calculated from each of the plurality of images of the same equipment.

[0113] [Supplementary Note 11] The management support system described in any one of Supplementary Notes 1 to 10, wherein the management target identification means determines that there is no equipment among the equipment registered in the database that corresponds to the equipment identified as the processing target, and the update means accepts input as to whether or not to newly register the equipment identified as the processing target in the database.

[0114] [Supplementary Note 12] The management support system according to any one of Supplementary Notes 1 to 11, further comprising an image selection means for selecting, from the successively captured images, an image of the equipment identified as the processing target that was captured before or after the image used to identify the processing target, and wherein the update means adds the selected image to the database as an image of the equipment identified as the management target.

[0115] [Supplementary Note 13] A management support method comprising: acquiring an image taken by a camera mounted on a mobile body and location information of the location where the image was taken; detecting one or more specified facilities on a road from the image; identifying a facility to be processed from the one or more specified facilities based on at least one of the position or size of each of the facilities in the image; calculating facility location information indicating the geographical position of the facility identified as the processing target based on the location information and at least one of the position or size of the facility identified as the processing target; identifying, as a management target, a facility corresponding to the facility identified as the processing target from among facilities registered in a database that manages road facilities based on the facility location information; and updating data of the facility identified as the management target.

[0116] [Supplementary Note 14] A recording medium that non-temporarily records a program that causes a computer to execute the following processes: acquiring an image taken by a camera mounted on a moving object and location information of the location where the image was taken; detecting one or more specified facilities on a road from the image; identifying a facility to be processed from the one or more specified facilities based on at least one of the position or size of each of the facilities in the image; calculating facility location information that indicates the geographical position of the identified facility based on the location information and at least one of the position or size of the facility identified as the processing target; identifying, as a management target, a facility that corresponds to the facility identified as the processing target from among facilities registered in a database that manages road facilities based on the facility location information; and updating data of the facility identified as the management target.

[0117] REFERENCE SIGNS LIST 100, 110 Management support system 101 Acquisition unit 102 Detection unit 103 Processing target identification unit 104 Position calculation unit 105 Management target identification unit 106 Update unit 10 Camera 11 Mobile object 20 Administrator terminal 30 Communication network 40 Storage 50 Database

Claims

1. An acquisition means for acquiring an image taken by a camera mounted on a mobile device and location information of the location where the image was taken, A detection means for detecting one or more predetermined facilities on the road from the aforementioned image, A means for identifying a processing target, which identifies the equipment to be processed from among the one or more predetermined pieces of equipment based on at least one of the position or size of each piece of equipment in the image, A location calculation means calculates equipment location information indicating the geographical location of the equipment identified as a processing target, based on the aforementioned location information and at least one of the position or size of the equipment identified as a processing target in the image; Based on the aforementioned equipment location information, a means for identifying equipment to be managed, which identifies equipment corresponding to the equipment identified as a target for processing among the equipment registered in the database for managing equipment on roads, Update means for updating data of the equipment identified as being managed, A management support system equipped with the following features.

2. The processing target identification means identifies the largest piece of equipment in the image as the processing target. The management support system according to claim 1.

3. The processing target identification means identifies the equipment closest to the corner of the lower image that is on the lane side of the moving body as the processing target. The management support system according to claim 1 or 2.

4. The detection means detects the depth of the equipment from the image, The processing target identification means identifies the equipment closest to the moving object as the processing target based on the depth. The management support system according to claim 1 or 2.

5. The position calculation means uses the position information as the equipment position information if the equipment to be processed is located on the side of the travel lane of the moving body, or if the size of the equipment in the image is larger than a predetermined value. The management support system according to claim 1 or 2.

6. The position calculation means calculates the position information of the equipment by correcting the position information by a predetermined distance perpendicular to the direction of travel of the moving body, if the equipment to be processed is located on the opposite side of the lane from the traveling lane of the moving body. The management support system according to claim 1 or 2.

7. The position calculation means calculates the equipment position information obtained by correcting the position information by a predetermined distance with respect to the direction of travel of the moving body if the position of the equipment to be processed in the image is more than a predetermined distance from the end of the image on the lane side of the moving body, or if the size of the equipment in the image is less than a predetermined distance. The management support system according to claim 1 or 2.

8. The aforementioned managed equipment identification means identifies that a piece of equipment corresponds to the equipment to be processed if the location indicated by the equipment location information and the location of the equipment registered in the database belong to the same or adjacent area within a mesh-like region obtained by dividing the map into intervals smaller than the installation interval of the equipment. The management support system according to claim 1 or 2.

9. The camera mounted on the mobile device captures an image, and the location information of the place where the image was taken is acquired. From the aforementioned image, one or more predetermined facilities on the road are detected. Based on at least one of the position or size of each of the aforementioned pieces of equipment in the aforementioned image, the piece of equipment to be processed is identified from among the one or more predetermined pieces of equipment. Based on the aforementioned location information and at least one of the position or size of the equipment identified as the processing target in the image, equipment location information indicating the geographical location of the equipment identified as the processing target is calculated. Based on the aforementioned equipment location information, among the equipment registered in the database for managing equipment on roads, equipment corresponding to the equipment identified as a target for processing is identified as a managed equipment. Update the data of the equipment identified as being under management. Management support methods.

10. The camera mounted on the mobile device captures an image, and the location information of the place where the image was taken is acquired. From the aforementioned image, one or more predetermined facilities on the road are detected. Based on at least one of the position or size of each of the aforementioned pieces of equipment in the aforementioned image, the piece of equipment to be processed is identified from among the one or more predetermined pieces of equipment. Based on the aforementioned location information and at least one of the position or size of the equipment identified as the processing target in the image, equipment location information indicating the geographical location of the identified equipment is calculated. Based on the aforementioned equipment location information, among the equipment registered in the database for managing equipment on roads, equipment corresponding to the equipment identified as a target for processing is identified as a managed equipment. Update the data of the equipment identified as being under management. A program that instructs a computer to perform a process.