Construction site state recording system, materials and equipment management system, and construction progress grasping system

Abstract

To provide a construction site state recording system capable of easily using a state in a construction site with a simple configuration.SOLUTION: A construction site state recording system 1 for recording a state in a construction site includes an identification tag arranged in the construction site, a photographing recording device 3 for continuously photographing the inside of the construction site by a site worker, a detection part 43 for detecting the identification tag from a plurality of photographed images, and a calculation part 44 for calculating an absolute position in the construction site of a photographing position to the photographed image from which the identification tag is detected. An identification tag photographing absolute position calculation unit 44, a relative path calculation unit 45 that calculates a relative position of a photographing position of each of a plurality of photographed images with respect to a photographing position of another photographed image photographed immediately before or after and calculates a relative path that is a movement path of the relative position, and an overall absolute position calculation unit 46 that moves the entire relative path so that the relative position in the relative path matches an absolute position of the photographing position of the photographed image and calculates an absolute position in a construction site with respect to each of the relative positions are provided.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a construction site status recording system that records the status within a construction site, an equipment and materials management system that uses the construction site status recording system to detect and manage the location of on-site equipment and materials within a construction site, and a construction progress monitoring system that uses the construction site status recording system to manage the progress of wall and ceiling construction within a construction site. [Background technology]

[0002] Work performed at a construction site includes site management work, which involves visually inspecting, managing, and recording the site's condition. Site management work may include, for example, managing the various machines and materials (hereinafter referred to as site materials and equipment) used in the work, and keeping track of the progress of the construction work. A system is being considered that manages such site materials and equipment and keeps track of the progress of the construction work by associating them with their locations within the construction site. Regarding the management of on-site equipment and materials, for example, Patent Document 1 discloses the configuration of a location management system that includes a location acquisition unit that acquires the locations of on-site equipment and materials at a construction site, and a display control unit that displays the locations of the on-site equipment and materials acquired by the location acquisition unit. In this configuration, the location acquisition unit includes a transmitter that is attached to the on-site equipment and materials and transmits radio waves, and a receiver that is attached to the building and receives the radio waves transmitted from the transmitter. In the configuration disclosed in Patent Document 1, a transmitter must be attached to each of the on-site equipment and materials, which requires time and cost to attach a transmitter to each of the many on-site equipment and materials.

[0003] In response to this, Patent Document 2 discloses an equipment and materials management system that detects and manages the positions of on-site equipment and materials within a construction site. The equipment and materials management system includes identification tags that are placed at different positions within the construction site and have identification information recorded on them, a pedestrian autonomous navigation means that has an autonomously operating sensor and acquires movement information of on-site workers, an imaging and recording device that photographs the construction site, a detection means that detects on-site equipment and materials and identification tags from images photographed by the imaging and recording device, an information integration unit that associates the movement information of on-site workers obtained by the pedestrian autonomous navigation means with the detection results of the on-site equipment and materials and identification tags by the detection means by time and stores them in a database, a position calculation unit that calculates the positions of the on-site equipment and materials by referring to the database, and an information display unit that displays the positions of the on-site equipment and materials calculated by the position calculation unit. In the configuration of Patent Document 2, there is no need to attach a transmitter to each of the on-site materials and equipment, so the locations of materials and equipment within a construction site can be managed accurately and at low cost.

[0004] Regarding management of construction progress, Patent Document 3 discloses a construction progress monitoring system for managing the progress of wall construction and ceiling construction within a construction site. The construction progress monitoring system includes: means for detecting identification tags placed at different positions within the construction site and for estimating the progress of construction work, including wall construction and ceiling construction, from images of the construction site taken by an image-taking device held or worn by the site worker; a construction range estimation unit for calculating the photographing position of the photographed image from movement information of the site worker acquired by pedestrian autonomous navigation means, using the position of the identification tag detected immediately before the photographed image in which construction work was detected as the base point, and estimating the location and range of the construction work based on the photographing position; and an information display unit for creating display data by overlaying the progress of the construction work on the location and range estimated by the construction range estimation unit on the construction drawings of the building, and displaying the display data on a display device.

[0005] In the above-mentioned Patent Documents 2 and 3, the on-site workers need to patrol the construction site while carrying both a pedestrian autonomous navigation means that acquires the movement information of the on-site workers and an imaging and recording device that takes images of the inside of the construction site. As such, the on-site workers need to operate both the pedestrian autonomous navigation means and the imaging and recording device, which may make them difficult to use conveniently. It is desirable to realize a construction site condition recording system that is simpler in configuration and easier to use when recording the condition within a construction site in relation to its location within the site, thereby simplifying site management tasks such as managing on-site equipment and materials and managing the progress of construction work. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2020-16466 [Patent Document 2] Japanese Patent Publication No. 2022-92364 [Patent Document 3] Japanese Patent Application Publication No. 2023-64882 Summary of the Invention [Problem to be solved by the invention]

[0007] The problem that the present invention aims to solve is to provide a construction site condition recording system, an equipment and materials management system using the construction site condition recording system, and a construction progress monitoring system using the construction site condition recording system, which have a simple configuration and can be easily used when recording the condition within a construction site in correspondence with a location within the construction site. [Means for solving the problem]

[0008] The present invention employs the following means to solve the above problems: That is, the present invention is a construction site condition recording system for recording the condition within a construction site, comprising: an identification tag arranged within the construction site; an image recording device configured to be held or attached by a site worker or a mobile object configured to be movable within the construction site, and configured to continuously photograph the construction site at predetermined time intervals to generate a plurality of photographed images; a detection unit that detects the identification tag from the plurality of photographed images; an identification tag photograph absolute position calculation unit that calculates, for the photographed image in which the identification tag has been detected, the absolute position within the construction site of the photographed position at which the photographed image was taken based on the identification tag photographed in the photographed image; A construction site status recording system is provided, comprising: a relative path calculation unit that calculates the relative position of the shooting position of each of a plurality of the photographed images relative to the shooting positions of other photographed images photographed immediately before or after the photographed images, based on the content photographed in each of the images, and calculates a relative path, which is the path along which the relative positions move; and an overall absolute position calculation unit that moves the entire relative path so as to align the relative position of the shooting position of the photographed image from which the identification tag was detected within the relative path with the absolute position of the shooting position of the photographed image, and calculates an absolute position within the construction site for each of the relative positions within the relative path. According to the above-described configuration, the photographing and recording device is held or attached by a site worker or a mobile body configured to be movable within the construction site, and, for example, as the site worker or mobile body patrols the construction site, photographs of the construction site are taken continuously at predetermined time intervals, and multiple photographed images are generated. Since identification tags are placed within the construction site, the plurality of photographed images captured as described above will include images in which the identification tags are photographed. The detection unit detects the identification tags from among the plurality of photographed images in which the identification tags are photographed. For photographed images in which the identification tags have been detected, the identification tag photograph absolute position calculation unit calculates the absolute position within the construction site of the photographed position at which the photographed image was photographed, based on the identification tags photographed in the photographed image. Furthermore, the relative path calculation unit calculates the relative position of each of the multiple captured images relative to the capturing positions of other captured images captured immediately before or after it, based on the content captured in each of the multiple captured images, and calculates a relative path, which is the path of movement of the relative positions. The relative path calculated in this way is found by calculating the relative positional relationship between the capturing positions of each of the consecutively captured images, and does not indicate an absolute position on the construction site. In response to this, the overall absolute position calculation unit moves the entire relative path so as to align the relative position of the shooting position of the photographed image in which the identification tag was detected with the absolute position of the shooting position of the photographed image, and calculates the absolute position within the construction site for each relative position within the relative path. In this way, even for photographed images in which the identification tag is not detected, the absolute position within the construction site of the shooting position of the photographed image can be calculated. In this way, the photographing position of every photographed image can be accurately identified as an absolute position within the construction site. Therefore, the photographed image representing the state within the construction site and the accurately calculated absolute position within the construction site as the photographing position of the photographed image can be recorded in correspondence with each other. When realizing the above-described configuration, the calculation of the shooting position of a captured image is basically all performed based on the content captured in the captured image. Therefore, there is no particular need to use other means, such as a pedestrian autonomous navigation means that acquires movement information of a field worker who patrols a site while holding or wearing a shooting and recording device, to determine the shooting position of the captured image. Therefore, the configuration can be simplified. Furthermore, when recording the conditions within a construction site using the above-described construction site condition recording system, it is basically sufficient for a site worker or a mobile object to carry or wear an image recording device and patrol the construction site, making it easy to use. In this way, a construction site condition recording system can be realized that has a simpler configuration and is easier to use when recording conditions within a construction site in association with positions within the construction site.

[0009] In one aspect of the present invention, the relative path calculation unit generates a plurality of mask images by masking a predetermined pixel area including a portion of each of the plurality of captured images in which the field worker or the moving object holding or wearing the image capturing and recording device is captured, extracts feature points from each of the plurality of mask images, and calculates the relative position of the capturing position of each of the plurality of captured images with respect to the capturing position of another captured image captured immediately before or after by comparing the pixel positions in each of the mask images of the feature points extracted in common in the mask images corresponding to each of the consecutively captured captured images. When continuously photographing a construction site using a photography and recording device to generate multiple photographed images, for example, if an omnidirectional camera is used or if the photography and recording device is positioned so that a site worker or moving object holding or wearing the photography and recording device is captured in the image, there is a possibility that the site worker or moving object will be photographed in approximately the same position in the multiple photographed images. According to the above configuration, the relative path calculation unit generates a plurality of mask images by masking a predetermined pixel area including a portion of each of the plurality of photographed images in which a field worker or a mobile object holding or wearing an image capturing and recording device is photographed, extracts feature points from each of the plurality of mask images, and calculates the relative position of the photographing position of each of the plurality of photographed images with respect to the photographing positions of other photographed images photographed immediately before or immediately after by comparing the pixel positions of the feature points commonly extracted in the mask images corresponding to each of the consecutively photographed photographed images. If a field worker or a mobile object carrying an image capturing and recording device is photographed in each of the plurality of photographed images, the feature points extracted from the field worker or the mobile object will be located at approximately the same positions in the plurality of photographed images, and the feature points may affect the calculation of the relative position of the photographing position of each of the plurality of photographed images with respect to the photographing positions of other photographed images photographed immediately before or immediately after. In contrast, in the above configuration, the image from which feature points are extracted is a mask image generated by masking a predetermined pixel area from the captured image, the area including the site worker or moving object. As a result of masking the predetermined pixel area including the area including the site worker or moving object, feature points are not extracted or are difficult to extract from that area. This prevents feature points from being extracted from the area corresponding to the site worker or moving object in the captured image, and as a result, prevents incorrect calculation of the relative position.

[0010] In another aspect of the present invention, the relative path calculation unit includes an object identification trained model trained as object detection to identify each object photographed in an input image by surrounding it with a rectangular outer frame, and a pixel region identification trained model trained as semantic segmentation to identify a pixel region corresponding to each object photographed in the input image, and the relative path calculation unit inputs each of the plurality of photographed images to the object identification trained model to identify the outer frame corresponding to the field worker or the moving object, and calculates the pixel region identified as corresponding to the field worker or the moving object in each of the plurality of photographed images. The part inside the outer frame is cut out and input into the pixel region identification trained model to identify the pixel region corresponding to the field worker or the moving object, the pixel region identified as corresponding to the field worker or the moving object in each of the plurality of captured images is masked to generate a plurality of mask images, feature points are extracted from each of the plurality of mask images, and the pixel positions in each of the mask images of the feature points commonly extracted in the mask images corresponding to each of the consecutively captured captured images are compared to calculate the relative position of the shooting position of each of the plurality of captured images with respect to the shooting position of the other captured image taken immediately before or after. When continuously photographing a construction site using a photography and recording device to generate multiple photographed images, for example, if an omnidirectional camera is used or if the photography and recording device is positioned so that a site worker or moving object holding or wearing the photography and recording device is captured in the image, there is a possibility that the site worker or moving object will be photographed in approximately the same position in the multiple photographed images. According to the above configuration, the relative path calculation unit inputs each of the plurality of captured images into an object identification trained model trained as object detection so as to identify each object captured in the input image by surrounding it with a rectangular outer frame, thereby identifying the outer frame corresponding to the field worker or the mobile object. The relative path calculation unit then cuts out the portion inside the outer frame identified as corresponding to the field worker or the mobile object from each of the plurality of captured images, and inputs this into a pixel region identification trained model trained as semantic segmentation so as to identify the pixel region corresponding to each object captured in the input image, thereby identifying the pixel region corresponding to the field worker or the mobile object. The relative path calculation unit then masks pixel areas identified as corresponding to the field worker or the moving object in each of the multiple captured images to generate multiple mask images, extracts feature points from each of the multiple mask images, and calculates the relative position of the shooting position of each of the multiple captured images with respect to the shooting positions of other captured images captured immediately before or immediately after by comparing the pixel positions in each of the mask images of the feature points commonly extracted in the mask images corresponding to each of the consecutively captured images. If each of the multiple captured images captures a field worker or a moving object carrying an image recording device, the feature points extracted from the field worker or the moving object will be located at approximately the same positions in the multiple captured images, and the feature points may affect the calculation of the relative position of the shooting position of each of the multiple captured images with respect to the shooting positions of other captured images captured immediately before or immediately after. In contrast, in the above configuration, the image from which feature points are extracted is a mask image generated by masking pixel areas identified as corresponding to field workers or moving objects from the captured image. As a result of masking the pixel areas identified as corresponding to field workers or moving objects in this mask image, feature points are not extracted or are difficult to extract from those areas. This prevents feature points from being extracted from areas in the captured image that correspond to field workers or moving objects, and as a result, prevents incorrect calculation of relative positions.

[0011] In another aspect of the present invention, the overall absolute position calculation unit calculates the distance between the absolute positions of the photographing positions of the photographed images in which the identification tag is successively detected, calculates a scale ratio by dividing the distance between the absolute positions by the distance between the relative positions of the photographing positions, calculates a scale-adjusted relative position by multiplying each of the relative positions in the relative path by the scale ratio, selects a first photographed image, a second photographed image, and a third photographed image from the photographed images in which the identification tag is detected, moves the entire relative path so that the scale-adjusted relative position of the photographing position of the first photographed image matches the absolute position of the photographing position of the first photographed image, and The entire relative path is rotated around the absolute position of the shooting position of the first captured image as a center so that the scale-adjusted relative position of the shooting position matches the absolute position of the shooting position of the second captured image, and the entire relative path is rotated around a line connecting the absolute position of the shooting position of the first captured image and the absolute position of the shooting position of the second captured image as a center so that the scale-adjusted relative position of the shooting position of the third captured image matches the absolute position of the shooting position of the third captured image, thereby aligning the relative position of the shooting position of the captured image in which the identification tag was detected in the relative path to the absolute position of the shooting position of the captured image. According to the above configuration, it is possible to appropriately execute a process of aligning the relative position of the shooting position of the captured image in which the identification tag is detected within the relative path with the absolute position of the shooting position of the captured image.

[0012] The present invention also provides an equipment and materials management system that includes a construction site condition recording system as described above and uses the construction site condition recording system to detect and manage the location of on-site equipment and materials within the construction site, wherein the detection unit detects the on-site equipment and materials together with the identification tag from among the multiple photographed images, and records in a database the correspondence between the photographed image in which the on-site equipment and materials were detected and the detected on-site equipment and materials. According to the above configuration, the equipment management system uses the construction site condition recording system to detect and manage the positions of on-site equipment within a construction site. The detection unit of the construction site condition recording system detects on-site equipment from captured images. The detection unit records a correspondence between the captured image in which the on-site equipment is detected and the detected on-site equipment in a database. Then, the construction site condition recording system processes the image to estimate the absolute position of the captured image for each captured image. Here, when on-site equipment is detected from a captured image, the on-site equipment should be located near the absolute position estimated by the construction site condition recording system to be the capture position of the image, at least within the range captured by the image recording device. Therefore, the absolute position inferred for the captured image in the above manner can be considered to be the position where the detected on-site equipment is installed. In this way, the location of field equipment can be automatically identified and managed.

[0013] The present invention also provides a construction progress assessment system that includes the construction site condition recording system described above and uses the construction site condition recording system to manage the progress of wall construction and ceiling construction within the construction site, wherein the detection unit detects construction work including the wall construction and ceiling construction work together with the identification tag from among the multiple photographed images, estimates the progress of the construction work, and records in a database the correspondence between the photographed image in which the construction work was detected and the progress of the detected construction work. According to the above configuration, the construction progress monitoring system uses the construction site condition recording system to manage the progress of wall construction and ceiling construction within a construction site. The detection unit of the construction site condition recording system detects construction work, including wall construction and ceiling construction work, from captured images. The detection unit records the correspondence between the captured image in which construction work is detected and the progress of the detected construction work in a database. Then, the construction site condition recording system processes the image to estimate the absolute position of the captured image for each captured image. Here, if construction work is detected from a captured image, the construction work should be performed near the absolute position estimated by the construction site condition recording system to be the capture position of the image, at least within the range captured by the image recording device. Therefore, the absolute position inferred for the captured image in the above manner can be considered to be the location where the detected construction work is being performed. In this way, the location of the construction work can be automatically identified, and the progress of the construction work can be managed. [Effects of the Invention]

[0014] According to the present invention, it is possible to provide a construction site condition recording system, an equipment and materials management system using the construction site condition recording system, and a construction progress monitoring system using the construction site condition recording system, which have a simple configuration and can be easily used when recording the condition within a construction site in correspondence with a location within the construction site. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a block diagram of a construction site condition recording system according to an embodiment of the present invention. [Figure 2] FIG. 1 is an explanatory diagram showing an example of a state in which a site worker is patrolling within a construction site. [Figure 3] FIG. [Figure 4] 10 is a flowchart of a construction site condition recording method using the construction site condition recording system. [Figure 5]FIG. 10 is an explanatory diagram showing the relationship between the identification tag and the relative position of the photographing position with respect to the identification tag, which is estimated based on the position of the identification tag, for each photographed image. [Figure 6] FIG. 6 is an explanatory diagram showing the photographing positions shown in FIG. 5 displayed on a map of the construction site as absolute positions within the construction site. [Figure 7] FIG. 10 is a diagram showing feature points extracted from each of consecutively captured images by SfM. [Figure 8] FIG. 10 is a diagram showing correspondences between feature points in successively captured images. [Figure 9] FIG. 10 is an explanatory diagram of a relative path formed by the relative positions of the respective photographing positions of a plurality of photographed images. [Figure 10] FIG. 10 is an explanatory diagram of a state in which a part of the imaging and recording device is masked. [Figure 11] FIG. 10 is an explanatory diagram regarding processing of a captured image when semantic segmentation is used. [Figure 12] FIG. 10 is an explanatory diagram regarding processing of a captured image when object detection is used. [Figure 13] FIG. 1 is an explanatory diagram illustrating processing of a captured image when both semantic segmentation and object detection are used. [Figure 14] 10 is an explanatory diagram showing the relationship between the absolute position of the photographing position estimated based on the position of the identification tag and the relative position of the photographing position of the photographed image in which the identification tag is detected within the relative path. FIG. [Figure 15] This is an explanatory diagram of a state in which a relative route is displayed on a map of a construction site, with the entire relative route moved so that the relative position of the shooting position of the photographed image in which the identification tag was detected is aligned with the absolute position of the shooting position of the photographed image. [Figure 16] FIG. 10 is an explanatory diagram showing a state before a process of superimposing a set of relative positions on a set of absolute positions is executed. [Figure 17] FIG. 10 is an explanatory diagram of a first method for matching a set of relative positions with a set of absolute positions. [Figure 18]FIG. 10 is an explanatory diagram of a second method for matching a set of relative positions with a set of absolute positions. [Figure 19] This is a continuation of Figure 18. [Figure 20] FIG. 10 is an explanatory diagram illustrating a case where, as a result of superimposing a set of relative positions on a set of absolute positions, many of the distances between corresponding points are equal to or greater than a threshold value. [Figure 21] FIG. 10 is an explanatory diagram illustrating a case where, as a result of superimposing a set of relative positions on a set of absolute positions, there are few sets of points where the distance between corresponding points is equal to or greater than a threshold value. [Figure 22] FIG. 10 is an explanatory diagram showing a case where there are multiple identification tags and, for an image taken at a position between the identification tags, there is a discrepancy between the absolute position of the photographing position calculated based on one of the identification tags and the absolute position of the photographing position calculated based on the other identification tag. [Figure 23] FIG. 23 is an explanatory diagram showing that weights are applied to the absolute positions of the imaging positions that are deviated in the situation of FIG. 22. [Figure 24] FIG. 10 is a diagram showing a state in which an absolute position calculated using the construction site condition recording system of the embodiment is displayed. [Figure 25] FIG. 1 is a block diagram of an equipment and materials management system that detects and manages the positions of on-site equipment and materials within a construction site using the construction site status recording system of the above embodiment. [Figure 26] 10A and 10B are diagrams illustrating an example of a captured image in which on-site equipment and materials have been detected by a detection unit. [Figure 27] FIG. 10 is a block diagram of a construction progress monitoring system that uses the construction site status recording system of the above embodiment to manage the progress of wall construction and ceiling construction within a construction site. [Figure 28] FIG. 10 is a diagram showing the estimated position and range of wall construction work. [Figure 29] FIG. 10 is a diagram showing the estimated position and range of ceiling construction work. DETAILED DESCRIPTION OF THE INVENTION

[0016] The present invention provides a construction site status recording system that records the status within a construction site, an equipment and materials management system that uses the construction site status recording system to detect and manage the location of on-site equipment and materials within a construction site, and a construction progress monitoring system that uses the construction site status recording system to manage the progress of wall and ceiling construction within a construction site. The construction site status recording system uses SfM technology to estimate the trajectory of on-site workers or moving objects from multiple images taken by them within the construction site, and estimates information about the captured images and the locations where they were taken within the construction site. Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Fig. 1 is a block diagram of a construction site condition recording system according to this embodiment, and Fig. 2 is an explanatory diagram showing an example of a state in which a site worker is patrolling within a construction site. As shown in FIG. 1, the construction site condition recording system 1 includes an image recording device 3 and a system main body 4. In the construction site condition recording system 1, the photographing and recording device 3 is carried by a site worker Q or a mobile body configured to be movable within the construction site G, who moves around the construction site G and records the condition within the construction site G. More specifically, while the site worker Q or the mobile body moves around the construction site G, the photographing and recording device 3 photographs the construction site G, and photographed images are generated and saved. As will be explained later, in the construction site condition recording system 1, the photographing position at which each photographed image was taken is identified as an absolute position within the construction site G. This makes it possible to know which location within the construction site G the photographed image was taken at, and so by viewing the saved records, information about the construction site G can be confirmed or shared among relevant parties.

[0017] 2, a plurality of identification tags M are placed at different positions within the construction site G. When the identification tag M is photographed by the photographing and recording device 3, the identification tag M is detected from the photographed image and used to identify the position where the photographed image was taken. In this embodiment, the identification tag M has a rectangular shape. For example, if the construction site G has multiple floors, the identification tag M is installed on each floor (floor). It is preferable to install the identification tag M near stairwells and elevator rooms, which are areas of high traffic within the construction site G and are likely to be the starting and ending points for people's movements on each floor. In particular, in this embodiment, the identification tag M is installed in multiple locations on each floor, including the locations mentioned above.

[0018] FIG. 3 is a diagram showing an example of a captured image of an identification tag. It is desirable to attach the identification tag M to the surface of a pillar or wall after construction has been completed. In particular, when the building to be constructed has multiple floors, it is common for pieces of paper with the floor numbers written on them to be posted on pillars or the like on each floor. In such cases, by attaching the identification tag M near the floor number notices, it is possible to install the identification tag M at the same time as posting the floor number notices. In this way, there is no need to patrol the building just to install the identification tag M. In particular, in buildings under construction, if the identification tag M is placed on a base material or the like while the walls, ceilings, floors, etc. are still in the middle of construction, it will be necessary to replace the identification tag M as the construction progresses. There is also a possibility that the identification tag M may be lost. Therefore, it is desirable to place the identification tag M on window glass where no further construction is required once it has been placed, or in places where finishing has been completed. Furthermore, if the identification tag M is attached with a magnet or adhesive, it will be easy to remove. 3, in this embodiment, the identification tag M is, for example, a two-dimensional barcode, an AR marker, etc. Each identification tag M is associated with individual identification information.

[0019] The photographing and recording device 3 is carried by a site worker Q and photographs the inside of a building construction site G. In this embodiment in particular, the photographing and recording device 3 is a so-called omnidirectional camera (360-degree camera) that can photograph the surrounding area in all directions (360 degrees). The photographing and recording device 3 may also be a portable digital video camera or the like. The photographing and recording device 3 is configured so that it can be held by the site worker Q. In particular, when an omnidirectional camera is used as the photographing and recording device 3, it is desirable to install the photographing and recording device 3 at a position higher than the site worker Q in order to prevent the site worker Q from being photographed and obstructing the filming as much as possible. Therefore, it is desirable that the photographing and recording device 3 is configured so that it can be fixed and worn by the site worker Q, for example, on top of a helmet. Alternatively, the photographing and recording device 3 may be fixed to the upper end of a support rod, and the site worker Q may hold the lower end of the support rod so that the photographing and recording device 3 is positioned above the head of the site worker Q. The photographing and recording device 3 may be configured to be attachable to a mobile body such as a drone or robot configured to be able to move along a predetermined route within the construction site G, instead of the on-site worker Q. Hereinafter, even if the photographing and recording device 3 is described as being carried by the on-site worker Q, the description also includes the case where the photographing and recording device 3 is carried by a mobile body.

[0020] The photographing and recording device 3 includes a photographing section 31, a photographed image storage section 32, and a photographed image output section 33. The photographing unit 31 includes an image pickup element using a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like. For example, when a site worker Q patrols the construction site G, the photographing unit 31 continuously photographs the construction site G at predetermined time intervals to generate a plurality of photographed images P. The photographing unit 31 may generate a plurality of photographed images P by capturing still images at predetermined short intervals, such as 0.5 seconds. Alternatively, the photographing unit 31 may generate a plurality of photographed images P by capturing a moving image. In this case, each frame captured at predetermined intervals, such as one to several seconds, within the moving image is considered to be a single still image, and the photographing unit 31 can be considered to generate a plurality of photographed images P captured at predetermined intervals. When the photographing unit 31 generates a plurality of photographed images P by capturing a moving image, the process of actually extracting and generating still images from the moving image may be performed by the photographing and recording device 3 or by the system main body 4, which will be described later. In the photographing and recording device 3, photographing by the photographing unit 31 is started and ended at any timing. For example, photographing by the photographing unit 31 may be started and ended at the timing when the site worker Q starts and ends his patrol of the construction site G.

[0021] A plurality of photographed images P photographed by the photographing unit 31 are stored in the photographed image storage unit 32. The captured image output unit 33 outputs the movement information of the field worker Q stored in the captured image storage unit 32 to the system main body 4. In this embodiment, the capturing of the image P by the photographing and recording device 3 and the processing of the image P in the system main body 4 are carried out in parallel. That is, when the image P is captured by the photographing and recording device 3, the image P is processed in real time so that it is transmitted to the system main body 4 one after another. For this reason, in this embodiment, the image P is transmitted to the system main body 4 by data transfer via, for example, a wireless LAN (Local Area Network) such as Wi-Fi or Bluetooth (registered trademark), a mobile phone communication network, or the like. The plurality of captured images P may not be transmitted to the system main body 4 in real time, but may be transmitted all at once to the system main body 4 after the site worker Q has completed his patrol. In this case, the captured image storage unit 32 may output movement information by data transfer via a connection cable or various types of portable memory. When the photographing and recording device 3 photographs the construction site G around the movement route of the site worker Q, if an identification tag M is placed within the photographing range, the identification tag M will appear in the photographed image P.

[0022] The system main body 4 is composed of a computer terminal such as a server or a personal computer, and performs required functions by executing a preset program. Functionally, the system main body 4 comprises a photographed image acquisition unit 41, a database 42, a detection unit 43, an identification tag photograph absolute position calculation unit 44, a relative path calculation unit 45, an overall absolute position calculation unit 46, and an output unit 47. The relative path calculation unit 45 also includes a pixel region identification trained model 51 and an object identification trained model 52.

[0023] The photographed image acquisition unit 41 acquires a plurality of photographed images P transmitted from the photographed image output unit 33 of the photographing and recording device 3, and stores them in a database . The database 42 stores a plurality of photographed images P and data used and generated in each process described below. The detection unit 43 detects the identification tag M from among the plurality of captured images P. The identification tag photographing absolute position calculation unit 44 calculates the absolute position within the construction site G of the photographing position where the photographed image P was photographed, based on the identification tag M photographed in the photographed image P in which the identification tag M was detected. Based on the contents captured in each of the multiple captured images P, the relative path calculation unit 45 calculates the relative position of the shooting position of each of the multiple captured images P relative to the shooting positions of other captured images captured immediately before or after, and calculates a relative path, which is the movement path of the relative positions. The overall absolute position calculation unit 46 moves the entire relative path so as to align the relative position of the shooting position of the captured image P in which the identification tag M was detected with the absolute position of the shooting position of the captured image P, and calculates the absolute position within the construction site G for each of the relative positions in the relative path. The output unit 47 outputs the results of the above processing to a display device (not shown), such as a display.

[0024] FIG. 4 is a flowchart of a construction site condition recording method using the construction site condition recording system 1 described above. In the following, the flow of the construction site condition recording method will be explained using the flowchart, and the processing of each of the above-mentioned detection unit 43, identification tag photograph absolute position calculation unit 44, relative path calculation unit 45, overall absolute position calculation unit 46, and output unit 47 will be explained in detail.

[0025] First, in the manner already described, an identification tag M is placed within the construction site G (step S1). Next, information about the identification tag M is registered in the database 42 of the system main body 4 (step S2). In this embodiment, the database 42 stores a drawing D as shown in FIG. 2 as information about the construction site G. An origin O is set at an arbitrary position on the drawing D, for example, at the position of the lower left corner in FIG. 2, and the drawing D is expressed as a two-dimensional coordinate system based on the origin O. Information about the absolute position, including the position, i.e., coordinates (absolute coordinates), of the identification tag M on this drawing D, i.e., within the construction site G, and the floor number on which the identification tag M is provided, and the posting direction, which is the direction in which the surface of the identification tag M faces, are associated with the identification information of the identification tag M and registered in the database 42.

[0026] After performing the above steps S1 and S2 as advance preparations, a site worker Q carries the photographing and recording device 3 and patrols the construction site G. In the photographing and recording device 3, the photographing unit 31 photographs the construction site G to generate a plurality of photographed images P, which are stored in the photographed image storage unit 32 (step S3). When patrolling the construction site G, it is desirable for the photographing and recording device 3 to pass close to the identification tags M as appropriate. However, it is not necessary for all identification tags M to be photographed in the photographed image P, and it is sufficient that at least one identification tag M is photographed in multiple photographed images P at any point during the patrol. The photographed image output unit 33 transmits the plurality of photographed images P stored in the photographed image storage unit 32 to the system main body 4 (step S4).

[0027] When the processing on the shooting and recording device 3 side as described above as steps S3 and S4 is completed, the shooting image acquisition unit 41 of the system main body 4 receives multiple shooting images P from the shooting image output unit 33 and stores them in the database 42 (step S5). As already explained, in this embodiment, the processes of taking the image P in step S3, transmitting the image P in step S4, and receiving the image P in step S5 are performed in real time, but the transmission and reception of the image P may also be performed after the site worker Q has completed his patrol.

[0028] Next, the detection unit 43 detects the identification tag M from among the plurality of captured images P (step S6). The detection unit 43 determines, for each of the plurality of captured images P, whether or not the identification tag M is captured in the captured image P.

[0029] The identification tag photographing absolute position calculation unit 44 calculates, for each of the photographed images P in which the identification tag M is detected among the multiple photographed images P, the absolute position within the construction site G of the photographing position where the photographed image P was photographed based on the identification tag M photographed in the photographed image P (step S7). Specifically, the identification tag photograph absolute position calculation unit 44 first estimates the relative position of the photographing position at which the photographed image P was taken with respect to the identification tag M. The identification tag photograph absolute position calculation unit 44 calculates the angle at which the photographed image P was taken from a frontal position of the identification tag M, for example, based on the positions of the corners Mc of the identification tag M, as shown in FIG. 3. When the identification tag M is photographed from the front, the corners Mc should be positioned in the photographed image P so that the identification tag M appears rectangular (rectangle or square) in the photographed image P. However, when the identification tag M is photographed obliquely from the front of the identification tag M, the lines connecting the corners Mc will not form a rectangle, but will instead be positioned in the photographed image P so that the lines form a distorted shape, such as a trapezoid. The identification tag photograph absolute position calculation unit 44 calculates the angle at which the photographed image P was taken from a frontal position of the identification tag M, based on the positions of the corners Mc. Furthermore, the identification tag photograph absolute position calculation unit 44 calculates the size of the identification tag M in the photographed image P, and based on this, calculates the distance from the identification tag M to the position where the photographed image P was photographed. The identification tag photograph absolute position calculation unit 44 calculates the relative position at which the photographed image P was photographed with respect to the identification tag M, based on the angle and distance calculated as described above. Fig. 5 is an explanatory diagram showing the relationship between each of the captured images, the identification tag, and the relative position of the capture position estimated based on the position of the identification tag with respect to the identification tag M. In Fig. 5, the capture position (relative position with respect to the identification tag M) where each of the captured images P in which the identification tag M1 was detected was captured is depicted with a black circle.

[0030] Next, for each of the multiple photographed images P in which the identification tag M is detected, the identification tag photograph absolute position calculation unit 44 refers to the database 42 and obtains information regarding the absolute position of the identification tag M, including the position and floor number of the identification tag M within the construction site G, and the posting direction, which is associated with the identification information of the identification tag M detected from the photographed image P. The identification tag photographing absolute position calculation unit 44 calculates, for each photographed image P in which the identification tag M is detected, the absolute position of the photographing position within the construction site G at which the photographed image P was photographed and the orientation of the photographed image P, based on the absolute position of the identification tag M within the construction site G, the display direction of the identification tag M, and the relative position at which the photographed image P was photographed with respect to the identification tag M. Fig. 6 is an explanatory diagram showing the photographing positions shown in Fig. 5 displayed on a map of the construction site as absolute positions within the construction site. In Fig. 6, the absolute positions PA of the photographing positions where each of the photographed images P in which the identification tag M1 was detected was photographed are depicted with black circles for the identification tag M1.

[0031] In parallel with the above steps S6 and S7, the relative path calculation unit 45 calculates the relative position of the shooting position of each of the multiple captured images P relative to the shooting positions of other captured images P captured immediately before or after it, based on the content captured in each of the multiple captured images P, and calculates a relative path, which is the movement path of the relative positions (step S8). This step S8 does not have to be executed in parallel with steps S6 and S7, and may be executed after the processing of steps S6 and S7 has been completed. In this embodiment, the relative path is calculated using SfM (Structure from Motion). In SfM, feature points within each of a plurality of captured images P are extracted, and the feature points are compared between, for example, consecutively captured images P to estimate whether there are any common feature points, and if so, which feature points are identical. Based on this, the positional relationship between the capture positions of the consecutively captured images P is calculated. Fig. 7 is a diagram showing feature points extracted from each of the consecutively captured images by SfM. In Fig. 7, the extracted feature points are indicated by circles for each of two consecutively captured images P1 and P2. Fig. 8 is a diagram showing the correspondence between feature points in consecutively captured images. In Fig. 8, feature points that are estimated to be common to consecutively captured images P1 and P2 are connected by lines.

[0032] The relative path calculation unit 45 compares the pixel positions of the commonly extracted feature points in each of the captured images P1 and P2 to calculate the relative position of the shooting position of each of the multiple captured images P with respect to the shooting position of the other captured image P captured immediately before or immediately after. That is, in this embodiment, the relative path calculation unit 45 calculates the relative position of the shooting position of each of the multiple captured images P with respect to the shooting position of the other captured image P captured immediately before or immediately after, based on the change or displacement of the pixel positions of the feature points. For example, the relative path calculation unit 45 calculates the relative position of the shooting position of the captured image P2 with respect to the shooting position of the other captured image P1 captured immediately before, by calculating the displacement or movement of the feature points of the captured image P1 to the position in the captured image P2 captured immediately after, through stereo image processing. FIG. 9 is an explanatory diagram of a relative path formed by the relative positions of the respective photographing positions of a plurality of photographed images. As described above, the relative path calculation unit 45 calculates the relative position PR of the shooting position of each of the multiple captured images P relative to the shooting positions of other captured images P that were captured either immediately before or immediately after the captured image P. The relative path calculation unit 45 calculates such relative positions PR among the multiple captured images P, for example, between all consecutively captured images P, and thereby calculates a relative path IR, which is the path along which the relative positions PR of the shooting positions of each of the multiple captured images P moved.

[0033] In this embodiment, as described above, the photographing and recording device 3 uses a so-called omnidirectional camera (360-degree camera) that can capture images in all directions (360 degrees), so that the site worker Q carrying the photographing and recording device 3 will almost certainly be captured in the photographed image P. If the site worker Q is always captured in the photographed image P in this way, feature points related to the site worker Q are extracted within the pixel area of ​​the site worker Q captured in the photographed image P. Since the site worker Q should basically be captured in approximately the same position in all the photographed images P, it is considered that the feature points related to the site worker Q will also be detected in approximately the same position. Here, as described above, in this embodiment, the relative path calculation unit 45 extracts feature points in the photographed image P, calculates the relative position PR of the photographed position of the photographed image P based on changes in the pixel position of the feature points, and calculates the relative path IR. Therefore, if there is a feature point whose position remains almost constant within successively captured images P, the relative position PR may not be estimated appropriately, which may have a negative impact on the accuracy of estimating the relative position PR. To address this, in this embodiment, the pixel area of ​​the captured image P where the site worker Q is captured is masked so that image processing is not performed on that area and feature points are not extracted. In this embodiment, masking a pixel area means setting that the pixel area is not to be subjected to image processing related to feature point extraction. In this embodiment, as will be explained below as mask form 1 to mask form 4, four types of forms are realized to mask the area in which the field worker Q is photographed, and these are used appropriately.

[0034] (Mask form 1) FIG. 10 is an explanatory diagram of a state in which a part of the image capturing and recording device is masked. In mask form 1, a pixel area corresponding to a predetermined, i.e., fixed range 34, including the portion of each of the multiple captured images P where the field worker Q holding or wearing the capture and recording device 3 is captured, is masked to generate multiple mask images. The relative path calculation unit 45 extracts feature points from each of the multiple mask images generated in this manner, and compares the pixel positions in each of the mask images of the feature points commonly extracted in the mask images corresponding to each of the consecutively captured images P, thereby calculating the relative position PR of the shooting position of each of the multiple captured images P with respect to the shooting position of another captured image P captured immediately before or after it, and obtains the relative path IR. In this way, feature points are extracted based on a mask image in which a predetermined pixel region including the portion where the field worker Q is photographed is excluded from the feature point extraction process, thereby preventing extraction of feature points related to the field worker Q. Therefore, a decrease in the estimation accuracy of the relative position PR is prevented.

[0035] (Mask form 2) In this embodiment, the relative path calculation unit 45 uses the pixel area identification learned model 51 to identify the pixel area in each of the multiple captured images P in which the field worker Q is captured, and then processes the captured image P based on this to mask the area. The pixel region identification trained model 51 is realized, for example, by multiple convolutional layers followed by multiple deconvolutional layers, and is machine-learned to use semantic segmentation to identify, for each object captured in an input image input to the pixel region identification trained model 51, the pixel region corresponding to that object.

[0036] FIG. 11 is an explanatory diagram regarding processing of a captured image when semantic segmentation is used. Specifically, the pixel region identification trained model 51 is trained using a training input image prepared in advance and a labeled image, in which each pixel region in the training input image is labeled, as training data corresponding to the training input image. When a training input image is input, the pixel region identification trained model 51 is trained by adjusting values ​​of parameters, weights, etc. using backpropagation or gradient descent so that the output result is close to the labeled image corresponding to the training input image. As a result, when a training input image is input, the pixel region identification trained model 51 is trained to output an inference result, i.e., a labeled image, that is close to the corresponding training data.

[0037] In this way, a pixel region identification trained model 51 is generated as a trained model that is used as a program module that is part of artificial intelligence software, with learning parameters such as filter parameters and weights learned. The pixel region identification learned model 51, which has been deep-learned as described above, is configured to output a labeled image PL when a captured image P is input, in which the pixel regions within the captured image P are labeled and each pixel is set to have a brightness value corresponding to the label assigned to that pixel, resulting in each pixel region within the captured image P being assigned a different color.

[0038] The relative path calculation unit 45 inputs each of the multiple captured images P into the pixel region identification trained model 51 to generate a labeled image PL for each of the multiple captured images P. Semantic segmentation does not go as far as identifying the type of object corresponding to each pixel region. Therefore, the relative path calculation unit 45 identifies a pixel region in the labeled image PL that includes one specific pixel PQ in the captured image P that is likely to contain the field worker Q. In this manner, the relative path calculation unit 45 identifies the pixel region RQ corresponding to the field worker Q in each of the multiple captured images P. Then, the relative path calculation unit 45 generates a masked image by masking the pixel region RQ corresponding to the field worker Q in each of the multiple captured images P to exclude it from the feature point extraction process. In this manner, the relative path calculation unit 45 generates multiple masked images from the multiple captured images P.

[0039] The relative path calculation unit 45 extracts feature points from each of the multiple mask images generated in this manner, and compares the pixel positions in each of the mask images of the feature points commonly extracted in the mask images corresponding to each of the consecutively captured images P, thereby calculating the relative position PR of the shooting position of each of the multiple captured images P with respect to the shooting position of another captured image P captured immediately before or after it, and obtains the relative path IR. In this way, feature points are extracted based on a mask image in which pixel regions where the field worker Q is thought to be photographed are excluded from the feature point extraction process, thereby preventing extraction of feature points related to the field worker Q. Therefore, a decrease in the estimation accuracy of the relative position PR is prevented. In particular, in mask form 2, the minimum area considered to correspond to the field worker Q is masked. Therefore, the number of feature values ​​that can be used to calculate the relative route IR in the captured image P increases, and the accuracy of the relative route IR can be improved.

[0040] (Mask form 3) The relative path calculation unit 45 uses the object identification trained model 52 to mask the area in each of the multiple captured images P in which the site worker Q is captured. The object identification trained model 52 is realized, for example, to have multiple convolutional layers, and is machine-trained to identify the approximate position of each object by surrounding each object photographed in the input image input to the object identification trained model 52 with a rectangular outer frame through object detection. FIG. 12 is an explanatory diagram regarding processing of a captured image when object detection is used. Specifically, the object identification trained model 52 is trained using a training input image prepared in advance and information about each object captured in the training input image (e.g., the position and type of the object) as training data corresponding to the training input image. When a training input image is input, the object identification trained model 52 performs machine learning by adjusting values ​​of parameters, weights, etc. using backpropagation or gradient descent so that the output result is close to the information about each object corresponding to the training input image. As a result, when a training input image is input, the object identification trained model 52 is trained to output an inference result close to the corresponding training data. For example, for each object in the input image, the object identification trained model 52 outputs the type of the object and an outer frame FR that surrounds the portion of the image corresponding to the object.

[0041] In this way, an object identification learned model 52 is generated as a learned model that is used as a program module that is part of artificial intelligence software, with learning parameters such as filter parameters and weights learned. The object identification learned model 52, which has been deep-learned as described above, is configured so that when a photographed image P is input, the object in the photographed image P is identified and the type of object and an outer frame FR that surrounds the part of the photographed image P that corresponds to the object are output.

[0042] The relative path calculation unit 45 inputs each of the multiple captured images P into the object identification learned model 52, and outputs, for each of the multiple captured images P, an outer frame FR that surrounds a portion of the captured image P that corresponds to the object. Next, based on the output information, the relative path calculation unit 45 identifies the outer frame FR that corresponds to the field worker Q in the captured image P by identifying an outer frame FR whose corresponding object is estimated to be a person. Then, the relative path calculation unit 45 generates a mask image by masking all pixels included in the outer frame FR that are inside the outer frame FR and excluding them from the feature point extraction process. In this way, the relative path calculation unit 45 generates multiple mask images from the multiple captured images P.

[0043] The relative path calculation unit 45 extracts feature points from each of the multiple mask images generated in this manner, and compares the pixel positions in each of the mask images of the feature points commonly extracted in the mask images corresponding to each of the consecutively captured images P, thereby calculating the relative position PR of the shooting position of each of the multiple captured images P with respect to the shooting position of another captured image P captured immediately before or after it, and obtains the relative path IR. In this way, feature points are extracted based on a mask image in which pixel regions where the field worker Q is thought to be photographed are excluded from the feature point extraction process, thereby preventing extraction of feature points related to the field worker Q. Therefore, a decrease in the estimation accuracy of the relative position PR is prevented. Furthermore, in mask form 2, if the position of field worker Q changes within the photographed image P and one specific pixel PQ in the photographed image P, which is thought to have a high probability of containing field worker Q, is no longer included in the pixel area corresponding to field worker Q, there is a possibility that a pixel area different from field worker Q will be erroneously estimated to correspond to field worker Q. In contrast, in mask form 3, a person within the photographed image P is identified and the pixel area corresponding to field worker Q is estimated, so that the pixel area corresponding to field worker Q can be reliably masked.

[0044] (Mask form 4) The relative path calculation unit 45 uses both the pixel area identification learned model 51 and the object identification learned model 52 to identify the pixel area in which the field worker Q is photographed in each of the multiple captured images P, and masks that area. FIG. 13 is an explanatory diagram regarding the processing of a captured image when both semantic segmentation and object detection are used. In this case, the relative path calculation unit 45 first inputs each of the multiple captured images P into the object identification learned model 52, and outputs, for each of the multiple captured images P, an outer frame FR that surrounds the portion of the captured image P that corresponds to the object. Next, based on the output information, the relative path calculation unit 45 identifies the outer frame FR that corresponds to the field worker Q in the captured image P by specifying, from among the outer frames FR, those that are estimated to be the corresponding object to be a person.

[0045] Next, the relative path calculation unit 45 generates a clipped image PC by clipping the portion inside the outer frame FR that has been identified as corresponding to the field worker Q. The relative path calculation unit 45 then inputs the clipped image PC to the pixel area identification trained model 51, which generates a labeled image PL for the clipped image PC. Because the clipped image PC is obtained by clipping the portion of the photographed image P that corresponds to the field worker Q, the pixel area with the largest area in the labeled image PL is considered to be the area corresponding to the field worker Q. Therefore, the relative path calculation unit 45 identifies the pixel area RQ that corresponds to the field worker Q in each of the multiple photographed images P by identifying the pixel area with the largest area in the labeled image PL. Then, the relative path calculation unit 45 generates a mask image by masking the pixel region RQ corresponding to the field worker Q in each of the multiple captured images P to exclude the pixel region RQ from the feature point extraction process. In this way, the relative path calculation unit 45 generates a plurality of mask images from the multiple captured images P.

[0046] The relative path calculation unit 45 extracts feature points from each of the multiple mask images generated in this manner, and compares the pixel positions in each of the mask images of the feature points commonly extracted in the mask images corresponding to each of the consecutively captured images P, thereby calculating the relative position PR of the shooting position of each of the multiple captured images P with respect to the shooting position of another captured image P captured immediately before or after it, and obtains the relative path IR. In this way, feature points are extracted based on a mask image in which pixel regions where the field worker Q is thought to be photographed are excluded from the feature point extraction process, thereby preventing extraction of feature points related to the field worker Q. Therefore, a decrease in the estimation accuracy of the relative position PR is prevented. In particular, in mask form 4, similar to mask form 2, the minimum area considered to correspond to the field worker Q is masked. Therefore, the number of feature values ​​that can be used to calculate the relative route IR in the captured image P increases, and the accuracy of the relative route IR can be improved. Furthermore, as with mask form 3, the person in the captured image P is identified and the pixel area corresponding to the field worker Q is estimated, so that the pixel area corresponding to the field worker Q can be reliably masked.

[0047] The relative path IR may be calculated using any of the above mask forms 1 to 4. In either case, as described above, the relative route IR contains only information regarding the relative positions PR of the shooting positions where the captured images P were captured. In other words, the relative route IR is route information that is set so that the relative positions PR of the shooting positions of the captured images P are continuous. Therefore, the relative route IR does not contain information regarding the absolute position or direction within the construction site G. Furthermore, the distance between the relative positions PR on the relative route IR is not necessarily the same as the distance at the actual construction site G. In other words, with respect to the shooting positions of two captured images P, the value of the scale ratio obtained by dividing the distance between the absolute positions PA corresponding to the shooting positions by the distance between the corresponding relative positions PR on the relative route IR is not necessarily 1.

[0048] Fig. 14 is an explanatory diagram showing the relationship between the absolute position of the photographing position estimated based on the position of the identification tag and the relative position of the photographing position of the photographed image in which the identification tag was detected within the relative route. Fig. 14 shows a state in which the absolute positions PA of the photographing positions within the construction site G obtained for a plurality of photographed images P, for example, six, correspond to the relative positions PR of the photographing positions within the relative route IR. Each of the absolute positions PA and PA is calculated based on the photographed images P. Therefore, the correspondence between which absolute position PA corresponds to which relative position PR within the relative route IR can be obtained via the photographed images P. The overall absolute position calculation unit 46, which will be described next, adjusts the scale ratio, rotation angle, and position of the entire relative route IR based on the correspondence between the absolute position PA and the relative position PR, so that there is a corresponding absolute position PA within the relative route IR, i.e., so that the relative position PR calculated corresponding to the captured image P in which the identification tag M was captured is superimposed on and aligned with the corresponding absolute position PA. In this way, for each relative position PR included in the relative route IR, the absolute position of the photographing position within the construction site G where that image P was photographed is calculated (step S9).

[0049] For this purpose, the overall absolute position calculation unit 46 first calculates the scale ratio by dividing the distance between the absolute positions PA of the photographing positions calculated for each of the photographed images P in which the identification tag M is successively detected by the distance between the corresponding relative positions PR on the relative path IR. For example, the scale ratio S between the absolute positions PA1, PA2 and the corresponding relative positions PR1, PR2 shown in FIG. 1、2 can be expressed by the following equation using the distance LA12 between the absolute position PA1 and the absolute position PA2 and the distance LR12 between the relative position PR1 and the relative position PR2. S 1、2 =LA12 / LR12 Similarly, the scale ratio S between the absolute positions PA2, PA3 and the corresponding relative positions PR2, PR3 2、3can be expressed by the following equation using the distance LA23 between the absolute position PA2 and the absolute position PA3 and the distance LR23 between the relative position PR2 and the relative position PR3. S 2、3 =LA23 / LR23

[0050] If the estimation accuracy of the absolute position PA and the relative position PR is high and ideal results are obtained, S 1、2 and S 2、3 , and other absolute positions PAn-1, absolute position PAn and relative position PRn-1, scale ratio S calculated at relative position PRn n-1、n However, in reality, the estimation includes errors, so the scale ratios between the shooting positions may be different from each other. For this reason, it is desirable to calculate a representative value of all the scale ratios between the photographing positions calculated as described above, and use this as the scale ratio S of the entire relative path IR to be used in subsequent calculations. The representative value may be calculated as an average value, for example. However, in this case, if there are significant outliers in the estimated absolute positions PA and relative positions PR, the outliers will also be used in calculating the average value, and the average value may not be appropriate due to the influence of the outliers. For this reason, it is more desirable to use the median value as the representative value.

[0051] The overall absolute position calculation unit 46 multiplies each of the relative positions PR in the relative path IR by the scale ratio S of the entire relative path IR calculated as described above, to calculate a scale-adjusted relative position, which is a relative position whose scale has been adjusted to match the absolute position PA. In the following description, the scale-adjusted relative position calculated as above will be treated as the relative position PR of the relative path IR. Thereafter, the overall absolute position calculation unit 46 performs a calculation to move the entire relative path IR so that the relative position PR (more precisely, the scale-adjusted relative position) of the shooting position of the captured image P in which the identification tag M was detected in the relative path IR is aligned with the absolute position PA of the shooting position of the captured image P. Figure 15 is an explanatory diagram showing the relative route displayed on a map of the construction site, with the entire relative route moved so that the relative position of the shooting position of the captured image from which the identification tag was detected is aligned with the absolute position of the shooting position of the captured image. In Figure 15, for each captured image P in which the identification tag M is detected, the position of the entire relative path IR is adjusted so that the relative positions PR1 to PR6 in the relative path IR overlap with the corresponding absolute positions PA1 to PA6, respectively. By doing this, even for a photographed image P in which the identification tag M is not detected and the absolute position PA is not estimated, it is possible to calculate the absolute position PA of the photographed position within the construction site G where the photographed image P was taken.

[0052] Next, with reference to Figures 16 to 21, the process of aligning the relative position PR of the shooting position of the captured image P in which the identification tag M is detected in the relative path IR as described above with the absolute position PA of the shooting position will be described in more detail. FIG. 16 is an explanatory diagram showing a state before the process of superimposing the set of relative positions PR on the set of absolute positions PA is executed. In the relative path IR, a relative position PR of the shooting position is calculated for each of the multiple captured images P. Among these multiple relative positions PR, only the captured image P in which the identification tag M was detected has a corresponding absolute position PA to be superimposed. Therefore, hereinafter, when it is stated that a set of relative positions PR is "superimposed" on a set of absolute positions PA, it means that the set of relative positions PR is superimposed between the corresponding shooting positions on the set of absolute positions PA, particularly limited to the captured image P in which the identification tag M was detected.

[0053] (First method for matching a set of relative positions PR with a set of absolute positions PA) FIG. 17 is an explanatory diagram of a first method for matching a set of relative positions with a set of absolute positions. In the first method, the overall absolute position calculation unit 46 first selects an arbitrary captured image Pn (not shown) from the captured images P in which the identification tag M is detected, and then superimposes the relative position PRn of the captured position estimated for the captured image Pn on the absolute position PAn of the captured position estimated for the captured image Pn. Next, the overall absolute position calculation unit 46 rotates the entire relative path IR around the superimposed absolute position PAn (relative position PRn). As explained with respect to the absolute position PA, the identification tag photograph absolute position calculation unit 44 calculates, for the photographed image P in which the identification tag M is detected, the absolute position at which the photographed image P was photographed within the construction site G and the orientation of the photographed image P. Therefore, the overall absolute position calculation unit 46 uses the orientation of this photographed image P to rotate the relative path IR so that the orientation of the relative position PRn matches the orientation of the absolute position PAn. In this first method, if the absolute position PA and the relative position PR include an error, the farther the photographing position of a photographed image P is from the absolute position PAn (relative position PRn) of the photographing position of an arbitrarily selected photographed image Pn, the larger the error in the absolute position calculated for the relative position PR of the photographing position of the photographed image P. Furthermore, if either the relative position PRn or the relative position PRn of the photographing position of an arbitrarily selected photographed image Pn is an outlier, the result will be that the set of relative positions PR will not overlap at all with the set of absolute positions PA.

[0054] (Second method for matching a set of relative positions PR with a set of absolute positions PA) FIG. 18 is an explanatory diagram of a second method for matching a set of relative positions with a set of absolute positions. In the second method, the overall absolute position calculation unit 46 arbitrarily selects three captured images P, namely, a first captured image, a second captured image, and a third captured image, from among a plurality of captured images P in which the identification tag M has been detected. These first, second, and third captured images are selected such that the absolute position PAs of the capturing position of the first captured image, the absolute position PAt of the capturing position of the second captured image, and the absolute position PAu of the capturing position of the third captured image are not on a straight line. Next, the overall absolute position calculation unit 46 translates the entire relative path IR so that the relative position PRs of the shooting position of the first captured image coincides with the absolute position PAs of the shooting position of the first captured image, resulting in a state as shown in Figure 18. Then, the overall absolute position calculation unit 46 rotates the entire relative path IR around the absolute position PAs (relative position PRs) of the shooting position of the first captured image so that the relative position PRt (scale-adjusted relative position) of the shooting position of the second captured image coincides with the absolute position PAt of the shooting position of the second captured image.

[0055] Figure 19 is a figure following Figure 18, and shows the state in which the entire relative path has been rotated around the absolute position of the shooting position of the first captured image so that the relative position of the shooting position of the second captured image matches the absolute position of the shooting position of the second captured image. Furthermore, the overall absolute position calculation unit 46 rotates the entire relative path IR around a line Lst connecting the absolute position PAs (relative position PRs) of the shooting position of the first captured image and the absolute position PAt (relative position PRt) of the shooting position of the second captured image so that the relative position PRu (scale-adjusted relative position) of the shooting position of the third captured image coincides with the absolute position PAu of the shooting position of the third captured image. In this way, the overall absolute position calculation unit 46 aligns the relative position PR of the photographing position of the photographed image P in which the identification tag M was detected within the relative path IR with the absolute position PA of the photographing position of that photographed image P. In this second method, when the absolute position PA and the relative position PR include an error, the effect of the error on the calculation result can be reduced more than in the first method.

[0056] However, as with the first method, the second method may also produce calculation results containing large errors if there are outliers in the absolute positions PA or relative positions PR estimated for three arbitrarily selected captured images P. To address this issue, applying RANSAC (Random Sample Consensus) may be considered. RANSAC is a technique for estimating a shape from a set of points by randomly extracting the minimum number of points required to define a shape and repeatedly determining whether a set of points is on the shape generated by the extracted points. In this embodiment, a shape is defined using three representative points, and whether the set of relative positions PR and the set of absolute positions PA overlap can be determined by calculating, for each captured image P, whether the distance between the relative position PR and the corresponding absolute position PA is within a threshold.

[0057] Fig. 20 is an explanatory diagram of a case where, as a result of superimposing a set of relative positions on a set of absolute positions, there are many cases where the distance between corresponding points is equal to or greater than a threshold. Fig. 21 is an explanatory diagram of a case where, as a result of superimposing a set of relative positions on a set of absolute positions, there are few cases where the distance between corresponding points is equal to or greater than a threshold. In this case, more accurate calculation results can be obtained by selecting three captured images P as shown in Fig. 21 rather than by selecting three captured images P as shown in Fig. 20. In this way, it is desirable to repeat the selection and determination of the three photographed images P, and ultimately adopt the one with the most overlapping points.

[0058] In both the first and second methods, the movement and rotation of the relative path IR can be represented by a rotation matrix that represents the movement and rotation. When the movement and rotation of the relative path IR are performed sequentially multiple times, these multiple movements and rotations can be represented by a single matrix obtained by multiplying the corresponding matrices. In this case, the calculations performed by the overall absolute position calculation unit 46 can be realized by representing the multiple relative positions PR included in the relative path IR as vectors and multiplying the vectors by the matrices that represent the movement and rotation of the relative path IR.

[0059] In the above, a case has been described in which the absolute positions of all of the relative positions PR included in the relative route IR are calculated based on the detection results of one identification tag M. However, there may be a case in which a plurality of identification tags M are provided, and different identification tags M are captured in different captured images P among a plurality of captured images P. Even in such a case, the absolute position for each of the relative positions PR included in the relative route IR can basically be calculated using the above-described processing. As already explained, the relative path calculation unit 45 calculates the relative path IR by calculating the relative positional relationship (relative position PR) between the respective shooting positions of successively captured images P using SfM. As already explained, in SfM, the relative position PR of the shooting position of a certain captured image P is calculated relative to the shooting position of the subsequent captured image P, and the relative position PR of the shooting position of the further subsequent captured image P is calculated relative to the relative position PR, and so on, thereby calculating the relative path IR by continuing to calculate the relative positions PR between successive captured images P. For this reason, as the calculation of the relative positions PR progresses, errors contained in the relative positions PR accumulate, resulting in larger errors. Here, in the overall absolute position calculation unit 46, with regard to the photographing position of the photographed image P in which the identification tag M is not captured, the absolute position PA within the construction site G for each relative position PR in the relative route IR is calculated using the absolute position PA of the photographing position of the photographed image P in which the identification tag M is detected as the base point. Therefore, in this embodiment, the absolute position PA calculated for each relative position PR in the relative route IR may deviate significantly from the original, correct position as it moves away from the identification tag M.

[0060] FIG. 22 is an explanatory diagram showing a case where there are multiple identification tags and, for an image taken at a position between the identification tags, there is a gap between the absolute position of the photographing position calculated based on one of the identification tags and the absolute position of the photographing position calculated based on the other identification tag. Here, an example will be described based on the example shown in Fig. 22. In Fig. 22, an identification tag M2 located on the left side of the figure is detected in consecutive images P among a plurality of captured images P, and the identification tag photography absolute position calculation unit 44 estimates the absolute positions PA of these photographing positions as a plurality of positions starting from absolute position PA7 and ending at absolute position PA8. Furthermore, an identification tag M is not detected in subsequent images P. In these images P, the relative path calculation unit 45 calculates a relative path IR, and based on the relative path IR, the absolute positions PA of the photographing positions are estimated as a plurality of positions starting from absolute position PA9 and ending at absolute position PA14. Then, in further subsequent images P, an identification tag M3 located on the right side of the figure is detected, and the identification tag photography absolute position calculation unit 44 estimates the absolute positions PA of these photographing positions as a plurality of positions starting from absolute position PA15 and ending at absolute position PA16. Hereinafter, the section from the absolute position PA (absolute position PA9) of the shooting position of the captured image P in which the identification tag M is no longer detected, such as from absolute position PA9 to absolute position PA14, to the absolute position PA (absolute position PA14) of the shooting position of the last captured image P in which the identification tag M was not detected, i.e., the image P captured immediately before the next captured image P in which the identification tag M was detected, will be referred to as the transition section TR.

[0061] As described above, in this embodiment, the relative position PR in the transition section TR and the absolute position PA calculated for that relative position PR may deviate significantly from their original positions as they move away from the identification tag M. Therefore, when calculating the absolute position PA for the relative position PR in the transition section TR, it is desirable to base the calculation on the identification tag M2 detected before the start of the transition section TR for the first half of the transition section TR. Similarly, it is desirable to base the calculation on the identification tag M3 detected after the end of the transition section TR for the second half of the transition section TR. In FIG. 22, the absolute position PA within the transition section TR is calculated in this manner. However, in this case, for the captured image P captured at the intermediate position of the transition section TR, there is a possibility that the deviation J between the absolute position PAm-2 calculated based on the identification tag M2 and the absolute position PAm-3 calculated based on the identification tag M3 will be large, resulting in a lack of continuity in the absolute positions PA calculated in the transition section TR.

[0062] Therefore, in this embodiment, for each relative position PR within the transition section TR, both the absolute position PA based on the identification tag M2 detected before the transition section TR begins and the absolute position PA based on the identification tag M3 detected after the transition section TR ends are calculated separately, and then the final absolute position PA is calculated taking both of these into consideration. In this case, weights are applied so that on the identification tag M2 side of the transition section TR, the absolute position PA estimated based on the identification tag M2 is given greater consideration than the absolute position PA estimated based on the identification tag M3, and so that on the identification tag M3 side, the absolute position PA estimated based on the identification tag M3 is given greater consideration than the absolute position PA estimated based on the identification tag M3. FIG. 23 is an explanatory diagram showing the application of weights to the absolute positions of the imaging positions that are deviated in the situation of FIG. As shown in Figure 23, the transition section TR is divided into a first transition section TR1 on the identification tag M2 side, a second transition section TR2 located midway between identification tags M2 and M3, and a third transition section TR3 on the identification tag M3 side.

[0063] In the first transition section TR1, the weighting is set so that it refers only to the absolute position PA estimated based on identification tag M2, and not to the absolute position PA estimated based on identification tag M3. That is, in the first transition section TR1, the absolute position PA that is the final calculation result after weighting is applied is the same as the absolute position PA estimated based on identification tag M2. Therefore, for absolute positions PA9 and PA10 included in the first transition section TR1 in Figure 23, the absolute positions PA9 and PA10 estimated based on identification tag M2 are output as the final shooting positions. In the third transition section TR3, the weighting is set so that it refers only to the absolute position PA estimated based on identification label M3, and not to the absolute position PA estimated based on identification label M2. That is, in the third transition section TR3, the absolute position PA that is the final calculation result after weighting is applied is the same as the absolute position PA estimated based on identification label M3. Therefore, in the absolute positions PA13-PA14 included in the third transition section TR3 in Fig. 23, the absolute positions PA13-PA14 estimated based on identification label M3 are output as the final shooting positions.

[0064] The weight is set as a function that takes a first value (e.g., 1) at the end of the second transition section TR2 on the identification tag M2 side, takes a second value (e.g., 0) at the end of the second transition section TR2 on the identification tag M3 side, and gradually changes from the first value to the second value as it moves from the identification tag M2 side to the identification tag M3 side. As a result, for example, with regard to the absolute position PA closest to the identification tag M2 side in the second transition section TR2, the final absolute position PA11 is calculated as a position closer to the absolute position PA11-2 between the absolute position PA11-2 estimated based on the identification tag M2 and the absolute position PA11-3 estimated based on the identification tag M3. Furthermore, with regard to the absolute position PA closest to the identification tag M3 in the second transition section TR2, the final absolute position PA12 is calculated as a position closer to the absolute position PA12-3 between the absolute position PA12-2 estimated based on the identification tag M2 and the absolute position PA12-3 estimated based on the identification tag M3. For the absolute positions PA between these two, the final absolute position PA is calculated to gradually move from a position closer to the absolute position PA estimated based on the identification tag M2 to a position closer to the absolute position PA estimated based on the identification tag M3 as it moves from the identification tag M2 side to the identification tag M3 side.

[0065] The ratio of the length of the first transition section TR1 and the length of the third transition section TR3 to the total length of the transition section TR may be, for example, 0.2, and the ratio of the length of the second transition section TR2 to the total length of the transition section TR may be, for example, 0.6. In the second transition section TR2, the weighting value can be set to change linearly. In this case, for example, if the weighting value w is 1 at the end of the second transition section TR2 on the identification tag M2 side and 0 at the end on the identification tag M3 side, the final value pf of the absolute position PA at each of the photographing positions can be expressed by the following equation: pf=pa2×w+pa3×(1-w) In the above equation, pa2 is the value of the absolute position PA estimated based on the identification tag M2, and pa3 is the value of the absolute position PA estimated based on the identification tag M3.

[0066] In this manner, the overall absolute position calculation unit 46 moves the entire relative route IR so as to align the relative position PR of the shooting position of the captured image P in which the identification tag M was detected with the absolute position PA of the shooting position of the captured image P, and calculates the absolute position PA within the construction site G for each of the relative positions PR in the relative route IR.

[0067] The total absolute position calculation unit 46 associates the absolute position PA calculated as described above with each captured image P and stores the absolute position PA in the database 42 (step S10). In this way, the state within the construction site G at the time when the site worker Q patrolled the construction site G, i.e., the contents of the captured image P, are recorded together with the absolute position PA within the construction site G where the captured image P was taken.

[0068] FIG. 24 is a diagram showing a state in which the absolute position calculated using the construction site state recording system 1 of this embodiment is displayed. As shown in FIG. 24, the output unit 47 displays the calculated value of the absolute position PA on a display device (not shown) such as a display, in a state where the value is superimposed on a drawing D of the construction site G. In this state, the construction site condition recording system 1 can be configured so that, for example, when a worker arbitrarily selects a displayed absolute position PA using an input device such as a mouse (not shown), the image P captured at the selected absolute position PA is displayed on the screen of the display device. In this way, the condition of any position within the construction site G at the time when the site worker Q patrolled the construction site G can be confirmed.

[0069] Next, we will explain the equipment and materials management system 70, which uses the construction site condition recording system 1 of this embodiment to detect and manage the locations of various equipment and materials (hereinafter referred to as site equipment and materials) used when carrying out work within the construction site G. FIG. 25 is a block diagram of an equipment and materials management system that detects and manages the positions of on-site equipment and materials within a construction site using the construction site condition recording system of this embodiment. The equipment and materials management system 70 includes the construction site condition recording system 1 as described above, and a site equipment and materials position specifying unit 71. More specifically, the equipment and materials management system 70 is configured by functionally realizing the site equipment and materials position specifying unit 71 inside the system main body 4 of the construction site condition recording system 1.

[0070] In the equipment and materials management system 70, the detection unit 43 detects the on-site equipment and materials in addition to the identification tag M from the multiple captured images P. The detection unit 43 can be realized to detect on-site equipment using a trained model that is deep-learned using training input images of on-site equipment and identification data of the on-site equipment captured in the training input images. In this case, the trained model is preferably configured using, for example, a convolutional neural network (CNN). The trained model is deep-trained so that when an image is input, if any on-site equipment is captured in the image, the trained model identifies the on-site equipment by outputting identification data corresponding to the on-site equipment. In this way, the trained model is used as a program module that is part of artificial intelligence software and has been trained with appropriate learning parameters. The detection unit 43 executes this trained model as a program on, for example, a CPU or GPU, to identify on-site equipment when a captured image P is input. In this embodiment, the identification data is, for example, name data of the site equipment or materials, but may also be an identification number associated with each site equipment or materials.

[0071] FIG. 26 is a diagram showing an example of a captured image P in which on-site equipment and materials have been detected by the detection unit. The detection unit 43 inputs the captured image P into, for example, a trained model configured as described above, and outputs identification data of the site equipment 100, thereby estimating the site equipment 100 captured in the captured image P. In Fig. 7, the detected site equipment 100 in the captured image P is shown surrounded by a frame Px. When the detection unit 43 detects the site equipment 100 from the captured image P, it records in the database 42 the correspondence between the captured image P in which the site equipment 100 was detected and the identification data of the detected site equipment 100.

[0072] When using the above-described equipment and materials management system 70 to search for, for example, a specific on-site equipment or material 100, the worker inputs information about the on-site equipment or material 100 that he or she is searching for into the equipment and materials management system 70 via an input device such as a keyboard (not shown). When information about the site equipment or material 100 being searched for is input, the site equipment or material position specifying unit 71 refers to the database 42 and searches for the photographed image P associated with the identification data corresponding to the input site equipment or material 100. Next, the site equipment or material position specifying unit 71 acquires the absolute position PA of the photographing position where the photographed image P was taken, calculated by the overall absolute position calculation unit 46 of the construction site condition recording system 1. In this way, the site equipment and material position specifying unit 71 estimates the position within the construction site G of the site equipment and material 100 that is the search target. The output unit 47 highlights and displays, for example, the position on the drawing D of the construction site G where the site equipment and materials 100 are estimated to be located.

[0073] Next, a construction progress monitoring system 80 will be described, which uses the construction site condition recording system 1 of this embodiment to manage the progress of wall construction and ceiling construction within a construction site. FIG. 27 is a block diagram of a construction progress monitoring system that uses the construction site status recording system of this embodiment to manage the progress of wall construction and ceiling construction within a construction site. The construction progress monitoring system 80 includes the construction site condition recording system 1 as described above, and a construction location identification unit 81. More specifically, the construction progress monitoring system 80 is configured by functionally realizing the construction location identification unit 81 inside the system main body 4 of the construction site condition recording system 1.

[0074] In the construction progress monitoring system 80, the detection unit 43 detects construction work including wall construction and ceiling construction work in addition to identification tags M from multiple captured images P, and if wall construction work or ceiling construction work is captured in a photograph, estimates the progress status of the construction work. The detection unit 43 can be realized to detect wall construction using a trained model for wall construction, which is obtained by deep learning using training input images related to wall construction and the type of wall construction captured in the training input images as training data. In this case, the trained model for wall construction can be an object detection model such as faster R-CNN, YOLO, or SSD (Single Shot MultiBox Detector). The types of wall construction work described above indicate the progress of wall construction work, such as wall underlayment work using LGS (Light Gauge Steel), insulation work, spraying work, board work, putty filling work, finishing work, etc. The trained model for wall construction is deep-trained so that when an image is input, if any wall construction is captured in the image, the trained model for wall construction estimates the progress of the wall construction by outputting the type of construction corresponding to that wall construction. In this way, the trained model for wall construction is used as a program module that is part of artificial intelligence software and has learned appropriate learning parameters. The detection unit 43 executes this trained model for wall construction as a program on, for example, a CPU or GPU, and estimates the progress of the wall construction when a captured image P is input. When the detection unit 43 detects wall construction from the photographed image P, it records in the database 42 the correspondence between the photographed image P in which the wall construction is detected and the progress status of the detected wall construction.

[0075] The detection unit 43 can also be realized to detect ceiling construction using a trained model for ceiling construction, which is obtained by deep learning using training input images related to ceiling construction and the type of ceiling construction captured in the training input images as training data. In this case, the trained model for ceiling construction can be an image recognition model such as VGG, Residual Network (Residual Network), or Xception. The above-mentioned types of ceiling construction work indicate the progress of the ceiling construction work, such as ceiling underlayment work, board work, equipment work, fireproof coating work, putty filling work, finishing work, and unfinished work. The trained model for ceiling work is deep-learned so that when an image is input, if any ceiling work is photographed in the image, it outputs the type of work corresponding to that ceiling work, thereby estimating the progress of the ceiling work. In this way, the trained model for ceiling work is used as a program module that is part of artificial intelligence software, and has learned appropriate learning parameters. The detection unit 43 executes this trained model for ceiling work as a program on, for example, a CPU or GPU, and estimates the progress of the ceiling work when a photographed image P is input. When the detection unit 43 detects ceiling construction work from the photographed image P, it records in the database 42 the correspondence between the photographed image P in which the ceiling construction work is detected and the progress status of the detected ceiling construction work.

[0076] The construction location specifying unit 81 specifies the area where construction is being carried out based on the content of the photographed image P in which construction is detected. FIG. 28 is a diagram showing the estimated position and range of wall construction. 28, based on a photographed image P in which wall construction work has been detected on a drawing D of the construction site G, the output unit 47 acquires the absolute position PA of the photographed position of the photographed image P and the progress status of the wall construction work from the database 42. The output unit 47 generates a range Wm representing the progress status so as to group together absolute positions PA that are close to each other and have the same progress status, and displays the range Wm on a display device (not shown) such as a display, for example, in a colored state according to the progress status.

[0077] FIG. 29 is a diagram showing the estimated position and range of ceiling construction work. 29, based on a captured image P in which ceiling construction work has been detected on a drawing D of the construction site G, the output unit 47 acquires the absolute position PA of the photographing position of the captured image P and the progress status of the ceiling construction work from the database 42. The output unit 47 generates a range Cm representing the progress status so as to group together absolute positions PA that are close to each other and have the same progress status, and displays the range Cm on a display device (not shown) such as a display, for example, colored in accordance with the progress status.

[0078] The construction site condition recording system 1 as described above is a construction site condition recording system 1 that records the condition within the construction site G, and includes an identification tag M placed within the construction site G, an image recording device 3 that is configured to be held or attached by a site worker Q or a mobile object configured to be movable within the construction site G and that continuously photographs the construction site G at predetermined time intervals to generate a plurality of photographed images P, a detection unit 43 that detects the identification tag M from the plurality of photographed images P, and an identification calculation unit 44 that calculates, for a photographed image P in which the identification tag M has been detected, the absolute position PA within the construction site G of the photographing position where the photographed image P was photographed based on the identification tag M photographed in the photographed image P. The system is equipped with a separate ticket photographing absolute position calculation unit 44, a relative path calculation unit 45 that calculates the relative position PR of the photographing position of each of the multiple photographed images P with respect to the photographing position of another photographed image P photographed immediately before or after it based on the content photographed in each of the multiple photographed images P, and calculates a relative path IR that is the movement path of the relative position PR, and an overall absolute position calculation unit 46 that moves the entire relative path IR so that the relative position PR of the photographing position of the photographed image P in which the identification tag M is detected in the relative path IR is aligned with the absolute position PA of the photographing position of the photographed image P, and calculates the absolute position PA within the construction site G for each of the relative positions PR in the relative path IR. According to the above-described configuration, the photographing and recording device 3 is held or attached by a site worker Q or a mobile body configured to be movable within the construction site G, and, for example, as the site worker Q or the mobile body patrols the construction site G, photographs of the construction site G are taken continuously at predetermined time intervals, and multiple photographed images P are generated. Since identification tags M are placed within the construction site G, the plurality of photographed images P captured as described above includes images in which the identification tags M are captured. The detection unit 43 detects the identification tags M from among the plurality of photographed images P in which the identification tags M are captured. For a photographed image P in which the identification tags M are detected, the identification tag photograph absolute position calculation unit 44 calculates the absolute position PA within the construction site G of the photographing position at which the photographed image P was captured, based on the identification tags M captured in the photographed image P. Furthermore, the relative path calculation unit 45 calculates the relative position PR of each of the multiple captured images P relative to the shooting position of another captured image P taken immediately before or after the shooting position of each of the multiple captured images P, based on the content captured in each of the multiple captured images P, and calculates a relative path IR, which is the movement path of the relative position PR. The relative path IR calculated in this way is found by calculating the relative positional relationship between the shooting positions of each of the continuously captured images P, and does not indicate an absolute position at the construction site G. In response to this, the overall absolute position calculation unit 46 moves the entire relative route IR so as to align the relative position PR of the shooting position of the photographed image P in which the identification tag M was detected in the relative route IR with the absolute position PA of the shooting position of the photographed image P, and calculates the absolute position PA within the construction site G for each of the relative positions PR in the relative route IR. In this way, even for a photographed image P in which the identification tag M is not detected, the absolute position PA within the construction site G of the shooting position of the photographed image P can be calculated. In this way, for all photographed images P, the photographed position of the photographed image P can be accurately identified as an absolute position PA within the construction site G. Therefore, the photographed image P as the state within the construction site G and the accurately calculated position within the construction site G as the photographed position of the photographed image P can be recorded in association with each other. When realizing the above configuration, the calculation of the shooting position of the captured image P is basically all performed based on the content captured in the captured image P. Therefore, in order to determine the shooting position of the captured image P, there is no particular need to use other means, such as a pedestrian autonomous navigation means that acquires movement information of a site worker Q who patrols the site while holding or wearing an image recording device 3. Therefore, the configuration can be simplified. Furthermore, when recording the conditions within the construction site G using the construction site condition recording system 1 as described above, it is basically sufficient for, for example, a site worker Q or a mobile object to hold or wear the photographing and recording device 3 and patrol the construction site G. Therefore, it can be used easily. In this way, a construction site condition recording system 1 can be realized that has a simpler configuration and is easier to use when recording the condition within the construction site G in correspondence with the position within the construction site G.

[0079] In addition, the relative path calculation unit 45 generates multiple mask images by masking a predetermined pixel area in each of the multiple captured images P that includes the portion where the field worker Q or moving object holding or wearing the image capturing and recording device 3 is captured, extracts feature points from each of the multiple mask images, and calculates the relative position PR of the capture position of each of the multiple captured images P with respect to the capture position of other captured images P captured immediately before or after it by comparing the pixel positions in each of the mask images of the feature points that are commonly extracted in the mask images corresponding to each of the continuously captured captured images P. When continuously photographing the construction site G using the photographing and recording device 3 to generate multiple photographed images P, for example, if an omnidirectional camera is used or if the photographing and recording device 3 is positioned so that the site worker Q or moving object holding or wearing the photographing and recording device 3 is captured in the image, there is a possibility that the site worker Q or moving object will be photographed in approximately the same position in the multiple photographed images P. According to the above configuration, the relative path calculation unit 45 generates multiple mask images by masking a predetermined pixel area including a portion of each of the multiple captured images P where the field worker Q or a moving object holding or wearing the photographing and recording device 3 is captured, extracts feature points from each of the multiple mask images, and compares the pixel positions of the feature points commonly extracted in the mask images corresponding to each of the consecutively captured photographed images P to calculate the relative position PR of the photographing position of each of the multiple captured images P with respect to the photographing position of another photographed image P captured immediately before or immediately after. If the field worker Q or a moving object carrying the photographing and recording device 3 is captured in each of the multiple captured images P, the feature points extracted from the field worker Q or the moving object will be located at approximately the same positions in the multiple captured images P, and the feature points may affect the calculation of the relative position PR of the photographing position of each of the multiple captured images P with respect to the photographing position of another photographed image P captured immediately before or immediately after. In contrast, in the above configuration, the image from which feature points are extracted is a mask image generated by masking a predetermined pixel area including the portion where the field worker Q or the moving object is photographed from the captured image P. As a result of masking the predetermined pixel area including the portion where the field worker Q or the moving object is photographed, this mask image is in a state where feature points are not (hard to) be extracted from that area. This prevents feature points from being extracted from the area corresponding to the field worker Q or the moving object in the captured image P, and as a result, prevents the relative position PR from being calculated correctly.

[0080] The relative path calculation unit 45 also includes an object identification trained model 52 that has been trained as an object detection method to identify each object photographed in the input image by surrounding it with a rectangular outer frame FR, and a pixel region identification trained model 51 that has been trained as a semantic segmentation method to identify each object photographed in the input image. The relative path calculation unit 45 inputs each of the multiple photographed images P to the object identification trained model 52, identifies the outer frame FR corresponding to the field worker Q or the moving object, and calculates the pixel region identification trained model 51 that has been trained as a semantic segmentation method to identify each of the multiple photographed images P that corresponds to the field worker Q or the moving object. The portion inside the outer frame FR is cut out and input into a pixel region identification learned model 51 to identify the pixel region RQ corresponding to the field worker Q or the moving body, the pixel region RQ identified as corresponding to the field worker Q or the moving body in each of the multiple captured images P is masked to generate multiple mask images, feature points are extracted from each of the multiple mask images, and the pixel positions in each mask image of the feature points commonly extracted in the mask images corresponding to each of the consecutively captured captured images P are compared to calculate the relative position PR of the shooting position of each of the multiple captured images P with respect to the shooting position of another captured image P taken immediately before or after it. When continuously photographing the construction site G using the photographing and recording device 3 to generate multiple photographed images P, for example, if an omnidirectional camera is used or if the photographing and recording device 3 is positioned so that the site worker Q or moving object holding or wearing the photographing and recording device 3 is captured in the image, there is a possibility that the site worker Q or moving object will be photographed in approximately the same position in the multiple photographed images P. Here, with the above-described configuration, the relative path calculation unit 45 inputs each of the multiple captured images P into an object identification trained model 52 that has been trained as object detection so as to identify each object captured in the input image by surrounding it with a rectangular outer frame FR, thereby identifying the outer frame FR corresponding to the field worker Q or the moving object. Next, the relative path calculation unit 45 cuts out the portion inside the outer frame FR identified as corresponding to the field worker Q or the moving object from each of the multiple captured images P, and inputs this into a pixel region identification trained model 51 that has been trained as semantic segmentation so as to identify a pixel region corresponding to each object captured in the input image, thereby identifying the pixel region RQ corresponding to the field worker Q or the moving object. The relative path calculation unit 45 then masks the pixel region RQ identified as corresponding to the field worker Q or the moving object in each of the multiple captured images P to generate multiple mask images, extracts feature points from each of the multiple mask images, and compares the pixel positions in each of the mask images corresponding to each of the consecutively captured captured images P of the feature points extracted in common in the mask images to calculate the relative position PR of the shooting position of each of the multiple captured images P with respect to the shooting position of another captured image P captured immediately before or after it. If the field worker Q or a moving object carrying the image capturing and recording device 3 is captured in each of the multiple captured images P, the feature points extracted from the field worker Q or the moving object will be located at approximately the same positions in the multiple captured images P, and the feature points may affect the calculation of the relative position PR of the shooting position of each of the multiple captured images P with respect to the shooting position of another captured image P captured immediately before or after it. In contrast, in the above configuration, the image from which feature points are extracted is a mask image generated by masking pixel regions RQ identified as corresponding to the field worker Q or a moving object from the captured image P. As a result of masking the pixel regions RQ identified as corresponding to the field worker Q or a moving object in this mask image, feature points are not (or are less likely to be) extracted from those regions. This prevents feature points from being extracted from the regions in the captured image P that correspond to the field worker Q or a moving object, and as a result, prevents the relative position PR from being calculated correctly.

[0081] The relative path calculation unit 45 also includes a pixel region identification trained model 51 that has been trained as semantic segmentation to identify the pixel region corresponding to each object photographed in the input image. The relative path calculation unit 45 inputs each of the multiple photographed images P into the pixel region identification trained model 51 to identify the pixel region RQ corresponding to the field worker Q or the moving body, masks the pixel region RQ in each of the multiple photographed images P that has been identified as corresponding to the field worker Q or the moving body to generate multiple mask images, extracts feature points from each of the multiple mask images, and calculates the relative position PR of the photographing position of each of the multiple photographed images P relative to the photographing position of another photographed image P photographed immediately before or after it by comparing the pixel positions in each mask image of the feature points extracted in common in the mask images corresponding to each of the consecutively photographed photographed images P. The relative path calculation unit 45 also includes an object identification trained model 52 that has been trained as an object detection model so as to identify each object photographed in the input image by surrounding it with a rectangular outer frame FR. The relative path calculation unit 45 inputs each of the multiple photographed images P into the object identification trained model 52 to identify the outer frame FR corresponding to the field worker Q or the moving body, masks the inside of the outer frame FR identified as corresponding to the field worker Q or the moving body in each of the multiple photographed images P to generate multiple mask images, extracts feature points from each of the multiple mask images, and calculates the relative position PR of the shooting position of each of the multiple photographed images P relative to the shooting position of other photographed images P photographed immediately before or after it by comparing the pixel positions in each mask image of the feature points extracted in common in the mask images corresponding to each of the photographed images P photographed successively. Even in this case, extraction of feature points from the field worker Q or the moving object is suppressed, and as a result, incorrect calculation of the relative position PR is suppressed.

[0082] Furthermore, the overall absolute position calculation unit 46 calculates the distance between the absolute positions PA of the photographing positions of the photographed images P in which the identification tag M is successively detected, calculates the scale ratio S by dividing the distance between the absolute positions PA by the distance between the relative positions PR of the photographing positions, calculates the scale-adjusted relative position by multiplying each of the relative positions PR in the relative path IR by the scale ratio S, selects the first photographed image, the second photographed image, and the third photographed image from the photographed images P in which the identification tag M is detected, moves the entire relative path IR so that the scale-adjusted relative position PRs of the photographing position of the first photographed image coincides with the absolute position PAs of the photographing position of the first photographed image, and calculates the scale-adjusted relative position by multiplying each of the relative positions PR in the relative path IR by the scale ratio S. The entire relative path IR is rotated around the absolute position PAs of the shooting position of the first captured image as the center so that the scale-adjusted relative position PRt of the first captured image coincides with the absolute position PAt of the shooting position of the second captured image, and the entire relative path IR is rotated around the line Lst connecting the absolute position PAs of the shooting position of the first captured image and the absolute position PAt of the shooting position of the second captured image as the center so that the scale-adjusted relative position PRu of the shooting position of the third captured image coincides with the absolute position PAu of the shooting position of the third captured image, thereby aligning the relative position PR of the shooting position of the captured image P in which the identification tag M was detected in the relative path IR with the absolute position PA of the shooting position of that captured image P. According to the above configuration, it is possible to appropriately execute a process of aligning the relative position PR of the shooting position of the captured image P in which the identification tag M is detected within the relative path IR with the absolute position PA of the shooting position of the captured image P.

[0083] Furthermore, the above-described equipment management system 70 is equipped with a construction site condition recording system 1, and is an equipment management system 70 that uses the construction site condition recording system 1 to detect and manage the position of site equipment 100 within the construction site G, and the detection unit 43 detects the site equipment 100 together with the identification tag M from a plurality of photographed images P, and records in the database 42 the correspondence between the photographed image P in which the site equipment 100 was detected and the detected site equipment 100. According to the above-described configuration, the equipment management system 70 uses the construction site condition recording system 1 to detect and manage the position of the site equipment 100 within the construction site G. The detection unit 43 of the construction site condition recording system 1 detects the site equipment 100 from the captured image P. The detection unit 43 records the correspondence between the captured image P in which the site equipment 100 is detected and the detected site equipment 100 in the database 42. Then, the construction site condition recording system 1 processes the image P to estimate the absolute position PA of the captured image P. Here, when the site equipment 100 is detected from the captured image P, the site equipment 100 should be located near the absolute position PA estimated by the construction site condition recording system 1 to be the capture position of the image P, at least within the range captured by the image recording device 3. Therefore, the absolute position PA inferred for the captured image P as described above can be regarded as the position where the detected site equipment 100 is installed. In this way, the location of the site equipment 100 can be automatically identified and managed.

[0084] Furthermore, the construction progress monitoring system 80 as described above is a construction progress monitoring system 80 that includes a construction site condition recording system 1 and uses the construction site condition recording system 1 to manage the progress of wall construction and ceiling construction within the construction site G, and the detection unit 43 detects construction work including wall construction and ceiling construction work together with an identification tag M from a plurality of photographed images P, estimates the progress of the construction work, and records in the database 42 the correspondence between the photographed image P in which the construction work was detected and the progress of the detected construction work. According to the above-described configuration, the construction progress monitoring system 80 manages the progress of wall and ceiling construction within the construction site G using the construction site condition recording system 1. The detection unit 43 of the construction site condition recording system 1 detects construction work, including wall and ceiling construction work, from the captured image P. The detection unit 43 records the correspondence between the captured image P in which construction work is detected and the progress of the detected construction work in the database 42. Then, the construction site condition recording system 1 processes the image P to estimate the absolute position PA of the captured image P. Here, if construction work is detected from the captured image P, the construction work should be performed near the absolute position PA estimated by the construction site condition recording system 1 to be the capture position of the image P, at least within the range captured by the image recording device 3. Therefore, the absolute position PA inferred for the captured image P as described above can be regarded as the location where the detected construction work is being performed. In this way, the location of the construction work can be automatically identified, and the progress of the construction work can be managed.

[0085] The construction site condition recording system, equipment and materials management system, and construction progress assessment system of the present invention are not limited to the above-described embodiments explained with reference to the drawings, and various other modifications are possible within the technical scope. For example, the construction site condition recording system 1 described above may be configured to be able to identify the position of the site worker Q in real time using the captured image P taken by the photographing and recording device 3, thereby constructing a site worker management system that detects and manages the position of the site worker Q within the construction site G. In addition to this, it is possible to select and discard the configurations given in the above embodiments and to change them to other configurations as appropriate. [Explanation of symbols]

[0086] 1 Construction Site Status Recording System IR Relative Route 3. Recording equipment Lst line 41 Image acquisition unit M Identification tag 42 Database P Photographed Images 43 Detector PA Absolute Position 44 Absolute position calculation unit for photographing identification tag PR Relative position 45 Relative path calculation unit PAs Absolute position where the first captured image was taken 46 Overall absolute position calculation unit PAt Absolute position where the second captured image was taken 70 Equipment and Materials Management System PAu Absolute position where the third image was taken 80 Construction Progress Monitoring System Q Field Worker 100 Field equipment RQ Pixel area corresponding to field workers or moving objects FR Outer Frame S Scale Ratio G Construction site

Claims

1. A construction site condition recording system for recording conditions within a construction site, an identification tag placed within the construction site; an image capturing and recording device configured to be held or attached by a site worker or a mobile body configured to be movable within the construction site, and configured to continuously capture images of the construction site at predetermined time intervals to generate a plurality of captured images; a detection unit that detects the identification tag from among the plurality of captured images; an identification tag photographing absolute position calculation unit that calculates, for the photographed image in which the identification tag is detected, the absolute position within the construction site of the photographing position at which the photographed image was taken based on the identification tag photographed in the photographed image; a relative path calculation unit that calculates a relative position of the shooting position of each of the plurality of photographed images relative to the shooting positions of other photographed images photographed immediately before or after the plurality of photographed images, based on the content photographed in each of the plurality of photographed images, and calculates a relative path that is a movement path of the relative position; an overall absolute position calculation unit that moves the entire relative path so as to align the relative position of the photographing position of the photographed image from which the identification tag was detected in the relative path with the absolute position of the photographing position of the photographed image, and calculates an absolute position within the construction site for each of the relative positions in the relative path; A construction site condition recording system comprising:

2. The relative path calculation unit masking a predetermined pixel area of ​​each of the plurality of captured images, the pixel area including a portion of the field worker or the moving object holding or wearing the image capturing and recording device, to generate a plurality of masked images; Feature points are extracted from each of the plurality of mask images, and pixel positions in each of the mask images of the feature points commonly extracted in the mask images corresponding to each of the successively captured images are compared to calculate the relative position of the photographing position of each of the plurality of captured images with respect to the photographing position of the other captured image photographed immediately before or immediately after.

2. The construction site condition recording system according to claim 1.

3. The relative path calculation unit an object identification trained model that is trained as an object detection model to identify each object captured in an input image by surrounding it with a rectangular outer frame; A pixel region identification trained model trained as semantic segmentation to identify a pixel region corresponding to each object captured in an input image; Equipped with The relative path calculation unit inputting each of the plurality of captured images into the object identification trained model to identify the outer frame corresponding to the field worker or the moving object; extracting a portion inside the outer frame identified as corresponding to the field worker or the moving object from each of the plurality of captured images, and inputting the extracted portion into the pixel region identification trained model to identify the pixel region corresponding to the field worker or the moving object; masking the pixel regions identified as corresponding to the field workers or the moving objects in each of the plurality of captured images to generate a plurality of mask images; Feature points are extracted from each of the plurality of mask images, and pixel positions in each of the mask images of the feature points commonly extracted in the mask images corresponding to each of the successively captured images are compared to calculate the relative position of the photographing position of each of the plurality of captured images with respect to the photographing position of the other captured image photographed immediately before or immediately after.

2. The construction site condition recording system according to claim 1.

4. The overall absolute position calculation unit calculating a distance between the absolute positions of the photographing positions of the photographed images at which the identification tag is successively detected, and dividing the distance between the absolute positions by the distance between the relative positions of the photographing positions to calculate a scale ratio; multiplying each of the relative positions in the relative path by the scale ratio to calculate a scaled relative position; selecting a first captured image, a second captured image, and a third captured image from the captured images in which the identification tag has been detected; moving the entire relative path so that the scale-adjusted relative position of the shooting position of the first photographed image coincides with the absolute position of the shooting position of the first photographed image; rotating the entire relative path around the absolute position of the shooting position of the first photographed image so that the scale-adjusted relative position of the shooting position of the second photographed image coincides with the absolute position of the shooting position of the second photographed image; Rotating the entire relative path around a line connecting the absolute position of the shooting position of the first photographed image and the absolute position of the shooting position of the second photographed image so that the scale-adjusted relative position of the shooting position of the third photographed image coincides with the absolute position of the shooting position of the third photographed image. By doing so, the relative position of the photographing position of the photographed image in which the identification tag is detected in the relative path is matched to the absolute position of the photographing position of the photographed image.

2. The construction site condition recording system according to claim 1.

5. 5. A construction site condition recording system according to claim 1, wherein the construction site condition recording system detects and manages the positions of on-site equipment and materials within the construction site, The detection unit detects the site equipment and materials together with the identification tag from among the plurality of photographed images, and records a correspondence between the photographed image in which the site equipment and materials are detected and the detected site equipment and materials in a database. An equipment and materials management system.

6. A construction progress monitoring system comprising the construction site condition recording system according to any one of claims 1 to 4, and using the construction site condition recording system to manage the progress of wall construction and ceiling construction within the construction site, The detection unit detects the construction work including the wall construction work and the ceiling construction work together with the identification tag from the plurality of photographed images, estimates the progress status of the construction work, and records the correspondence between the photographed image in which the construction work was detected and the progress status of the detected construction work in a database. A construction progress monitoring system characterized by:

Images