Attachment status acquisition system

Abstract

The present invention appropriately and easily acquires information indicating the attachment status of a material at a construction site.　An attachment status acquisition system 1 is used for acquiring information indicating the attachment status of a material, said system comprising: a position information acquisition unit 11 that acquires position information indicating the three-dimensional position of a time series related to lifting performed by a crane 100; a time band identification unit 12 that, on the basis of the position information, identifies a removal time band in which the material lifted by the crane 100 is removed from the crane 100; an attachment position estimation unit 15 that, on the basis of the position information in the removal time band, estimates a position where the material lifted by the crane 100 is attached at the construction site; a correspondence information acquisition unit 16 that acquires correspondence information in which information for identifying the material and the position where the material is attached are associated with each other; and a material identification unit 17 that compares the estimated position with the position indicated by the correspondence information to identify the attached material.

Description

Installation status acquisition system

[0001] The present invention relates to an installation status acquisition system that acquires information indicating the installation status of materials at a construction site.

[0002] Patent Document 1 discloses that, when materials are transported by a crane, information is read from an IC (Integrated Circuit) tag attached to the materials.

[0003] Patent No. 6132148

[0004] In managing the construction of steel frame erection work at a construction site, it is important to manage the progress of the installation of materials such as steel frames (for example, exterior materials or PCa (Precast Concrete) members) in order to reduce the occurrence of rework and waiting time in the construction. As shown in Patent Document 1, it is conceivable to attach IC tags to the materials in advance, read information about the materials from the IC tags, and use this information to manage the progress of the installation of the materials.

[0005] However, attaching IC tags to materials must be done manually, for example. Such a method requiring manual labor is a burden for workers, and further labor-saving measures are desired.

[0006] One embodiment of the present invention has been made in consideration of the above, and aims to provide an installation status acquisition system that can appropriately and easily acquire information indicating the installation status of materials at a construction site.

[0007] In order to achieve the above-mentioned object, an installation status acquisition system according to one embodiment of the present invention is an installation status acquisition system that acquires information indicating the installation status of materials at a construction site, and comprises: a position information acquisition means that acquires position information indicating the three-dimensional position of the material lifted by the crane over time; a time zone identification means that identifies, for each lifted material, the removal time zone in which the material lifted by the crane was removed from the crane based on the position information acquired by the position information acquisition means; an installation position estimation means that estimates the position at the construction site where the material lifted by the crane will be installed based on the position information for the removal time zone identified by the time zone identification means among the position information acquired by the position information acquisition means; a correspondence information acquisition means that acquires correspondence information that is prepared in advance and that corresponds information that identifies the material and the position at which the material should be installed; and a material identification means that compares the position estimated by the installation position estimation means with the position indicated by the correspondence information acquired by the correspondence information acquisition means to identify the installed material.

[0008] In an installation status acquisition system according to one embodiment of the present invention, the location where the material lifted by the crane will be installed at the construction site is estimated based on the location information during the removal time period, and the estimated location is compared with the location indicated by the correspondence information to identify the installed material. Therefore, the installation status acquisition system according to one embodiment of the present invention can appropriately acquire information about the installed material without the need to attach an IC tag or the like to the material. In other words, the installation status acquisition system according to one embodiment of the present invention can appropriately and easily acquire information indicating the installation status of materials at the construction site.

[0009] The time period specification means may estimate, for each lifted material, a lifting time period during which the material was lifted by the crane and a lifting return time period during which the crane will move after the lifted material is removed from the crane, based on the location information acquired by the location information acquisition means, and specify a removal time period by comparing the altitude indicated by the location information during the estimated lifting time period with the altitude indicated by the location information during a time period between the estimated lifting time period and the lifting return time period. This configuration makes it possible to appropriately and reliably specify a removal time period. As a result, it is possible to appropriately and reliably obtain information indicating the installation status of materials at a construction site.

[0010] The installation position estimation means may identify the number of materials lifted by the crane in one lifting operation based on the position information for the removal time period identified by the time period identification means, among the position information acquired by the position information acquisition means, and estimate the installation position at the construction site for each of the materials. With this configuration, even when multiple materials are lifted in one lifting operation, information indicating the installation status of the materials at the construction site can be appropriately and easily acquired.

[0011] The installation position estimation means may cluster the position information for the removal time period identified by the time period identification means, among the position information acquired by the position information acquisition means, to identify the number of materials lifted by the crane in one lifting. With this configuration, the number of materials lifted in one lifting can be appropriately and reliably identified. As a result, even when multiple materials are lifted in one lifting, information indicating the installation status of the materials at the construction site can be appropriately and reliably acquired.

[0012] The time period specification means specifies, for each lifted material, at least a portion of a lifting time period during which the material was lifted by the crane based on the location information acquired by the location information acquisition means, and the attachment status acquisition system further includes an image acquisition means for acquiring, for each lifting time period, a plurality of images of the lifted material taken at a plurality of times included in at least a portion of the lifting time period specified by the time period specification means, and a type discrimination means for discriminating the type of material lifted for each lifting time period from the plurality of images acquired by the image acquisition means, and the material specification means may specify the attached material based also on the type of material discriminated by the type discrimination means. With this configuration, information indicating the attachment status of the material at the construction site can be appropriately and easily acquired according to the type of the lifted material.

[0013] The attachment position estimation means may estimate a plurality of positions at the construction site where a material lifted by the crane will be attached, and the material identification means may calculate, for each material related to the correspondence information, the distance between each of the plurality of positions estimated by the attachment position estimation means and the position indicated by the correspondence information, calculate a score for the material corresponding to the plurality of positions based on a weight according to the calculated distance, and identify the attached material based on the calculated score. With this configuration, the attached material can be more appropriately identified, and as a result, information indicating the attachment status of the materials at the construction site can be more appropriately obtained.

[0014] According to one embodiment of the present invention, information indicating the installation status of materials at a construction site can be appropriately and easily acquired.

[0015] 1 is a diagram showing the configuration of an attachment status acquisition system according to an embodiment of the present invention. FIG. 2 is an image of an example of a hook block of a crane according to an embodiment. FIG. 3 is a graph showing an example of position information. FIG. 4 is a graph showing an example of altitude indicated by position information, and an example of a class of work related to lifting by time. FIG. 5 is a diagram showing an example of a class of work related to lifting. FIG. 6 is a graph showing an example of altitude indicated by position information. FIG. 7 is a diagram showing an example of a multi-slung lift. FIG. 8 is a graph showing an example of coordinates indicating the three-dimensional position where materials included in the correspondence information should be attached. FIG. 9 is a diagram showing an example of information used to generate the correspondence information. FIG. 10 is a flowchart showing processing executed in an attachment status acquisition system according to an embodiment of the present invention. FIG. 11 is a graph showing an example of the three-dimensional distance between a plurality of estimated attachment positions and the position where each material should be attached. FIG. 12 is an example of a k-distance graph.

[0016] Hereinafter, an embodiment of the attachment status acquisition system according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals and duplicated explanations will be omitted.

[0017] 1 shows an installation status acquisition system 1 according to this embodiment. The installation status acquisition system 1 is a system (device) that acquires information indicating the installation status of materials (components) at a construction site. In this embodiment, the installation of materials at a construction site is performed by lifting the materials using a crane 100.

[0018] The construction site is, for example, a steel frame erection work site. However, the construction site may be any site where materials are lifted by crane 100 and then installed. At the construction site, the lifting of materials by crane 100 and the installation of the lifted materials may be performed in the same manner as in the past.

[0019] The crane 100 is, for example, a tower crane used at a construction site. For example, the crane 100 is provided with a boom 101, and lifting is performed using a hook 102 suspended from the tip of the boom 101. The hook 102 is attached to and supported by a hook block (weight block) 103. The hook block 103 has a sling (wire) 104 attached thereto, and is suspended from the boom 101 by the sling 104.

[0020] Figure 2 shows an image of a portion of the hook block 103. The hook block 103 is equipped with a pulley, and a sling 104 is wound around the pulley and supported by the sling 104. The slings 104 extend from both sides of the pulley of the hook block 103 toward the boom 101. The hook block 103 includes two plate-like members 103a that sandwich the pulley and support and protect it. The main surfaces of the two plate-like members 103a face each other with a gap between them.

[0021] Note that the crane 100 may have a configuration other than that described above. The crane 100 may also be a crane other than a tower crane. For example, the crane 100 may be a stationary crane other than a tower crane, or a mobile crane such as a crawler crane or a crawler crane. The crane 100 may be the same as a conventional crane.

[0022] The materials to be installed at the construction site are, for example, materials (components) that constitute the structure to be built. Furthermore, the materials are not limited to those described above, and may be any materials that are installed after being lifted by the crane 100 at the construction site. Examples of the types of materials include decks, steel columns, stairs, reinforcing bars, temporary materials, steel girders, steel beams, steel-related materials, and exterior materials (e.g., curtain walls and extruded cement boards). Furthermore, the types of materials may be steel frame components other than those described above that are used in steel frame erection work at the construction site. Furthermore, the types of materials may be PCa or the like, as long as they are directly installed by the crane 100. Furthermore, the materials may be deck plates.

[0023] An installation position corresponding to each material is preset for each material. Information indicating the installation status of materials acquired by the installation status acquisition system 1 is information indicating materials that have been installed at the preset position. Information indicating the installation status of materials is used for construction management, such as progress management at the construction site.

[0024] 1, the installation status acquisition system 1 includes a server device 10, a positioning meter 20, and a camera 30. The server device 10 is a device that acquires data from the positioning meter 20 and the camera 30, and generates and acquires information indicating the installation status of materials from the acquired data. Specific functions of the server device 10 will be described later.

[0025] Specifically, the server device 10 is a computer including hardware such as a CPU (Central Processing Unit) and a memory. The functions of the server device 10, which will be described later, are realized by these components operating through programs or the like. The server device 10 may be realized by a single computer, or may be realized by a computer system in which multiple computers are connected to each other via a network. The server device 10 may also be realized by a cloud system.

[0026] The server device 10, the positioning meter 20, and the camera 30 each have a communication function, and are capable of transmitting data from the positioning meter 20 and the camera 30 to the server device 10. For example, the positioning meter 20 and the camera 30 may have a wireless communication function and transmit data to the server device 10 via wireless communication. The positioning meter 20 and the camera 30 may also be powered by a battery. Using a positioning meter 20 and a camera 30 that have a wireless communication function and are battery-powered makes it easy to retrofit the positioning meter 20 and the camera 30.

[0027] The positioning meter 20 is a detection means for detecting a time-series three-dimensional position related to the lifting of the crane 100. For example, the positioning meter 20 detects the three-dimensional position of the hook 102 of the crane 100. The three-dimensional position detected by the positioning meter 20 is, for example, an altitude (m), latitude, and longitude. The positioning meter 20 detects the time-series three-dimensional position and transmits position information indicating the detected time-series three-dimensional position to the server device 10. The position information transmitted to the server device 10 is configured to allow the timing at which the three-dimensional position was detected to be known. The positioning meter 20 detects the three-dimensional position at a time interval (for example, every second) appropriate for obtaining information indicating the attachment status of the material.

[0028] The positioning meter 20 is provided, for example, on the hook 102 of the crane 100 (in the vicinity of the hook 102). Specifically, as shown in Figures 1 and 2, the positioning meter 20 is fixed to the top of the hook block 103 (above the upper end of the hook block 103) and at a position sandwiched between two slings 104 that support the hook block 103. At this position, there is a relatively large space protected by the hook block 103 and the slings 104. Therefore, there is little risk that the positioning meter 20 placed at this position will come into contact with materials or buildings and be damaged.

[0029] The positioning meter 20 is housed in a housing (box, main body) together with a battery for powering the positioning meter 20. To be precise, FIG. 2 shows the housing that houses the positioning meter 20. The housing may be one that matches the dimensions of the hook 102 (hook block 103). By using a housing with an upper cover that opens upward, the battery can be replaced while the housing is attached to the hook block 103. By installing the housing on top of the hook block 103, there is no risk of a large load being applied to the housing, causing it to break and causing the material to fall, compared to a housing that is attached between the hook 102 and the material to be lifted and relays the two.

[0030] The positioning device 20 may be the same as a conventional positioning device. For example, the positioning device 20 may be a device that acquires its own three-dimensional position information with high accuracy by RTK (real-time kinematic) positioning.

[0031] The three-dimensional position detected by the positioning meter 20 may be a three-dimensional position related to the lifting of the crane 100, and may be a three-dimensional position other than the three-dimensional position of the hook 102. Furthermore, the arrangement of the positioning meter 20 may be other than that described above, as long as the three-dimensional position related to the lifting of the crane 100 can be detected.

[0032] The camera 30 is an imaging device that captures images of materials lifted by the crane 100. For example, the camera 30 is positioned near the hook 102 of the crane 100 so that the materials being lifted are captured in the captured image. The camera 30 may also be positioned so that the hook 102 is captured in the captured image in addition to the materials being lifted. Specifically, as shown in FIGS. 1 and 2 , the camera 30 is fixed between two plate-like members 103a of the hook block 103. By placing the camera 30 in this position protected by the hook block 103, damage to the camera 30 due to contact with the outside can be avoided. Furthermore, since this position is close to the lifted material suspended by the hook 102, higher-quality images of the materials can be captured. The attachment status acquisition system 1 may also include multiple cameras 30. For example, as shown in FIG. 1 , two cameras 30 may be positioned symmetrically with respect to the main surface of the plate-like member 103a when viewed from the direction of the main surface. By providing a plurality of cameras 30, even if the material is difficult to see in the image from one camera 30, the image from another camera 30 can be used to make up for it.

[0033] The camera 30 may be powered by a battery stored in the housing that houses the positioning meter 20. By powering the positioning meter 20 and the camera 30 from a battery stored in the housing, the positioning meter 20 and the camera 30 can continue to operate for several days without battery replacement. Alternatively, power may be supplied to the camera 30 by other methods. The camera 30 may be placed in a location other than the above so that the material being lifted is captured in the captured image. For example, the camera 30 may be attached to a part that is not protected by the hook block 103 or the like as described above, such as the outside of the hook block 103. In this case, the camera 30 needs to be protected by, for example, placing it in a sturdy housing to prevent it from being broken and flying away. The camera 30 may be made small so as not to come into contact with other objects, and a safety wire may be attached to prevent the camera 30 from falling.

[0034] The camera 30 takes images in time series and transmits the images obtained by the image capture to the server device 10. The images transmitted to the server device 10 are configured so that the timing at which the images were captured can be determined. The camera 30 captures images at time intervals appropriate for obtaining information indicating the installation status of the material. For example, the camera 30 captures images at the same timing as the detection of the three-dimensional position by the positioning meter 20. Therefore, the camera 30 may capture images even when the crane 100 is not lifting materials. The camera 30 may also capture video. The camera 30 may be the same as a conventional camera.

[0035] Next, a description will be given of the functions of the server device 10. As shown in Fig. 1, the server device 10 is configured to include a location information acquisition unit 11, a time period identification unit 12, an image acquisition unit 13, a type determination unit 14, an attachment position estimation unit 15, a corresponding information acquisition unit 16, and a material identification unit 17.

[0036] The position information acquisition unit 11 is a position information acquisition means for acquiring position information indicating the time-series three-dimensional position of the lifting of the crane 100. The position information acquisition unit 11 receives and acquires position information transmitted from the positioning meter 20. The position information acquisition unit 11 may convert the latitude and longitude in the position information transmitted from the positioning meter 20 into a predetermined coordinate system (for example, a planar rectangular coordinate system to which the construction site belongs, in which all coordinate axes are in meters (m)). In this case, the converted position information is used in the following processing. Figure 3 illustrates an example of position information. Each point in Figure 3 is an individual three-dimensional position indicated by the position information. The axes (X-axis, Y-axis, and Z-axis) of the graph in Figure 3 are coordinate axes in three-dimensional space. The position information acquisition unit 11 outputs the acquired position information to the time zone identification unit 12, the image acquisition unit 13, and the installation position estimation unit 15.

[0037] The time period identification unit 12 is a time period identification means that identifies, for each lifted material, a removal time period during which the material lifted by the crane was removed from the crane, based on the location information acquired by the location information acquisition unit 11. The time period identification unit 12 may estimate, for each lifted material, a lifting time period during which the material was lifted by the crane 100 and a lifting return time period during which the crane 100 moves after the lifted material is removed from the crane 100, based on the location information acquired by the location information acquisition unit 11, and identify the removal time period by comparing the altitude indicated by the location information during the estimated lifting time period with the altitude indicated by the location information during a time period between the estimated lifting time period and the lifting return time period. The time period identification unit 12 may identify, for each lifted material, at least a portion of the lifting time period during which the material was lifted by the crane 100, based on the location information acquired by the location information acquisition unit 11.

[0038] For example, the time period identification unit 12 identifies the above-mentioned time period as follows: The time period identification unit 12 inputs location information from the location information acquisition unit 11. The time period identification unit 12 classifies lifting-related work at each time into one of five classes based on the location information. For classification, altitude (position in the vertical direction) of the three-dimensional position in time series indicated in the location information is used. Figure 4 shows a graph of an example of the altitude in time series indicated in the location information. In the graph of Figure 4, the horizontal axis is the time axis and the vertical axis is the altitude axis. Figure 5 shows some of the classified classes. The five classified classes are "rigging / ground lifting," "lifting," "attaching / removing," and "lifting return" shown in Figure 5, as well as "standby," which is not shown in Figure 5.

[0039] "Slinging and lifting" refers to the work of attaching the material to be lifted to the hook 102 and lifting it (hoisting the sling 104 to lift the material away from the ground and temporarily stopping it). Normally, slinging also includes the work of removing the material from the hook 102, but in this embodiment, it refers only to the work of attaching the material. "Lifting" refers to the work of transporting the material in a vertical direction by lifting the sling 104 while the material is attached to the hook 102, to the attachment position, i.e., the work of lifting the material. In this embodiment, "lifting" refers to the work of transporting the material upward in a vertical direction by lifting the crane 100. The "lifting" time period is the lifting time period during which the material is lifted by the crane 100.

[0040] "Attaching and unattaching" is the work of attaching materials that have been lifted to the attachment position and removing them from the hook 102. Note that the attachment in "attaching and unattaching" refers to, for example, attaching materials to a building or the like that is being constructed, or placing materials at a location where they will be used. The "attaching and unattaching" time period is the time period during which materials lifted by the crane 100 are removed from the crane 100.

[0041] Furthermore, the attachment position in "lifting" and "attaching / removing" is the position where the attachment is performed. "Lifting return" is the work of moving the hook 102 in the vertical direction to retrieve the next material to be lifted when no material is attached to the hook 102. In this embodiment, "lifting return" is the work of moving the hook 102 downward in the vertical direction. The "lifting return" time period is the lifting return time period during which the crane moves after the lifted material has been removed from the crane. "Waiting" is a state in which work is stopped.

[0042] As shown in Figure 5, when lifting one material, the work usually takes place in the following order: "slinging and lifting off the ground," "lifting," "attaching and removing the sling," and "lifting return." Therefore, the work in the order of "slinging and lifting off the ground," "lifting," "attaching and removing the sling," and "lifting return" is considered to be the work related to lifting one material (i.e., one material lifting unit). Note that materials do not necessarily have to be lifted one at a time; multiple materials may be lifted together. Figure 4 shows the time period T for the work unit related to lifting the above-mentioned material, the time period T1 for "slinging and lifting off the ground," the time period T2 for "lifting," the time period T3 for "attaching and removing the sling," and the time period T4 for "lifting return."

[0043] In this embodiment, the M-shaped change in altitude over time, as shown in the graph in Figure 4, is considered to represent the above-mentioned series of operations related to the lifting of materials. The M-shaped change in altitude over time is as follows: During the "lifting" time period T2, the altitude increases. During the "attaching and unattaching" time period T3, the altitude peaks at the start and end of the operation, with the altitude decreasing in between. During the "lifting return" time period T4, the altitude decreases. The reason why the altitude of the operations related to the lifting of materials exhibits an M-shaped change over time is that the hook 102 of the crane 100 is basically moved above the building, straddling the exterior wall of the building, and must be lowered to the installation altitude when attaching the material. Any other change in altitude does not correspond to the operations related to the lifting of materials. For example, an altitude change in which the crane rises to the highest point, stops at the highest point, and then descends to the original altitude corresponds to a temporary break, etc.

[0044] The time period identification unit 12 classifies each time into one of five classes based on pre-stored classification rules as follows: From the location information, the time period identification unit 12 extracts time periods in which the altitude increases by more than a predetermined altitude range (ascent altitude, for example, a value in meters) for a predetermined continuous time length (ascent altitude determination time, for example, a value in seconds), and sets these as candidates for the "lifting" time period. From the location information, the time period identification unit 12 extracts time periods in which the altitude decreases by more than a predetermined altitude range (descent altitude, for example, a value in meters) for a predetermined continuous time length (descent altitude determination time, for example, a value in seconds), and sets these as candidates for the "lifting return" time period. From these candidates, the time period identification unit 12 selects pairs of candidate time periods for "lifting" and "lifting return" that meet the conditions as the actual "lifting" and "lifting return."

[0045] The time period identification unit 12 searches for a candidate time period for "lifting" that satisfies the conditions for actual "lifting return" from the candidate time period for "lifting return" that follows the candidate time period, as follows: First, the time period identification unit 12 determines whether the candidate time period for "lifting return" satisfies the conditions, starting from the earliest time period after the candidate time period for "lifting". Therefore, the time period identification unit 12 determines the lowest point altitude (hereinafter referred to as "lowest point altitude") of the time period between the candidate time period for "lifting" to be determined and the candidate time period for "lifting return".

[0046] Next, the time zone determination unit 12 determines the candidate combination of time zones for "lifting" and "lifting return" to be the actual combination of time zones for "lifting" and "lifting return" if the candidate combination of time zones for "lifting" and "lifting return" to be judged satisfies the following four conditions (a) to (d).

[0047] (a) The length of time between the end time of the candidate time slot for "lifting" and the start time of the candidate time slot for "lifting return" is equal to or greater than a preset threshold (allowable time, e.g., a value in seconds). This is to exclude immediate continuous movements from ascent (candidate time slot for "lifting" T2c) to descent (candidate time slot for "lifting return" T4c), as shown in Figure 6(a). By setting the threshold to a time length during which it is practically difficult to perform "attaching and unattaching," e.g., 30 seconds, it is possible to exclude such times when nothing is being lifted.

[0048] (b) The difference in altitude between the lowest point of the candidate "lifting" time period and the lowest point of the candidate "lifting return" time period is less than a predetermined specified altitude width (threshold) (the allowable altitude difference between the lowest point of lifting and the lowest point of lifting return, for example, a value in meters). This is to detect lifting units such as those shown in Figure 6(b). Such lifting units are seen during multiple lifting, and two "lifting return" time periods can occur by lowering the altitude after installing the first beam (material) and then installing the second and subsequent beams in succession. In this case, the descent immediately after installing the first beam can be excluded, and the period from installing the second beam until the complete return to the original altitude can be extracted as one lifting unit. The specified altitude width (threshold) can be set to, for example, 2 meters.

[0049] (c) The descent height range from the highest point of the altitude in the candidate "lifting" time slot to the "lowest point altitude" is equal to or greater than a predetermined specified altitude range (threshold) (the descent height from the highest lifting point to the lowest point for removing the rope, for example, a value in meters). This descent height range corresponds to the depth of the central depression in the M-shaped configuration. This makes it possible to confirm that the worker is performing installation work and not taking a break. The specified altitude range (threshold) can be set to, for example, 1 meter.

[0050] (d) The ascending altitude range from the lowest altitude point in the candidate "lifting" time period to the "lowest point altitude" is equal to or greater than a predetermined altitude range (threshold) (the descending altitude from the lowest point of lifting to the lowest point of lifting, e.g., a value in meters). The ascending altitude range is an indicator of the altitude of the central depression in the M-shaped configuration above the ground. This prevents false detection of "lifting" and "lifting return" for movements that repeatedly ascend and descend to the lifting start height, such as movements from candidate "lifting" time period T2c to candidate "lifting return" time period T4c in Figure 6(c) (such as removing or transferring a jig without installing materials). The predetermined altitude range (threshold) can be set to, for example, 5 m.

[0051] If the combination of candidate time periods for "lifting" and "returning" that is the subject of judgment does not satisfy any one of the four conditions (a) to (d) above, the time period identification unit 12 judges whether the combination of the candidate time period for "lifting" and the next candidate time period for "returning" satisfies the above conditions. The time period identification unit 12 repeats the above judgment for the candidate time period for "lifting" that is the subject of judgment until a candidate time period for "returning" that satisfies the four conditions (a) to (d) is found. If the candidate time period for "lifting" that is the subject of judgment cannot be found to satisfy the four conditions (a) to (d), the time period identification unit 12 does not use the candidate time period for "lifting" as the time period for the actual "lifting" and "returning."

[0052] The time period identification unit 12 performs the above determination for all candidate time periods for "lifting." When the time period identification unit 12 determines that a combination of candidate time periods for "lifting" and "lifting return" is a combination of actual time periods for "lifting" and "lifting return," it determines the time period between these combinations as a time period for "attaching and removing." This makes it possible to capture the M-shaped shape seen in typical height changes during lifting.

[0053] As described above, the time period determination unit 12 estimates candidate time periods for "lifting" and "lifting return," compares the altitude of the candidate time period for "lifting" with the altitude of the time period between the candidate time periods for "lifting" and "lifting return," and determines the time periods for "lifting" and "lifting return."

[0054] If there is a time period between the actual "lifting" time period and the previous actual "lifting return" time period, the time period specification unit 12 determines that time period as a "slinging / lifting to the ground" time period. Due to the nature of the classification rule, a "slinging / lifting to the ground" time period does not necessarily occur for each lifting unit.

[0055] The time zone identification unit 12 calculates the speed (moving average) of the hook 102 of the crane 100 at each time point from the time-series position information using the following procedures (e) and (f): (e) Determine the time change in the position information to calculate the speed at each time point. Note that if the position information is not suitable for calculating the speed, the position information may be converted to one suitable for calculating the speed. For example, the latitude and longitude included in the position information are converted to a plane rectangular coordinate system to which the construction site belongs, and the unit is set to meters. (f) Calculate the moving average of the speed to remove noise caused by temporary fluctuations such as wind.

[0056] The time period identification unit 12 compares the calculated speed with a preset specified value, and identifies a time period in which the calculated speed is equal to or less than the specified value as "standby."

[0057] The time period identification unit 12 may identify the above time period using a method other than the above (for example, using a classification rule other than the above) as long as the method is based on the location information input from the location information acquisition unit 11.

[0058] The time period specification unit 12 outputs information indicating the specified time period for each class to the image acquisition unit 13, the type determination unit 14, and the attachment position estimation unit 15. Note that the output of the information indicating the time period for each class to the image acquisition unit 13, the type determination unit 14, and the attachment position estimation unit 15 may be the output of only information indicating the time period for the class used in the functional unit to which the information is output.

[0059] The image acquisition unit 13 is an image acquisition means that acquires, for each lifting time period, multiple images of the lifted materials taken at multiple times that are included in at least a portion of the lifting time period identified by the time period identification unit 12.

[0060] The image acquisition unit 13 may acquire a plurality of images of the lifted material taken at a plurality of timings included in a time period during which the altitude indicated by the altitude information is equal to or higher than a predetermined altitude during at least a part of the lifting time period. The image acquisition unit 13 may acquire a plurality of images of the lifted material taken at a predetermined number of timings at equal intervals. The image acquisition unit 13 may acquire images obtained by imaging using the camera 30.

[0061] The images acquired by the image acquisition unit 13 are intended to determine the type of material lifted during the time period identified by the time period identification unit 12 as the lifting time period. For example, the image acquisition unit 13 acquires images for this purpose as follows. The image acquisition unit 13 inputs information indicating at least a part of the lifting time period, for example, the "lifting" time period as described above, from the time period identification unit 12. The image acquisition unit 13 inputs location information from the location information acquisition unit 11. The image acquisition unit 13 inputs and acquires images from the camera 30.

[0062] For each lifting time period, the image acquisition unit 13 selects, from among the images input from the camera 30, multiple images taken at multiple times within the "lifting" time period, which is at least a part of the lifting time period identified by the time period identification unit 12, during which the altitude indicated by the location information input from the location information acquisition unit 11 is equal to or higher than a predetermined altitude, as images to be used to determine the type of material.

[0063] For example, five images taken at equal intervals during a "lifting" time period when the altitude is equal to or higher than a preset altitude are used to determine the type of material. The first and last of the above images may or may not coincide with the start and end times of the time period.

[0064] The reason for using images taken during a time period when the altitude is above a predetermined altitude as described above is that images taken at a lower altitude may result in the lifted material being mistaken for other components on the ground.By using images taken during the time period when the lifted material has reached a certain altitude and the background has become distant, the type of lifted material can be properly identified.

[0065] The timings of the images used to determine the type of material do not necessarily have to be as described above, and may be multiple timings that are included in at least a part of each lifting time period identified by the time period identification unit 12. If multiple cameras 30 are provided, multiple images may be included in one timing. The image acquisition unit 13 outputs the multiple images used to determine the type of material for each lifting time period to the type identification unit 14.

[0066] The type discrimination unit 14 is a type discrimination means that discriminates the type of material lifted for each lifting time period from the multiple images acquired by the image acquisition unit 13.

[0067] The type discrimination unit 14 may primarily discriminate the type of the lifted material from each of the multiple images, and then discriminate the type of the lifted material based on multiple discrimination results from the primary discrimination. The type discrimination unit 14 may calculate a score for each type of material for the lifted material from each of the multiple images acquired by the image acquisition unit 13, calculate a discrimination score for each type of material from the multiple scores calculated for each type of material using a predetermined calculation formula, and discriminate the type of the lifted material based on the calculated discrimination score. The type discrimination unit 14 may detect a portion of the multiple images acquired by the image acquisition unit 13 that includes the hook 102, cut out a portion of the image corresponding to the detected portion, and discriminate the type of the lifted material from the cut-out image. For example, the type discrimination unit 14 may discriminate the type of the lifted material for each lifting time period as follows.

[0068] The types of materials identified by the type identification unit 14 are the types described above. Note that the types of materials are not limited to the above and may be arbitrarily set.

[0069] The type discrimination unit 14 inputs information indicating the lifting time period from the time period specification unit 12. The type discrimination unit 14 inputs a plurality of images for each lifting time period from the image acquisition unit 13. The images input from the image acquisition unit 13 are images taken at the time when the crane 100 is lifting materials, and therefore include the hook 102 and the lifted materials.

[0070] The type determination unit 14 cuts out a portion of the image input from the image acquisition unit 13 that shows the material to be lifted as a partial image. This cutting may be performed, for example, by detecting a portion that shows the hook 102 and based on the detected portion. This detection may be performed using conventional image analysis technology. Alternatively, a predetermined portion (position in the image) that shows the material to be lifted may be cut out based on the installation position of the camera 30, etc. Cutting may also be performed using methods other than those described above. The partial image is an image of a predetermined size and shape (e.g., a rectangle such as a square). The position of the partial image in the image is, for example, a position where the portion showing the hook 102 is centered on the partial image. The type determination unit 14 determines the type of material lifted during the lifting time period using the cut-out partial image and a learning model (image recognition model) used to determine the type of material.

[0071] By using a partial image as described above, it is possible to exclude from the image anything other than the material being lifted, enabling more appropriate identification. It is not necessary to crop the partial image, and the type identification unit 14 may make the identification by directly using the image input from the image acquisition unit 13. For example, depending on the installation position of the camera 30 and the size of the hook 102, the image acquired by the image acquisition unit 13 may be a close-up image of the material being lifted, so there is no need to crop the partial image.

[0072] The type discrimination unit 14 primarily discriminates the type of the lifted material from each of the multiple images. For example, the type discrimination unit 14 performs the temporary discrimination using a learning model previously generated by machine learning. The learning model inputs an image and outputs information related to the type of material depicted in the image. For example, the learning model may be configured to include a neural network. The neural network may be multi-layered, i.e., generated by deep learning. The neural network may also be a CNN (convolutional neural network). CNN is strong in image data and is the most commonly used method in the field of image recognition, and using a CNN enables more appropriate discrimination.

[0073] For example, the learning model includes an input layer, an intermediate layer, and an output layer. The input layer has neurons equal to the number of pixels in the image, and each neuron receives a pixel value of the image. The output layer has neurons equal to the number of material types to be identified, and each neuron outputs a confidence score for each type. The confidence score indicates the degree to which the material depicted in the image is of that type. For example, the confidence score is a numerical value between 0 and 1, and the higher the confidence score, the greater the degree to which the material depicted in the image is of that type. The above learning model can be generated by conventional machine learning using learning images in which the type of material depicted is known in advance.

[0074] The type discrimination unit 14 stores a learning model in advance. The type discrimination unit 14 inputs an image used to discriminate the type of material into the learning model and primarily discriminates the type of the lifted material based on the image. The type discrimination unit 14 inputs the image into the learning model and obtains, as output from the learning model, a certainty factor for each type of material for the image. The type discrimination unit 14 determines the type with the highest certainty factor among the certainty factors for each type output from the learning model as the type of material discriminated from the image.

[0075] The type discrimination unit 14 performs the above primary discrimination on all images taken during the same lifting time period (for example, images taken during the above five times). The type discrimination unit 14 determines the type that is most frequently identified among the images as the type of material lifted during that lifting time period. In other words, the type discrimination unit 14 finally determines the type of material by majority vote. For example, if the primary type discrimination of each image results in the most frequent discrimination of steel beams, the final decision is made that the material is a steel beam. If multiple cameras 30 are installed and multiple images are included at one time period, it is sufficient to perform a temporary type discrimination from each image. In other words, it is sufficient to perform the same number of temporary type discriminations as there are images.

[0076] By identifying the type of material being lifted from multiple images, it is possible to properly identify the type of material even when it is not possible to properly identify the type from a single image. In other words, it is possible to reduce the influence of the appearance of the primary image and improve image recognition accuracy.

[0077] Furthermore, the type discrimination unit 14 may discriminate the type of material without performing primary discrimination. For example, for one lifting time period, the type discrimination unit 14 may calculate a discrimination score for each type of material from the certainty factor for each type of lifted material using a preset calculation formula, and discriminate the type of the lifted material based on the calculated discrimination score. Specifically, the average of the calculated certainty factors for each type may be used as the discrimination score. For example, when only one image is used, the material may be discriminated as an incorrect type, "other," but by taking the average of the certainty factors, it can be discriminated as the correct type, "steel beam."

[0078] Alternatively, the classification score may be determined by adding up all of the highest confidence scores for each image for each type. The type classification unit 14 determines the type with the highest classification score as the type of material lifted during the lifting time period.

[0079] The type discrimination unit 14 may also discriminate the type of material lifted during a lifting time period using a learning model that inputs all of the images related to a single lifting time period and outputs a single piece of information related to the type of material in response to the input of the multiple images. The learning model may be, for example, a convolutional long short term memory (LSTM). In this case, the output from the learning model is, for example, a confidence score, which is a score for each type, similar to the learning model described above. The type discrimination unit 14 determines the type with the highest confidence score as the type of material lifted during the lifting time period. This type of discrimination enables discrimination based on the overall characteristics of the multiple images (discrimination that takes into account characteristics commonly included in the multiple images or the movement (such as rotation in the air) seen in a specific material).

[0080] The type discrimination unit 14 outputs information indicating the type of material lifted during each lifting time period (the ``lifting'' time period), i.e., for each time period of the work unit related to the lifting of materials, to the attachment position estimation unit 15 and the material identification unit 17.

[0081] The attachment position estimation unit 15 is an attachment position estimation means that estimates positions at the construction site where materials lifted by the crane 100 will be attached, based on position information for the removal time period specified by the time period specification unit 12 out of the position information acquired by the position information acquisition unit 11. The attachment position estimation unit 15 may specify the number of materials lifted by the crane 100 in one lifting operation, based on position information for the removal time period specified by the time period specification unit 12 out of the position information acquired by the position information acquisition unit 11, and may estimate the position at the construction site where each of the materials will be attached. The attachment position estimation unit 15 may cluster the position information for the removal time period specified by the time period specification unit 12 out of the position information acquired by the position information acquisition unit 11, to specify the number of materials lifted by the crane 100 in one lifting operation.

[0082] For example, the attachment position estimation unit 15 estimates the position where the material is attached as follows: The attachment position estimation unit 15 stores rules for estimating the attachment position of the material in advance, and estimates the attachment position of the material in accordance with the rules. The attachment position estimation unit 15 inputs position information from the position information acquisition unit 11. The attachment position estimation unit 15 inputs information indicating the time period for each class from the time period identification unit 12. The attachment position estimation unit 15 inputs information indicating the type of material lifted for each "lifting" time period from the type determination unit 14.

[0083] The attachment position estimation unit 15 estimates the position where the material will be attached during each "attaching / removing" time period indicated by the information input from the time period specification unit 12, from the position information input from the position information acquisition unit 11. At this time, the type of the lifted material indicated by the information input from the type determination unit 14 may be used, as follows:

[0084] The attachment position estimation unit 15 extracts position information for each time period of "attaching and unattaching" from the position information input from the position information acquisition unit 11. The attachment position estimation unit 15 estimates the position where the material will be attached for each time period based on the extracted position information for each time period of "attaching and unattaching". The attachment position estimation unit 15 clusters the multiple positions included in the extracted position information. For example, the clustering is performed by DBSCAN (Density-Based Spatial Clustering of Applications with Noise).

[0085] DBSCAN is a method of clustering data according to its density. In the clustering results of DBSCAN, points (positions) whose density is equal to or greater than a threshold are grouped in the same cluster, and any other points are considered noise. The clusters obtained by DBSCAN clustering correspond to positions where the hook 102 of the crane 100 is temporarily stopped. This is because the hook 102 always temporarily stops when attaching materials. In DBSCAN, points with high density are grouped as clusters, and as a result, clusters are formed at three-dimensional positions where the crane is thought to have stopped for attachment.

[0086] For materials of the same type, multiple materials may be lifted simultaneously by the crane 100. This is called "multiple lifting" or "lantern lifting," for example. The number of clusters generated by the clustering described above may exceed the number of materials being lifted simultaneously. Therefore, the attachment position estimation unit 15 reduces the number of clusters to be equal to or less than the upper limit of the number of materials that can be lifted simultaneously. The upper limit of the number of clusters is set in advance, for example, to three. Note that multiple lifting often consists of at most three pieces (three materials). Although rare, there are also cases where the number is four. The upper limit of the number of clusters may be set according to the construction site. The upper limit of the number of clusters may also be set according to the type of material. The type of material is the class type determined by the type determination unit 14 for the "lifting" time period immediately preceding the "attaching and unattaching" time period used to estimate the attachment position of the material.

[0087] The attachment position estimation unit 15 determines the position (coordinates) of the center of gravity of the positions (coordinates) of each cluster as the representative point of the cluster. In order to reduce the number of clusters, the attachment position estimation unit 15 determines that two representative points whose distance from each other is less than a predetermined value (for example, 1 m) represent the attachment of the same material, and deletes one of them. This is because the set of point clouds that form a cluster does not completely overlap at one point, so it is necessary to determine a single point as the representative position. More specifically, for two representative points whose coordinates are (x1, y1, z1) and (x2, y2, z2), respectively, the attachment position estimation unit 15 calculates the three-dimensional distance between the two points (d = √((x2 - x1) ２ +(y2-y1) ２ +(z2-z1) ２ )) is calculated, and one of the points is deleted for representative points where the distance is less than 1 m, for example.

[0088] The representative point to be deleted is the one with the higher altitude of the two representative points. The process of deleting representative points is performed in order of ascending altitude from the combination of two representative points including the representative point with the lowest altitude. This is because there is a case where the robot temporarily waits directly above the attachment point just before attaching the material, and the representative point with the lower altitude is considered to correspond to the attachment position of the material.

[0089] If the number of representative points after the deletion (P1, P2 shown in FIG. 3 ) is equal to or less than the upper limit of the number of clusters, the attachment position estimation unit 15 determines the representative points after the deletion as positions corresponding to the attachment positions of the materials. If the number of representative points after the deletion exceeds the upper limit of the number of clusters, the attachment position estimation unit 15 extracts positions corresponding to the attachment positions of the materials from the representative points after the deletion. For example, the attachment position estimation unit 15 determines the representative points (P1 shown in FIG. 3 ) corresponding to the upper limit of the number of clusters as positions corresponding to the attachment positions of the materials in order of the distance (distance D1 in FIG. 3 ) from the end time of the "attachment / unhooking" time period (the time corresponding to the right end of time period T3 in FIG. 4 ) (the start point of the "lifting return" and the position directly above the lifting yard). This is because the hook 102 of the crane 100 basically travels from the lifting yard on the ground to the attachment point and back, and therefore the farther the distance from the lifting yard, the more likely it is that the position corresponds to the attachment position of the materials. This prevents a situation where someone is temporarily waiting for the next lifting near the lifting yard from being mistakenly detected as an attachment.

[0090] For example, for a pillar or girder, one representative point corresponds to the attachment position of the material. Therefore, the upper limit of the number of clusters for a pillar or girder is set to 1. Unlike pillars and girders, two or three sub-girders may be attached in one lift, i.e., they may be hung in series. Therefore, for a sub-girder, three representative points correspond to the attachment positions of the material. Therefore, the upper limit of the number of clusters for a sub-girder is set to 3.

[0091] Next, the attachment position estimation unit 15 determines a point that is lower in altitude by a preset wire length (distance D2 shown in FIG. 3) from the position corresponding to the attachment position of the material identified as above as the estimated attachment position of the material (P3 shown in FIG. 3). The wire length is the length of the wire (or rope) that connects the hook 102 of the crane 100 to the material when the material is lifted by the crane 100. The preset wire length may be determined according to the type of material.

[0092] When multiple positions correspond to the mounting positions of multiple materials in a single lift, i.e., when multiple materials are being hoisted, the mounting position estimation unit 15 estimates the mounting positions of each material according to a rule previously set for that case. The rule includes, for example, a correspondence between the order in which the materials are mounted and the length from a position (representative point) corresponding to the mounting position of the material to the mounting position. For example, when multiple beams are hoisted, beams that are attached to the same floor are often hoisted together. Therefore, according to the following rule, the mounting height of one material is used as a reference, and the heights of other materials are aligned to that reference.

[0093] For example, as shown in FIG. 7 , in the case of a double-barreled lifting, a shorter wire is added in addition to the long wire used in the single-barreled lifting, and in the case of a triple-barreled lifting, the second and third materials are hung above and below the same long wire. Therefore, in the case of a double-barreled lifting, the height is unified to that of the first attached material, and in the case of a triple-barreled lifting, the height is unified to that of the second attached material. The attachment position estimation unit 15 sets the attachment position of the first material in a double-barreled lifting to a point lower in altitude by the predetermined length (m) of the long wire from the position corresponding to the attachment position of the material (representative point), and stores the altitude of this point. The attachment position estimation unit 15 sets the altitude of the attachment position of the second material in a double-barreled lifting to the stored altitude. Similarly, in the case of a triple-barreled lifting, the attachment position estimation unit 15 sets the attachment position of the second material to a point lower in altitude by the predetermined length (m) of the long wire from the position corresponding to the attachment position of the material (representative point), and sets the altitude of the attachment positions of the first and third materials to the altitude of this point.

[0094] The attachment position estimation unit 15 may estimate the attachment position of the material by a method other than the above, as long as it is based on the position information for each time period of "attaching / removing" among the position information acquired by the position information acquisition unit 11. The attachment position estimation unit 15 outputs information indicating the estimated attachment position for each material to the material identification unit 17.

[0095] The correspondence information acquisition unit 16 is correspondence information acquisition means that acquires correspondence information that is prepared in advance and associates information that identifies a material with a position where the material is to be attached.

[0096] The correspondence information is, for example, information that associates, for each material, information that identifies the material (e.g., an ID that is preset for each material) with coordinates that indicate the three-dimensional position where the material should be attached at the construction site. The coordinate system in the correspondence information is associated with (e.g., the same coordinate system as) the coordinate system of the position information acquired by the position information acquisition unit 11. Figure 8 shows an example of coordinates that indicate the three-dimensional position where the material included in the correspondence information should be attached. Each point in Figure 8 indicates the coordinates that correspond to an individual material. The information that identifies the material may be associated with information that indicates the type of the material.

[0097] The correspondence information may be generated from BIM (Building Information Modeling) data as shown in FIG. 9A and prepared in advance. For example, for a material to be attached by the crane 100 from the BIM data, the center point of the upper part of the material when attached (e.g., point P4 shown in FIG. 9B) is set as the three-dimensional position where the material should be attached. More precisely, the center point is the geometric center of gravity of the member in the planar dimensions. Because columns and beams are simple steel members, it is assumed that the dimensional center of gravity coincides with the actual center of gravity. The height of the center point is the top surface of the material. This is because, when the crane 100 hoists the material, the hook 102 is hoisted so that it is directly above the center of gravity of the material. Therefore, for shaft members such as columns and beams, it is assumed that the hook 102 of the crane 100 is directly above the center point of the member.

[0098] At this time, the BIM coordinate system is converted into the coordinate system of the construction site (the coordinate system of the position information acquired by the position information acquisition unit 11, for example, a planar rectangular coordinate system). The BIM data also includes information that identifies the material and information that indicates the type of material for each material. As described above, the information that identifies the material and the information that indicates the type of material are associated with the three-dimensional position where the material should be attached, and this is used as correspondence information.

[0099] The correspondence information acquisition unit 16 acquires the correspondence information, for example, by accepting an input operation of the user for the correspondence information. Alternatively, the correspondence information acquisition unit 16 may acquire BIM data and generate and acquire the correspondence information from the BIM data as described above. That is, the correspondence information acquisition unit 16 may prepare the correspondence information in advance. Furthermore, the correspondence information may be acquired by a method other than the above. Furthermore, the correspondence information may not be information generated from BIM data. The correspondence information acquisition unit 16 outputs the acquired correspondence information to the material identification unit 17.

[0100] The material identification unit 17 is a material identification means that identifies the attached material by comparing the position estimated by the attachment position estimation unit 15 with the position indicated by the correspondence information acquired by the correspondence information acquisition unit 16. The material identification unit 17 may also identify the attached material based on the type of material determined by the type determination unit 14.

[0101] The material identification unit 17 identifies the attached material as follows: The material identification unit 17 inputs information indicating the estimated attachment position for each material from the attachment position estimation unit 15. The material identification unit 17 inputs the correspondence information from the correspondence information acquisition unit 16. For each estimated attachment position of a material indicated by the information input from the attachment position estimation unit 15, the material identification unit 17 compares the estimated attachment position with the position where the material should be attached, indicated in the correspondence information input from the correspondence information acquisition unit 16, and identifies the attached material based on the comparison.

[0102] The material identification unit 17 calculates the three-dimensional distance between the estimated installation position of the material and the position where each material indicated in the correspondence information should be installed. The calculation of the three-dimensional distance may be performed in the same manner as the calculation of the distance between the representative points described above. The material identification unit 17 identifies, among the materials indicated in the correspondence information, the material with the shortest calculated three-dimensional distance as the material installed at the construction site. In other words, among the positions indicated in the correspondence information, the material located closest to the estimated installation position of the material is identified as the material installed at the construction site.

[0103] When the type determination unit 14 has determined the type of material for which the estimated attachment position has been obtained by the attachment position estimation unit 15, the material identification unit 17 may identify the attached material using the type. In this case, the material identification unit 17 inputs information indicating the type of material from the type determination unit 14. When comparing the estimated attachment position of the material with the position where the material should be attached, which is indicated in the correspondence information, the material identification unit 17 may limit the positions to be compared in the correspondence information to only positions related to materials of the same type as the type of the material.

[0104] Furthermore, the material identification unit 17 may identify the attached material by comparing the estimated attachment position with the position indicated by the correspondence information using a method other than the above.

[0105] The material identification unit 17 outputs information indicating the identification result. For example, the material identification unit 17 outputs information identifying the material whose installation has been completed (for example, the ID of the material) as the information indicating the identification result. For example, the material identification unit 17 transmits the information indicating the identification result to a terminal that records the installation registration of the material. Alternatively, the material identification unit 17 displays the information on a display device provided in the server device 10. The user can understand the material whose installation has been completed by referring to the output information. Furthermore, the material identification unit 17 may update the BIM data based on the information indicating the identification result. These are the functions of the server device 10.

[0106] Next, the processing executed by the installation status acquisition system 1 according to this embodiment will be described using the flowchart in Fig. 10. In this processing, the positioning meter 20 detects the three-dimensional position of the material lifted by the crane 100 (S01). The camera 30 captures an image of the material lifted by the crane 100 (S02). The detection by the positioning meter 20 (S01) and the capture of the image by the camera 30 (S02) are performed continuously during the time period when information indicating the installation status of the material at the construction site is to be acquired.

[0107] In the server device 10, the location information acquisition unit 11 acquires location information indicating the three-dimensional time-series position of the lifting operation by the crane 100, which is the result of detection by the positioning device 20 (S03). Next, the time period identification unit 12 identifies the time period of each class related to the lifting operation by the crane 100 for each lifted material based on the location information (S04). The time periods include at least the "attaching / unhooking" time period, which is the time period during which the lifted material is removed from the crane 100. The time periods may also include the "lifting" time period, which is the time period during which the lifting operation was performed. Next, the image acquisition unit 13 acquires, for each lifted material, multiple images of the lifted material taken by the camera 30 at multiple times included in the "lifting" time period identified by the time period identification unit 12 (S05). Next, the type identification unit 14 identifies the type of each lifted material from the multiple images (S06).

[0108] Next, the attachment position estimation unit 15 estimates the position at the construction site where the material lifted by the crane 100 will be attached, based on the position information for the "attaching and unattaching" time period among the position information (S07). Furthermore, the correspondence information acquisition unit 16 acquires correspondence information, which is prepared in advance and associates information identifying the material with the position where the material is to be attached (S08). The correspondence information acquisition unit 16 acquires the correspondence information (S08) independently of the above-described processes of S01 to S07, and may be performed before the material identification unit 17 identifies the attached material (S09). Next, the material identification unit 17 compares the estimated attachment position of the material with the position indicated by the correspondence information to identify the attached material (S09). Next, the material identification unit 17 outputs information indicating the identification result of the attached material (S10). The above is the processing executed by the attachment status acquisition system 1 according to this embodiment.

[0109] In this embodiment, the location where the material lifted by the crane 100 will be attached at the construction site is estimated based on the location information during the "attaching and unattaching" time period, which is the removal time period, and the estimated location is compared with the location indicated by the correspondence information to identify the attached material. Therefore, according to this embodiment, information about the attached material can be appropriately obtained without the need to attach an IC tag or the like to the material. According to this embodiment, the recording of material attachment registration can be automated. In other words, according to this embodiment, information indicating the attachment status of materials at the construction site can be appropriately and easily obtained.

[0110] Furthermore, as in this embodiment, the time period determination unit 12 may estimate, based on the position information, a "lifting" time period (or a candidate for the lifting time period) during which the crane lifts the material, and a "lifting return" time period (or a candidate for the lifting return) during which the crane moves after the lifted material is removed from the crane, for each lifted material. The time period determination unit 12 may then compare the altitude indicated by the position information during the "lifting" time period (or a candidate for the lifting time period) with the altitude indicated by the position information during the time period between these time periods (or a candidate for the lifting time period), thereby determining the "attaching and unattaching" time period. This configuration allows the "attaching and unattaching" time period, which is the time period during which the material is removed, to be determined appropriately and reliably. As a result, information indicating the attachment status of materials at a construction site can be obtained appropriately and reliably. However, the "attaching and unattaching" time period does not need to be determined as described above; it can be determined by any method based on the position information.

[0111] Furthermore, as in this embodiment, the attachment position estimation unit 15 may identify the number of materials lifted by the crane 100 in one lifting operation based on the position information for the "attaching and unattaching" time period, and estimate the attachment position at the construction site for each of the materials. With this configuration, even when multiple materials are lifted in one lifting operation, it is possible to appropriately and easily obtain information indicating the attachment status of the materials at the construction site.

[0112] In this case, the attachment position estimation unit 15 may cluster the position information during the "attaching and unattaching" time period among the position information to identify the number of materials lifted by the crane 100 in one lifting. This configuration makes it possible to appropriately and reliably identify the number of materials lifted in one lifting. As a result, even when multiple materials are lifted in one lifting, it is possible to appropriately and reliably obtain information indicating the attachment status of materials at the construction site. However, it is not necessary to identify the number of materials lifted by the crane 100 in one lifting as described above. Furthermore, it is not necessary to identify the number of materials lifted by the crane 100 in one lifting.

[0113] Furthermore, as in this embodiment, the time period identification unit 12 identifies the "lifting" time period for each lifted material based on the location information, and the installation status acquisition system 1 further includes an image acquisition unit 13 that acquires multiple images of the lifted material taken at multiple times that include at least a portion of the "lifting" time period, and a type identification unit 14 that identifies the type of material lifted for each "lifting" time period from the multiple acquired images, and the material identification unit 17 may identify the installed material based on the identified type of material. This configuration makes it possible to appropriately and easily acquire information indicating the installation status of materials at a construction site based on the type of lifted material. However, the type of material does not necessarily need to be used to identify the installed material, and in that case, the installation status acquisition system 1 does not need to be equipped with a configuration for identifying the type of material.

[0114] In this embodiment, the installation status acquisition system 1 includes the server device 10, the positioning meter 20, and the camera 30, but the installation status acquisition system 1 may be configured only with the server device 10. In that case, the installation status acquisition system 1 only needs to acquire the above-mentioned necessary data from the positioning meter and the camera that are not included in the installation status acquisition system 1.

[0115] Next, modified examples of the embodiment of the present invention will be described. Note that the modified examples described below may be implemented by replacing part of the above-described embodiment with or adding to it.

[0116] In the above-described embodiment, one representative point per cluster is used to identify the attached material. This method allows for a high accuracy rate of identifying materials when a single column or girder is hung. However, this method may have a low accuracy rate when the material spacing is small, such as when multiple sub-girders are hung, or when an outlier is included in the cluster. Taking this into consideration, the attached material may be identified as follows.

[0117] The attachment position estimation unit 15 estimates a plurality of positions at the construction site where one material lifted by the crane 100 will be attached. Furthermore, the material identification unit 17 calculates, for each material related to the correspondence information, the distance between each of the plurality of positions estimated by the attachment position estimation unit 15 and the position indicated by the correspondence information, calculates a score for each combination of the material indicated by the correspondence information and one material corresponding to the plurality of positions based on a weight according to the calculated distance, and identifies the attached material based on the calculated score.

[0118] Specifically, the attachment position estimation unit 15 and the material identification unit 17 perform the following processing, for example. In the above-described embodiment, the attachment position estimation unit 15 estimates multiple estimated attachment positions, which are multiple positions used to identify attached materials, based on positions included in a cluster corresponding to a representative point that is determined to be the position corresponding to the attachment position of one material. For example, for each position included in the above-described cluster, the attachment position estimation unit 15 determines, as in the above-described embodiment, a point that is lower in altitude from that position by a predetermined wire length as the estimated attachment position of the material. Since there can be multiple positions included in a cluster for one material, the above-described estimated attachment position for one material can be multiple positions.

[0119] The attachment position estimation unit 15 may estimate the multiple estimated attachment positions for each material by a method other than the above, as long as it is based on the position information for each time period of "attaching and removing" from the position information acquired by the position information acquisition unit 11. The attachment position estimation unit 15 outputs information indicating the multiple estimated attachment positions estimated for each material to the material identification unit 17.

[0120] The material identification unit 17 receives information indicating the multiple estimated attachment positions estimated for each material from the attachment position estimation unit 15. The material identification unit 17 calculates the three-dimensional distance between each of the multiple estimated attachment positions estimated for each material and the position where each material is to be attached, as indicated in the correspondence information. FIG. 11 shows an example of the three-dimensional distance between the multiple estimated attachment positions and the position where each material is to be attached. In FIG. 11, the length of the line connecting the triangles indicating the multiple estimated attachment positions and the circle indicating the position where the material is to be attached represents the three-dimensional distance. The darker the line, the shorter the three-dimensional distance, i.e., the closer the estimated attachment position is to the position where the material is to be attached. The calculation of the three-dimensional distance may be performed in the same manner as the calculation of the distance between the representative points described above.

[0121] The material identification unit 17 calculates a weight corresponding to the calculated three-dimensional distance for each material indicated in the correspondence information and for each of a plurality of estimated attachment positions for that material. The material identification unit 17 stores in advance rules such as formulas for calculating weights from three-dimensional distances, and calculates weights in accordance with these rules. For example, the material identification unit 17 calculates the weight w(d) from the three-dimensional distance using a probability density function expressed by the following formula: In the above formula, d is the calculated three-dimensional distance. σ is a preset value. This formula assumes a Gaussian distribution (normal distribution) centered on the position where each material is to be attached. For example, if the position coordinate system has all coordinate axes in meters (m) and σ is 0.5, the above weights represent weights assuming that there is a 95% probability that the estimated attachment position (cluster point) will be within 1.0 m of the position where the material is to be attached.

[0122] The weights are set to be larger as the calculated three-dimensional distance decreases, i.e., as the estimated attachment position approaches the location where the material is to be attached. Furthermore, the weights do not necessarily need to be based on a Gaussian distribution as in the above formula, but may be any weights that can appropriately identify the attached material.

[0123] The material identification unit 17 calculates a score for one material corresponding to a plurality of estimated installation positions from the calculated weights for each of the materials indicated in the correspondence information. The material identification unit 17 stores in advance rules such as formulas for calculating scores from weights, and calculates scores in accordance with the rules. For example, the material identification unit 17 calculates a score P(cluster) for one material corresponding to a plurality of estimated installation positions from the weight w(d) using the following formula: In the above formula, Σ ｃｌｕｓｔｅｒ w(d) is the sum of the scores w(d) of all the multiple estimated installation positions for one material corresponding to multiple estimated installation positions and for the target material indicated in the correspondence information. ｂｉｍ Σ ｃｌｕｓｔｅｒ w(d) is the Σ ｃｌｕｓｔｅｒ For one material corresponding to multiple estimated attachment positions, Σ ｂｉｍ Σ ｃｌｕｓｔｅｒ Since w(d) (the denominator of P(cluster)) is the same value, P(cluster) = Σ ｃｌｕｓｔｅｒ That is, P(cluster) may be only the numerator value of the above formula.

[0124] The score increases as the number of estimated installation locations with high scores among the materials indicated in the correspondence information, i.e., the number of estimated installation locations close to the location indicated in the correspondence information, increases. Furthermore, the score does not necessarily have to be calculated using the above formula, as long as it can appropriately identify the installed material.

[0125] For one material corresponding to multiple estimated installation locations, the material identification unit 17 identifies the material with the highest calculated score among the materials indicated in the correspondence information as the material installed at the construction site.

[0126] By estimating multiple estimated attachment positions and identifying the attached materials as in this modified example, it is possible to prevent a decrease in the accuracy rate even in cases where there is a risk of a low accuracy rate, as described above. In other words, according to this modified example, it is possible to more appropriately identify the attached materials, and as a result, it is possible to more appropriately obtain information indicating the attachment status of materials at the construction site.

[0127] In the above-described embodiment, the wire length used in calculating the estimated attachment position by the attachment position estimating unit 15 is a preset length. The attachment position estimating unit 15 may (automatically) calculate and set the wire length for each lifting of a material based on the position information acquired by the position information acquiring unit 11. For example, the wire length may be calculated as follows.

[0128] The positioning meter 20 detects acceleration simultaneously with the time-series three-dimensional position related to the lifting of the crane 100. The acceleration may be angular acceleration. For example, the positioning meter 20 detects the acceleration of the hook 102 of the crane 100. Information indicating the acceleration detected by the positioning meter 20 is acquired by the server device 10 in the same manner as information indicating the three-dimensional position.

[0129] The attachment position estimation unit 15 calculates the wire length from the information indicating the acceleration and the information input from the position information acquisition unit 11 and the time period identification unit 12. The attachment position estimation unit 15 stores in advance rules for calculating the wire length, and calculates the wire length in accordance with the rules.

[0130] The attachment position estimation unit 15 identifies the ground removal point, which is the timing of the removal, from the information input from the time period determination unit 12 and the information indicating the acceleration. Ground removal refers to the lifting of materials from the ground during lifting work. For example, the attachment position estimation unit 15 determines the point at which the vertical (Z-axis) acceleration reaches its maximum within a preset time period (e.g., within 60 seconds) from the start of the "lifting" time period as the ground removal point. The attachment position estimation unit 15 sets the height of the hook 102 from the ground based on the position information of the ground removal point as the wire length used to calculate the estimated attachment position.

[0131] By calculating and setting the wire length as described above, an appropriate estimated installation position can be calculated, and as a result, information indicating the installation status of materials at a construction site can be obtained more appropriately.

[0132] In addition, when multiple components are hoisted in a manner that calculates the wire length as described above, the altitudes of the other components may be unified to the altitude of the component attached at the lowest position. For example, the attachment position estimation unit 15 sets the attachment position of the first component to a point that is lower in altitude from the position (representative point) corresponding to the attachment position of the component by the predetermined length (m) of the long wire, and stores the altitude of this point. The attachment position estimation unit 15 sets the altitudes of the attachment positions of the second and subsequent components to the stored altitude. In the manner that calculates the wire length as described above, always using the component attached at the lowest position as the reference simplifies the algorithm used by the attachment position estimation unit 15 to estimate the estimated attachment position.

[0133] In the above-described embodiment, an example has been shown in which DBSCAN-based position clustering is used to estimate the estimated attachment position by the attachment position estimation unit 15. DBSCAN-based clustering uses a threshold value for the distance between points (positions). Appropriate clustering cannot be performed unless the threshold value is set appropriately. For example, an inappropriate setting of the threshold value may result in the creation of many small clusters or the calculation of clusters that are too large. If appropriate clustering cannot be performed, there is a risk that the attached material cannot be properly identified. In other words, the DBSCAN distance threshold value affects the accuracy of identifying the attached material.

[0134] In consideration of the above, the attachment position estimation unit 15 may (automatically) determine the threshold value based on a plurality of positions included in the extracted position information for each time period of "attaching / removing balls." For example, the threshold value may be determined as follows.

[0135] The attachment position estimation unit 15 generates a k-distance graph from multiple positions included in the extracted position information. The k-distance graph calculates the distance to the kth closest point for each point (position) and arranges the distances in ascending order. k is a preset value. The value of k is preferably set taking into account the minimum number of positions (minimum number of points) included in a cluster (threshold value for the number of positions included in a cluster). FIG. 12 shows an example of a k-distance graph generated from multiple positions included in the extracted position information. In the distance graph of FIG. 12, the horizontal axis represents the positions (points within the cluster) arranged in ascending order, and the vertical axis represents the distance to the kth closest point. One cluster corresponds to the position where one material is attached. Therefore, the minimum number of positions (minimum number of points) included in one cluster may be a number corresponding to the length of time (e.g., 30 seconds) that the hook 102 is considered to need to be stopped to attach one material. Therefore, if the location information is at one-second intervals as described above, the minimum number of locations (minimum score) included in one cluster may be 30 (the number of location information included in 30 seconds). Also, the value of k may be the same as the minimum number of locations (minimum score) included in one cluster.

[0136] In a k-distance graph generated from multiple locations included in extracted location information, such as the k-distance graph shown in Figure 12, there are points where the value suddenly increases, and the graph tends to have two main gradients. Therefore, the distance to the kth closest point at the intersection of the lines representing the two gradients is set as the distance threshold.

[0137] For example, the attachment position estimation unit 15 calculates two straight lines (regression lines) that best fit the k-distance graph using an existing algorithm (for example, RANSAC (Random Sample Consensus)). The two calculated straight lines have different gradients. The different gradients are the gradient (first half on the horizontal axis of the k-distance graph) before the gradient trends differ, and the gradient (second half on the horizontal axis of the k-distance graph) after the gradient trends differ. FIG. 12 shows the first half fitting straight line (Line 1) and the second half fitting straight line (Line 2). The attachment position estimation unit 15 finds the intersection of the two straight lines, and sets the distance to the kth closest point at the intersection as the distance threshold.

[0138] By setting the DBSCAN distance threshold as described above, it is possible to properly identify the installed materials, and as a result, it is possible to more appropriately obtain information indicating the installation status of materials at the construction site.

[0139] In the above-described embodiment, when the number of representative points after deletion exceeds the upper limit of the number of clusters, the attachment position estimation unit 15 may extract positions corresponding to the attachment positions of materials from the representative points after deletion as follows: This is to eliminate clusters that are generated by unexpected behavior, such as lifting materials and then not returning to the material storage area but instead carrying temporary materials such as nets.

[0140] The attachment position estimation unit 15 determines the representative points (P1 shown in Figure 3) of the upper limit number of clusters as positions corresponding to the attachment positions of materials, in order of distance from the position at the start time of the "attachment / removal" time period (the time corresponding to the left end of time period T3 in Figure 4). At this time, the clusters are arranged in chronological order based on the start time of the "attachment / removal" time period, and if there are clusters located more than 5 meters above or 10 meters below the first cluster, those representative points are deleted first. This prevents behavior other than material attachment from being erroneously detected as attachment, as described above.

[0141] In the above-described embodiment, the time period identification unit 12 identifies the time period related to lifting using the altitude (Z-axis value) included in the position information. However, the time period identification unit 12 may identify the time period related to lifting by taking into account the horizontal position (horizontal movement distance) in addition to the altitude included in the position information. For example, the time period related to lifting may be identified three-dimensionally by adding the horizontal direction, rather than one-dimensionally in the vertical direction. For example, the time period identification unit 12 uses the horizontal position to identify the time period related to lifting as follows.

[0142] When extracting candidates for the "lifting" time period, the time period identification unit 12 may determine, in addition to the altitude condition, whether the horizontal position has moved horizontally by more than a predetermined movement amount (e.g., a value in meters) during the time period to be extracted, based on the position information. In this case, for example, if the altitude condition is satisfied or the horizontal position has moved by more than the predetermined movement amount during the time period to be extracted, the time period identification unit 12 determines the time period as a candidate for the "lifting" time period. Note that the horizontal movement used in the above determination may be in any of the predetermined horizontal directions (e.g., the X-axis direction or the Y-axis direction).

[0143] When extracting candidate time periods for "lifting and returning," the time period identification unit 12 may determine, in addition to the altitude condition, whether the horizontal position has moved horizontally by more than a predetermined movement amount (e.g., a value in meters) during the time period to be extracted, based on the position information. In this case, for example, if the altitude condition is satisfied or the horizontal position has moved by more than the predetermined movement amount during the time period to be extracted, the time period identification unit 12 determines the time period as a candidate time period for "lifting and returning." Note that the horizontal movement used in the above determination may be in any of the predetermined horizontal directions (e.g., the X-axis direction or the Y-axis direction). Note that when candidate time periods for "lifting" and "lifting and returning" are extracted using the horizontal position condition, one time period may correspond to both a candidate time period for "lifting" and a candidate time period for "lifting and returning." In this case, the time period identification unit 12 determines that one time period is a candidate time period for "lifting" and a candidate time period for "lifting return," and determines that the pair of candidate time periods for "lifting" and "lifting return" described above is the actual "lifting" and "lifting return." Since this is a lifting operation, it is likely that there is a slight difference in altitude between the point where the lifting begins and the point where it is installed. Even if the candidate time periods for "lifting" and "lifting return" are extracted using the horizontal position condition as described above, the altitude condition is used when determining the actual "lifting" and "lifting return." Therefore, even if the difference in altitude between the "lifting" and "lifting return" time periods is small, the lifting operation can be extracted with generally sufficient accuracy.

[0144] As described above, if the horizontal position (horizontal movement distance) is used to identify the time period related to lifting, it becomes possible to properly detect the lifting of materials at low positions using a mobile crane (e.g., crawler crane), etc. As a result, it is possible to more properly obtain information indicating the installation status of materials at a construction site.

[0145] The installation status acquisition system of the present disclosure has the following configuration: [1] An installation status acquisition system for acquiring information indicating the installation status of materials at a construction site, comprising: position information acquisition means for acquiring position information indicating three-dimensional positions over time related to lifting by a crane, time period identification means for identifying, for each lifted material, a removal time period in which the material lifted by the crane was removed from the crane based on the position information acquired by the position information acquisition means, installation position estimation means for estimating a position at the construction site where the material lifted by the crane will be installed, based on position information for the removal time period identified by the time period identification means among the position information acquired by the position information acquisition means, correspondence information acquisition means for acquiring correspondence information prepared in advance that associates information identifying the material with a position where the material is to be installed, and material identification means for identifying the installed material by comparing the position estimated by the installation position estimation means with the position indicated by the correspondence information acquired by the correspondence information acquisition means. [2] The installation status acquisition system according to [1], wherein the time period specification means estimates, for each lifted material, a lifting time period in which the material was lifted by the crane and a lifting return time period in which the crane will move after the lifted material is removed from the crane based on the position information acquired by the position information acquisition means, and specifies the removal time period by comparing the altitude indicated by the position information during the estimated lifting time period with the altitude indicated by the position information during the time period between the estimated lifting time period and the lifting return time period. [3] The installation status acquisition system according to [1] or [2], wherein the installation position estimation means specifies the number of materials lifted by the crane in one lifting based on the position information during the removal time period specified by the time period specification means among the position information acquired by the position information acquisition means, and estimates the position at which each of the materials will be installed at the construction site.[4] The installation status acquisition system according to [3], wherein the installation position estimation means clusters position information for a removal time period identified by the time period identification means among the position information acquired by the position information acquisition means, and identifies the number of materials lifted by the crane in one lifting. [5] The time period identification means identifies, for each lifted material, at least a portion of a lifting time period in which the material was lifted by the crane based on the position information acquired by the position information acquisition means, and the installation status acquisition system further comprises: image acquisition means for acquiring, for each of the lifting time periods, a plurality of images of the lifted material taken at a plurality of times included in at least a portion of the lifting time period identified by the time period identification means; and type identification means for identifying the type of material lifted for each lifting time period from the plurality of images acquired by the image acquisition means, and the material identification means identifies the installed material based also on the type of material identified by the type identification means. [6] An installation status acquisition system according to any one of [1] to [5], wherein the installation position estimation means estimates a plurality of positions at which one material lifted by a crane will be installed at a construction site, and the material identification means calculates, for each material related to the correspondence information, the distance between each of the plurality of positions estimated by the installation position estimation means and the position indicated by the correspondence information, calculates a score for the one material corresponding to the plurality of positions based on a weight according to the calculated distance, and identifies the installed material based on the calculated score.

[0146] 1...Installation status acquisition system, 10...Server device, 11...Location information acquisition unit, 12...Time period identification unit, 13...Image acquisition unit, 14...Type discrimination unit, 15...Installation position estimation unit, 16...Corresponding information acquisition unit, 17...Material identification unit, 20...Positioning meter, 30...Camera, 100...Crane, 101...Boom, 102...Hook, 103...Hook block, 104...Sling

Claims

1. An installation status acquisition system for acquiring information indicating the installation status of materials at a construction site, comprising: position information acquisition means for acquiring position information indicating three-dimensional positions over time related to the lifting of materials by a crane; time zone identification means for identifying, for each lifted material, a removal time zone in which the material lifted by the crane was removed from the crane based on the position information acquired by the position information acquisition means; installation position estimation means for estimating a position at the construction site where the material lifted by the crane will be installed based on position information for the removal time zone identified by the time zone identification means among the position information acquired by the position information acquisition means; correspondence information acquisition means for acquiring correspondence information prepared in advance that associates information identifying the material with the position where the material is to be installed; and material identification means for identifying the installed material by comparing the position estimated by the installation position estimation means with the position indicated by the correspondence information acquired by the correspondence information acquisition means.

2. The installation status acquisition system of claim 1, wherein the time period identification means estimates, based on the location information acquired by the location information acquisition means, the lifting time period during which the material was lifted by the crane for each lifted material, and the lifting return time period during which the crane will move after the lifted material is removed from the crane, and identifies the removal time period by comparing the altitude indicated by the location information during the estimated lifting time period with the altitude indicated by the location information during the time period between the estimated lifting time period and the lifting return time period.

3. An installation status acquisition system as described in claim 1 or 2, wherein the installation position estimation means identifies the number of materials lifted by the crane in one lifting based on the position information acquired by the position information acquisition means, including the position information during the removal time period identified by the time period identification means, and estimates the position at which each of the materials will be installed at the construction site.

4. The installation status acquisition system described in claim 3, wherein the installation position estimation means clusters the position information acquired by the position information acquisition means, among the position information during the removal time period identified by the time period identification means, to identify the number of materials lifted by the crane in one lifting.

5. The time period identification means identifies, for each lifted material, at least a portion of the lifting time period during which the material was lifted by the crane based on the location information acquired by the location information acquisition means; and the installation status acquisition system further comprises: an image acquisition means for acquiring, for each of the lifting time periods, a plurality of images of the lifted material taken at a plurality of times included in at least a portion of the lifting time period identified by the time period identification means; and a type discrimination means for discriminating the type of material lifted for each lifting time period from the plurality of images acquired by the image acquisition means; and the material identification means identifies the installed material also based on the type of material discriminated by the type discrimination means.

6. The installation status acquisition system described in claim 1 or 2, wherein the installation position estimation means estimates multiple positions at the construction site where a single material lifted by a crane will be installed, and the material identification means calculates, for each material related to the correspondence information, the distance between each of the multiple positions estimated by the installation position estimation means and the position indicated by the correspondence information, calculates a score for the single material corresponding to the multiple positions based on a weight according to the calculated distance, and identifies the installed material based on the calculated score.

Images