Lifting achievement acquisition system

Abstract

To appropriately acquire information indicating crane lifting achievement.SOLUTION: A lifting achievement acquisition system 1 for acquiring information indicating lifting achievement of a crane 100 which lifts a material comprises: an altitude information acquisition unit 11 that acquires altitude information indicating time-series altitude related to lifting of the crane 100; a time zone identification unit 12 that identifies at least a part of a lifting time zone during which material lifting is carried out by the crane 100 for each lifted material on the basis of the acquired altitude information; an image acquisition unit 13 that acquires a plurality of images of the lifted material captured at a plurality of timings included in at least a part of the identified lifting time zone; and a type determination unit 14 that determines the type of the material lifted during each lifting time zone from the acquired images.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a lifting record acquisition system that acquires information indicating the lifting record of a crane that lifts materials. [Background technology]

[0002] In streamlining construction work, it is important to understand and improve the operating status of materials, equipment, and workers. In particular, lifting work using cranes such as tower cranes can become a bottleneck in the process. Therefore, there is a need to improve the efficiency of crane lifting work. In order to improve the efficiency of crane lifting work, it is first necessary to obtain information showing the lifting performance of the crane. Examples of crane lifting performance include the time period during which the lifting work was performed by the crane and the type of materials lifted during that time period.

[0003] The following Patent Document 1 discloses a method for acquiring information on the lifting performance of a crane. In this method, an image of the material when it is lifted by the crane is used to determine the type of the material. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US Patent Application Publication No. 2021 / 0292132 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the method shown in Patent Document 1, there is a risk that the type of material may not be properly identified depending on the influence of light at the time of imaging and the appearance of the material in the image, etc. As a result, there is a risk that the obtained information showing the lifting performance of the crane may not be appropriate.

[0006] The present invention has been made in consideration of the above, and has an object to provide a lifting record acquisition system that can appropriately acquire information indicating the lifting record of a crane. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the lifting performance acquisition system of the present invention is a lifting performance acquisition system that acquires information indicating the lifting performance of a crane that lifts materials, and comprises an altitude information acquisition means that acquires altitude information that indicates the chronological altitude related to the lifting of materials by the crane, a time zone identification means that identifies at least a portion of the lifting time zone in which the material was lifted by the crane for each lifted material based on the altitude information acquired by the altitude information acquisition means, an image acquisition means that acquires, for each lifting time zone, a plurality of images of the lifted materials taken at a plurality of times that fall within at least a portion of the lifting time zone identified by the time zone identification means, and a type discrimination means that discriminates the type of material lifted for each lifting time zone from the plurality of images acquired by the image acquisition means.

[0008] In the lifting record acquisition system according to the present invention, the type of material is determined from a plurality of images of the material lifted during at least a portion of the lifting time period. Therefore, the type of material can be appropriately determined. As a result, the lifting record acquisition system according to the present invention can appropriately acquire information indicating the lifting record of the crane.

[0009] The time period specification means may specify a time period during which the material is being transported in the vertical direction by the lifting of the crane as at least a part of the lifting time period. With this configuration, the multiple images used for the material type can be made appropriate. As a result, information showing the lifting performance of the crane can be obtained appropriately and reliably.

[0010] The image acquisition means may acquire a plurality of images of the lifted material taken at a plurality of times during at least a portion of the lifting time period during which the altitude indicated by the altitude information is equal to or higher than a preset altitude. With this configuration, the plurality of images used for the type of material can be made appropriate. As a result, information indicating the lifting performance of the crane can be acquired appropriately and reliably.

[0011] The image acquisition means may acquire a plurality of images of the lifted material taken at a predetermined number of equal intervals. With this configuration, the plurality of images used for the type of material can be appropriate. As a result, information showing the lifting performance of the crane can be appropriately and reliably acquired.

[0012] The type discrimination means may primarily discriminate the type of the lifted material from each of the multiple images, and discriminate the type of the lifted material based on the multiple discrimination results from the primary discrimination. With this configuration, the type of material can be discriminated appropriately and reliably. As a result, information indicating the lifting performance of the crane can be obtained appropriately and reliably.

[0013] The type discrimination means may calculate a score for each type of material lifted from each of the multiple images acquired by the image acquisition means, calculate a discrimination score for each type of material from the multiple scores calculated for each type of material using a preset calculation formula, and discriminate the type of the lifted material based on the calculated discrimination score. This configuration makes it possible to more appropriately discriminate the type of material. As a result, it is possible to more appropriately acquire information indicating the lifting performance of the crane.

[0014] The type discrimination means may detect a portion of the crane hook in the multiple images acquired by the image acquisition means, cut out a portion of the image corresponding to the detected portion, and discriminate the type of the lifted material from the cut-out image. With this configuration, the type of material can be more appropriately discriminated. As a result, information indicating the lifting performance of the crane can be more appropriately acquired.

[0015] The crane may further include a detection means for detecting the time-series altitude related to the lifting of the crane, which is provided at the upper part of the hook block supporting the hook of the crane and at a position sandwiched between two slings supporting the hook block, and the altitude information acquisition means may acquire altitude information detected by the detection means. With this configuration, the altitude information can be acquired reliably and appropriately. Also, damage to the detection means during lifting, etc. can be prevented.

[0016] The hook block may further include an imaging means provided between two plate-like members that support the hook of the crane and whose main surfaces face each other with a gap therebetween, and the image acquisition means may acquire images captured by the imaging means. With this configuration, images can be acquired reliably and appropriately. Also, damage to the imaging means during lifting, etc. can be prevented. Effect of the Invention

[0017] According to the present invention, information indicating the lifting performance of a crane can be appropriately acquired. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram showing a configuration of a lifting performance acquisition system according to an embodiment of the present invention. [Diagram 2] 1 is an image of an example hook block of a crane according to an embodiment. [Diagram 3] 11 is a graph showing an example of information indicating lifting performance. [Figure 4] 11 is a graph showing an example of altitude information and classes of lifting-related work by time. [Diagram 5] 13 is a diagram showing timings related to images used to determine the type of material, the images, and confidence levels for each type of material calculated from the images; FIG. [Figure 6] FIG. 13 is a diagram showing an example of an image used to determine the type of material. [Figure 7] FIG. 13 is a diagram showing examples of images for each type of material. [Figure 8] 4 is a flowchart showing a process executed in the lifting performance acquisition system according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an embodiment of a lifting performance acquisition system according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated description will be omitted.

[0020] FIG. 1 shows a lifting record acquisition system 1 according to this embodiment. The lifting record acquisition system 1 is a system (apparatus) that acquires information indicating the lifting record of a crane 100 that lifts materials. The crane 100 is, for example, a tower crane used at construction sites and the like. For example, the crane 100 is provided with a boom 101, and lifting is performed using a hook 102 suspended from the tip of the boom 101. The hook 102 is attached to and supported by a hook block (weight block) 103. The hook block 103 has a sling (wire) 104 attached thereto, and is suspended from the boom 101 by the sling 104.

[0021] An image of a portion of the hook block 103 is shown in Figure 2. The hook block 103 is equipped with a pulley, and a sling 104 is wound around the pulley and supported by the sling 104. The slings 104 extend from both sides of the pulley of the hook block 103 toward the boom 101. The hook block 103 includes two plate-like members 103a that sandwich the pulley and support and protect it. The main surfaces of the two plate-like members 103a face each other with a gap between them.

[0022] Note that the crane 100 may have a configuration other than that described above. Also, the crane 100 may be a crane other than a tower crane. For example, the crane 100 may be a stationary crane other than a tower crane, or a mobile crane such as a crawler crane or a crawler crane. The crane 100 may be the same as a conventional crane.

[0023] The lifting record is the time period during which the lifting operation of materials was performed by the crane 100, and the type (kind) of materials lifted during that time period. The time period during which the lifting operation was performed is indicated, for example, by the start time of the lifting (e.g., the time when the movement of the hook 102 starts to lift the materials, or the time when the materials leave the ground) and the end time (e.g., the time when the materials are removed from the crane (removed from the load), or the time when the materials land). The type of material is, for example, the type of material (component) that constitutes the structure to be built. Specifically, the types of materials are decks, steel columns, stairs, reinforcing bars, temporary materials, steel girders, steel girders, steel-related materials, exterior materials, etc. Note that the types of materials are not limited to those mentioned above and may be set arbitrarily.

[0024] FIG. 3 shows an example of information indicating lifting performance acquired by the lifting performance acquisition system 1. In the graph of FIG. 3, the horizontal axis is the time axis. The hatched portion of the graph of FIG. 3 is the time period during which materials are lifted by the crane 100. The lower graph of FIG. 3 shows the actual lifting performance, and the upper graph shows the lifting performance acquired (estimated) by the lifting performance acquisition system 1. The information indicating the lifting performance acquired by the lifting performance acquisition system 1 is used, for example, to improve the efficiency of the lifting work by the crane 100. Specifically, the information indicating the lifting performance is used to understand the discrepancy between the lifting plan and the lifting performance, and the cause of the construction delay (process delay) is investigated and action improvement is implemented. For example, if three steel columns were planned to be lifted in the morning, but only two were lifted, it can be understood that there was some kind of trouble that caused the delay.

[0025] As shown in Fig. 1, the lifting record acquisition system 1 includes a server device 10, a positioning meter 20, and a camera 30. The server device 10 is a device that acquires data from the positioning meter 20 and the camera 30, generates a lifting record from the acquired data, and acquires information indicating the lifting record. Specific functions of the server device 10 will be described later.

[0026] Specifically, the server device 10 is a computer including hardware such as a CPU (Central Processing Unit) and a memory. The functions of the server device 10 described below are realized by these components operating according to programs or the like. The server device 10 may be realized by one computer, or may be realized by a computer system in which a plurality of computers are connected to each other via a network. The server device 10 may also be realized by a cloud system.

[0027] The server device 10, the position meter 20, and the camera 30 each have a communication function, and are capable of transmitting data from the position meter 20 and the camera 30 to the server device 10. For example, the position meter 20 and the camera 30 may have a wireless communication function and transmit data to the server device 10 via wireless communication. The position meter 20 and the camera 30 may also be powered by a battery. By using the position meter 20 and the camera 30 that have a wireless communication function and are battery-powered, it becomes easy to retrofit the position meter 20 and the camera 30.

[0028] The positioning meter 20 is a detection means for detecting the time-series altitude related to the lifting of the crane 100. For example, the positioning meter 20 detects the altitude of the hook 102 of the crane 100. The positioning meter 20 detects the time-series altitude and transmits altitude information indicating the detected time-series altitude to the server device 10. The altitude information transmitted to the server device 10 is configured to allow the timing at which the altitude was detected to be known. The positioning meter 20 detects the altitude at a time interval appropriate for obtaining the lifting performance.

[0029] The positioning meter 20 is provided, for example, on the hook 102 of the crane 100 (in the vicinity of the hook 102). Specifically, as shown in Figs. 1 and 2, the positioning meter 20 is fixed to the upper part of the hook block 103 (above the upper end of the hook block 103) and at a position sandwiched between two slings 104 supporting the hook block 103. At this position, there is a relatively large space protected by the hook block 103 and the slings 104. Therefore, there is little risk that the positioning meter 20 placed at this position will come into contact with materials or buildings and be damaged.

[0030] The positioning meter 20 is stored in a housing (box, main body) together with a battery for powering the positioning meter 20. To be precise, FIG. 2 shows the housing that stores the positioning meter 20. The housing may be one that matches the dimensions of the hook 102 (hook block 103). By using a housing with an upper cover that opens upward, the battery can be replaced while the housing is attached to the hook block 103. By installing the housing on top of the hook block 103, there is no risk of a large load being applied to the housing, causing it to break and causing the material to fall, compared to a case in which a housing is attached between the hook 102 and the material to be lifted and relayed between the two.

[0031] The positioning device 20 may be the same as a conventional positioning device. For example, the positioning device 20 may be a device that acquires its own three-dimensional position information with high accuracy by RTK (real-time kinematic) positioning.

[0032] The altitude detected by the positioning meter 20 may be an altitude other than the altitude of the hook 102 as long as it is an altitude related to the lifting of the crane 100. The arrangement of the positioning meter 20 may be other than the above as long as it is capable of detecting the altitude related to the lifting of the crane 100. The values ​​on the vertical axis of the graph in Fig. 3 are an example of a time series of altitude detected by the positioning meter 20. In the graph in Fig. 3, the horizontal axis is the time axis, and the vertical axis is the altitude axis.

[0033] The camera 30 is an imaging means for imaging the material lifted by the crane 100. For example, the camera 30 is arranged at the hook 102 of the crane 100 (near the hook 102) so that the material to be lifted is captured in the captured image. The camera 30 may also be arranged so that the hook 102 is captured in the captured image in addition to the material to be lifted. Specifically, as shown in FIG. 1 and FIG. 2, the camera 30 is fixed and arranged between two plate-like members 103a of the hook block 103. By arranging the camera 30 at the position protected by the hook block 103, damage to the camera 30 due to contact between the camera 30 and the outside can be avoided. In addition, since the position is close to the lifted material suspended by the hook 102, it is possible to obtain an image of the material with higher image quality. The lifting performance acquisition system 1 may also include a plurality of cameras 30. For example, as shown in FIG. 1, two cameras 30 may be arranged at symmetrical positions when viewed from the direction of the main surface of the plate-like member 103a. By providing multiple cameras 30, even if the material is difficult to see in the image from one camera 30, the image from another camera 30 can be used to compensate.

[0034] The camera 30 may be powered from a battery stored in the housing that houses the positioning meter 20. By supplying power to the positioning meter 20 and the camera 30 from a battery stored in the housing, the positioning meter 20 and the camera 30 can be operated for several days without changing the battery. Alternatively, the camera 30 may be powered by other methods. The camera 30 may be placed in a location other than the above so that the lifted material is captured in the captured image. For example, the camera 30 may be attached to a part that is not protected by the hook block 103 as described above, for example, the outside of the hook block 103. In this case, it is necessary to protect the camera 30, such as by putting the camera 30 in a sturdy housing to prevent it from scattering due to breakage. The camera 30 may be made small so as not to come into contact with other objects, and a fall prevention wire may be attached to prevent the camera 30 from falling.

[0035] The camera 30 takes images in a time series and transmits the images obtained by the image capture to the server device 10. The images transmitted to the server device 10 are configured so that the timing at which the images were captured can be understood. The camera 30 captures images at appropriate time intervals for obtaining lifting performance data. For example, the camera 30 captures images at the same timing as the altitude detection by the positioning meter 20. Therefore, the camera 30 may capture images even when the crane 100 is not lifting materials. The camera 30 may also capture video. The camera 30 may be the same as a conventional camera.

[0036] Next, a description will be given of the functions of the server device 10. As shown in FIG.

[0037] The altitude information acquisition unit 11 is an altitude information acquisition means that acquires altitude information indicating the time-series altitude related to the lifting of the crane 100. The altitude information acquisition unit 11 may acquire altitude information detected by the positioning meter 20. The altitude information acquisition unit 11 receives and acquires altitude information transmitted from the positioning meter 20. The altitude information acquisition unit 11 outputs the acquired altitude information to the time zone identification unit 12 and the image acquisition unit 13.

[0038] The time zone specifying unit 12 is a time zone specifying means that specifies at least a portion of the lifting time zone in which the material was lifted by the crane 100 for each lifted material, based on the altitude information acquired by the altitude information acquiring unit 11. The time zone specifying unit 12 may specify, as at least a portion of the lifting time zone, the time zone in which the material is transported in the altitude direction by the lifting of the crane. For example, the time zone specifying unit 12 specifies the above-mentioned lifting time zone as follows.

[0039] The time zone identification unit 12 inputs altitude information from the altitude information acquisition unit 11. The time zone identification unit 12 classifies lifting-related work at each time into one of five classes based on the chronological altitude indicated by the altitude information. Figure 4 shows a graph related to the classified classes. In the graph of Figure 4, the horizontal axis is the time axis and the vertical axis is the altitude axis. Note that the two graphs shown in Figure 4 are graphs of the same altitude at the same time. The five classes are "rigging and lifting", "lifting", "attaching and removing", "lifting return" and "standby".

[0040] "Slinging and lifting" refers to the work of attaching the material to be lifted to the hook 102 and lifting it from the ground (hoisting the sling 104 to move the material away from the ground and temporarily stopping it). Normally, slinging also includes the work of removing the material from the hook 102, but in this embodiment, it refers only to the work of attaching the material. "Lifting" refers to the work of transporting the material in the vertical direction by lifting the sling 104 with the material attached to the hook 102 to the attachment position, that is, the work of lifting the material. In this embodiment, "lifting" refers to the work of transporting the material upward in the vertical direction by lifting the crane 100. "Attaching and removing" refers to the work of attaching the material that has been transported to the attachment position by lifting and removing it from the hook 102. The attachment in "attaching and removing" refers to, for example, attaching the material to a building being constructed or placing the material at a position where it will be used. Additionally, the attachment position in "lifting" and "attaching / removing" is the position where the attachment is performed. "Returning to lift" is the work of moving the hook 102 in the vertical direction to retrieve the next material to be lifted when no material is attached to the hook 102. In this embodiment, "returning to lift" is the work of moving the hook 102 downward in the vertical direction. "Waiting" is the state in which work is stopped.

[0041] As shown in Fig. 4, when lifting one material, the work is usually performed in the order of "rigging and lifting", "lifting", "attaching and removing the rigging", and "returning the lift". Therefore, the work in the order of "rigging and lifting", "lifting", "attaching and removing the rigging", and "returning the lift" is considered to be the work related to the lifting of one material (i.e., the lifting unit of one material). In this embodiment, the time period determined to be the work related to the lifting of this one material is the lifting time period during which the lifting of one material was performed.

[0042] The time zone identification unit 12 classifies each time into one of five classes as follows. The time zone identification unit 12 takes a moving average of the time series altitude indicated by the altitude information at a preset time interval. The time zone identification unit 12 uses the time series altitude after the moving average to classify each time into one of five classes. The moving average is taken in order to remove temporary changes in the time series altitude. In the following explanation, the time series altitude is taken after the moving average. However, if it is considered that the class classification can be performed appropriately without taking the moving average, the class classification may be performed using the time series altitude indicated by the altitude information as it is without taking the moving average.

[0043] The class of the time period when the slope of the altitude against the time is a positive value equal to or greater than a preset certain value is designated as "lifting". The class of the time period when the slope is a negative value equal to or greater than a preset certain value is designated as "lifting return". The class of the time period from "lifting return" to "lifting" is designated as "rigging and lifting". The class of the time period from "lifting" to "lifting return" is designated as "attaching and removing". The class of the time period when the altitude does not change for a preset period of time or more is designated as "standby". Note that "standby" takes priority over "rigging and lifting" and "attaching and removing".

[0044] The time zone identifying unit 12 outputs information indicating at least a part of the identified lifting time zone to the image acquiring unit 13 and the type identifying unit 14. For example, the time zone identifying unit 12 outputs information indicating the "lifting" time zone within the lifting time zone to the image acquiring unit 13. The time zone identifying unit 12 also outputs information indicating each lifting time zone as a whole to the type identifying unit 14. At this time, the time zone identifying unit 12 may also output information indicating the class at each time of each lifting time zone to the type identifying unit 14.

[0045] The image acquiring unit 13 is an image acquiring means for acquiring, for each lifting time period, a plurality of images of the lifted materials taken at a plurality of timings included in at least a portion of the lifting time period identified by the time period identifying unit 12. The image acquiring unit 13 may acquire a plurality of images of the lifted materials taken at a plurality of timings included in a time period during which the altitude indicated by the altitude information is equal to or higher than a preset altitude, during at least a portion of the lifting time period. The image acquiring unit 13 may acquire a plurality of images of the lifted materials taken at a preset number of timings at equal intervals. The image acquiring unit 13 may acquire images acquired by imaging using the camera 30.

[0046] The images acquired by the image acquiring unit 13 are intended to determine the type of material lifted in the time period identified by the time period identifying unit 12 as the lifting time period. For example, the image acquiring unit 13 acquires images for this purpose as follows. The image acquiring unit 13 inputs information indicating at least a part of the lifting time period, for example, the "lifting" time period as described above, from the time period identifying unit 12. The image acquiring unit 13 inputs altitude information from the altitude information acquiring unit 11. The image acquiring unit 13 inputs and acquires images from the camera 30.

[0047] For each lifting time period, the image acquisition unit 13 selects, from among the images input from the camera 30, a number of images taken at a number of times within the "lifting" time period, which is at least a part of the lifting time period identified by the time period identification unit 12, during which the altitude indicated by the altitude information input from the altitude information acquisition unit 11 is equal to or higher than a preset altitude, as images to be used to identify the type of material.

[0048] For example, as shown in the graph on the left side of Fig. 5, within the "lifting" time period (the hatched time period), images taken at five equally spaced points (five points indicated by black circles) during the time period when the altitude is equal to or higher than a preset altitude are used to determine the type of material. Note that the first and last of the above points may or may not match the start and end times of the above time period.

[0049] The reason why images taken during a time period when the altitude is equal to or higher than a preset altitude are used as described above is because images taken at a lower altitude may erroneously identify the lifted material as other components on the ground. By using images taken during the above time period when the lifted material has reached a certain altitude and the background is in the distance, the type of lifted material can be properly identified.

[0050] The timings related to the images used to distinguish the type of materials do not necessarily have to be as described above, and may be multiple timings that are included in at least a part of the lifting time period identified by the time period identification unit 12 for each lifting time period. If multiple cameras 30 are provided, multiple images may be included in one timing. The image acquisition unit 13 outputs the multiple images used to distinguish the type of materials for each lifting time period to the type identification unit 14.

[0051] The type discrimination unit 14 is a type discrimination means for discriminating the type of the lifted material for each lifting time period from the multiple images acquired by the image acquisition unit 13. The type discrimination unit 14 may primarily discriminate the type of the lifted material from each of the multiple images, and discriminate the type of the lifted material based on multiple discrimination results from the primary discrimination. The type discrimination unit 14 may calculate a score for each type of material for the lifted material from each of the multiple images acquired by the image acquisition unit 13, calculate a discrimination score for each type of material from the multiple scores calculated for each type of material using a preset calculation formula, and discriminate the type of the lifted material based on the calculated discrimination score. The type discrimination unit 14 may detect a portion of the multiple images acquired by the image acquisition unit 13 in which the hook 102 is captured, cut out a portion corresponding to the detected portion from the image, and discriminate the type of the lifted material from the cut-out image. For example, the type discrimination unit 14 discriminates the type of materials lifted for each lifting time period as follows.

[0052] The type discrimination unit 14 inputs information indicating the lifting time zone from the time zone specification unit 12. The type discrimination unit 14 inputs a plurality of images for each lifting time zone from the image acquisition unit 13. The images input from the image acquisition unit 13 are images captured at the time when the crane 100 is lifting materials, and therefore include the hook 102 and the lifted materials.

[0053] As shown in FIG. 6, the type discrimination unit 14 cuts out a portion of the image I1 input from the image acquisition unit 13 in which the material to be lifted is shown as a partial image I2. The cutout may be performed, for example, by detecting a portion in which the hook 102 is shown and performing the cutout according to the detected portion. The detection may be performed by a conventional image analysis technique. Alternatively, a portion (position in the image) set in advance as a portion in which the material to be lifted is shown may be cut out according to the installation position of the camera 30, etc. Cutout may also be performed by a method other than the above. The partial image I2 is an image of a preset size and shape (for example, a rectangle such as a square). The position of the partial image I2 in the image I1 is, for example, a position in which the portion in which the hook 102 is shown is the center of the partial image I2. The type discrimination unit 14 uses the cut out partial image I2 and a learning model (image recognition model) used to discriminate the type of material to discriminate the type of material to discriminate the type of material lifted during the lifting time period.

[0054] By using partial image I2 as described above, it is possible to exclude from the image anything other than the material to be lifted, enabling more appropriate discrimination. It is not necessary to crop partial image I2, and type discrimination unit 14 may make discrimination by directly using the image input from image acquisition unit 13. For example, depending on the installation position of camera 30 and the size of hook 102, the image acquired by image acquisition unit 13 may be a close-up image of the material to be lifted, and therefore there is no need to crop partial image I2.

[0055] The type discrimination unit 14 primarily discriminates the type of the lifted material from each of the multiple images. For example, the type discrimination unit 14 performs temporary discrimination using a learning model that has been generated in advance by machine learning. The learning model inputs an image and outputs information related to the type of material depicted in the image. For example, the learning model may be configured to include a neural network. The neural network may be multi-layered, that is, generated by performing deep learning. The neural network may also be a CNN (convolutional neural network). CNN is strong in image data and is the most commonly used method in the field of image recognition, and the use of CNN enables more appropriate discrimination.

[0056] For example, the learning model includes an input layer, an intermediate layer, and an output layer. The input layer has neurons equal to the number of pixels in an image, and pixel values ​​of the image are input to each neuron. The output layer has neurons equal to the number of types of materials to be identified, and each neuron outputs a confidence score for each type. The confidence indicates the degree to which the material depicted in the image is of that type. For example, the confidence is a numerical value between 0 and 1, and the higher the confidence value, the greater the degree to which the material depicted in the image is of that type. The above learning model can be generated by conventional machine learning using learning images in which the type of material depicted in the image is known in advance.

[0057] The type discrimination unit 14 stores a learning model in advance. The type discrimination unit 14 inputs an image used for discriminating the type of material to the learning model, and primarily discriminates the type of the lifted material based on the image. The type discrimination unit 14 inputs the image to the learning model, and obtains a certainty factor for each type of material for the image as an output from the learning model. The type discrimination unit 14 determines the type with the highest certainty factor among the certainty factors for each type output from the learning model as the type of material discriminated from the image. Figure 7 shows examples of images for each type of material (deck, steel column, staircase, steel girder, steel beam, exterior material, and others). The graph on the right side of Figure 5 shows an example of the certainty factor for each type. The solid line in the graph on the right side of Figure 5 indicates the certainty factor for each type calculated from each image.

[0058] The type discrimination unit 14 performs the above primary discrimination for all images taken during the same lifting time period (images taken during five times in the example shown in FIG. 5). The type discrimination unit 14 determines the type that is most frequently identified among the number of images as the type of material lifted during that lifting time period. That is, the type discrimination unit 14 finally determines the type of material by majority vote. For example, as in the graph on the right side of FIG. 5, if the material is most frequently identified as a steel beam by the primary type discrimination of each image, it is finally determined to be a steel beam. If multiple cameras 30 are provided and multiple images are included in one time period, a temporary type discrimination can be performed from each image. That is, a temporary type discrimination can be performed for the number of images.

[0059] By identifying the type of lifted material from multiple images, even if the appropriate type cannot be identified from a single image, the type of material can be identified appropriately. In other words, the influence of the appearance of the primary image is reduced, and image recognition accuracy can be improved.

[0060] Also, the type discrimination unit 14 may discriminate the type of material without performing primary discrimination. For example, the type discrimination unit 14 may calculate a discrimination score for each type of material from the certainty factor for each type of lifted material for one lifting time period using a preset calculation formula, and discriminate the type of the lifted material based on the calculated discrimination score. Specifically, the average of the calculated certainty factors for each type may be used as the discrimination score. The graph on the right side of FIG. 5 shows an example of the average certainty factor for each type. The dashed line in the graph on the right side of FIG. 5 is the average certainty factor for each type calculated from each image. As shown in the graph on the right side of FIG. 5, with only one image, the material is discriminated as the wrong type, which is "other type," but by taking the average of the certainty factors, the material can be discriminated as the correct type, which is the steel beam type.

[0061] Alternatively, the highest certainty for each image may be added up for each type to determine the discrimination score. The type discrimination unit 14 determines the type with the highest discrimination score as the type of material lifted during the lifting time period.

[0062] The type discrimination unit 14 may also discriminate the type of material lifted during a lifting time period by using a learning model that inputs all of a plurality of images related to one lifting time period and outputs one piece of information related to the type of material in response to the input of the plurality of images. The learning model is, for example, a convolutional LSTM (Long Short Term Memory). In this case, the output from the learning model is, for example, a confidence level, which is a score for each type, similar to the above-mentioned learning model. The type discrimination unit 14 determines the type with the highest confidence level as the type of material lifted during the lifting time period. This type of discrimination enables discrimination based on the overall characteristics of the plurality of images (discrimination taking into account characteristics commonly included in the plurality of images or movements (such as rotation in the air) seen in a specific material), and as a result, information indicating the lifting performance of the crane 100 can be more appropriately acquired.

[0063] The type discrimination unit 14 outputs the information indicating the lifting time zone input from the time zone specification unit 12 and the information indicating the type of material lifted during the determined lifting time zone as information indicating the lifting performance of the crane 100 (for example, the information shown in FIG. 3). The type discrimination unit 14 may also input information indicating the class at each time of each lifting time zone from the time zone specification unit 12 and output this information as information indicating the lifting performance of the crane 100. This information makes it possible to grasp not only the presence or absence of lifting, but also more detailed performance including the time of the breakdown of work related to lifting, such as "rigging and lifting off the ground," "lifting," "attaching and removing the rigging," and "lifting return."

[0064] For example, the type discrimination unit 14 transmits information indicating the lifting performance to a terminal of a user or the like (for example, a terminal in an office used by a construction manager). Alternatively, the type discrimination unit 14 causes the information to be displayed on a display device provided in the server device 10. The display of the information indicating the lifting performance may be a visualization of what material was lifted and when in chronological order. In addition, statistics such as the number of lifts per hour for each type of material may also be displayed. The user can understand the lifting performance of the crane 100 by referring to the output information.

[0065] The discrimination by the type discrimination unit 14 and the information showing the lifting performance of the crane 100 based on the discrimination may be performed at a preset timing. For example, the type discrimination unit 14 updates the information to the latest information every 30 minutes and outputs it. Also, each functional unit performs processing at an appropriate timing so that such output can be performed. Also, the type discrimination unit 14 may discriminate the type of material by a method other than the above, so long as it uses images taken at multiple times included in the lifting time period. The above are the functions of the server device 10.

[0066] Next, the process executed by the lifting record acquisition system 1 according to this embodiment will be described with reference to the flowchart in Fig. 8. In this process, the positioning meter 20 detects the altitude related to the lifting by the crane 100 (S01). In addition, the camera 30 captures an image of the material lifted by the crane 100 (S02). Note that the detection by the positioning meter 20 (S01) and the capture of the image by the camera 30 (S02) are performed continuously during the time period in which information indicating the lifting record of the crane 100 is to be acquired.

[0067] In the server device 10, the altitude information acquisition unit 11 acquires altitude information indicating the time-series altitude related to the lifting of the crane 100, which is the detection result by the positioning meter 20 (S03). Next, the time zone identification unit 12 identifies at least a part of the lifting time zone during which the material was lifted by the crane 100 (the "lifting" time zone in the above example) for each lifted material based on the altitude information (S04). Next, the image acquisition unit 13 acquires multiple images of the lifted material taken by the camera 30 at multiple timings included in the time zone identified by the time zone identification unit 12 for each lifting time zone (S05).

[0068] Next, the type discrimination unit 14 discriminates the type of material lifted for each lifting time period from the multiple images (S06). Next, the type discrimination unit 14 outputs information indicating the lifting performance of the crane 100, including the discrimination result (S07). The above is the process executed by the lifting performance acquisition system 1 according to this embodiment.

[0069] In this embodiment, the type of material is determined from a plurality of images of the material lifted during at least a part of the lifting time period (the "lifting" time period in the above example). This allows the type of material to be properly determined even when the appropriate type cannot be determined from one image as described above (for example, when the material is temporarily hidden by a jig or when the attitude of the material is difficult to determine). By using images at a plurality of timings, images that improve the above-mentioned difficult-to-determine situation, such as when the material rotates and becomes easier to see, can be used for determination. As a result, according to this embodiment, information indicating the lifting performance of the crane 100 can be properly acquired. Furthermore, according to this embodiment, information indicating the lifting performance can be acquired automatically and rationally, and the labor required for visual recording by an operator can be reduced. In other words, according to this embodiment, the labor required for acquiring information indicating the lifting performance can be reduced. Therefore, this embodiment can contribute to improving the efficiency of the lifting work by the crane 100.

[0070] Also, as in this embodiment, a time period during which materials are being transported in the vertical direction by the lifting of the crane 100 (the "lifting" time period in the above example) may be specified as at least a part of the lifting time period. With this configuration, the multiple images used for the type of material can be made to more reliably show the materials. In other words, the multiple images used for the type of material can be made appropriate. As a result, information showing the lifting performance of the crane 100 can be obtained appropriately and reliably. However, the specified time period does not have to be the above, and may be any time period that is at least a part of the lifting time period and that allows for the acquisition of images that can be used to determine the type of lifted material.

[0071] Also, as in this embodiment, multiple images of the lifted materials taken at multiple timings during at least a part of the lifting time period when the altitude is equal to or higher than a preset altitude may be used to determine the type of material. Also, multiple images of the lifted materials taken at multiple timings at a preset number of equal intervals may be used to determine the type of material. With these configurations, the multiple images used for the type of material can be made appropriate. As a result, information indicating the lifting performance of the crane 100 can be obtained appropriately and reliably. However, the timing of the images does not have to be as described above, and may be multiple timings included in the lifting time period.

[0072] Also, as in this embodiment, the type of the lifted material may be primarily determined from each of the multiple images, and the type of the lifted material may be determined based on the multiple determination results from the primary determination. For example, the type of material may be finally determined by majority vote as described above. With this configuration, the type of material can be properly and reliably determined. As a result, information indicating the lifting performance of the crane 100 can be properly and reliably obtained. However, in determining the type of the lifted material, it is not necessarily necessary to perform a temporary determination for each image.

[0073] Also, as in this embodiment, a score for each type of lifted material may be calculated from each of the multiple images, a discrimination score for each type of material may be calculated from the multiple calculated scores for each type of material using a preset calculation formula, and the type of lifted material may be discriminated based on the calculated discrimination score. With this configuration, the type of material can be discriminated more appropriately. As a result, information indicating the lifting performance of the crane 100 can be obtained more appropriately. However, the type of lifted material does not necessarily have to be discriminated as described above.

[0074] Also, as in this embodiment, a portion of the image showing the hook 102 may be detected, a portion corresponding to the detected portion may be cut out from the image, and the type of the lifted material may be determined from the cut-out image. With this configuration, the type of material may be determined more appropriately. As a result, information showing the lifting performance of the crane 100 may be obtained more appropriately. For example, even if the image shows something other than the material to be determined, the type of material may be determined appropriately. However, in cases where the type of material can be determined appropriately without cutting out, the above configuration is not necessarily required.

[0075] Also, as in this embodiment, the lifting performance acquisition system 1 may further include a positioning meter 20, which is a detection means for detecting the time-series altitude related to the lifting of the crane, and is provided at a position between the upper part of the hook block 103 of the crane 100 and the two slings 104 supporting the hook block 103, and acquire altitude information detected by the positioning meter 20. With this configuration, altitude information can be acquired reliably and appropriately. Also, damage to the positioning meter 20 during lifting, etc. can be prevented. However, the positioning meter 20 does not necessarily have to have the above configuration.

[0076] Furthermore, as in this embodiment, the lifting record acquisition system 1 may further include a camera 30, which is an imaging means provided between two plate-like members 103a of the hook block 103 of the crane 100, and the image acquired by the imaging of the camera 30 may be used to determine the type of material. This configuration can ensure that images can be acquired. Also, damage to the camera 30 during lifting, etc. can be prevented. However, the camera 30 does not necessarily have to have the above configuration.

[0077] In this embodiment, the lifting record acquisition system 1 includes the server device 10, the positioning meter 20, and the camera 30, but the lifting record acquisition system 1 may be composed of only the server device 10. In that case, the lifting record acquisition system 1 may acquire the above-mentioned necessary data from the positioning meter and the camera that are not included in the lifting record acquisition system 1.

[0078] The lifting performance acquisition system of the present disclosure has the following configuration. [1] A lifting performance acquisition system for acquiring information indicating the lifting performance of a crane that lifts materials, An altitude information acquiring means for acquiring altitude information indicating a time series of altitudes related to the lifting of the crane; a time zone specification means for specifying, for each lifted material, at least a portion of a lifting time zone during which the material was lifted by the crane based on the altitude information acquired by the altitude information acquisition means; an image acquisition means for acquiring a plurality of images of the lifted material taken at a plurality of timings included in at least a portion of the lifting time period identified by the time period identification means for each of the lifting time periods; A type discrimination means for discriminating the type of the material lifted for each lifting time period from the plurality of images acquired by the image acquisition means; A lifting performance acquisition system equipped with [2] The lifting performance acquisition system described in [1], wherein the time period identification means identifies, as at least a part of the lifting time period, a time period during which materials are being transported in a vertical direction by lifting with a crane. [3] The lifting performance acquisition system described in [1] or [2], wherein the image acquisition means acquires multiple images of the lifted materials taken at multiple times during at least a portion of the lifting time period during which the altitude indicated by the altitude information is equal to or higher than a predetermined altitude. [4] A lifting performance acquisition system described in any of [1] to [3], wherein the image acquisition means acquires multiple images of the lifted material taken at multiple timings at equal intervals of a preset number. [5] A lifting performance acquisition system described in any of [1] to [4], wherein the type discrimination means primarily discriminates the type of the lifted material from each of the multiple images, and discriminates the type of the lifted material based on the multiple discrimination results from the primary discrimination. [6] A lifting performance acquisition system described in any of [1] to [5], wherein the type discrimination means calculates a score for each type of material lifted from each of the multiple images acquired by the image acquisition means, calculates a discrimination score for each type of material from the multiple scores calculated for each type of material using a predetermined calculation formula, and discriminates the type of material lifted based on the calculated discrimination score. [7] A lifting performance acquisition system described in any of [1] to [6], wherein the type discrimination means detects a portion of the crane hook in the multiple images acquired by the image acquisition means, cuts out a portion from the image corresponding to the detected portion, and discriminates the type of material lifted from the cut-out image. [8] The crane further includes a detection means for detecting the time-series altitude related to the lifting of the crane, the detection means being provided at the upper part of the hook block supporting the hook of the crane and at a position sandwiched between the two slings supporting the hook block; The lifting performance acquisition system according to any one of [1] to [7], wherein the altitude information acquisition means acquires altitude information detected by the detection means. [9] The hook block further includes an imaging means provided between two plate-like members that support the hook of the crane and whose main surfaces face each other with a gap therebetween, The lifting performance acquisition system according to any one of [1] to [8], wherein the image acquisition means acquires an image captured by the imaging means. [Explanation of symbols]

[0079] 1...lifting performance acquisition system, 10...server device, 11...altitude information acquisition unit, 12...time zone identification unit, 13...image acquisition unit, 14...type discrimination unit, 20...positioning meter, 30...camera, 100...crane, 101...boom, 102...hook, 103...hook block, 104...sling.

Claims

1. A lifting performance acquisition system that acquires information indicating the lifting performance of a crane that lifts materials, An altitude information acquiring means for acquiring altitude information indicating a time series of altitudes related to the lifting of the crane; a time zone specification means for specifying, for each lifted material, at least a portion of a lifting time zone during which the material was lifted by the crane based on the altitude information acquired by the altitude information acquisition means; an image acquisition means for acquiring a plurality of images of the lifted material taken at a plurality of timings included in at least a portion of the lifting time period identified by the time period identification means for each of the lifting time periods; A type discrimination means for discriminating the type of the material lifted for each lifting time period from the plurality of images acquired by the image acquisition means; A lifting performance acquisition system equipped with

2. 2. The lifting performance acquisition system according to claim 1, wherein the time period specification means specifies, as at least a part of the lifting time period, a time period during which materials are being transported in an elevation direction by lifting with a crane.

3. The lifting performance acquisition system described in claim 1 or 2, wherein the image acquisition means acquires multiple images of the lifted materials taken at multiple times during at least a portion of the lifting time period during which the altitude indicated by the altitude information is equal to or higher than a predetermined altitude.

4. 3. The lifting record acquisition system according to claim 1 or 2, wherein the image acquisition means acquires a plurality of images of the lifted material taken at a preset number of equal intervals.

5. The lifting performance acquisition system described in claim 1 or 2, wherein the type discrimination means primarily discriminates the type of the lifted material from each of the multiple images, and discriminates the type of the lifted material based on the multiple discrimination results from the primary discrimination.

6. The lifting performance acquisition system described in claim 1 or 2, wherein the type discrimination means calculates a score for each type of material for the lifted materials from each of the multiple images acquired by the image acquisition means, calculates a discrimination score for each type of material from the multiple scores calculated for each type of material using a predetermined calculation formula, and discriminates the type of the lifted materials based on the calculated discrimination score.

7. The lifting performance acquisition system described in claim 1 or 2, wherein the type discrimination means detects a portion of the multiple images acquired by the image acquisition means that includes a crane hook, cuts out a portion of the image corresponding to the detected portion, and discriminates the type of material lifted from the cut-out image.

8. The crane further includes a detection means for detecting a time-series altitude related to the lifting of the crane, the detection means being provided at an upper portion of a hook block supporting the hook of the crane and at a position sandwiched between two slings supporting the hook block, The lifting performance acquisition system according to claim 1 or 2, wherein the altitude information acquisition means acquires altitude information detected by the detection means.

9. The hook block further includes an imaging means provided between two plate-like members that support the hook of the crane and whose main surfaces face each other with a gap therebetween, The lifting performance acquisition system according to claim 1 or 2, wherein the image acquisition means acquires an image captured by the imaging means.

Images