Suspended load recognition system, suspended load recognition method, and suspended load recognition program

Abstract

This system provides a load recognition system that allows for easy monitoring of the status of suspended loads during crane operations. [Solution] The suspended load recognition system 1 for recognizing a suspended load C from a crane 100 comprises an imaging device 2 provided at the tip of the jib 104 of the crane 100 with an imaging range directed downward, and a control device 3 that calculates the outer shape of the suspended load C based on an image 10 captured by the imaging device 2. The suspended load C is suspended from the crane 100 via a hook 108. The control device 3 recognizes the hook 108 from the image 10 and calculates the outer shape of the suspended load C based on the hook 108 recognized in the image 10.

Description

Technical Field

[0001] The present disclosure relates to a suspended load recognition system, a suspended load recognition method, and a suspended load recognition program.

Background Art

[0002] Conventionally, in crane operations, a method is known in which an imaging device is provided at the tip of the boom of a crane, and an image of the crane's working area is taken from above by this imaging device to grasp the situation of the working area (for example, Patent Document 1).

Prior Art Documents

Patent Documents

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

Means for Solving the Problems

[0007] According to this disclosure, it is possible to provide a suspended load recognition system, a suspended load recognition method, and a suspended load recognition program that can easily grasp the status of a suspended load during crane operation. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows the schematic configuration of a crane to which the load recognition system according to the embodiment is applied. [Figure 2] Functional block diagram of the suspended load recognition system according to this embodiment [Figure 3] Flowchart for the derivation of the proximity warning area by the suspended load recognition system. [Figure 4] Flowchart of the subroutine processing in step S105 of Figure 3 [Figure 5] Flowchart of worker approach detection control implemented by the suspended load recognition system [Figure 6] This figure shows an example of an image of the work area of ​​a crane operation taken by an imaging device. [Figure 7] Figure 6 shows an example of various regions derived from the image illustrated in the diagram. [Figure 8] Figure 6 shows an example of the helmet region derived in the image provided as an example. [Figure 9] Diagram illustrating the overview of worker approach detection and control. [Figure 10] Hardware configuration diagram of the control unit for the suspended load recognition system. [Modes for carrying out the invention]

[0009] The embodiments will be described below with reference to the attached drawings. To facilitate understanding of the explanation, the same reference numerals are used for identical components in each drawing whenever possible, and redundant explanations are omitted.

[0010] Figure 1 is a diagram showing the schematic configuration of a crane 100 to which the suspended load recognition system 1 according to the embodiment is applied. In Figure 1, the vertical direction corresponds to the vertical direction of the working area of ​​the crane operation. In Figure 1, the left-right direction corresponds to the horizontal direction of the working area of ​​the crane operation.

[0011] The suspended load recognition system 1 according to this embodiment is a system for recognizing a suspended load C that is suspended by the crane 100 during operation of the crane 100, and for understanding the status of the suspended load C.

[0012] Figure 1 illustrates a mobile crane as an example of the crane 100. The crane 100 according to this embodiment is a device that can move the suspended load to any height and any horizontal position, and includes, for example, gantry cranes, fixed cranes installed in ports and factories, and cranes mounted on crane ships.

[0013] The mobile crane 100 illustrated in Figure 1 comprises an upper slewing body 101 and a lower traveling body 102. The lower traveling body 102 is a self-propelled element having a traveling device such as tracks. The upper slewing body 101 is installed on top of the lower traveling body 102 and is an element that can rotate mainly around a slewing axis aligned with the vertical direction.

[0014] On the upper slewing body 101, a cab 103 for an operator to operate the crane and a boom 104 are installed. The boom 104 is a longitudinal member that is connected to the upper slewing body 101 so as to be rotatable in the vertical direction. The base of the boom 104 is connected to the upper slewing body 101, and a rotation axis extending in the horizontal direction is provided at the connection portion, and it is rotatable about this rotation axis. Thereby, the boom 104 rotates around the rotation axis with the base as the center, and the tip portion is arranged to be movable in the vertical direction.

[0015] Further, the crane 100 has a first hoisting device 105 and a second hoisting device 106. Generally, two hoisting devices are provided in a mobile crane, and the specifications such as the rated load and the hoisting speed are different between the two hoisting devices. Depending on the load of the suspended load and the working range, they are used separately, or by simultaneous operation, it is possible to stably handle a large suspended load.

[0016] In addition, in FIG. 1, the symbol 105 representing the first hoisting device is attached to the first sheave installed at the tip portion of the boom 104. However, the actual structure of the first hoisting device 105 includes a first drum around which the first wire 107 is wound, and a first winch that generates a driving force for rotating the first drum, which are provided on the upper slewing body 101 of the crane 100 and the like. Further, the first wire 107 drawn out from the first drum extends from the base side to the tip side of the boom 104 along the extending direction of the boom 104, is wound around the first sheave, and is hung downward. A first hook 108 is connected to the tip (lower end) of the first wire 107, and a suspended load C or the like is suspended from the first hook 108 via a rope, a wire, or the like.

[0017] Similarly, in FIG. 1, a reference numeral 106 representing the second hoisting device is attached to the second sheave installed at the tip of the jib 104. However, the actual structure of the second hoisting device 106 includes a second drum around which the second wire 109 is wound, and a second winch that generates a driving force for rotating the second drum, which are provided on the upper slewing body 101 of the crane 100 or the like. Further, the second wire 109 drawn from the second drum extends from the base side to the tip side of the jib 104 along the extending direction of the jib 104, is wound around the second sheave, and is suspended downward. A second hook 110 is connected to the tip (lower end) of the second wire 109, and a suspended load C or the like is suspended from the second hook 110 via a rope, a wire, or the like.

[0018] In the example of FIG. 1, the second sheave of the second hoisting device 106 is arranged on the more tip side in the extending direction of the jib 104 than the first sheave of the first hoisting device 105. However, the positional relationship between the first hoisting device 105 and the second hoisting device 106 may be configured differently from the example of FIG. 1.

[0019] In the example of FIG. 1, a case where the handling of the suspended load C is performed using the first hoisting device 105 is illustrated. In this case, the hoisting device to be recognized by the suspended load recognition system 1 according to the present embodiment is the first hoisting device 105. Therefore, in the following description, the description will be focused on the first hoisting device 105 among the two hoisting devices, and the "first wire 107" of the first hoisting device 105 may be simply referred to as "wire 107", and the "first hook 108" may be simply referred to as "hook 108".

[0020] As shown in FIG. 1, the suspended load recognition system 1 according to the present embodiment includes an imaging device 2 and a control device 3.

[0021] The imaging device 2 includes, for example, a camera, a CCD, or the like. The imaging device 2 is provided at the tip of the jib 104 of the crane 100 with the imaging range facing downward. The imaging device 2 outputs the information of the image 10 (see, for example, FIG. 6) acquired by imaging to the control device 3.

[0022] The control device 3 calculates the outline of the suspended load C based on the image 10 captured by the imaging device 2. As described above, the suspended load C is suspended from the jib 104 of the crane 100 via a hook 108. In this embodiment in particular, the control device 3 recognizes the hook 108 from the image 10 and calculates the outline of the suspended load C based on the hook 108 recognized in the image 10. When recognizing the hook 108, a method for detecting the region containing the hook 108 in the image 10 may be applied, or a method for detecting the outline of the hook 108 may be applied.

[0023] More specifically, the control device 3 can recognize areas with the same color tone around the hook 108 detected in the image 10 captured by the imaging device 2 as areas that form the outline of the suspended load C. Alternatively, a method may be applied to recognize "areas in which feature points in the captured image 10 move in the same way as the hook 108" as areas that form the outline of the suspended load C. For example, when the upper slewing body 101 of the crane 100 rotates, the lower traveling body 102 and objects placed on the ground surface G, such as traffic cones (registered trademark), rotate relative to the upper slewing body 101, wire 107, and hook 108 in the image 10. The aforementioned "areas in which feature points in the captured image 10 move in the same way as the hook 108" excludes such lower traveling body 102 and ground surface objects, and includes areas that do not move relative to the hook 108.

[0024] Furthermore, when the control device 3 detects the hook 108 from the image 10, it can, for example, detect the wire 107 hanging down from the tip of the jib 104 in the image 10, and based on the wire 107 detected in the image 10, detect the hook 108 attached to the tip (lower end) of the wire 107.

[0025] Next, referring to Figure 2 and subsequent figures, the functions of the suspended load recognition system 1 according to this embodiment will be further described.

[0026] Figure 2 is a functional block diagram of the suspended load recognition system 1 according to the embodiment. As shown in Figure 2, the control device 3 of the suspended load recognition system 1 includes, as elements related to the various functions described above, an image capture control unit 31, a wire vector calculation unit 32, a hook area detection unit 33, a suspended load shape detection unit 34, a suspended load height calculation unit 35, a suspended load projection shape detection unit 36, an approach warning area calculation unit 37, and a worker approach detection unit 38.

[0027] The imaging control unit 31 controls the operation of the imaging device 2. The imaging control unit 31 operates the imaging device 2 to acquire an image 10 of the work area of ​​the crane operation. The imaging control unit 31 outputs the information of the acquired image 10 to the wire vector calculation unit 32, hook area detection unit 33, suspended load shape detection unit 34, suspended load height calculation unit 35, suspended load projection shape detection unit 36, etc.

[0028] The wire vector calculation unit 32 calculates the wire vector 11 (see Figure 7) within the image 10. Here, "wire vector 11" refers to information including the position, length, and direction of the wire 107 within the image 10. The wire vector calculation unit 32 outputs the calculated wire vector 11 information to the hook region detection unit 33, the suspended load projection shape detection unit 36, and the like.

[0029] The hook region detection unit 33 detects the region surrounding the outline of the hook 108 in the image 10 (hereinafter referred to as the "hook region 12"; see Figure 7, etc.) based on the image 10 and the wire vector 11. Note that other detection methods may be applied to determine the hook region 12, such as detecting a region that includes at least a part of the hook 108 in the image 10, or detecting a region that follows the outline of the hook 108. The hook region detection unit 33 outputs the detected hook region 12 information to the suspended load shape detection unit 34, the suspended load height calculation unit 35, etc.

[0030] The suspended load shape detection unit 34 detects the area surrounding the outer shape of the suspended load C in image 10 (hereinafter referred to as the "suspended load area 13"; see Figure 7, etc.) based on image 10 and the hook area 12. The suspended load shape detection unit 34 outputs the information of the detected suspended load area 13 to the suspended load projection shape detection unit 36.

[0031] The suspended load height calculation unit 35 calculates the height T3 (see Figure 1) of the suspended load C from the ground G (see Figure 1) based on the image 10 and the hook region 12. The suspended load height calculation unit 35 outputs the calculated information of the height T3 of the suspended load C to the suspended load projection shape detection unit 36.

[0032] The suspended load projection shape detection unit 36 ​​calculates the area surrounding the outline of the projection F (see Figure 6, etc.) of the suspended load C formed on the ground G (hereinafter referred to as the "projection area 14"; see Figure 7, etc.) based on the image 10, the wire vector 11, the suspended load area 13, and the height T3 of the suspended load C (see Figure 1). The suspended load projection shape detection unit 36 ​​outputs the calculated projection area 14 information to the proximity warning area calculation unit 37.

[0033] The proximity warning area calculation unit 37 calculates, based on the projection area 14, the area where workers P1 and P2 (see Figure 1) on the ground G are deemed to be too close to the suspended load C and therefore in danger (hereinafter referred to as the "proximity warning area 15"; see Figure 7, etc.). The proximity warning area calculation unit 37 outputs the calculated proximity warning area 15 information to the worker proximity detection unit 38.

[0034] The worker proximity detection unit 38 detects the intrusion of workers P1 and P2 into the proximity warning area 15 based on the proximity warning area 15. When the worker proximity detection unit 38 detects intrusion, it can notify the intruded workers or the worker operating the crane 100 that intrusion into the proximity warning area 15 has been detected, using, for example, sound, light, or text installed on or around the crane 100.

[0035] Figure 3 is a flowchart of the proximity warning area derivation control performed by the suspended load recognition system 1.

[0036] In step S101, the imaging control unit 31 activates the imaging device 2 to acquire an image 10 of the working area of ​​the crane operation.

[0037] Figure 6 shows an example of an image 10 of the work area during crane operation, captured by the imaging device 2. As described above, the imaging device 2 is mounted at the tip of the jib 104 of the crane 100 with its imaging range facing downwards, so the captured image 10 includes the first wire 107, first hook 108, second wire 109, and second hook 110 that are suspended downwards from the tip of the jib 104. Also, because the crane is in operation, one of the two hoisting devices (the first hoisting device 105 in the examples of Figures 1 and 6) is transporting the suspended load C. Therefore, the image 10 includes the shape of the suspended load C, which is suspended from the first wire 107 by the first hook 108, as viewed from the position of the imaging device 2 above. In the example of Figure 6, since the suspended load C is positioned vertically below the optical axis of the imaging device 2, only the top surface of the suspended load C is included in the image 10.

[0038] Furthermore, in the examples of Figures 1 and 6, the suspended load C is lifted to an arbitrary height above the ground G and suspended from the crane 100. Therefore, Image 10 also includes the projection F of the suspended load C onto the ground G. In this embodiment, "projection F" refers to the shape of the suspended load C formed when the outer edge of the suspended load C, viewed from vertically above, is moved onto the ground G vertically below. Projection F can also be described as the outer shape of the portion of the suspended load C that covers the ground G when the suspended load C is lowered onto the ground G while maintaining the posture of the upper slewing body 101 and jib 104 of the crane 100.

[0039] Furthermore, image 10 also includes workers P1 and P2 who are within the work area. At this time, since the imaging device 2 is definitely capturing images of the work area from above, above workers P1 and P2, image 10 also includes images of the helmets H1 and H2 that workers P1 and P2 are wearing on their heads.

[0040] Returning to Figure 3, in step S102, the wire vector calculation unit 32 calculates the wire vector 11 based on the image 10 acquired in step S101.

[0041] Figure 7 shows an example of various regions 11 to 15 derived in the image 10 illustrated in Figure 6. The wire vector 11 is information including the position, length, and direction of the wire from which the suspended load C is suspended. In the examples of Figures 1, 6, and 7, the suspended load C is suspended from the first wire 107, so the wire vector calculation unit 32 calculates the wire vector 11 of the first wire 107. In Figure 7, the region of the wire vector 11 extracted from the image 10 by the wire vector calculation unit 32 is enclosed by a dotted line.

[0042] In step S102, the wire vector calculation unit 32 can detect the wire 107 being used to suspend the load C using any machine learning model, such as a pre-trained image recognition neural network, and derive the wire vector 11.

[0043] Returning to Figure 3, in step S103, the hook region detection unit 33 detects the hook 108 being used to suspend the load C, based on the image 10 acquired in step S101 and the wire vector 11 derived in step S102. The hook region detection unit 33 derives the area surrounding the outline of the hook in use (first hook 108) as the hook region 12, for example, as shown by the dotted circle in Figure 7.

[0044] In step S103, the hook region detection unit 33 can derive the hook region 12 of the hook 108 used to suspend the load C in the vicinity of the wire vector 11 shown in Figure 7 (particularly preferably the vicinity of the lower end of the wire vector 11) using any machine learning model, such as a pre-trained image recognition neural network.

[0045] In step S104, the suspended load shape detection unit 34 detects the shape of the suspended load C suspended by the crane 100 based on the image 10 acquired in step S101 and the hook region 12 derived in step S103. In the examples of Figures 1, 6, and 7, the shape of the top surface of the suspended load C is rectangular. Therefore, the suspended load shape detection unit 34 derives the area surrounding the outer shape of the suspended load C as the suspended load region 13, for example, as shown by the dotted rectangle in Figure 7.

[0046] In step S104, the suspended load shape detection unit 34 can derive the shape of the suspended load C as the suspended load region 13, using an arbitrary machine learning model, such as a pre-trained image recognition neural network, centered on the hook region 12 detected in step S103.

[0047] In step S105, the load height calculation unit 35 calculates the height T3 of the load C from the ground G. In this step, the subroutine processing shown in Figure 4 is performed. Figure 4 is a flowchart of the subroutine processing in step S105 of Figure 3.

[0048] In the subroutine processing, first in step S201, the helmets H1 and H2 of workers P1 and P2 are detected. Based on the image 10 acquired in step S101, the suspension load height calculation unit 35 derives the area surrounding each helmet H1 and H2 as the helmet area 16.

[0049] Figure 8 shows an example of the helmet region 16 derived in image 10 illustrated in Figure 6. The helmet region 16 is the area surrounding the outlines of the helmets H1 and H2 worn by workers P1 and P2, respectively, in image 10. In Figure 8, the helmet region 16 corresponding to each helmet H1 and H2 is illustrated as a dotted square with helmets H1 and H2 as its center.

[0050] In step S201, the suspended load height calculation unit 35 can derive helmet regions 16 corresponding to each helmet H1 and H2 using any machine learning model, such as a pre-trained image recognition neural network.

[0051] In step S201, instead of using a machine learning model, any other method may be used, such as using an existing image recognition model to detect workers P1 and P2 and then performing image recognition on helmets H1 and H2 in their vicinity.

[0052] Returning to Figure 4, in step S202, the height T1 (see Figure 1) of the imaging device 2 is calculated based on the size of the helmet region 16 detected in step S201. The suspended load height calculation unit 35 can calculate the current height T1 of the imaging device 2 using any method, such as comparing it with a reference value when the imaging device 2 is directly above the helmets H1 and H2, using information such as the number of pixels of the helmet region 16 on the image 10.

[0053] In step S202, as shown in Figure 8, a reference marker 4 may be placed at a reference height such as the ground G, and the height T1 of the imaging device 2 may be calculated based on the size of the reference marker 4 instead of the helmet area 16. In this case, the height T1 of the imaging device 2 can be calculated using any method, such as comparing it with a reference value when the imaging device 2 is directly above the reference marker 4, based on, for example, the number of pixels on the image 10 of one reference marker 4, or the number of pixels on the image 10 between two reference markers 4.

[0054] In step S203, the height T2 of the hook 108 from the ground G (see Figure 1) is calculated based on the height T1 of the imaging device 2 calculated in step S202 and the size of the hook region 12 detected in step S103. The suspended load height calculation unit 35 derives the number of pixels when the hook 108 is on the ground G, i.e., the minimum number of pixels in the hook region 12 according to the height T1 of the imaging device 2, based on the height T1 of the imaging device 2 calculated in step S202. This minimum value can be derived such that it becomes smaller as the height T1 of the imaging device 2 increases and larger as it decreases. Then, the height T2 of the hook 108 can be calculated by comparing it with the minimum value, for example, by finding the difference between the number of pixels in the hook region 12 on the image 10 and the minimum value.

[0055] In step S204, the height T3 of the suspended load C (see Figure 1) is calculated based on the height T2 of the hook 108 calculated in step S203. The suspended load height calculation unit 35 uses the height T2 of the hook 108 to convert it to the height T3 of the suspended load C from the ground G. The suspended load height calculation unit 35 is provided in advance with information such as the height dimension of the suspended load C and the vertical distance between the hook 108 and the suspended load C from the worker or system administrator, and can derive the height T3 of the suspended load C from the height T2 of the hook 108 using these given dimensional values. When the processing of step S204 is completed, the process returns to the main flow in Figure 3.

[0056] Returning to Figure 3, in step S106, the suspended load projection shape detection unit 36 ​​detects the projected shape of the suspended load C based on the image 10 acquired in step S101, the wire vector 11 derived in step S102, the suspended load region 13 derived in step S104, and the height T3 of the suspended load C calculated in step S105. In the examples of Figures 1, 6, and 7, the shape of the top surface of the suspended load C is rectangular, and the side of the suspended load C is not visible in image 10. Therefore, it is considered that the bottom surface of the suspended load C is at least the same as the top surface, or a similar rectangular shape with a smaller area than the top surface, or at least within the outer shape of the top surface and a shape other than a rectangle that is different from the top surface. In any case, it is considered that the shape of the projection F of the suspended load C formed on the ground G will be a rectangular shape similar to the top surface of the suspended load C. Therefore, the suspended load projection shape detection unit 36 ​​derives the region surrounding the outline of the projection F of the suspended load C as the projection region 14, for example, as shown by the dotted rectangle in Figure 7.

[0057] Unlike the examples in Figures 1, 6, and 7, if the side of the suspended load C is also visible in Image 10, it is highly likely that the bottom surface of the suspended load C is larger than the top surface. In this case, in this step, the shape of the projection F is derived based on the outline of the suspended load C in Image 10, including the top and side surfaces of the suspended load C.

[0058] In step S106, the suspended load projection shape detection unit 36 ​​can derive the projection F of the suspended load C formed on the ground G by moving a figure identical in shape to the suspended load region 13 to a position on the image 10 that is a height T3 lower from the position of the suspended load region 13 along the direction of the wire vector 11, i.e., on the ground G. Then, the region surrounding the outline of the projection F thus derived can be derived as the projection region 14.

[0059] In step S107, the proximity warning area calculation unit 37 calculates the proximity warning area 15 based on the projection area 14 derived in step S106. The proximity warning area calculation unit 37 can form the proximity warning area 15 by taking a predetermined distance outside the projection area 14, for example, as shown by the dotted ellipse in Figure 7. This predetermined distance can be determined by using safety distance information provided in advance by workers, system administrators, etc. When the processing in step S107 is completed, this control flow ends.

[0060] Figure 5 is a flowchart of the worker approach detection control performed by the suspended load recognition system 1. Each process in the flowchart of Figure 5 is performed by the worker approach detection unit 38 of the control device 3. Furthermore, as a prerequisite for performing the flowchart of Figure 5, the approach warning area derivation control shown in Figure 3 has already been performed, and the worker approach detection unit 38 has already obtained information on the approach warning area 15 from the approach warning area calculation unit 37.

[0061] In step S301, workers P1 and P2 are detected in image 10. The worker proximity detection unit 38 can derive worker regions 17 surrounding workers P1 and P2, for example, by using an existing image recognition model. Figure 9 is a diagram illustrating the overview of worker proximity detection control. In the example in Figure 9, worker regions 17 are formed in a rectangular shape surrounding workers P1 and P2.

[0062] Returning to Figure 5, in step S302, the proximity warning area 15 in image 10 is compared with the worker area 17 derived in step S301.

[0063] In step S303, it is determined whether or not a worker has entered the proximity warning area 15. If no worker has entered (NO in step S303), the process returns to step S301.

[0064] On the other hand, if it is determined that an intrusion has occurred (YES in step S303), the process proceeds to step S304, where a notification of worker intrusion is issued. When the worker proximity detection unit 38 detects an intrusion, it can use any element, such as sound, light, or text, to notify the intruded worker, the worker operating the crane 100, or the administrator of the suspended load recognition system 1 that an intrusion into the proximity warning area 15 has been detected. Once the processing in step S304 is complete, this control flow ends, and the process returns to step S101 in Figure 3.

[0065] In the example shown in Figure 9, a portion of worker P1's worker area 17 overlaps with the proximity warning area 15. Therefore, the worker proximity detection unit 38 determines that worker P1 has entered the proximity warning area 15 and issues a notification to that effect. On the other hand, worker P2's worker area 17 is separate from the proximity warning area 15, so the worker proximity detection unit 38 determines that worker P2 has not entered the proximity warning area 15.

[0066] Figure 10 is a hardware configuration diagram of the control device 3 of the suspended load recognition system 1. As shown in Figure 10, the control device 3 can be physically configured as a computer system including a processor 201 which includes a CPU (Central Processing Unit) and GPU, RAM (Random Access Memory) 202 and ROM (Read Only Memory) 203 which are main memory devices, input devices 204 such as a keyboard and mouse, output devices 205 such as a display, a communication module 206 which is a data transmission and reception device such as a network card, and auxiliary storage devices 207 such as a hard disk. The functions of the control device 3 described above are realized by loading predetermined computer software (suspended load recognition program) onto the hardware such as the processor 201 and RAM 202, thereby operating the communication module 206, input devices 204 and output devices 205 under the control of the processor 201, and reading and writing data to RAM 202 and auxiliary storage devices 207.

[0067] Next, the effects of the suspended load recognition system 1 according to this embodiment will be described.

[0068] The suspended load recognition system 1 according to this embodiment is a suspended load recognition system that recognizes a suspended load C suspended from a crane 100, and comprises an imaging device 2 provided at the tip of the jib 104 of the crane 100 with an imaging range facing downward, and a control device 3 that calculates the outer shape of the suspended load C based on an image 10 captured by the imaging device 2. The suspended load C is suspended from the crane 100 via a hook 108. The control device 3 recognizes the hook 108 from the image 10 and calculates the outer shape of the suspended load C based on the hook 108 recognized in the image 10.

[0069] In crane operations, the suspended load C can have various shapes. Therefore, directly detecting the suspended load C from image 10 presents various considerations and limits the detection accuracy. On the other hand, the hook 108 used in the crane 100 is usually formed in a similar shape, so it is considered easier to extract from image 10 compared to the suspended load C. Furthermore, since the suspended load C is suspended by the hook 108, it is considered highly likely that the suspended load C is basically located around the hook 108. Therefore, in this embodiment, the hook 108 is recognized in image 10 first, and then the outline of the suspended load C is calculated in image 10 based on the recognized hook 108. This configuration makes it possible to quickly calculate the outline of the suspended load C. As a result, the status of the suspended load C during crane operation can be easily grasped.

[0070] Furthermore, in methods such as Patent Document 1, which use images taken from above of the crane's work area to understand the condition of the work area, it is necessary to install another imaging device on the suspended load. Therefore, in conventional methods, it is necessary to attach or detach some element to the suspended load each time crane work is performed, which may reduce the work efficiency of the crane. In contrast, in this embodiment, it is not necessary to install elements such as an imaging device on the suspended load C itself. The outer shape of the suspended load C can be calculated using only the image 10 taken by the imaging device 2. As a result, in this embodiment, it is possible to understand the condition of the work area of ​​the crane, especially the condition of the suspended load C, without reducing work efficiency.

[0071] Furthermore, in the suspended load recognition system 1 according to this embodiment, it is preferable that the control device 3 is configured to recognize the area with the same color tone around the hook 108 detected in the image 10 as the suspended load C. With this configuration, the detection range of the suspended load C can be limited to the area around the hook 108 and narrowed down to an area with the same color tone, so the outer shape of the suspended load C (i.e., the suspended load area 13) can be detected quickly.

[0072] Furthermore, in the suspended load recognition system 1 according to this embodiment, it is preferable that the control device 3 detects the wire 107 hanging from the tip of the jib 104 in the image 10, and then detects the hook 108 attached to the tip of the wire 107 based on the wire 107 detected in the image 10. In most cases, the hook 108 used in a crane 100 is attached to the lower end of the wire 107 that is suspended from the tip of the jib 104 of the crane 100. Therefore, when detecting the hook 108, it is considered highly likely that it is located near the lower end of the wire 107. Accordingly, in this embodiment, by first detecting the wire 107, it is possible to detect the hook 108 more quickly.

[0073] Furthermore, in typical crane operations, the wire 107 connected to the hook 108 in use is longer than the wire 109 connected to the unused hook 110. Therefore, by detecting either wire 107 or wire 109 first and selecting the object near the end of the longer wire as the hook in use, it becomes possible to quickly detect the hook 108 in use.

[0074] Furthermore, in the suspended load recognition system 1 according to this embodiment, it is preferable that the control device 3 is configured to calculate the height T1 of the imaging device 2 from the ground G based on the size of the helmets H1, H2 of workers P1, P2 or the reference marker 4 included in the image 10. The size of the helmets H1, H2 of workers P1, P2 and the reference marker 4 in the image 10 are relative to the height of the imaging device 2. The higher the position of the imaging device 2, the smaller these appear in the image 10, and the lower the position of the imaging device 2, the larger they tend to appear in the image 10. Therefore, with this configuration, the height T1 of the imaging device 2 can be calculated accurately based on the image 10.

[0075] Furthermore, in the suspended load recognition system 1 according to this embodiment, it is preferable that the control device 3 is configured to calculate the height T2 of the hook 108 from the ground G based on the size of the hook 108 included in the image 10. The size of the hook 108 in the image 10 is relative to the height T2 of the hook 108 from the ground G. The higher the height T2 of the hook 108 is, the further it is from the ground G, the larger the hook 108 will appear in the image 10, and the lower the height T2 of the hook 108 is, the smaller the hook 108 will appear in the image 10. Therefore, with this configuration, the height T2 of the hook 108 can be calculated accurately based on the image 10.

[0076] Furthermore, in the suspended load recognition system 1 according to this embodiment, it is preferable that the control device 3 calculates the suspension height position T3 of the suspended load C based on the height T1 of the imaging device 2 and the height T2 of the hook 108, and calculates a projection region 14 that surrounds the outline of the projection F of the suspended load C formed on the ground G based on the suspension height position T3 and the wire vector 11 indicating the hanging direction of the wire 107 detected in the image 10. With this configuration, the projection region 14 of the suspended load C can be calculated with high accuracy based on the image 10.

[0077] Furthermore, in the suspended load recognition system 1 according to this embodiment, it is preferable that the control device 3 is configured to calculate the minimum horizontal distance between the suspended load C and workers P1 and P2 based on the position and outline of the projection area 14 in the image 10 and the positions of workers P1 and P2, or the helmets H1 and H2 of workers P1 and P2. With this configuration, the minimum horizontal distance between the suspended load C and workers P1 and P2 can be calculated with high accuracy based on the image 10.

[0078] Furthermore, in the suspended load recognition system 1 according to this embodiment, it is preferable that the control device 3 is configured to notify the operator of approach to the suspended load C when the minimum horizontal distance between the suspended load C and the operators P1 and P2 is below a predetermined threshold. This configuration allows for accurate detection of the operators P1 and P2 approaching the suspended load C, thereby improving the safety of crane operations.

[0079] The embodiments have been described above with reference to specific examples. However, this disclosure is not limited to these specific examples. Modifications made to these specific examples by those skilled in the art are also included within the scope of this disclosure, as long as they retain the features of this disclosure. The elements, their arrangement, conditions, shapes, etc., of each of the aforementioned specific examples are not limited to those illustrated and can be modified as appropriate. The elements of each of the aforementioned specific examples can be combined in different ways as appropriate, as long as no technical inconsistencies arise.

[0080] In the above embodiment, the configuration is illustrated in which the outer shape of the suspended load C (i.e., the suspended load area 13) is detected over the entire area of ​​the image 10 (see Figure 6, etc.) of the crane operation area captured by the imaging device 2. However, the control device 3 may be configured to limit the detection range of the suspended load C to a part of the image 10. This limitation may be performed autonomously by the control device 3, or it may be manually specified by a worker or the like. This configuration makes it possible to quickly detect the suspended load area 13 and process it more efficiently.

[0081] Furthermore, in the case of a configuration that limits the detection range of the suspended load area 13 as described above, it is preferable for the control device 3 to automatically set the detection range of the outer shape of the suspended load C based on the detection position of the hook 108 in the image 10 (i.e., the hook area 12). As described above, the imaging device 2 is installed so that its field of view is vertically downward, so in most cases the hook 108 is positioned overlapping the center of the suspended load C. Therefore, the suspended load area 13 is likely to be located around the hook area 12, and by setting the detection range as described above, it becomes possible to detect the suspended load area 13 even more quickly.

[0082] Furthermore, it is preferable that the control device 3 limits the detection range of the hook 108 to the area around the lower end of the wire 107. Generally, in a crane 100, the hook 108 is attached to the lower end of the wire 107, so by limiting the detection range of the hook as described above, it becomes possible to quickly detect the hook area 12 and process it more efficiently.

[0083] Furthermore, the control device 3 may be configured to calculate the planar shape and three-dimensional position of the suspended load C based on its external shape and the suspension height position T3. This allows for high-precision determination of the three-dimensional position of the suspended load C, enabling smoother crane operations. When calculating the three-dimensional position of the suspended load C, information such as the height dimension of the suspended load C and the distance from the hook 108 may also be considered. Additionally, a sub-camera different from the imaging device 2 may be used to photograph the side of the suspended load C, and the image from the sub-camera may be utilized. These configurations also make it possible to derive the three-dimensional shape of the suspended load C. [Explanation of Symbols]

[0084] 1. Suspended Load Recognition System 2. Imaging device 3. Control device 31 Imaging control unit 34. Suspended load shape detection unit 4. Reference Marker 10 images 11 Wire vector 12 Hook Area 13 Hanging load area 14 Projection area 15 Approach Warning Area 16 Helmet Area 17 Worker area 100 Cranes 104 Jib 107 First wire (wire) 108 First hook (hook) C Hanging load P1, P2 workers H1, H2 helmets G ground F projection

Claims

1. A load recognition system that recognizes a load suspended from a crane, An imaging device is provided at the tip of the jib of the crane, with the imaging range directed downwards. A control device that calculates the external shape of the suspended load based on an image captured by the aforementioned imaging device, Equipped with, The aforementioned load is suspended from the crane via a hook. The control device recognizes the hook from the image and calculates the external shape of the suspended load based on the hook recognized in the image. Suspended load recognition system.

2. The suspension load recognition system according to claim 1, wherein the control device recognizes the area around the hook detected in the image, which has the same color tone, as the suspension load.

3. The control device detects a wire hanging from the tip of the jib in the image, and detects the hook attached to the tip of the wire based on the wire detected in the image. The suspended load recognition system according to claim 1.

4. The control device calculates the height of the imaging device from the ground based on the size of the worker's helmet or reference marker included in the image. The suspended load recognition system according to claim 3.

5. The control device calculates the height of the hook from the ground based on the size of the hook included in the image. The suspended load recognition system according to claim 4.

6. The control device calculates the suspension height position of the suspended load based on the height of the imaging device and the height of the hook, and calculates a projection area that surrounds the outline of the projection of the suspended load formed on the ground based on the suspension height position and the wire vector indicating the direction of sagging of the wire detected in the image. The suspended load recognition system according to claim 5.

7. The control device calculates the minimum horizontal distance between the suspended load and the worker based on the position and shape of the projection area in the image and the position of the worker or the helmet. The suspended load recognition system according to claim 6.

8. The control device notifies of approach to the suspended load when the minimum horizontal distance is below a predetermined threshold. The suspended load recognition system according to claim 7.

9. The control device limits the detection range of the outer shape of the suspended load within the image to a portion of it. The suspended load recognition system according to claim 1.

10. The control device automatically sets the detection range for the outer shape of the suspended load based on the detection position of the hook in the image. The suspended load recognition system according to claim 1.

11. The control device limits the detection range of the hook to the area around the lower end of the wire. The suspended load recognition system according to claim 3.

12. The control device calculates the planar shape and three-dimensional position of the suspended load based on the external shape of the suspended load and the suspension height position. The suspended load recognition system according to claim 6.

13. A method for recognizing a suspended load that is lifted from a crane, An image acquisition step in which an image is taken by an imaging device provided at the tip of the crane's jib with an imaging range directed downwards, and an image including the suspended load is acquired; A load shape detection step which calculates the external shape of the suspended load based on the image taken in the image acquisition step, Includes, The aforementioned load is suspended from the crane via a hook. In the aforementioned load shape detection step, the hook is recognized from the image, and the outer shape of the load is calculated based on the hook recognized in the image. Hanging load recognition method.

14. A load recognition program that recognizes a load suspended from a crane, An image acquisition function that acquires an image including the suspended load, captured by an imaging device provided at the tip of the crane's jib with an imaging range directed downwards, A load shape detection function that calculates the external shape of the suspended load based on the image captured by the image acquisition function, To make this a reality on a computer, The aforementioned load is suspended from the crane via a hook. The aforementioned load shape detection function recognizes the hook from the image and calculates the outer shape of the load based on the hook recognized in the image. A program for recognizing suspended loads.

Images