Lifting support system for industrial machinery, lifting support method, and program

Abstract

To provide a lifting support system that can assist lifting operations performed by lifting operators. [Solution] The rigging support system SYS, which assists in the rigging work of a suspended load SL, comprises an imaging device CM that images the suspended load SL, a calculation device AU that identifies the center of gravity of the suspended load SL based on the image captured by the imaging device CM, and a display device DS that serves as an output device to communicate to the rigging worker the center of gravity of the suspended load SL identified by the calculation device AU, or the position of the hook 7 for lifting the suspended load SL calculated based on the center of gravity of the suspended load SL.

Description

Technical Field

[0001] The present disclosure relates to a ball hanging work support system, a ball hanging work support method, and a program for a working machine.

Background Art

[0002] Conventionally, a sway reduction ball hanging guide system that can reduce the sway of a suspended load generated when lifting the suspended load with a crane is known (see Patent Document 1). This system is configured to reduce the sway of the suspended load caused by the deflection of the boom generated when lifting the suspended load. Specifically, this system takes into account the deflection of the boom generated when lifting the suspended load, and the position of the center of mass (center of gravity) of the suspended load is farther from the crane than the position directly below the tip of the boom when starting to lift the suspended load (the sway reduction ball hanging position). It is configured to present the sway reduction ball hanging position to the crane operator.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the above-described system is not configured to present useful information to the ball hanging operator. Therefore, the above-described system cannot support the ball hanging work by the ball hanging operator at all.

Means for Solving the Problems

[0005] A rigging support system according to one embodiment of the present disclosure is a rigging support system for supporting rigging work of a suspended load, comprising: an imaging device for imaging the suspended load; a calculation device for identifying the center of gravity of the suspended load based on the image captured by the imaging device; and an output device for communicating to a rigging worker the center of gravity of the suspended load identified by the calculation device, or the position of a hook for lifting the suspended load calculated based on the center of gravity of the suspended load. [Effects of the Invention]

[0006] The aforementioned rigging support system can assist rigging workers with their rigging tasks. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic diagram showing an example configuration of a rigging work support system according to the present disclosure. [Figure 2] Figure 1 is a block diagram showing an example configuration of a rigging support system. [Figure 3] A flowchart illustrating an example of the process for supporting rigging operations. [Figure 4] This figure shows an example of a work site where rigging work is performed. [Figure 5] This figure shows another example of a work site where rigging work is performed. [Figure 6] This figure shows an example of an image captured by a camera attached to the tip of a boom. [Figure 7] This figure shows an example of the state of a suspended load before it is released from the ground. [Figure 8] This flowchart shows an example of the process for re-determining the center of gravity. [Figure 9] This flowchart shows another example of the process for re-determining the center of gravity. [Modes for carrying out the invention]

[0008] The embodiments of this disclosure will be described below with reference to the drawings. Furthermore, the embodiments described below are illustrative and not limiting to the invention, and not all features or combinations thereof described as embodiments are necessarily essential to the invention. In each drawing, identical or corresponding components are denoted by the same or corresponding reference numerals, and their descriptions may be omitted.

[0009] First, with reference to Figures 1 and 2, a lifting support system SYS, which is a system for work machines according to the embodiment of this disclosure, will be described. Figure 1 is a schematic diagram showing an example configuration of the lifting support system SYS. Figure 2 is a block diagram showing an example configuration of the lifting support system SYS.

[0010] The slinging support system SYS is a system that supports slinging operations performed by slinging workers. In the illustrated example, the slinging support system SYS mainly consists of a work machine 100, a support device 200, and a management device 300. Each of the work machine 100, support device 200, and management device 300 is equipped with a communication device TD and is directly or indirectly connected to each other via an information communication network IN such as a mobile phone network, satellite network, or short-range wireless communication network. Each of the work machine 100, support device 200, and management device 300 that constitute the slinging support system SYS may be one unit or multiple units. In the illustrated example, the slinging support system SYS includes one work machine 100, one support device 200, and one management device 300.

[0011] In the illustrated example, the work machine 100 is a mobile crane, with an upper rotating body 3 rotatably mounted on a crawler-type lower traveling body 1 via a slewing mechanism 2. A boom 4 is attached to the upper rotating body 3. A wire rope 5 extends downward from the tip of the boom 4, and a hook 7 is suspended from the wire rope 5 via a hook bracket 6. The hook bracket 6 has a pulley (not shown) inside for which the wire rope 5 is attached.

[0012] An imaging device CM (first imaging device CM1) is attached to the tip of boom 4. In the illustrated example, the first imaging device CM1 is a monocular camera and is positioned to image the space vertically below. The first imaging device CM1 may also be an imaging device capable of measuring the distance between the first imaging device CM1 and the subject, such as a stereo camera, RGBD camera, or LiDAR.

[0013] The work machine 100 can raise the hook 7 and lift the load SL by winding up the wire rope 5 with a front winch (not shown). The work machine 100 can also lower the hook 7 and lower the load SL by unwinding the wire rope 5 with the front winch.

[0014] The upper rotating body 3 is equipped with a cabin 8 serving as a driver's cab and is also fitted with a power source such as an engine. The upper rotating body 3 is also fitted with a calculation unit AU (work machine side controller AU1), a display device DS (first display device DS1), an input device ID (first input device ID1), and a communication device TD (first communication device TD1), etc. The power source may be an electric motor driven by a battery or an external power supply. The first input device ID1 may be a touch panel, a hardware button, or a microphone for voice input.

[0015] The machine-side controller AU1 is configured to control the machine 100. In the illustrated example, the machine-side controller AU1 is an example of a processing circuit as a control device (arithmetic unit AU), and is composed of a computer equipped with a CPU, RAM, NVRAM, and ROM. The machine-side controller AU1 reads the program corresponding to each functional element from ROM and loads it into RAM, and causes the CPU to execute the corresponding processing. However, each functional element may be composed of hardware, or it may be composed of a combination of software and hardware.

[0016] The support device 200 is a device that supports the ball-hanging operation by the ball-hanging operator. In the illustrated example, the support device 200 is a portable client computer (such as a notebook PC, a tablet PC, or a mobile terminal device such as a smartphone) carried by the ball-hanging operator, and includes an arithmetic unit AU (client-side controller AU2), an imaging device (second imaging device CM2), a display device DS (second display device DS2), an input device ID (second input device ID2), and a communication device TD (second communication device TD2). Note that the support device 200 may function as a server. Also, the support device 200 may be a wearable terminal device such as an XR (extended reality) goggle.

[0017] The client-side controller AU2 is configured to be able to control the support device 200. In the illustrated example, the client-side controller AU2 is another example of a processing circuit as the arithmetic unit AU, and is composed of a computer equipped with a CPU, a RAM, a NVRAM, a ROM, etc. Then, the client-side controller AU2 reads out the program corresponding to each functional element from the ROM and reads it into the RAM, and causes the CPU to execute the corresponding processing. However, each functional element may be composed of hardware, or may be composed of a combination of software and hardware.

[0018] In the illustrated example, the support device 200 is a smartphone, the second imaging device CM2 is the out-camera of the smartphone, the second display device DS2 is an organic EL display, and the second input device ID2 is a touch panel. Note that the second imaging device CM2 may be an imaging device capable of measuring the distance between the second imaging device CM2 and the subject, such as a stereo camera, a RGBD camera, or a LiDAR. Also, the second input device ID2 may be a microphone for voice input.

[0019] The management device 300 is a device that manages the work by the working machine 100. In the illustrated example, the management device 300 is a server computer installed in a management center or the like located at a place away from the work site (working machine 100), and includes an arithmetic unit AU (server-side controller AU3), a display device DS (third display device DS3), an input device ID (third input device ID3), and a communication device TD (third communication device TD3). Note that the management device 300 may be a portable computer (for example, a portable terminal device such as a notebook PC, a tablet PC, or a smartphone).

[0020] In the illustrated example, the management device 300 is a desktop PC, the third display device DS3 is a liquid crystal display, and the third input device ID3 is a keyboard and a mouse.

[0021] The server-side controller AU3 is configured to be able to control the management device 300. In the illustrated example, the server-side controller AU3 is another example of a processing circuit as an arithmetic unit AU, and is composed of a computer including a CPU, a RAM, a NVRAM, a ROM, and the like. Then, the server-side controller AU3 reads out a program corresponding to each functional element from the ROM and reads it into the RAM, and causes the CPU to execute the corresponding processing. However, each functional element may be composed of hardware, or may be composed of a combination of software and hardware.

[0022] Next, referring to Figure 3, an example of the flow of the process by which the rigging support system SYS assists rigging workers in rigging operations (hereinafter referred to as "rigging support process") will be explained. Figure 3 is a flowchart of an example of the flow of the rigging support process. In the illustrated example, the client-side controller AU2 of the support device 200 starts this rigging support process when a predetermined operation is performed by the operator using the second input device ID2. The predetermined operation is, for example, touching a predetermined icon. In the illustrated example, the predetermined icon is a startup icon for launching the rigging support application. Note that the predetermined operation may also be pressing a predetermined hardware button such as a work support start button, or uttering a predetermined keyword.

[0023] First, the client-side controller AU2 acquires an image of the suspended load SL (step ST1). In the illustrated example, when the startup icon is touched, the client-side controller AU2 displays the initial screen of the rigging support application and also displays a text image that says, "Please take a picture of the suspended load."

[0024] Upon seeing this text image, the rigging operator will use the second imaging device CM2 mounted on the support device 200 to image the suspended load SL.

[0025] Subsequently, the client-side controller AU2, having acquired an image of the suspended load SL, identifies the suspended load SL (step ST2). In the illustrated example, the client-side controller AU2 identifies the suspended load SL using known image recognition technology. The client-side controller AU2 may also present the rigging worker with a text image such as "Please trace the outline of the suspended load image" to facilitate the identification of the suspended load SL.

[0026] Subsequently, the client-side controller AU2, having identified the suspended load SL, acquires information about the suspended load SL (step ST3). In the illustrated example, the client-side controller AU2 accesses the management device 300 and reads the information about the identified suspended load SL stored in the management device 300's storage device. This is because, if the suspended load SL is a standard product such as building material, the information about the suspended load SL is typically stored in the management device 300's storage device. The information about the identified suspended load SL may include the position (three-dimensional coordinates) of its center of mass (center of gravity). In this case, the three-dimensional coordinates of the center of gravity may be stored in advance as relative coordinates to multiple three-dimensional coordinates on the surface of the suspended load SL. This is to ensure that the three-dimensional coordinates of the center of gravity can be uniquely identified when multiple three-dimensional coordinates on the surface of the suspended load SL are identified. Alternatively, the three-dimensional coordinates (relative coordinates) of the center of gravity may be dynamically determined based on at least one of the dimensions, material, weight, and part number of the suspended load SL, which are stored in advance.

[0027] Alternatively, the client-side controller AU2 may acquire images of the suspended load SL from other viewpoints. In this case, the client-side controller AU2 may display a text image to the rigging worker such as, "Please photograph the suspended load from the left, right, and rear sides." Note that "front," "rear," "left," "right," "up," and "down" each represent the direction as seen from the rigging worker's perspective. The same applies to the following descriptions.

[0028] Subsequently, the client-side controller AU2, having acquired information about the suspended load, identifies the position of the center of gravity of the suspended load SL (step ST4). In the illustrated example, the client-side controller AU2 associates each of the multiple three-dimensional coordinates on the surface of the suspended load SL received from the management device 300 with a point on the image of the suspended load SL. Then, the client-side controller AU2 associates the three-dimensional coordinate of the center of gravity of the suspended load SL received from the management device 300 with a single point (two-dimensional coordinate) on the image of the suspended load SL.

[0029] Alternatively, the client-side controller AU2 may identify the shape of the suspended load SL from the acquired image of the suspended load SL, and then determine the position of the center of gravity of the suspended load SL from the identified shape. Specifically, the client-side controller AU2 may, for example, generate a three-dimensional model of the suspended load SL by using multiple images obtained by imaging the suspended load SL from multiple viewpoints and known three-dimensional reconstruction techniques. Three-dimensional reconstruction techniques are techniques that estimate three-dimensional information from two-dimensional images captured by the imaging device CM, and include, for example, photogrammetry or NeRF (Neural Radiance Fields) techniques. Based on this three-dimensional model, the client-side controller AU2 may then identify a point (two-dimensional coordinate) on the image of the suspended load SL that corresponds to the center of gravity of the suspended load SL.

[0030] One method for determining the position of the center of gravity of a suspended load SL from an image of the suspended load SL or a three-dimensional model of the suspended load SL is to use machine learning techniques. Specifically, the client-side controller AU2 can use a machine learning model generated by analyzing (learning) a large amount of images or three-dimensional models to output the position of the center of gravity in the currently input image of the suspended load SL or the three-dimensional model of the suspended load SL. The machine learning model is, for example, a neural network model, which is formed using backpropagation.

[0031] Subsequently, the client-side controller AU2, having identified the position of the center of gravity of the suspended load SL, displays a work support image (step ST5). In the illustrated example, the client-side controller AU2 superimposes a graphic image (e.g., a circular image) representing the position of the center of gravity of the suspended load SL as a work support image (center of gravity image) on top of the image of the suspended load SL. The image of the suspended load SL may be a still image or a moving image currently being captured by the second imaging device CM2. If the image of the suspended load SL is a moving image, the display position of the center of gravity image changes in accordance with the movement of the rigging worker carrying the support device 200, that is, in accordance with the movement of the image of the suspended load SL displayed on the second display device DS2.

[0032] The rigger can recognize the position of the center of gravity of the suspended load SL by viewing the center of gravity image superimposed on the image of the suspended load SL displayed on the second display device DS2. Therefore, the rigger can guide the hook 7 to a position directly above the center of gravity of the suspended load SL.

[0033] Next, referring to Figure 4, we will explain how rigging worker WK uses the rigging support system SYS to confirm the position of the center of gravity of the suspended load SL. Figure 4 is a diagram showing an example of a work site where rigging work is performed.

[0034] Specifically, in the work site shown in Figure 4, the second rigger WK2 and the third rigger WK3 are attempting to attach the rigging wire SW to the load SL, while the first rigger WK1 is using the support device 200 to confirm the position of the center of gravity of the load SL. The first rigger WK1 is carrying a tablet PC, which is being used as the support device 200.

[0035] More specifically, the first rigging worker WK1 is capturing images of the suspended load SL using the second imaging device CM2 of the support device 200. The images captured by the second imaging device CM2 (captured images) are transmitted to the management device 300. The management device 300 applies known image recognition technology to the images received from the support device 200 to identify what the suspended load SL is. If details such as the dimensions, weight, or center of gravity of the object to be crane work (suspended load SL) have already been registered, the management device 300 can determine the center of gravity of the suspended load SL by identifying which of the previously registered objects the suspended load SL shown in the captured image is. Previously registered objects include, for example, containers, wall materials, or flooring materials.

[0036] Alternatively, the management device 300 may apply known image recognition techniques to one or more captured images received from the support device 200 to recognize the three-dimensional shape of the suspended load SL and determine the position of the center of gravity of the suspended load SL.

[0037] Subsequently, the information regarding the position of the center of gravity of the suspended load SL (center of gravity position information) identified by the management device 300 is transmitted to the support device 200. Based on the received center of gravity position information, the support device 200 superimposes a work support image SG onto the image displayed on the second display device DS2 (on the image captured by the second imaging device CM2) at an appropriate position. The work support image SG is, for example, a graphic image showing the position of the center of gravity of the suspended load SL. The work support image SG may also be a graphic image showing the position where the hook 7 should be positioned (typically the position directly above the center of gravity of the suspended load SL).

[0038] The first rigger WK1 can confirm the position of the center of gravity of the suspended load SL by viewing the work support image SG displayed on the second display device DS2. Therefore, the first rigger WK1 can, for example, instruct the operator of the work machine 100 via the communication device TD to move the hook 7 to a position directly above the center of gravity of the suspended load SL. Specifically, the first rigger WK1 can guide the hook 7 to a position directly above the center of gravity of the suspended load SL by conveying information such as, "Please move the hook 7 30 cm to the right and 30 cm backward" to the operator of the work machine 100. This information may be voice information or text information. The rigging work support system SYS may automatically issue the same instruction to the operator of the work machine 100 without receiving instructions from the first rigger WK1. Furthermore, the rigging work support system SYS may be configured to allow constant voice communication between the rigger WK and the operator of the work machine 100. In other words, both the work machine 100 and the support device 200 may be equipped with devices necessary for voice communication, such as a microphone and a speaker.

[0039] In the example described above, the identification of the suspended load SL and the location of the center of gravity of the suspended load SL are performed by the server-side controller AU3 of the management device 300. However, at least one of these identifications may be performed by the work machine-side controller AU1 of the work machine 100, or by the client-side controller AU2 of the support device 200.

[0040] Next, referring to Figure 5, we will describe another method by which the first rigger WK1 uses the rigging support system SYS to confirm the position of the center of gravity of the suspended load SL. Figure 5 is a diagram showing another example of a work site where rigging work is performed. The first rigger WK1 in the work site shown in Figure 5 differs from the first rigger WK1 in the work site shown in Figure 4 in that he is wearing AR (Augmented Reality) glasses as a support device 200. The method by which the first rigger WK1 confirms the position of the center of gravity of the suspended load SL in the work site shown in Figure 5 is otherwise the same as the method by which the first rigger WK1 confirms the position of the center of gravity of the suspended load SL in the work site shown in Figure 4. Therefore, the explanation of the common parts will be omitted below, and the differences will be explained in detail.

[0041] Specifically, the AR glasses, which serve as the support device 200, are equipped with an out-camera positioned to capture images of the area in front and a pair of small displays, each corresponding to one of the eyes of the first lifting worker WK1. Based on the center of gravity position information received from the management device 300, the AR glasses superimpose a work support image SG onto the image displayed on the second display device DS2 (the pair of small displays) (on the image captured by the second imaging device CM2 (out-camera)) at an appropriate position. The work support image SG is a graphic image indicating the position of the center of gravity of the suspended load SL.

[0042] The first lifting worker WK1 can confirm the position of the center of gravity of the suspended load SL by looking at the work support image SG displayed on the second display device DS2. In other words, in this configuration, the first lifting worker WK1 can confirm the position of the center of gravity of the suspended load SL without taking their eyes off the suspended load SL and without having both hands occupied by the support device 200.

[0043] Next, with reference to Figure 6, another example of how the control device 300 identifies the position of the suspended load SL or its center of gravity will be described. Figure 6 shows an example of an image GM captured by a first imaging device CM1 attached to the tip of the boom 4 of the work machine 100. The first imaging device CM1 is attached to the tip of the boom 4 so as to capture images vertically downward to acquire images of the hook 7 and its surroundings.

[0044] Specifically, image GM is an image captured by the first imaging device CM1 located above the suspended load SL, and includes image G5 of the wire rope 5, image G6 of the hook bracket 6, and image GSL of the suspended load SL.

[0045] The management device 300 is configured to identify what the suspended load SL is using the image GM received from the support device 200 (the image captured by the second imaging device CM2), and is further configured to identify the position of the center of gravity of the suspended load SL.

[0046] This configuration allows the management device 300 to identify the type of suspended load SL with higher accuracy compared to when it identifies the type of suspended load SL based solely on images received from the support device 200 (images captured by the second imaging device CM2). This is because it can additionally utilize images that cannot be captured by the second imaging device CM2 mounted on the support device 200 carried by the first lifting worker WK1 (images of the suspended load SL taken from directly above). The same applies to the accuracy of identifying the position of the center of gravity of the suspended load SL.

[0047] However, the management device 300 may be configured to identify what the suspended load SL is based solely on the image captured by the first imaging device CM1, and may also be configured to identify the position of the center of gravity of the suspended load SL. Alternatively, the management device 300 may additionally or substitutely utilize images captured by another imaging device other than the first imaging device CM1 and the second imaging device CM2. That is, the management device 300 may be configured to identify what the suspended load SL is based on at least one of the images captured by the first imaging device CM1, the images captured by the second imaging device CM2, and the images captured by the other imaging device, and may also be configured to identify the position of the center of gravity of the suspended load SL. The other imaging device may be, for example, an imaging device attached to a structure such as a steel tower at the work site, or an imaging device attached to an aircraft such as a multicopter flying above the suspended load SL.

[0048] Next, referring to Figure 7, the effects of the rigging support system SYS assisting rigging work by rigging worker WK will be explained. Figure 7 shows an example of the state of the suspended load SL before it is lifted off the ground. Lifting off the ground of the suspended load SL means the work of lifting the suspended load SL away from the installation surface IS (the ground on which the suspended load SL is placed). In the example shown in Figure 7, the suspended load SL is a combination of the first suspended load SL1 and the second suspended load SL2, with the second suspended load SL2 stacked on top of the first suspended load SL1. Also, four rigging wires SW (first rigging wire SW1 to fourth rigging wire SW4) are attached between the suspended load SL and the hook 7. Note that in Figure 7, the third rigging wire SW3 is hidden behind the first rigging wire SW1 and is not visible, and the fourth rigging wire SW4 is hidden behind the second rigging wire SW2 and is not visible. Furthermore, the length of the first lifting wire SW1 is approximately the same as the length of the third lifting wire SW3, and the length of the second lifting wire SW2 is approximately the same as the length of the fourth lifting wire SW4. In addition, the lengths of the first lifting wire SW1 and the third lifting wire SW3 are greater than the lengths of the second lifting wire SW2 and the fourth lifting wire SW4, respectively.

[0049] In the example shown in Figure 7, the hook 7 is positioned directly above the combined center of gravity GC of the suspended load SL, and each of the four lifting wires SW (the first lifting wire SW1 to the fourth lifting wire SW4) is slightly slack. This state is achieved through the lifting operation support process and will be referred to as the "ready state" below.

[0050] In the example shown in Figure 7, the first rigging worker WK1 (see Figure 4) can easily identify the position of the combined center of gravity GC of the suspended load SL, which is derived from the combined center of gravity of the first center of gravity GC1 and the second center of gravity GC2, by looking at the work support image SG displayed on the second display device DS2. The first center of gravity GC1 is the center of gravity of the first suspended load SL1, and the second center of gravity GC2 is the center of gravity of the second suspended load SL2. In other words, the first rigging worker WK1 can easily identify the position of the combined center of gravity GC of the suspended load SL, which is difficult to determine from the appearance of the suspended load SL.

[0051] Furthermore, the first lifting worker WK1, upon viewing the work support image SG, can estimate in advance the length of each of the four lifting wires SW (first lifting wire SW1 to fourth lifting wire SW4) in the ready state. Therefore, the first lifting worker WK1 can, for example, inform the second lifting worker WK2 (see Figure 4) and the third lifting worker WK3 (see Figure 4) of the lengths of the four lifting wires SW in the ready state before each of the four lifting wires SW is attached to the hook 7. Thus, the lifting work support system SYS can prevent the use of lifting wires SW of inappropriate lengths.

[0052] Furthermore, since the appropriate length of each of the four lifting wires SW can be determined in advance, the first lifting worker WK1 can appropriately identify the temporary position of the hook 7 (a position different from the position in the ready state (target position)) for each of the four lifting wires SW that is easy to attach. Figure 7 shows the hook 7 at that temporary position as hook 7T with a dashed line. Note that the temporary position of hook 7 is, for example, a position that can be reached by the second lifting worker WK2 and the third lifting worker WK3, respectively.

[0053] Specifically, the first rigging worker WK1 can guide the operator of the work machine 100 through a communication device TD or the like so that the hook 7 moves to the temporary position. The rigging support system SYS may be configured to display a graphic image SG on the display device DS as a work support image, indicating at least one of the target position and temporary position of the hook 7. In this case, the rigging support system SYS may omit the display of a graphic image indicating the center of gravity of the suspended load SL.

[0054] Subsequently, the second lifting operator WK2 and the third lifting operator WK3 can attach each of the four lifting wires SW to the hook 7 while the hook 7 is in a temporary position. After confirming that each of the four lifting wires SW is properly attached to the hook 7, the first lifting operator WK1 can guide the operator of the work machine 100 via the communication device TD or the like to move the hook 7 to the target position.

[0055] Afterward, the first lifting operator WK1 can, after confirming that the hook 7 is positioned at the target location, instruct the operator of the work machine 100 via the communication device TD or the like to begin operations to lift the load SL.

[0056] As a result, the work machine 100 can lift the suspended load SL straight up with almost no oscillation of the suspended load SL, thereby achieving ground clearance of the suspended load SL.

[0057] Next, referring to Figure 8, we will describe an example of the process of re-determining the center of gravity of the suspended load SL, which was identified in the rigging work support process (hereinafter, "center of gravity re-determining process"). Figure 8 is a flowchart showing an example of the flow of the center of gravity re-determining process. In the illustrated example, the client-side controller AU2 of the support device 200 starts this center of gravity re-determining process when a predetermined operation using the second input device ID2 is performed by the first rigging worker WK1 (see Figure 4). The predetermined operation is, for example, touching a predetermined icon.

[0058] First, the client-side controller AU2 acquires a series of images showing the lifting of the suspended load SL (step ST11). In the illustrated example, when a predetermined icon is touched, the client-side controller AU2 displays a text image such as "Please photograph the suspended load" on the second display device DS2. Upon seeing this text image, the first rigging worker WK1 points the second imaging device CM2 at the ready suspended load SL and starts recording a video.

[0059] The operator of the work machine 100 starts raising the hook 7 in response to instructions from the rigging support system SYS or the first rigging worker WK1. Each of the four rigging wires SW (see Figure 3) changes from a slightly slack state to a taut state as the hook 7 rises.

[0060] Subsequently, the client-side controller AU2 detects the movement of each lifting wire switch (step ST12). The movement of the lifting wire switches includes, for example, the amount of movement (magnitude of movement) of the lifting wire switches, or the tension of the lifting wire switches. For example, the client-side controller AU2 can detect, based on a series of images captured by the second imaging device CM2 before the lifting load SL is released from the ground, when each of the four lifting wire switches became tensioned.

[0061] Subsequently, the client-side controller AU2 re-determines the center of gravity of the suspended load SL based on the movement of each lifting wire SW (step ST13). For example, if the client-side controller AU2 determines that the time when the first lifting wire SW1 became taut was delayed by a predetermined amount of time compared to the other three lifting wire SWs, it determines that the hook 7 is not directly above the center of gravity of the suspended load SL. In this case, the client-side controller AU2 re-determines the center of gravity by bringing the current center of gravity closer to the lower end of the first lifting wire SW1. Note that the determination of whether or not the hook 7 is directly above the center of gravity of the suspended load SL may be performed using machine learning techniques.

[0062] In this case, the client-side controller AU2 may change the display position of the work support image SG (a graphic image indicating the position of the center of gravity of the suspended load SL) which is superimposed on the image of the suspended load SL, based on the re-identified center of gravity position.

[0063] Furthermore, the operator of the work machine 100 stops raising the hook 7 in response to instructions from the rigging support system SYS or the first rigging worker WK1, and then lowers the hook 7 to the original target position.

[0064] Subsequently, the first rigging worker WK1 can instruct the operator of the work machine 100 via the communication device TD or the like to move the hook 7 to a position directly above the re-identified center of gravity. Alternatively, the rigging support system SYS may automatically issue the same instruction to the operator of the work machine 100 without receiving instructions from the first rigging worker WK1. Or, the first rigging worker WK1 may adjust the length of each of the four rigging wires SW based on the re-identified center of gravity.

[0065] Through this center of gravity re-identification process, the rigging support system SYS can determine whether the hook 7 is directly above the center of gravity of the load SL when it starts to rise, even after the center of gravity of the load SL has been identified in a ready state. Therefore, even after the hook 7 has started to rise, the rigging support system SYS can inform the first rigger WK1 that the hook 7 is not directly above the center of gravity of the load SL. Furthermore, the rigging support system SYS can clearly present the re-identified center of gravity to the first rigger WK1. Therefore, the rigging support system SYS can prevent the load SL from swinging after being lifted while the hook 7 is not directly above the center of gravity of the load SL.

[0066] Next, with reference to Figure 9, another example of the center of gravity re-determination process will be described. Figure 9 is a flowchart showing another example of the flow of the center of gravity re-determination process. The center of gravity re-determination process shown in Figure 9 differs from the center of gravity re-determination process shown in Figure 8 in that it re-determines the center of gravity of the suspended load SL based on the movement of the suspended load SL when a part of the suspended load SL leaves the installation surface IS before the suspended load SL is lifted off the ground. However, in other respects, it is the same as the center of gravity re-determination process shown in Figure 8. Therefore, the explanation of the common parts will be omitted below, and the differences will be explained in detail. Note that "when a part of the suspended load SL leaves the installation surface IS" means, in other words, "when another part of the suspended load SL has not yet left the installation surface IS."

[0067] First, the client-side controller AU2 acquires a series of images representing the lifting of the suspended load SL, similar to the center of gravity re-identification process shown in Figure 8 (step ST21).

[0068] Subsequently, the client-side controller AU2 detects the movement of the suspended load SL when a portion of it leaves the mounting surface IS (step ST22). The movement of the suspended load SL includes tilting, translation, or rotation of the suspended load. For example, based on a series of images captured by the second imaging device CM2, the client-side controller AU2 can detect that the lower left front corner of the roughly rectangular suspended load SL has lifted up, and that the suspended load SL has rotated counterclockwise around the vertical axis with the lower right rear corner of the suspended load SL as the pivot point.

[0069] Subsequently, the client-side controller AU2 re-determines the center of gravity of the suspended load SL based on its movement (step ST23). For example, if the client-side controller AU2 determines that the suspended load SL, which is roughly rectangular in shape, has rotated counterclockwise around its vertical axis with the lower right rear corner as the pivot point, it determines that the hook 7 is not directly above the center of gravity of the suspended load SL. Specifically, the client-side controller AU2 determines that the correct center of gravity is to the left and rearward of the current incorrect center of gravity. In this case, the client-side controller AU2 re-determines the center of gravity by moving the current incorrect center of gravity further to the left and rearward.

[0070] In this case, the client-side controller AU2 may change the display position of the work support image SG (a graphic image indicating the position of the center of gravity of the suspended load SL) which is superimposed on the image of the suspended load SL, based on the re-identified center of gravity position.

[0071] Furthermore, the operator of the work machine 100 stops raising the hook 7 in response to instructions from the rigging support system SYS or the first rigging worker WK1, and then lowers the hook 7 to the original target position.

[0072] Subsequently, the first rigging worker WK1 can instruct the operator of the work machine 100, via the communication device TD or the like, to move the hook 7 to a position directly above the re-identified center of gravity. The rigging support system SYS may also automatically issue the same instruction to the operator of the work machine 100 without receiving instructions from the first rigging worker WK1.

[0073] This re-determination of the center of gravity allows the slinging support system SYS to inform the first slinging worker WK1 that the hook 7 is not directly above the center of gravity of the load SL. Furthermore, the slinging support system SYS can clearly present the re-determined center of gravity to the first slinging worker WK1. Therefore, the slinging support system SYS can prevent the load SL from swinging after being lifted, which would occur if the hook 7 were not directly above the center of gravity of the load SL.

[0074] In the example shown in Figure 9, the client-side controller AU2 is configured to detect the movement of the suspended load SL based on a series of images captured by the second imaging device CM2. However, if a detection device for detecting the position and orientation of the hook 7 is provided on the hook 7, the client-side controller AU2 may be configured to detect the movement of the suspended load SL based on the output of that detection device. Specifically, the client-side controller AU2 may, for example, receive data output by the detection device from the work machine 100 via a communication device TD, and estimate the movement of the suspended load SL from the movement of the hook 7 detected based on the received data. The detection device for detecting the position and orientation of the hook 7 may consist of, for example, a GNSS compass, a gyro sensor, an acceleration sensor, or a combination thereof.

[0075] Through this center of gravity re-determination process, the rigging support system SYS can determine whether the hook 7 is directly above the center of gravity of the load SL even after the center of gravity of the load SL has been determined in a ready state, when a part of the load SL leaves the installation surface. Therefore, even after the lifting of the load SL has begun, the rigging support system SYS can inform the first rigger WK1 that the hook 7 is not directly above the center of gravity of the load SL. Furthermore, the rigging support system SYS can clearly present the re-determined center of gravity to the first rigger WK1. Therefore, the rigging support system SYS can prevent the load SL from swinging after being lifted while the hook 7 is not directly above the center of gravity of the load SL.

[0076] As described above, the rigging support system SYS according to the embodiment of this disclosure includes an imaging device CM that images the suspended load SL, a calculation device AU that identifies the center of gravity of the suspended load SL based on the image captured by the imaging device CM, and an output device that transmits to the rigging worker WK the center of gravity of the suspended load SL identified by the calculation device AU, or the position of the hook for lifting the suspended load SL calculated based on the center of gravity of the suspended load SL (see, for example, hook 7T in Figure 7). The calculation device AU is implemented by, for example, at least one of the work machine side controller AU1, the client side controller AU2, and the server side controller AU3. In the illustrated example, the calculation device AU is the client side controller AU2. The imaging device CM for imaging the suspended load SL is, for example, at least one of the following: a first imaging device CM1 attached to the work machine 100, a second imaging device CM2 mounted on the support device 200, an imaging device attached to a structure such as a steel tower at the work site, and an imaging device attached to an aircraft such as a multicopter flying above the suspended load SL. In the illustrated example, the imaging device CM is the second imaging device CM2. The output device may be a laser irradiation device, projection device, or projector attached to the work machine 100 or a structure such as a steel tower at the work site. In this case, for example, the laser irradiation device as a display device DS may shine a laser at one or more positions on the surface of the suspended load SL corresponding to the position of the center of gravity of the suspended load SL, thereby indicating the position of the center of gravity of the suspended load SL to the rigging worker WK. The output device may also be an audio output device such as a speaker that conveys various information audibly to the rigging worker WK, or a vibration generating device that conveys various information tactilely to the rigging worker WK. The slinging support system SYS may be configured to generate images, sounds, or vibrations corresponding to the distance between the position of the hook 7 and the center of gravity of the suspended load SL (distance on a virtual plane parallel to the installation surface). For example, the slinging support system SYS may output intermittent sounds, the smaller the distance between the position of the hook 7 and the center of gravity of the suspended load SL, the smaller the output interval.

[0077] This configuration supports the rigging work by the rigging worker WK by providing information about the position of the center of gravity of the suspended load SL. This is because the rigging worker WK can attach the rigging wire SW to the hook 7, which is positioned directly above the center of gravity of the suspended load SL. Therefore, this configuration has the effect of suppressing swaying or detachment of the suspended load SL during ground lifting, or detachment of the suspended load SL during transport. In other words, this configuration has the effect of ensuring that rigging work is performed appropriately without relying on the experience, skills, know-how, or intuition of a skilled rigging worker. Furthermore, the rigging worker WK can eliminate the cumbersome task of repeatedly making fine adjustments to the position of the hook 7 relative to the suspended load SL while slightly lifting the load SL with the work machine 100. Therefore, this configuration has the effect of reducing the burden on the rigging worker WK. Furthermore, this configuration has the effect of reducing the time required for rigging work by rigging operator WK.

[0078] Furthermore, the output device may be a display device DS that displays a work support image SG showing the center of gravity position of the suspended load SL identified by the calculation device AU, or the position of the hook 7 for lifting the suspended load SL (see hook 7T in Figure 7) calculated based on the center of gravity position of the suspended load SL. In this case, the work support image SG is displayed in a position visible to the rigging worker.

[0079] This configuration has the effect of making it easier for the rigging worker WK to understand the position of the center of gravity of the suspended load SL, compared to communicating the position of the center of gravity of the suspended load SL to the rigging worker WK aurally or tactilely.

[0080] Furthermore, the work support image SG may be superimposed on the image captured by the imaging device CM, as shown in Figure 4. However, the work support image SG may be an image displayed independently in a different area from the area where the image of the suspended load SL is displayed. In this case, the work support image SG may be a combination of a CG image representing the shape of the suspended load SL and an image superimposed on the CG image representing the position of the center of gravity. Alternatively, the work support image SG may be a text image such as, "The center of gravity is located 30 cm to the right, 50 cm down, and 40 cm behind the upper left front vertex of the suspended load SL."

[0081] The configuration in which the work support image SG is superimposed on the image captured by the imaging device CM has the effect of clearly communicating the position of the center of gravity of the suspended load SL to the rigging worker WK, compared to a configuration in which the image is displayed in a separate area from the area where the image is displayed.

[0082] Furthermore, the computing unit AU is typically configured to determine the center of gravity of the suspended load SL before the lifting wire SW for lifting the load SL is attached to the hook 7.

[0083] This configuration offers the advantage of efficiently supporting the lifting operation by the lifting worker WK, compared to the case where the center of gravity of the suspended load SL is determined after the lifting wire SW is attached to the hook 7. This is because it reduces the frequency of having to reattach the lifting wire SW to the hook 7.

[0084] Furthermore, the imaging device CM may capture images of the hook 7 rising while multiple lifting wires SW for lifting the suspended load SL are attached to the hook 7. In this case, the computing device AU may detect the movement of at least one of the multiple lifting wires SW when the hook 7 rises while multiple lifting wires SW are attached to the hook 7, based on the images captured by the imaging device CM. The computing device AU may then be configured to re-determine the center of gravity of the suspended load SL based on the detected movement.

[0085] This configuration has the effect of more reliably suppressing swaying or detachment of the suspended load SL during lifting, or detachment of the suspended load SL during transport. This is because it is possible to determine whether the hook 7 is directly above the center of gravity of the suspended load SL just before it lifts up, and it is possible to inform the rigging worker WK if the hook 7 is not directly above the center of gravity of the suspended load SL.

[0086] Furthermore, the imaging device CM may capture images of the state of the suspended load SL when a portion of it leaves the installation surface IS. In this case, the computing device AU may be configured to detect the movement of the suspended load SL when a portion of it leaves the installation surface IS based on the image captured by the imaging device CM, and to re-determine the center of gravity of the suspended load SL based on that movement. Preferably, this re-determining of the center of gravity is performed before the suspended load SL is lifted off the ground, that is, before the suspended load SL leaves the installation surface IS. If the computing device AU determines, based on that movement, that the hook 7 is not directly above the center of gravity of the suspended load SL, it can stop the rising of the hook 7 and prompt the person in charge (slinging worker WK or operator of the work machine 100) to lower the suspended load SL onto the installation surface IS. The computing device AU can also prompt the person in charge to correct the position of the hook 7.

[0087] This configuration has the effect of more reliably suppressing swaying or detachment of the suspended load SL during ground lifting, or detachment of the suspended load SL during transport. This is because it is possible to determine whether the hook 7 is directly above the center of gravity of the suspended load SL after a part of the suspended load SL has been lifted and before ground lifting is performed, and it is possible to inform the rigging worker WK if the hook 7 is not directly above the center of gravity of the suspended load SL.

[0088] Furthermore, a lifting operation support method for supporting the lifting operation of a suspended load SL according to an embodiment of the present disclosure includes the steps of: an imaging device CM imaging the suspended load SL; a calculation device AU identifying the center of gravity position of the suspended load SL based on the image captured by the imaging device CM; and an output device communicating to the lifting worker WK the center of gravity position of the suspended load identified by the calculation device AU, or the position of the hook 7 for lifting the suspended load SL calculated based on the center of gravity position of the suspended load SL.

[0089] This method has the effect of suppressing swaying or detachment of the suspended load SL during ground clearance, or detachment of the suspended load SL during transport.

[0090] Furthermore, the rigging support program according to the embodiment of this disclosure is a program that causes a computer to execute a method for supporting rigging work of a suspended load SL, and causes the computer to execute the following steps: causing an imaging device CM to image the suspended load SL; causing a calculation device AU to identify the center of gravity position of the suspended load SL based on the image captured by the imaging device CM; and causing an output device to output information that conveys to the rigging worker WK the center of gravity position of the suspended load SL identified by the calculation device AU, or the position of the hook 7 for lifting the suspended load SL calculated based on the center of gravity position of the suspended load SL.

[0091] This program has the effect of suppressing swaying or detachment of the suspended load SL during ground clearance, or detachment of the suspended load SL during transport.

[0092] Furthermore, in the above-described embodiment, the work machine 100 is a mobile crane, but it may also be a fixed crane. Alternatively, the work machine 100 may be a shovel (excavator) equipped with a crane function. That is, the work machine 100 may be a shovel equipped with a hook housed on the rear side of the bucket. Also, the rigging support system SYS does not necessarily include at least one of the work machine 100 and the control device 300. For example, the rigging support system SYS may consist of a combination of the support device 200 and the control device 300, a combination of the work machine 100 and the support device 200, or the support device 200 alone.

[0093] Furthermore, in the above-described embodiment, the rigging support system SYS is configured to support rigging work when lifting the load SL using the hook 7 of the work machine 100. However, it may also be configured to support rigging work when lifting the load SL using a chain block such as a manual chain block or an electric chain block.

[0094] Preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the embodiments described above, nor is it limited to the embodiments described later. Various modifications or substitutions can be applied to the embodiments described above or later without departing from the scope of the present invention. Furthermore, features described separately can be combined as long as no technical inconsistencies arise. [Explanation of Symbols]

[0095] 1...Lower traveling body 2...Slewing mechanism 3...Upper slewing body 4...Boom 5...Wire rope 6...Hook bracket 7, 7T...Hook 8...Cabin 100...Working machine 200...Support device 300...Management device AU...Calculation unit AU1...Working machine side controller AU2...Client side controller AU3...Server side controller CM...Imaging device CM1...First imaging device CM2...Second imaging device CM3...Third imaging device DS...Display device DS1...First display device DS2...Second display device DS3...Third display device GC...Composite center of gravity GC1...First center of gravity GC2...Second center of gravity ID...Input device ID1...First input device ID2...Second input device ID3...Third input device SL...Suspended load SL1...First suspended load SL2...Second suspended load SW...Lifting wire SW1...First lifting wire SW2...Second lifting wire SW3...Third lifting wire SW4...Fourth lifting wire SYS...Lifting work support system TD...Communication device TD1...First communication device TD2...Second communication device TD3...Third communication device WK...Lifting worker WK1...First lifting worker WK2...Second lifting worker WK3...Third lifting worker

Claims

1. A lifting support system that assists in lifting loads, An imaging device for imaging the aforementioned suspended load, A calculation device that determines the center of gravity of the suspended load based on the image captured by the imaging device, The system includes an output device that communicates to the rigging operator the position of the center of gravity of the suspended load identified by the calculation device, or the position of the hook for lifting the suspended load calculated based on the center of gravity of the suspended load. Slinging and lifting support system.

2. The output device is a display device that displays a work support image showing the center of gravity position of the suspended load specified by the calculation device, or the position of the hook for lifting the suspended load calculated based on the center of gravity position of the suspended load. The lifting operation support system according to claim 1.

3. The aforementioned work support image is superimposed on the image captured by the imaging device. The lifting operation support system according to claim 2.

4. The calculation device is configured to determine the center of gravity of the suspended load before the lifting wire for lifting the load is attached to the hook. The lifting operation support system according to claim 1.

5. The imaging device captures images of the hook rising while multiple lifting wires for lifting the load are attached to the hook. The calculation device is configured to re-determine the center of gravity of the suspended load based on the image captured by the imaging device. The lifting operation support system according to claim 1.

6. The imaging device captures images of the state when a part of the suspended load leaves the installation surface. The calculation device is configured to re-determine the center of gravity of the suspended load based on the image captured by the imaging device. The lifting operation support system according to claim 1.

7. A method for supporting lifting operations, which assists in lifting operations for suspended loads, The imaging device takes an image of the suspended load, The steps include: a computing device determining the center of gravity of the suspended load based on the image captured by the imaging device; The process includes the step of an output device communicating to a rigging worker the position of the center of gravity of the suspended load identified by the calculation device, or the position of the hook for lifting the suspended load calculated based on the center of gravity of the suspended load. Sling work support method.

8. A program that causes a computer to execute a method to assist in the lifting and rigging of suspended loads, The steps include causing the imaging device to image the suspended load, The steps include: causing a computing device to determine the center of gravity of the suspended load based on the image captured by the imaging device; The computer is made to perform the following steps: output to an output device information that conveys to the rigging worker the position of the center of gravity of the suspended load, which has been specified by the calculation device, or the position of the hook for lifting the suspended load, which is calculated based on the center of gravity of the suspended load; program.

Images