Lifting support system and program for industrial machinery

Abstract

To provide a lifting support system that can assist lifting operations performed by lifting operators. [Solution] The rigging support system SYS, which assists in the rigging work of a suspended load SL, comprises an imaging device CM that images the suspended load SL, a first display device DS1 that can be seen by a support worker EX located at a distance from the rigging worker WK performing the rigging work, a second display device DS2 that can be seen by the rigging worker WK, a first input device ID1 used by the support worker EX when inputting work support information to assist the rigging work performed by the rigging worker WK, and a control device CR that displays the work support information input through the first input device ID1 on the second display device DS2 along with the image captured by the imaging device CM.

Description

Technical Field

[0001] The present disclosure relates to a ball hanging work support system and a program for a working machine.

Background Art

[0002] Conventionally, a crane having a function of assisting an operator of a crane who receives instructions verbally from a ball hanging worker is known (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the above-mentioned crane cannot assist the ball hanging work by the ball hanging worker at all.

Means for Solving the Problems

[0005] A ball hanging work support system according to an embodiment of the present disclosure is a ball hanging work support system for assisting the ball hanging work of a suspended load, and includes an imaging device for imaging the suspended load, a first display device visible to a work support person located at a position away from the ball hanging worker who performs the ball hanging work, a second display device visible to the ball hanging worker, a first input device used when the work support person inputs work support information for assisting the ball hanging work by the ball hanging worker, and a control device that causes the second display device to display the work support information input through the first input device together with the image captured by the imaging device.

Effects of the Invention

[0006] The above-mentioned ball hanging work support system can assist the ball hanging work by the ball hanging worker. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic diagram showing an example configuration of a rigging work support system according to the present disclosure. [Figure 2] Figure 1 is a block diagram showing an example configuration of a rigging support system. [Figure 3] A flowchart illustrating an example of the process for supporting rigging operations. [Figure 4] This figure shows an example of a work site where rigging work is performed. [Figure 5] This flowchart shows another example of the process flow for supporting rigging operations. [Figure 6] This is a schematic diagram showing another configuration example of the rigging work support system according to the present disclosure. [Modes for carrying out the invention]

[0008] The embodiments of this disclosure will be described below with reference to the drawings. Furthermore, the embodiments described below are illustrative and not limiting to the invention, and not all features or combinations thereof described as embodiments are necessarily essential to the invention. In each drawing, identical or corresponding components are denoted by the same or corresponding reference numerals, and their descriptions may be omitted.

[0009] First, with reference to Figures 1 and 2, a lifting support system SYS, which is a system for work machines according to the embodiment of this disclosure, will be described. Figure 1 is a schematic diagram showing an example configuration of the lifting support system SYS. Figure 2 is a block diagram showing an example configuration of the lifting support system SYS.

[0010] The slinging support system SYS is a system that supports slinging work performed by slinging workers WK. In the illustrated example, the slinging support system SYS mainly consists of a first support device 100, a second support device 200, a work machine 300, and a management device 400. Each of the first support device 100, the second support device 200, the work machine 300, and the management device 400 is equipped with a communication device TD and is directly or indirectly connected to each other via an information communication network IN such as a mobile phone communication network, a satellite communication network, or a short-range wireless communication network. Each of the first support device 100, the second support device 200, the work machine 300, and the management device 400 that constitute the slinging support system SYS may be one unit or multiple units. In the illustrated example, the slinging support system SYS includes one first support device 100, one second support device 200, one work machine 300, and one management device 400.

[0011] The first support device 100 is a device that assists rigging work performed by rigging worker WK. In the illustrated example, the first support device 100 is a portable client computer (e.g., a notebook PC, tablet PC, or smartphone) carried by a work supporter EX who is located away from rigging worker WK, and includes a control device CR (first client-side controller CR1), an imaging device (first imaging device CM1), a display device DS (first display device DS1), an input device ID (first input device ID1), and a communication device TD (first communication device TD1). The work supporter EX is typically a skilled rigging worker who remotely assists rigging work performed by rigging worker WK at the work site from a location away from the work site. The first support device 100 may also function as a server. Furthermore, the first support device 100 may be a wearable terminal device such as XR (augmented reality) goggles.

[0012] The first client-side controller CR1 is configured to control the first support device 100. In the illustrated example, the first client-side controller CR1 is an example of a processing circuit as a control device CR, and is composed of a computer equipped with a CPU, RAM, ROM, and a non-volatile storage device. The first client-side controller CR1 reads the program corresponding to each functional element from ROM and loads it into RAM, and causes the CPU to execute the corresponding processing. However, each functional element may be composed of hardware, or it may be composed of a combination of software and hardware.

[0013] In the illustrated example, the first support device 100 is a smartphone, the first imaging device CM1 is the smartphone's rear camera, the first display device DS1 is an organic EL display, and the first input device ID1 is a touch panel. The first imaging device CM1 may be an imaging device capable of measuring the distance between the first imaging device CM1 and the subject, such as a stereo camera, RGBD camera, or LiDAR. The first input device ID1 may be a microphone for voice input.

[0014] The second support device 200 is a device that assists the rigging work performed by the rigging worker WK. In the illustrated example, the second support device 200 is a portable client computer (e.g., a notebook PC, tablet PC, or smartphone) carried by the rigging worker WK, and includes a control device CR (second client controller CR2), an imaging device (second imaging device CM2), a display device DS (second display device DS2), an input device ID (second input device ID2), and a communication device TD (second communication device TD2). The second support device 200 may also function as a server. Furthermore, the second support device 200 may be a wearable terminal device such as XR (augmented reality) goggles.

[0015] The second client-side controller CR2 is configured to control the second support device 200. In the illustrated example, the second client-side controller CR2 is another example of a processing circuit as a control device, and is configured by a computer including a CPU, a RAM, a ROM, a non-volatile storage device, and the like. Then, the second client-side controller CR2 reads a program corresponding to each functional element from the ROM and loads it into the RAM, and causes the CPU to execute the corresponding processing. However, each functional element may be configured by hardware, or may be configured by a combination of software and hardware.

[0016] In the illustrated example, the second support device 200 is a smartphone, the second imaging device CM2 is the out-camera of the smartphone, the second display device DS2 is an organic EL display, and the second input device ID2 is a touch panel. Note that the second imaging device CM2 may be an imaging device such as a stereo camera, an RGBD camera, or a LiDAR that can measure the distance between the second imaging device CM2 and the subject. Also, the second input device ID2 may be a microphone for voice input.

[0017] The working machine 300 is a mobile crane, and an upper revolving body 3 is mounted on a crawler-type lower traveling body 1 via a slewing mechanism 2 so as to be able to revolve. A boom 4 is attached to the upper revolving body 3. A wire rope 5 extends downward from the tip of the boom 4, and a hook 7 is suspended from the wire rope 5 via a hook bracket 6. The hook bracket 6 has a pulley (not shown) for passing the wire rope 5 inside.

[0018] An imaging device CM (third imaging device CM3) is attached to the tip of the boom 4. In the illustrated example, the third imaging device CM3 is a monocular camera and is arranged so as to be able to image the space vertically below. Note that the third imaging device CM3 may be an imaging device such as a stereo camera, an RGBD camera, or a LiDAR that can measure the distance between the third imaging device CM3 and the subject.

[0019] The work machine 300 can lift the hook 7 to lift the suspended load SL by winding up the wire rope 5 with a front winch (not shown). Also, the work machine 300 can lower the hook 7 to lower the suspended load SL by paying out the wire rope 5 with the front winch.

[0020] The upper revolving body 3 is provided with a cabin 8 as a driver's cab and is equipped with a power source such as an engine. Also, on the upper revolving body 3, a control device CR (work machine side controller CR3), a display device DS (third display device DS3), an input device ID (third input device ID3), a communication device TD (third communication device TD3), etc. are attached. Note that the power source may be an electric motor driven by a battery or an external power source. Also, the work machine 300 may be a remotely operated work machine.

[0021] The work machine side controller CR3 is configured to be able to control the work machine 300. In the illustrated example, the work machine side controller CR3 is another example of a processing circuit as the control device CR and is composed of a computer equipped with a CPU, RAM, ROM, a non-volatile storage device, etc. Then, the work machine side controller CR3 reads out the program corresponding to each functional element from the ROM and reads it into the RAM, and causes the CPU to execute the corresponding processing. However, each functional element may be composed of hardware or may be composed of a combination of software and hardware.

[0022] In the illustrated example, the third display device DS3 is an organic EL display, and the third input device ID3 is a touch panel. Note that the third input device ID3 may be a microphone for voice input.

[0023] The management device 400 is a device that manages the work performed by the work machine 300. In the illustrated example, the management device 400 is a server computer installed in a management center or the like, located away from the work site (work machine 300), and includes a control device CR (server-side controller CR4), a display device DS (fourth display device DS4), an input device ID (fourth input device ID4), and a communication device TD (fourth communication device TD4). The management device 400 may also be a portable computer (for example, a notebook PC, tablet PC, or a mobile terminal device such as a smartphone).

[0024] In the illustrated example, the management device 400 is a desktop PC, the fourth display device DS4 is a liquid crystal display, and the fourth input device ID4 is a keyboard and mouse.

[0025] The server-side controller CR4 is configured to control the management device 400. In the illustrated example, the server-side controller CR4 is yet another example of a processing circuit as a control device CR, and is composed of a computer equipped with a CPU, RAM, ROM, and non-volatile memory device. The server-side controller CR4 reads the program corresponding to each functional element from ROM and loads it into RAM, and causes the CPU to execute the corresponding processing. However, each functional element may be composed of hardware, or it may be composed of a combination of software and hardware.

[0026] Furthermore, the server-side controller CR4 may have a machine learning unit ML. The machine learning unit ML is configured to learn from the content of the support provided by the work supporter EX. The machine learning is, for example, machine learning using a machine learning model such as a neural network model formed using backpropagation technology. The machine learning unit ML may be implemented by at least one of the first client-side controller CR1, the second client-side controller CR2, the work machine-side controller CR3, and the server-side controller CR4, or by other independent control devices.

[0027] Next, referring to Figure 3, an example of the flow of the process by which the rigging support system SYS assists rigging workers in rigging operations (hereinafter referred to as "rigging support process") will be described. Figure 3 is a flowchart of an example of the flow of the rigging support process. In the illustrated example, the first client-side controller CR1 of the first support device 100 starts this rigging support process when a predetermined operation using the first input device ID1 is performed by the support worker EX. The first client-side controller CR1 may also start this rigging support process when it receives a support request signal from the second support device 200. The second support device 200 may, for example, send a support request signal to the first support device 100 when a predetermined operation using the second input device ID2 is performed by the rigging worker WK. The predetermined operation is, for example, touching a predetermined icon. In the illustrated example, the predetermined icon is a startup icon for launching the rigging support application. The predetermined operation may also be pressing a predetermined hardware button such as a work support start button, or uttering a predetermined keyword, i.e., performing a predetermined voice input.

[0028] First, the first client-side controller CR1 acquires an image of the suspended load SL (step ST1). In the illustrated example, when the activation icon is touched, the first client-side controller CR1 outputs an image acquisition command to the second support device 200. Upon receiving the image acquisition command, the second client-side controller CR2 of the second support device 200 displays the initial screen of the rigging work support application on the second display device DS2, and also displays a text image such as "Please acquire an image of the suspended load."

[0029] Upon seeing this text image, the rigging worker WK uses the second imaging device CM2 mounted on the second support device 200 to image the suspended load SL. Subsequently, the second client-side controller CR2, having acquired the image of the suspended load SL (suspended load image), transmits the suspended load image to the first support device 100. The suspended load image may be a still image or a moving image. The first client-side controller CR1 of the first support device 100, upon receiving the suspended load image, displays the suspended load image on the first display device DS1. The suspended load image may be an image of the suspended load SL taken from above by the third imaging device CM3 attached to the tip of the boom 4 of the work machine 300. Alternatively, the suspended load image may be an image of the suspended load SL taken by another imaging device, such as an imaging device attached to a structure such as a steel tower installed at the work site, or an imaging device attached to an aircraft such as a multicopter flying over the work site. Alternatively, the suspended load image may be any combination of multiple images taken from various viewpoints as described above.

[0030] Subsequently, the first client-side controller CR1 prompts the work support worker EX to input work support information (step ST2). In the illustrated example, the first client-side controller CR1 displays a text image such as "Please input work support information" on the first display device DS1.

[0031] Work support information is information to support the lifting work performed by the lifting operator WK. Specifically, work support information includes graphic images showing the position of the center of mass (center of gravity) of the suspended load SL, graphic images showing the position on the suspended load SL where one end of the lifting wire SW should be attached, graphic images showing the position where the hook 7 should be positioned, or graphic images representing the length of the lifting wire SW. Note that graphic images may also be icons.

[0032] When a worker EX views a suspended load image displayed on the first display device DS1, they can select a desired position on the suspended load image by, for example, touching a position on the touch panel, which is the first input device ID1, to a desired position on the suspended load image. The desired position on the suspended load image could be, for example, the position corresponding to the center of gravity of the suspended load SL, the position where one end of the lifting wire SW should be attached, or the position where the hook 7 should be positioned. Alternatively, the worker EX can indicate the desired arrangement and length of the lifting wire SW by tracing a desired virtual line on the suspended load image with their finger. Alternatively, the worker EX can select a desired range on the suspended load image by tracing a virtual circle with their finger that encloses the desired range on the suspended load image. The desired range on the suspended load image could be, for example, the range corresponding to the center of gravity of the suspended load SL, the range where one end of the lifting wire SW should be attached, or the range where the hook 7 should be positioned. Furthermore, if the first input device ID1 is a touchpad or mouse, operations involving touching the touch panel with a finger can be replaced by click or drag operations performed while moving the cursor on the screen.

[0033] The first client-side controller CR1 overlays graphic images representing points touched by the work support worker EX's fingers, or lines traced by the work support worker EX's fingers, as work support information onto the image of the suspended load displayed on the first display device DS1.

[0034] Subsequently, the first client-side controller CR1 instructs the second support device 200 to display the suspended load image and work support information (step ST3). This is so that the rigging worker WK can see the suspended load image and work support information displayed on the second display device DS2 of the second support device 200. The image (suspended load image and work support information) displayed on the second display device DS2 may also be displayed on at least one of the third display device DS3 and the fourth display device DS4. Furthermore, the display characteristics of the image (suspended load image and work support information) displayed on the second display device DS2 (suspended load image and work support information), such as size, color, and appearance (from which the suspended load SL is viewed), may be the same as or different from the image (suspended load image and work support information) displayed on the first display device DS1. The same applies to the images (suspended load image and work support information) displayed on the third display device DS3 and the fourth display device DS4, respectively.

[0035] When rigger WK sees the image of the suspended load and work support information displayed on the second display device DS2, he can efficiently carry out the rigging work by utilizing the knowledge of the skilled worker, work supporter EX. For example, if the work support information indicates the position of the center of gravity of the suspended load SL, rigger WK can instruct the operator VP of the work machine 300 to position the hook 7 directly above the position of the center of gravity indicated by the work support information.

[0036] The suspended load image may be a moving image captured by the second imaging device CM2 mounted on the second support device 200. In this case, the display position of the work support information (indicating the position of the center of gravity of the suspended load SL) set in the first support device 100 may be changed in accordance with changes in the appearance of the suspended load image, that is, in accordance with the movement of the rigging worker WK carrying the second support device 200 (second imaging device CM2). This is to ensure that the work support information can continuously indicate the position of the center of gravity of the suspended load SL. Furthermore, the work support worker EX may correct the display position of the work support information in real time. This is because the work support worker EX may change their opinion regarding the position of the center of gravity of the suspended load SL by viewing the suspended load image captured from a different viewpoint after inputting the work support information. Changes to the display position of the work support information on the first display device DS1 may be reflected in real time in the changes to the display position of the work support information on the second display device DS2.

[0037] Furthermore, the rigging support system SYS may be configured to enable constant voice communication between the rigging worker WK, the support worker EX, and the operator VP who operates the work machine 300. That is, the first support device 100, the second support device 200, and the work machine 300 may all be equipped with devices necessary for voice communication, such as microphones and speakers.

[0038] Next, referring to Figure 4, we will explain how a support worker EX assists a rigging worker WK with rigging work using the rigging work support system SYS. Figure 4 is a diagram showing an example of a work site where rigging work is performed. In the example shown in Figure 4, the support worker EX assists the rigging worker WK with rigging work from a remote location far from the work site.

[0039] Specifically, in the work site shown in Figure 4, rigger WK is attempting to attach the rigging wire SW to the load SL, and is using the second support device 200 to confirm the position of the center of gravity of the load SL. Rigger WK is carrying a tablet PC, which is the second support device 200.

[0040] More specifically, the rigging worker WK is taking an image of the suspended load SL with the second imaging device CM2 of the second support device 200. The image of the suspended load captured by the second imaging device CM2 (the captured image) is then transmitted to the management device 400. The management device 400 then transmits (transfers) the suspended load image received from the second support device 200 to the first support device 100.

[0041] Upon receiving the suspended load image from the management device 400, the first support device 100 displays the suspended load image on the first display device DS1 so that the work support worker EX can view the suspended load image. The work support worker EX then touches the position on the suspended load image with their finger where they believe the center of gravity of the suspended load SL is located, determining the position where the work support information SG will be displayed. In the illustrated example, the work support information SG is a graphic image (a circular image) indicating the position of the center of gravity of the suspended load SL, and is displayed so as to be located inside the suspended load SL, which is actually invisible.

[0042] The first client-side controller CR1 may generate a three-dimensional model of the suspended load SL by, for example, using multiple images obtained by imaging the suspended load SL from multiple viewpoints and known three-dimensional reconstruction techniques. Three-dimensional reconstruction techniques are techniques for estimating three-dimensional information from two-dimensional images captured by the imaging device CM, and include, for example, photogrammetry or NeRF (Neural Radiance Fields) techniques. The work support worker EX may then identify a point (two-dimensional coordinate) on the image of the suspended load SL that corresponds to the center of gravity of the suspended load SL, based on the three-dimensional model. Specifically, the work support worker EX may determine the position where the work support information SG is displayed after viewing the three-dimensional model of the suspended load SL from any direction. In this case, the work support worker EX may fine-tune the position where the work support information SG is displayed using various GUIs (Graphical User Interfaces) provided by the first support device 100.

[0043] Subsequently, the information regarding the position of the center of gravity of the suspended load SL identified by the work support worker EX (center of gravity position information) is transmitted to the second support device 200. Based on the received center of gravity position information, the second support device 200 superimposes the work support information SG onto the image displayed on the second display device DS2 (on the image captured by the second imaging device CM2) at an appropriate position. The work support information SG may be a graphic image showing the position of the center of gravity of the suspended load SL, a graphic image showing the position where the hook 7 should be positioned (typically directly above the center of gravity of the suspended load SL), or a graphic image showing the position on the suspended load SL where the rigging wire SW should be attached.

[0044] The rigging worker WK can confirm the position of the center of gravity of the suspended load SL, as presented by the work support person EX, by viewing the work support information SG displayed on the second display device DS2. Therefore, the rigging worker WK can, for example, instruct the operator VP of the work machine 300 via the communication device TD to move the hook 7 to a position directly above the center of gravity of the suspended load SL. Specifically, the rigging worker WK can guide the hook 7 to a position directly above the center of gravity of the suspended load SL by transmitting information such as, "Move the hook 7 30 cm to the right and 30 cm backward" to the operator VP of the work machine 300. Note that "right" and "back" refer to the direction as viewed from the rigging worker WK. The same applies to "left," "front," "up," and "down," and is also true in the following descriptions. In addition, the work support person EX may, if necessary, give instructions to the operator VP of the work machine 300 via the communication device TD.

[0045] Next, with reference to Figure 5, another example of the rigging support process will be described. Figure 5 is a flowchart showing another example of the flow of the rigging support process. The rigging support process shown in Figure 5 differs from the rigging support process shown in Figure 3 in that it has steps ST4 and ST5 which are executed after step ST3, but is otherwise the same as the rigging support process shown in Figure 3. Therefore, the explanation of the common parts will be omitted below, and the differences will be explained in detail.

[0046] The first client-side controller CR1 instructs the second support device 200 to display the suspended load image and work support information, and then stores the pre-support image and post-support image in association (step ST4). This is to allow verification of the support provided by the work supporter EX, or to allow the machine learning unit ML to learn the content of the support provided by the work supporter EX. The pre-support image is an image showing the state of the suspended load SL before support is provided by the work supporter EX, and the post-support image is an image showing the state of the suspended load SL after support is provided by the work supporter EX.

[0047] Specifically, if rigger WK requests confirmation from support worker EX after independently attaching the rigging wire SW to the load SL, the rigging wire SW is already attached to the load SL when support worker EX inputs the support information SG. In this case, the pre-support image is, for example, an image captured by the second imaging device CM2 before support worker EX inputs the support information SG, and includes images of the load SL and rigging wire SW at that time. The pre-support image may also include an image of the hook 7. On the other hand, the post-support image will show the state of the load SL after any changes made by rigger WK based on the position of the center of gravity of the load SL provided by support worker EX, such as the placement of the rigging wire SW, the length of the rigging wire SW, or the position of the hook 7. If rigger WK does not make any changes to the placement of the rigging wire SW, the length of the rigging wire SW, or the position of the hook 7 after support has been provided by support worker EX, the content of the post-support image will be the same as the content of the pre-support image. The first client-side controller CR1 stores, for example, the image of the suspended load acquired in step ST1 as the pre-support image, and stores the image captured by the second imaging device CM2 as the post-support image when the lifting of the hook 7 by the work machine 300 begins.

[0048] Subsequently, the first client-side controller CR1 stores the pre-ground-cutting image and the post-ground-cutting image in association (step ST5). This is to allow verification of the support provided by the work supporter EX, or to enable the machine learning unit ML to learn the content of the support provided by the work supporter EX. The pre-ground-cutting image shows the state of the suspended load SL before ground-cutting is performed, and the post-ground-cutting image shows the state of the suspended load SL after ground-cutting is performed. Furthermore, ground-cutting of the suspended load SL means lifting the suspended load SL away from the installation surface IS (the ground on which the suspended load SL is placed).

[0049] Specifically, the pre-ground-cut image is an image captured by the second imaging device CM2 after the lifting of the hook 7 to which the lifting wire SW is attached begins, but before the suspended load SL is lifted into the air. This image includes images of the suspended load SL, the lifting wire SW, and the hook 7 at that point in time. On the other hand, the post-ground-cut image is an image captured by the second imaging device CM2 after the suspended load SL has left the installation surface IS. This image includes images of the suspended load SL, the lifting wire SW, and the hook 7 at that point in time.

[0050] The content of the pre-ground cut image is typically the same as the content of the post-support image. The content of the pre-ground cut image includes, for example, the arrangement of the lifting wire SW, the length of the lifting wire SW, or the position of the hook 7. Therefore, the first client-side controller CR1 may store the pre-support image and the post-support image in association, and instead of storing the pre-ground cut image and the post-ground cut image in association, it may store the pre-support image, the post-support image, and the post-ground cut image in association.

[0051] Through this process, the first client-side controller CR1 can collect data to verify the content of the support provided by the work support worker EX each time the work support worker EX assists the rigging worker WK. Alternatively, the first client-side controller CR1 can collect sample data for the machine learning unit ML to learn the content of the support provided by the work support worker EX each time the work support worker EX assists the rigging worker WK. Furthermore, the first client-side controller CR1 can perform various judgments regarding the suspended load SL. Specifically, by comparing the pre-support image with the post-support image, the first client-side controller CR1 can determine whether the rigging worker WK changed the placement of the rigging wire SW, the length of the rigging wire SW, or the position of the hook 7 based on the work support information SG. In addition, by comparing the pre-ground-cutting image with the post-ground-cutting image, the first client-side controller CR1 can determine whether the suspended load SL swayed or fell off (hereinafter referred to as "swayed, etc.") during ground-cutting. Furthermore, the first client-side controller CR1 can determine, before ground cutting is performed, whether or not there is a risk of load swinging or other issues occurring in the suspended load SL during ground cutting, based on pre-ground cutting images and a machine learning model.

[0052] Furthermore, if the results of machine learning by the machine learning unit ML are available, the work support worker EX may check whether the work support information SG that he or she has entered will suppress the occurrence of load swinging, etc., before presenting the work support information SG to the rigging worker WK. Alternatively, the work support worker EX may determine the display position of the work support information SG by referring to the arrangement of the rigging wire SW, the length of the rigging wire SW, or the position of the hook 7, etc., which are derived by the machine learning unit ML based on the shape of the suspended load SL.

[0053] Next, with reference to Figure 6, another configuration example of the slinging support system SYS will be described. Figure 6 is a schematic diagram showing another configuration example of the slinging support system SYS. The slinging support system SYS shown in Figure 6 differs from the slinging support system SYS shown in Figure 1 in that it includes one first support device 100, two second support devices 200 (second support device 200A and second support device 200B), two work machines 300 (work machine 300A and work machine 300B), one management device 400, and one remote control device 500. However, the slinging support system SYS shown in Figure 6 is the same as the slinging support system SYS shown in Figure 1 in other respects. Therefore, the explanation of the common parts will be omitted below, and the differences will be explained in detail.

[0054] Two work machines 300 (work machine 300A and work machine 300B), located at different locations from each other, are operated sequentially by a single remote operator OP who operates a remote control device 500. In other words, the remote operator OP can manage two work sites simultaneously. The second support device 200A is a tablet PC carried by rigger WK1, who performs rigging work on the suspended load SL (suspended load SL1) lifted by work machine 300A. The second support device 200B is a tablet PC carried by rigger WK2, who performs rigging work on the suspended load SL (suspended load SL2) lifted by work machine 300B.

[0055] The remote control device 500 is located in a place far from both the first work site where the work machine 300A is located and the second work site where the work machine 300B is located, and is also called the remote control room. In other words, the remote operator OP remotely controls the work machine 300 from a location far from both work sites. Similarly, the work support worker EX assists the rigging work performed by the rigging worker WK from a location far from both work sites and the remote control device 500. Furthermore, the first work site and the second work site are located far apart from each other.

[0056] The remote control device 500 includes an operating device 42, an operating sensor 43, a control device CR (remote controller CR5), a display device DS (fifth display device DS5), and a communication device TD (fifth communication device TD5), etc. The remote control device 500 also has an operating seat DR where the remote operator OP sits to remotely operate the work machine 300A and the work machine 300B, respectively.

[0057] The fifth communication device TD5 is configured to control communication with the communication device TD attached to the work machine 300.

[0058] The remote controller CR5 is yet another example of a processing circuit as a control unit CR, and consists of a computer equipped with a CPU, RAM, ROM, and non-volatile storage device. The remote controller CR5 reads the program corresponding to each functional element from ROM and loads it into RAM, causing the CPU to execute the corresponding process. However, each functional element may be composed of hardware, or a combination of software and hardware.

[0059] The fifth display device DS5 displays a screen based on information transmitted from the work machine 300 so that the remote operator OP can visually inspect the area around the work machine 300. The information transmitted from the work machine 300 is, for example, an image captured by one or more imaging devices attached to the work machine 300. By viewing the image displayed on the fifth display device DS5, the remote operator OP can confirm the situation of the work site, including the area around the work machine 300. In the illustrated example, the fifth display device DS5 is a liquid crystal display, but it may also be an XR (augmented reality) goggle or the like.

[0060] The operating device 42 is a device used by the remote operator OP to operate actuators mounted on the work machine 300. The actuators include at least one of a hydraulic actuator and an electric actuator. In the illustrated example, the operating device 42 includes an operating lever, a travel lever, and a travel pedal. The operating lever includes a slewing lever, a boom operating lever, and a hook lifting lever.

[0061] The operating device 42 is equipped with an operation sensor 43 for detecting the operation of the operating device 42. The operation sensor 43 is, for example, a tilt sensor that detects the tilt angle of the operating lever, or an angle sensor that detects the oscillation angle of the operating lever around its pivot axis. The operation sensor 43 may also consist of other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operation sensor 43 outputs information regarding the detected operation of the operating device 42 to the remote controller CR5. The remote controller CR5 generates an operation signal based on the received information and transmits the generated operation signal to the work machine 300. This allows the remote operator OP to remotely operate the work machine 300 using the remote control device 500.

[0062] In the example shown in Figure 6, when the control device 400 receives a support request signal from the second support device 200A carried by the rigging worker WK1, it forwards the support request signal to the first support device 100 carried by the work support worker EX.

[0063] Subsequently, the first support device 100 transmits an image request signal to the second support device 200A in order to acquire an image of the suspended load SL1 (first suspended load image). Upon receiving the image request signal, the second support device 200A displays a text image such as "Please take an image of the suspended load" on the second display device DS2, prompting the rigging worker WK1 to take an image of the first suspended load.

[0064] The second support device 200A, having acquired the first suspended load image, transmits the first suspended load image to the first support device 100. The first support device 100 displays the received first suspended load image on the first display device DS1 and prompts the work support worker EX to input work support information SG (work support information SG1). The work support worker EX inputs the work support information SG1 and transmits it to the second support device 200A.

[0065] Upon receiving the work support information SG1, the second support device 200A superimposes the work support information SG1 onto the image of the suspended load SL1 displayed on the second display device DS2. After confirming the work support information SG1, the rigging worker WK1 attaches one end of the rigging wire SW (rigging wire SW1) to the suspended load SL1.

[0066] Furthermore, the rigging worker WK1 requests the remote operator OP to operate the work machine 300A via the communication device TD. Specifically, rigging worker WK1 requests the remote operator OP to move the hook 7 (hook 7A) to the desired position. The work support person EX may, as needed, request the remote operator OP to perform the necessary operations. For example, if the work support person EX determines that rigging worker WK1 is requesting an incorrect operation from the remote operator OP, they may request the remote operator OP to stop that operation. Upon receiving the request from rigging worker WK1, the remote operator OP remotely operates the work machine 300A via the communication device TD and positions the hook 7A at the position indicated by rigging worker WK1.

[0067] Subsequently, the rigger WK1 attaches the other end of the rigging wire SW1 to the hook 7A, and after preparing the load SL1, requests the remote operator OP to raise the hook 7A. The "ready to lift load SL1" state means, for example, that one end of the rigging wire SW1 is attached to the load SL1 and the other end of the rigging wire SW1 is attached to the hook 7A, that is, that the load is ready to be lifted by the hook 7A. After that, if the load SL1 is lifted off the ground without causing any swaying of the load, the rigger WK1 requests the remote operator OP to transport the load SL1 to the desired destination position. In this way, the rigger WK1 can transport the load SL1.

[0068] Subsequently, when the control device 400 receives a support request signal from the second support device 200B carried by the rigging worker WK2, it forwards that support request signal to the first support device 100 carried by the work support worker EX, just as it would when receiving a support request signal from the second support device 200A. The subsequent workflow is also the same as when receiving a support request signal from the second support device 200A.

[0069] Specifically, the first support device 100 transmits an image request signal to the second support device 200B in order to acquire an image of the suspended load SL2 (second suspended load image). Upon receiving the image request signal, the second support device 200B displays a text image such as "Please take an image of the suspended load" on the second display device DS2, prompting the rigging worker WK2 to take an image of the second suspended load.

[0070] The second support device 200B, having acquired the second suspended load image, transmits the second suspended load image to the first support device 100. The first support device 100 displays the received second suspended load image on the first display device DS1 and prompts the work support worker EX to input work support information SG (work support information SG2). The work support worker EX inputs the work support information SG2 and transmits it to the second support device 200B.

[0071] Upon receiving the work support information SG2, the second support device 200B superimposes the work support information SG2 onto the image of the suspended load SL2 displayed on the second display device DS2. After confirming the work support information SG2, the rigging worker WK2 attaches one end of the rigging wire SW (rigging wire SW2) to the suspended load SL2.

[0072] Furthermore, the rigging worker WK2 requests the remote operator OP to operate the work machine 300B via the communication device TD. Specifically, rigging worker WK2 requests the remote operator OP to move the hook 7 (hook 7B) to a desired position. Upon receiving the request from rigging worker WK2, the remote operator OP remotely operates the work machine 300B via the communication device TD and positions the hook 7B at the position indicated by rigging worker WK2.

[0073] Subsequently, the rigger WK2 attaches the other end of the rigging wire SW2 to the hook 7B, and after preparing the suspended load SL2, requests the remote operator OP to raise the hook 7B. If the suspended load SL2 is lifted off the ground without causing any swaying, the rigger WK2 requests the remote operator OP to transport the suspended load SL2 to the desired target position. In this way, the rigger WK2 can transport the suspended load SL2.

[0074] As described above, by using the rigging support system SYS shown in Figure 6, a single support worker EX can sequentially support rigging operations performed at multiple work sites from a remote location. Similarly, a remote operator OP can remotely operate work machines 300 installed at multiple work sites from a remote location. Therefore, both rigging worker WK1 at the first work site and rigging worker WK2 at the second work site can safely and efficiently carry out rigging operations by utilizing the experience and knowledge of the support worker EX, who is a skilled worker located remotely. Furthermore, both rigging worker WK1 at the first work site and rigging worker WK2 at the second work site can safely and efficiently carry out rigging operations by utilizing the operational skills of the remote operator OP, who is a skilled operator located remotely. Note that the remote operator OP and the support worker EX may be the same person.

[0075] As described above, the rigging support system SYS, according to an embodiment of the present disclosure, which assists in rigging operations of a suspended load SL, includes, as shown in Figure 1, an imaging device CM for imaging the suspended load SL, a first display device DS1 that can be viewed by a support worker EX located at a distance from the rigging worker WK performing the rigging work, a second display device DS2 that can be viewed by the rigging worker WK, a first input device ID1 used by the support worker EX when inputting work support information to assist the rigging work performed by the rigging worker WK, and a control device CR that displays the work support information input through the first input device ID1 on the second display device DS2 along with the image captured by the imaging device CM.

[0076] This configuration allows the rigging support system SYS to assist rigging workers WK by, for example, showing them work support information provided by a skilled rigging worker, work supporter EX. Specifically, even if the shape of the suspended load SL is complex and it is difficult to determine the position of its center of gravity at a glance, rigging worker WK can be informed of the position of the center of gravity of the suspended load SL by work supporter EX. Therefore, even if rigging worker WK has little experience in rigging, for example, they can accurately perform the rigging work by attaching the rigging wire SW to the hook 7 positioned directly above the center of gravity of the suspended load SL. In other words, rigging worker WK can appropriately determine the position of the hook 7 to which the rigging wire SW is attached by utilizing the experience, skills, know-how, and intuition of work supporter EX. As a result, the SYS rigging support system ensures that the load SL is lifted while maintaining its weight balance, thereby suppressing load sway or detachment during lifting or transport. Furthermore, riggers WK can reduce the tedious repetition of tasks such as repeatedly lifting and lowering the load SL to fine-tune the position of the rigging wire SW or the hook 7. Additionally, since the support worker EX can assist riggers WK from a remote location, it can efficiently support multiple riggers WK at multiple work sites. In other words, the support worker EX can manage multiple work sites simultaneously.

[0077] Furthermore, the rigging support system SYS may have a first communication device TD1 used by the support worker EX and a second communication device TD2 used by the rigging worker WK. The control device CR may be configured to display work support information on the second display device DS2 along with the image captured by the imaging device CM when voice communication is established between the first communication device TD1 and the second communication device TD2.

[0078] This configuration allows the rigging worker WK to communicate with the work support worker EX via voice while simultaneously viewing the work support information set by the work support worker EX on the second display device DS2. As a result, the rigging worker WK can understand the support provided by the work support worker EX more accurately than when voice communication is not used. Specifically, the rigging worker WK can, for example, proceed with the rigging work while viewing the same image as the work support worker EX, similar to a web conference, and while consulting with the work support worker EX. Furthermore, the rigging worker WK can proceed with the rigging work while viewing the same image as both the work support worker EX and the operator VP of the work machine 300, similar to a web conference, and while consulting with both the work support worker EX and the operator VP of the work machine 300.

[0079] Furthermore, the rigging support system SYS may have a third display device DS3 that is visible to the onboard operator VP who operates the work machine 300 that lifts the load SL. The first input device ID1 may be used when the work supporter EX inputs operation support information to assist the onboard operator VP in operating the work machine 300. The control device CR may be configured to display the operation support information input through the first input device ID1 on the third display device DS3 along with the image captured by the imaging device CM. In other words, the work supporter EX may support the onboard operator VP of the work machine 300 by inputting operation support information equivalent to work support information, similar to when supporting the rigging worker WK.

[0080] With this configuration, the rigging support system SYS can, for example, support the operation of the work machine 300 by the onboard operator VP by presenting operation support information provided by the skilled operator EX to the onboard operator VP. Specifically, even if the onboard operator VP of the work machine 300 has little experience operating the work machine 300, they will be able to accurately position the hook 7 directly above the center of gravity of the suspended load SL.

[0081] Furthermore, as shown in Figure 4, the control device CR may be configured to store, in association, a pre-ground-cutting image showing the suspended load SL and the hook 7 used to lift the suspended load SL when the lifting of the suspended load SL begins after the work support information SG is displayed on the second display device DS2, and a post-ground-cutting image showing the suspended load SL and the hook 7 when the ground-cutting of the suspended load SL is performed thereafter.

[0082] This configuration allows the slinging support system SYS to record in history how slinger WK responded to advice from support worker EX, or to what extent WK followed EX's advice. Specifically, the slinging support system SYS can record in history whether or not slinger WK moved the position of hook 7 according to support worker EX's advice. Therefore, this configuration allows for later verification of whether the support worker EX provided regarding the position of hook 7 was appropriate, or whether the slinging work performed by slinger WK was appropriate, ultimately leading to improvements in the support provided by support worker EX. Furthermore, this configuration allows for machine learning to analyze the content of the support provided by support worker EX.

[0083] Furthermore, as shown in Figure 4, the control device CR may be configured to store, in association, a pre-ground-cutting image showing the suspended load SL and the sling wire SW attached to the suspended load SL when the lifting of the suspended load SL begins after the work support information SG is displayed on the second display device DS2, and a post-ground-cutting image showing the suspended load SL and the sling wire SW when the suspended load SL is subsequently cut off from the ground.

[0084] This configuration allows the slinging support system SYS to record, for example, whether slinger WK changed the placement or length of the sling wire SW in accordance with the advice of support worker EX. Therefore, this configuration allows for later verification of whether the support provided by support worker EX regarding the placement or length of the sling wire SW was appropriate, or whether the slinging work performed by slinger WK after receiving that support was appropriate, which in turn enables improvements to the support provided by support worker EX. Furthermore, this configuration allows for machine learning to analyze the content of the support provided by support worker EX.

[0085] Furthermore, the image captured by the imaging device CM may include at least one of the images showing the top surface of the suspended load SL and the side surface of the suspended load SL.

[0086] This configuration has the effect of increasing the effectiveness of support provided by the work support system EX. This is because the work support system EX can provide more appropriate support when the image of the suspended load is appropriate. For example, if the work support system EX can use an image taken from a viewpoint that makes it easy to identify the center of gravity of the suspended load SL, it can identify the center of gravity of the suspended load SL more accurately than when an image taken from a viewpoint that makes it easy to identify the center of gravity of the suspended load SL is not available.

[0087] Furthermore, the rigging work support program according to the embodiment of this disclosure is a program that causes a computer to execute a method for supporting rigging work of a suspended load SL, and is configured to cause a computer to execute the following steps: causing an imaging device CM to image the suspended load SL; displaying the image captured by the imaging device CM on a first display device DS1 that is visible to a work support person EX located at a distance from the rigging worker WK performing the rigging work; displaying the image captured by the imaging device CM on a second display device DS2 that is visible to the rigging worker WK; inputting work support information SG to support the rigging work by the rigging worker WK to the work support person EX via a first input device ID1; and displaying the work support information SG input via the first input device ID1 along with the image captured by the imaging device CM on the second display device DS2.

[0088] This program has the effect of enabling a skilled rigger (work supporter EX) to support riggers (work supporter WK) by making the work support information presented by the skilled rigger (work supporter EX) visible to the unskilled rigger (work supporter WK).

[0089] Furthermore, in the above-described embodiment, the work machine 300 is a mobile crane, but it may also be a fixed crane. Alternatively, the work machine 300 may be a shovel (excavator) equipped with a crane function. That is, the work machine 300 may be a shovel equipped with a hook housed on the rear side of the bucket. Also, the rigging support system SYS does not have to include at least one of the work machine 300 and the control device 400. For example, the rigging support system SYS may consist of a combination of the first support device 100, the second support device 200 and the control device 400, or a combination of the first support device 100, the second support device 200 and the work machine 300, or a combination of the first support device 100 and the second support device 200.

[0090] Furthermore, in the above-described embodiment, the rigging support system SYS is configured to support rigging work when lifting the load SL using the hook 7 of the work machine 300. However, it may also be configured to support rigging work when lifting the load SL using a chain block such as a manual chain block or an electric chain block.

[0091] Preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the embodiments described above, nor is it limited to the embodiments described later. Various modifications or substitutions can be applied to the embodiments described above or later without departing from the scope of the present invention. Furthermore, features described separately can be combined as long as no technical inconsistencies arise. [Explanation of symbols]

[0092] 1...Lower traveling body 2...Slewing mechanism 3...Upper slewing body 4...Boom 5...Wire rope 6...Hook bracket 7, 7A, 7B...Hook 8...Cabin 42...Operating device 43...Operating sensor 100...First support device 200, 200A, 200B...Second support device 300, 300A, 300B...Working machine 400...Management device 500...Remote control device CR...Control device CR1...First client-side controller CR2...Second client-side controller CR3...Working machine-side controller CR4...Server-side controller CR5...Remote controller CM...Imaging device CM1...First imaging device CM2...Second imaging device CM3...Third imaging device DR...Operator's seat DS...Display device DS1...First display device DS2...Second display device DS3...Third display device DS4...Fourth display device DS5...Fifth display device EX...Work support person ID...Input device ID1...First input device ID2...Second input device ID3...Third input device ID4...Fourth input device IN...Information and communication network OP...Remote operator SG, SG1, SG2...Work support information SL, SL1, SL2...Suspended load SW, SW1, SW2...Lifting wire SYS...Lifting work support system TD...Communication device TD1...First communication device TD2...Second communication device TD3...Third communication device TD4...Fourth communication device TD5...Fifth communication device VP...Onboard operator WK, WK1, WK2... Slinging workers

Claims

1. A lifting support system that assists in lifting loads, An imaging device for imaging the aforementioned suspended load, A first display device that can be seen by a work support person located at a distance from the rigging worker performing the aforementioned rigging work, The aforementioned second display device, which is visible to the rigging worker, A first input device used by the aforementioned rigging worker when the aforementioned rigging worker inputs work support information to assist the aforementioned rigging work, The system includes a control device that displays the work support information input through the first input device on the second display device along with the image captured by the imaging device, Slinging and lifting support system.

2. The first communication device used by the aforementioned work supporter, It has a second communication device used by the aforementioned rigging worker, The control device, when voice communication is established between the first communication device and the second communication device, displays the work support information on the second display device along with the image captured by the imaging device. The lifting operation support system according to claim 1.

3. The system has a third display device that can be seen by the operator of the work machine that lifts the aforementioned load, The first input device is used when the work support person inputs operation support information to assist the operator's operation. The control device causes the operation support information input through the first input device to be displayed on the third display device along with the image captured by the imaging device. The lifting operation support system according to claim 1.

4. The control device is configured to store, in association with, a pre-ground-cutting image showing the load and the hook used to lift the load, taken when the lifting of the load is started after the work support information is displayed on the second display device, and a post-ground-cutting image showing the load and the hook, taken when the ground-cutting of the load is performed thereafter. The lifting operation support system according to claim 1.

5. The control device is configured to store, in association with, a pre-ground-cutting image showing the load and the lifting wire attached to the load when the lifting of the load begins after the work support information is displayed on the second display device, and a post-ground-cutting image showing the load and the lifting wire when the load is subsequently cut off from the ground. The lifting operation support system according to claim 1.

6. The image captured by the imaging device includes at least one of an image showing the top surface of the suspended load and an image showing the side surface of the suspended load. The lifting operation support system according to claim 1.

7. A program that causes a computer to execute a method to assist in the lifting and rigging of suspended loads, The steps include causing the imaging device to image the suspended load, The steps include: displaying the image captured by the imaging device on a first display device visible to a work support person located at a distance from the rigging worker performing the rigging work; The steps include displaying the image captured by the imaging device on a second display device visible to the rigging worker, The steps include: causing the work support worker to input work support information to assist the rigging worker through the first input device; The computer is instructed to perform the following steps: display the work support information input through the first input device on the second display device along with the image captured by the imaging device; program.

Images