Strike process support system, strike process support method and program for work machine

Abstract

A rigging support system is provided, which assists in the rigging process for a suspended load and includes an image acquisition device that captures an image of the suspended load; a calculating device that identifies a center of gravity position of the suspended load based on the image captured by the image acquisition device; and an output device that notifies a rigger of the center of gravity position of the suspended load identified by the calculating device or of the position of a hook for lifting the suspended load. The position of the hook is calculated based on the center of gravity position of the suspended load.

Description

BACKGROUND 1. Technical field

[0001] The present disclosure relates to an impact support system, an impact support method and a program for a working machine. 2. Description of the state of the art

[0002] In related prior art, an anti-sway stop guidance system is known that is capable of reducing the oscillation of a suspended load lifted by a crane (see Japanese unexamined patent publication no. 2019-214464). This system is configured to reduce the oscillation of a suspended load caused by boom deflection that occurs when the load is lifted. In particular, the system is configured to take into account boom deflection that occurs when a load is lifted and indicates to a crane operator an anti-sway stop position where the center of mass (center of gravity) of the suspended load is farther from the crane than a position directly below the tip of the boom at the start of the load lift.

[0003] However, the system described above is not configured to provide useful information to a hanger. Therefore, the system cannot provide support for a hanger operation performed by the hanger. SUMMARY

[0004] A rigging support system according to one embodiment of the present invention is a rigging support system that assists a rigging operation for a suspended load, comprising an image acquisition device that captures an image of the suspended load; a computing device that identifies a center of gravity position of the suspended load based on the image captured by the image acquisition device; and an output device that communicates to a rigger the center of gravity position of the suspended load identified by the computing device or a position of a hook for lifting the suspended load. The position of the hook is calculated based on the center of gravity position of the suspended load.

[0005] The above-described stop-run support system can assist a stop run performed by a stopper, BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 is a schematic diagram illustrating a configuration example of an impact support system according to an embodiment of the present disclosure; Fig. 2 is a block diagram that shows the configuration example of the in Fig. 1 illustrated impact support system; Fig. 3 is a flowchart illustrating an example of a sequence of attack process support preparation; Fig. 4 is a diagram illustrating an example of a construction site where a striking operation is carried out; Fig. 5 is a diagram illustrating another example of a construction site where a jacking operation is carried out; Fig. Figure 6 is a diagram illustrating an example of an image taken by a camera mounted at the tip of a boom; Fig. Figure 7 is a view illustrating an example of the state of a suspended load before the suspended load is lifted from the ground; Fig. 8 is a flowchart illustrating an example of a new identification processing process from centroid position; and Fig. Figure 9 is a flowchart illustrating another example of the new identification processing process from centroid position. DETAILED DESCRIPTION

[0006] The following describes embodiments of the present disclosure with reference to the drawings. The embodiments described below are not intended to limit the invention, but rather to illustrate it, and all features and combinations thereof described as embodiments are not essential to the invention. In the drawings, the same or corresponding components are identified by the same or corresponding reference numerals, and their description may be omitted.

[0007] First, with reference to the Fig. 1 and Fig. 2 a stop-action support system SYS is described, which is a system for a working machine according to an embodiment of the present disclosure. Fig. Figure 1 is a schematic diagram illustrating a configuration example of the SYS impact support system. Fig. Figure 2 is a block diagram illustrating the configuration example of the SYS impact support system.

[0008] The SYS rigging support system is a system that assists a rigging operation performed by a rigger. In the illustrated example, the SYS rigging support system primarily comprises a working machine 100, a support device 200, and an administration device 300. The working machine 100, the support device 200, and the administration device 300 each include a communication device TD and are directly or indirectly connected to each other via an information communication network IN, such as a cellular network, a satellite communication network, or a short-range radio communication network. Each of the working machine 100, the support device 200, and the administration device 300 included in the SYS rigging support system can be a single device or multiple devices.In the example shown, the SYS stop operation support system comprises a working machine 100, a support device 200 and an administration device 300.

[0009] In the illustrated example, the working machine 100 is a mobile crane and comprises a superstructure 3, which is rotatably mounted on a chassis 1 via a swivel mechanism 2. A boom 4 is attached to the superstructure 3. A wire rope 5 hangs from a tip of the boom 4, and a hook 7 is suspended from the wire rope 5 via a hook bracket 6. The hook bracket 6 has an internal roller (not illustrated) over which the wire rope 5 passes.

[0010] An image acquisition device CM (first image acquisition device CM1) is mounted at the tip of the boom 4. In the illustrated example, the first image acquisition device CM1 is a monocular camera and is positioned to capture an image of a space vertically below it. The first image acquisition device CM1 could, for example, be a stereo camera, a red-green-blue depth camera (RGB-D camera), or a light detection and ranging (LiDAR) device capable of measuring the distance between the first image acquisition device CM1 and an object.

[0011] The working machine 100 is capable of lifting a suspended load SL by winding the wire rope 5 with a front winch (not shown) and thereby raising the hook 7. The working machine 100 is also capable of lowering the suspended load SL by unwinding the wire rope 5 with the front winch (not shown) and thereby lowering the hook 7.

[0012] The superstructure 3 is equipped with a cabin 8, which serves as an operator compartment, and with a drive source such as a motor. Furthermore, the superstructure 3 is equipped with a computing device AU (a machine-side control AU1), a display device DS (a first display device DS1), an input device ID (a first input device ID1), a communication device TD (a first communication device TD1), and the like. It should be noted that the drive source can be an electric motor powered by a battery or an external power supply. The first input device ID1 can be a touch panel, a hardware button, or a microphone for voice input.

[0013] The machine-side control AU1 is configured to control the machine 100. In the illustrated example, the machine-side control AU1 is an example of a processing circuit that serves as a control device (computing unit AU) and is implemented by a computer comprising a central processing unit (CPU), random access memory (RAM), non-volatile random access memory (NVRAM), read-only memory (ROM), and the like. The machine-side control AU1 reads programs corresponding to the respective functional elements from the ROM, loads the programs into the RAM, and instructs the CPU to perform the corresponding processing. However, each functional element can be implemented in hardware or by a combination of software and hardware.

[0014] The Support Device 200 is a device that assists an attack operation performed by an attacker. In the illustrated example, the Support Device 200 is a portable client computer (e.g., a mobile endpoint device such as a laptop personal computer (PC), a tablet PC, or a smartphone) carried by an attacker and includes a Computing Device AU (a client-side controller AU2), an Image Capture Device (a second Image Capture Device CM2), a Display Device DS (a second Display Device DS2), an Input Device ID (a second Input Device ID2), and a Communication Device TD (a second Communication Device TD2). It should be noted that the Support Device 200 can function as a server. The Support Device 200 can be a portable endpoint device, such as extended reality (XR) glasses.

[0015] The client-side controller AU2 is configured to control the support device 200. In the illustrated example, the client-side controller AU2 is another example of a processing circuit acting as a computing device AU, implemented by a computer comprising a CPU, RAM, NVRAM, ROM, and the like. The client-side controller AU2 reads programs corresponding to specific functional elements from the ROM, loads the programs into RAM, and instructs the CPU to execute the corresponding processes. Each functional element can be implemented in hardware or by a combination of software and hardware.

[0016] In the illustrated example, the support device 200 is a smartphone, the second image acquisition device CM2 is a camera integrated into the smartphone, the second display device DS2 is an organic electroluminescent (EL) display, and the second input device ID2 is a touch panel. It should be noted that the second image acquisition device CM2 could be, for example, a stereo camera, an RGB-D camera, or a LiDAR device capable of measuring the distance between the second image acquisition device CM2 and an object. The second input device ID2 could be a microphone for voice input.

[0017] The management device 300 is a device that manages an operation performed by the work machine 100. In the illustrated example, the management device 300 is a server computer installed in an administrative center or the like, located remotely from the construction site (the work machine 100), and comprises a computing device AU (a server-side controller AU3), a display device DS (a third display device DS3), an input device ID (a third input device ID3), and a communication device TD (a third communication device TD3). It should be noted that the management device 300 can be a portable computer (e.g., a mobile device such as a laptop PC, a tablet PC, or a smartphone).

[0018] In the illustrated example, the management device 300 is a desktop PC, the third display device DS3 is a liquid crystal display, and the third input device ID3 is a keyboard and a mouse.

[0019] The server-side controller AU3 is configured to control the management device 300. In the illustrated example, the server-side controller AU3 is yet another example of a processing circuit acting as a computing device AU, implemented by a computer comprising a CPU, RAM, NVRAM, ROM, and the like. The server-side controller AU3 reads programs corresponding to specific functional elements from the ROM, loads the programs into RAM, and instructs the CPU to perform the appropriate processing. Each functional element can be implemented in hardware or by a combination of software and hardware.

[0020] Next, with reference to Fig. 3 An example of a processing flow is described in which the SYS rigging support system supports a rigging operation performed by a rigger (hereinafter referred to as "rigging support processing"). Fig. Figure 3 is a flowchart illustrating an example of a stroke support processing sequence. In the illustrated example, the client-side controller AU2 of support device 200 initiates stroke support processing when a predefined operation is performed by a user of support device 200 using the second input device ID2. The predefined operation is, for example, touching a predefined symbol. In the illustrated example, the predefined symbol is a start symbol for initiating a stroke support application. It should be noted that the predefined operation could be pressing a predefined hardware button, such as a stroke support start button, or speaking a predefined keyword.

[0021] First, the client-side controller AU2 captures an image of the attached load SL (step ST1). In the illustrated example, when the start icon is touched, the client-side controller AU2 displays a start screen of the stop operation support application and also shows a text message such as "Please capture the attached load".

[0022] After this text message is displayed, a slinger who is the user of the support device 200 captures an image of the attached load SL using the second image capture device CM2, which is mounted on the support device 200.

[0023] After capturing the image of the attached load SL, the client-side controller AU2 identifies the attached load SL (step ST2). In the illustrated example, the client-side controller AU2 identifies the attached load SL using a known image recognition technique. It should be noted that the client-side controller AU2 can display a text message to the operator, such as "Track the outline of the attached load in the image." This is intended to facilitate the identification of the attached load SL.

[0024] After identifying the attached load SL, the client-side controller AU2 acquires information about the attached load SL (step ST3). In the illustrated example, the client-side controller AU2 accesses the management device 300 and reads information about the identified attached load SL stored in a memory device of the management device 300. This is because, if the attached load SL is a standardized product such as a building material, information about the attached load SL is typically pre-stored in the memory device of the management device 300. The information regarding the identified attached load SL can include the position (three-dimensional coordinates) of the center of mass (center of gravity).In this case, the three-dimensional coordinates of the center of gravity can be pre-stored, for example, as relative coordinates defined with respect to multiple three-dimensional coordinates on the surface of the suspended load SL. This allows the three-dimensional coordinates of the center of gravity to be uniquely determined when multiple three-dimensional coordinates are identified on the surface of the suspended load SL. Alternatively, the three-dimensional coordinates (relative coordinates) of the center of gravity can be dynamically determined based on at least one of the previously stored dimensions, material, weight, product number, or similar attributes of the suspended load SL.

[0025] Alternatively, the client-side controller AU2 can capture an image of the attached load SL taken from a different angle. In this case, the client-side controller AU2 can display a text message to the rigger, such as "Please pick up the attached load from the left, right, and back of the attached load." It should be noted that the terms "front," "back," "left," "right," "top," and "bottom" represent directions from the rigger's perspective. The same applies to the following description.

[0026] After acquiring information about the attached load SL, the client-side controller AU2 identifies the position of the center of gravity of the attached load SL (step ST4). In the illustrated example, the client-side controller AU2 links each of the multiple three-dimensional coordinates on the surface of the attached load SL, received from the management device 300, to a corresponding point on the image of the attached load SL. Subsequently, the client-side controller AU2 links the three-dimensional coordinates of the center of gravity of the attached load SL, received from the management device 300, to a corresponding point (two-dimensional coordinates) on the image of the attached load SL.

[0027] Alternatively, the client-side controller AU2 can identify the shape of the suspended load SL from the captured image of the suspended load SL and identify the position of the center of gravity of the suspended load SL from the identified shape. In particular, the client-side controller AU2 can, for example, generate a three-dimensional model of the suspended load SL by using multiple images obtained by capturing images of the suspended load SL from multiple viewpoints and a known three-dimensional reconstruction technique. The three-dimensional reconstruction technique is a technique for estimating three-dimensional information from two-dimensional images captured by the image acquisition device CM and includes, for example, a photogrammetry technique or a Neural Radiance Field (NeRF) technique.Then, the client-side controller AU2 can identify a point (two-dimensional coordinates) on the image of the attached load SL that corresponds to the center of gravity of the attached load SL, based on the identification of the three-dimensional model.

[0028] To identify the center of gravity of the attached load SL from images or a three-dimensional model of the attached load SL, a machine learning technique is used. Specifically, the client-side controller AU2 can utilize a trained model, generated by analyzing (training) a large number of images or three-dimensional models, to output the center of gravity position based on the input images or three-dimensional models of the attached load SL. This machine learning model could be, for example, a neural network model trained using backpropagation.

[0029] After identifying the position of the center of gravity of the suspended load SL, the client-side controller AU2 displays an operation support image (step ST5). In the illustrated example, the client-side controller AU2 displays a graphic image (e.g., a circular image) representing the position of the center of gravity of the suspended load SL as the operation support image (center of gravity image) superimposed on the image of the suspended load SL. The image of the suspended load SL can be a still image or a moving image currently being captured by the second image acquisition device CM2. If the image of the suspended load SL is a moving image, the display position of the center of gravity image changes according to the movement of the sling carrying the support device 200, that is, according to the movement of the image of the suspended load SL displayed on the second display device DS2.

[0030] The rigger can identify the position of the center of gravity of the suspended load SL by viewing the center of gravity image superimposed on the image of the suspended load SL displayed on the second display device DS2. Therefore, the rigger can guide the hook 7 to a position directly above the center of gravity of the suspended load SL.

[0031] Next, with reference to Fig. 4 a procedure is described in which the rigger WK checks the position of the center of gravity of the suspended load SL using the rigging support system SYS. Fig. Figure 4 is a diagram illustrating an example of a construction site where a jacking operation is carried out.

[0032] In particular, attach to the in Fig. Figure 4 illustrates a construction site in which a second rigger WK2 and a third rigger WK3 attach a sling SW to the suspended load SL, while a first rigger WK1 checks the position of the center of gravity of the suspended load SL using the support device 200. It should be noted that the first rigger WK1 carries a tablet PC as the support device 200.

[0033] In particular, the first rigger WK1 uses the second image acquisition device CM2 of the support device 200 to capture an image of the suspended load SL. The image captured by the second image acquisition device CM2 (captured image) is transmitted to the management device 300. The management device 300 applies a known image recognition technique to the image received from the support device 200 to identify the type of object that is the suspended load SL. In a case where details such as the dimensions, weight, or center of gravity of a target object (the suspended load SL) of the crane operation are already registered, the management device 300 can identify the center of gravity of the suspended load SL by identifying which of the registered objects is the suspended load SL shown in the captured image.The registered items include, for example, containers, wall materials and flooring materials.

[0034] Alternatively, the management device 300 can apply a known image recognition technique to one or more captured images received by the support device 200, analyze the three-dimensional shape of the attached load SL, and identify the position of the center of gravity of the attached load SL.

[0035] The information (information on the center of gravity position) relating to the center of gravity of the attached load SL identified by the management device 300 is then transmitted to the support device 200. The support device 200 overlays and displays an operation support image SG at a suitable position on the image displayed on the second display device DS2 (on the image captured by the second image capture device CM2), based on the received center of gravity position information. The operation support image SG is, for example, a graphic image showing the center of gravity position of the attached load SL. It should be noted that the operation support image SG can be a graphic image indicating a position where hook 7 should be positioned (typically a position directly above the center of gravity of the attached load SL).

[0036] The first rigger WK1 can check the position of the center of gravity of the suspended load SL by viewing the operation support screen SG displayed on the second display device DS2. Therefore, for example, the first rigger WK1 can instruct the operator of the work machine 100, via the communication device TD or similar, to move the hook 7 to a position directly above the center of gravity of the suspended load SL. Specifically, the first rigger WK1 can move the hook 7 to a position directly above the center of gravity of the suspended load SL by informing the operator of the work machine 100 with information such as "Move the hook 7 30 cm to the right and 30 cm back." This information can be provided verbally or as text.It should be noted that the SYS lashing support system can automatically issue essentially the same instruction to the operator of the work machine 100, regardless of any instruction from the first lasher, WK1. The SYS lashing support system can be configured to ensure continuous voice communication between the lasher, WK, and the operator of the work machine 100. This means that the work machine 100 and the support device 200 can each include devices necessary for voice communication, such as a microphone and a speaker.

[0037] In the example above, the server-side controller AU3 of the management device 300 identifies the type of object of the attached load SL and identifies the position of the center of gravity of the attached load SL. However, at least one of the identification processes can be performed by the work machine-side controller AU1 of the work machine 100 or by the client-side controller AU2 of the support device 200.

[0038] Next, with reference to Fig. 5 another procedure is described in which the first rigger WK1 checks the position of the center of gravity of the suspended load SL using the rigging process support system SYS. Fig. 5 is a diagram illustrating another example of a construction site where a rigging operation is carried out. The first rigger WK1 at the in Fig. The construction site shown in section 5 features an augmented reality (AR) headset, which serves as the support device 200, similar to the first WK1 attacker at the site in Fig. The construction site illustrated in section 4 differs. Otherwise, the procedure corresponds to checking the position of the center of gravity of the suspended load SL by the first rigger WK1 at the point shown. Fig. Figure 5 illustrated the construction site procedure for checking the position of the center of gravity of the suspended load SL by the first rigger WK1 at the Fig. The construction site in question is illustrated in section 4. Therefore, redundant descriptions are omitted below, and the difference is described in detail.

[0039] In particular, the AR glasses, which serve as the support device 200, comprise a camera designed to capture a front view image and a pair of small displays, each intended for the eyes of the first operator WK1. The AR glasses then overlay an operation support image SG at a suitable position onto an image displayed on the second display device DS2 (the pair of small displays), that is, onto an image captured by the second image capture device CM2 (the camera), based on information about the center of gravity position received from the management device 300. The operation support image SG is a graphic image indicating the position of the center of gravity of the attached load SL.

[0040] The first rigger WK1 can check the position of the center of gravity of the suspended load SL by viewing the operation support image SG displayed on the second display device DS2. That is, with this configuration, the first rigger WK1 can check the position of the center of gravity of the suspended load SL without taking his or her eyes off the suspended load SL and without having both hands occupied by the support device 200.

[0041] Next, with reference to Fig. 6 describes another example of a method by which the management device 300 identifies the attached load SL or the position of its center of gravity. Fig. Figure 6 is a diagram showing an example of an image GM taken by the first image acquisition device CM1, which is mounted at the tip of the boom 4 of the working machine 100. The first image acquisition device CM1 is mounted at the tip of the boom 4 to take an image of a vertically downward-facing view and to capture an image of the hook 7 and its surroundings.

[0042] In particular, image GM is an image taken by the first image acquisition device CM1 above the suspended load SL and includes an image G5 of the wire rope 5, an image G6 of the hook support 6 and an image GSL of the suspended load SL.

[0043] The management device 300 is configured to use not only an image (an image taken by the second image acquisition device CM2) received by the support device 200, but also the image GM to identify what type of object the attached load SL is, and furthermore to identify the position of the center of gravity of the attached load SL.

[0044] With this configuration, the management device 300 can identify the type of attached load SL with greater accuracy than in a case where the type of attached load SL is identified solely based on the image received by the support device 200 (the image captured by the second image acquisition device CM2). This is because an image (an image of the attached load SL taken directly from above) that cannot be captured by the second image acquisition device CM2, which is mounted on the support device 200 and carried by the first slinger WK1, can be used in addition.

[0045] The same applies to the accuracy of identifying the position of the center of gravity of the attached load SL.

[0046] However, the management device 300 can be configured to identify the attached load SL based solely on an image acquired by the first image acquisition device CM1, and can further be configured to identify the position of the center of gravity of the attached load SL. Alternatively, the management device 300 can additionally or alternatively use an image acquired by an image acquisition device other than the first image acquisition device CM1 and the second image acquisition device CM2.This means that the management device 300 can be configured to identify the type of suspended load SL and can further be configured to identify the position of the center of gravity of the suspended load SL based on at least one image taken by the first image acquisition device CM1, the image taken by the second image acquisition device CM2, or the image taken by the other image acquisition device. It should be noted that the other image acquisition device could, for example, be an image acquisition device attached to a structure such as a steel tower at a construction site, or an image acquisition device attached to an aircraft such as a multicopter flying over the suspended load SL.

[0047] Next, with reference to Fig. 7 an effect is described which is obtained by the SYS striking process support system, which supports a striking process carried out by the striker WK. Fig. Figure 7 is a diagram illustrating an example of the state of the suspended load SL before it is lifted from the ground. It should be noted that "lifting from the ground" of the suspended load SL refers to an operation in which the suspended load SL is separated (lifted) from an installation surface IS (a ground surface on which the suspended load SL is placed). In the diagram shown... Fig. In the illustrated example 7, the suspended load SL is a combination of a first suspended load SL1 and a second suspended load SL2, and the second suspended load SL2 is stacked on top of the first suspended load SL1. Four slings SW (from a first sling SW1 to a fourth sling SW4) are attached between the suspended load SL and the hook 7. Fig. In diagram 7, the third anchor rope SW3 is not visible behind the first anchor rope SW1, and the fourth anchor rope SW4 is not visible behind the second anchor rope SW2. The length of the first anchor rope SW1 is essentially the same as the length of the third anchor rope SW3, and the length of the second anchor rope SW2 is essentially the same as the length of the fourth anchor rope SW4. The lengths of the first anchor rope SW1 and the third anchor rope SW3 are greater than the lengths of the second anchor rope SW2 and the fourth anchor rope SW4.

[0048] At the in Fig. In the illustrated example 7, hook 7 is located directly above the combined center of gravity GC of the suspended load SL, and each of the four slings SW (from the first sling SW1 to the fourth sling SW4) is in a slightly slackened state. This state is implemented by the sling support processing and is referred to below as a "ready state".

[0049] At the in Fig. The 7 illustrated example shows the first flagger WK1 (see Fig. 4) Easily identify the position of the combined center of gravity GC of the suspended load SL, derived from the combined center of gravity of a first center of gravity GC1 and a second center of gravity GC2, by viewing the operation support image SG displayed on the second display device DS2. It should be noted that the first center of gravity GC1 is the center of gravity of the first suspended load SL1, and the second center of gravity GC2 is the center of gravity of the second suspended load SL2. That is, the first rigger WK1 can easily identify the position of the combined center of gravity GC of the suspended load SL, which is difficult to determine based on the external appearance of the suspended load SL.

[0050] Furthermore, the first rigger WK1, who views the process support image SG, can estimate in advance the length of each of the four rigging ropes SW (from the first rigging rope SW1 to the fourth rigging rope SW4) in the ready state. For example, the first rigger WK1 can then inform the second rigger WK2 (see Fig. 4) and the third flagger WK3 (see Fig. 4) Notify the system of the lengths of the four sling ropes SW in the ready state before the four sling ropes SW are attached to the hook 7. Therefore, the slinging support system SYS can prevent the use of a sling rope SW of an unsuitable length.

[0051] Since the appropriate length of each of the four rigging ropes SW can be known in advance, the first rigger WK1 can determine a suitable temporary position (a position that differs from the position (target position) in the ready state) of the hook 7 to which each of the four rigging ropes SW can be easily attached. Fig. Figure 7 illustrates hook 7 in its temporary position, indicated by hook 7T in dashed lines. It should be noted that the temporary position of hook 7 is, for example, a position reachable by the hands of the second hitter WK2 and the third hitter WK3.

[0052] In particular, the first rigger WK1 can control an operation performed by the operator of the work machine 100 via the communication device TD or the like, such that the hook 7 moves into the temporary position. It should be noted that the rigging operation support system SYS can be configured to display a graphic image on the display device DS, showing at least one of the target position or the temporary position of the hook 7 as the operation support image SG. In this case, the rigging operation support system SYS can omit the display of a graphic image showing the center of gravity position of the suspended load SL.

[0053] Afterwards, the second rigger WK2 and the third rigger WK3 can hook each of the four sling ropes SW onto hook 7 while hook 7 is in the temporary position. Then, after verifying that each of the four sling ropes SW is appropriately hooked onto hook 7, the first rigger WK1 can carry out an operation performed by the operator of the working machine 100 via the communication device TD or the like, so that hook 7 moves to the target position.

[0054] Then, after verifying that hook 7 is positioned at the target position, the first rigger WK1 can instruct the operator of the working machine 100 via the communication device TD or the like to start a process to lift the attached load SL.

[0055] As a result, the working machine 100 can lift the attached load SL directly upwards and lift the attached load SL off the ground with little or no oscillation of the attached load SL.

[0056] Next, an example of processing with reference to Fig. 8 described in which the position of the center of gravity of the attached load SL, which was identified during the attachment process support processing, is re-identified (hereinafter referred to as the “center of gravity position re-identification processing”). Fig. Figure 8 is a flowchart illustrating an example of a center of gravity position re-identification processing sequence. In the illustrated example, the client-side controller AU2 of the support device 200 starts the center of gravity position re-identification processing when a predetermined operation is performed by the first rigger WK1 using the second input device ID2 (see Figure 8). Fig. 4) The predetermined action is, for example, touching a predetermined symbol.

[0057] First, the client-side controller AU2 captures a series of images depicting a state in which the attached load SL is being lifted (step ST11). In the illustrated example, when a predetermined symbol is touched, the client-side controller AU2 causes the second display device DS2 to display a text message such as "Please pick up the attached load." After this text message is displayed, the first slinger WK1 directs the second image capture device CM2 toward the attached load SL and begins capturing a moving image.

[0058] The operator of the work machine 100 starts lifting the hook 7 in response to an instruction from the rigging process support system SYS or the first rigger WK1. Each of the four rigging ropes SW (see Fig. 3) changes from a slightly loosened state to a tense state when hook 7 is lifted.

[0059] The client-side controller AU2 then detects the movement of each sling SW (step ST12). The movement of the sling SW includes, for example, the magnitude of the movement, the tension of the sling SW, or similar parameters. For instance, before the suspended load SL is lifted from the ground, the client-side controller AU2 can detect, based on the image series captured by the second image acquisition device CM2, the point in time at which each of the four slings SW becomes taut.

[0060] The client-side controller AU2 then re-identifies the center of gravity position of the suspended load SL based on the movement of each sling SW (step ST13). For example, if the client-side controller AU2 determines that the time at which the first sling SW1 becomes taut is delayed by a predetermined time or more compared to the times at which the other three slings SW become taut, the client-side controller AU2 determines that hook 7 is not directly above the center of gravity position of the suspended load SL. In this case, the client-side controller AU2 re-identifies the center of gravity position by moving the current center of gravity position closer to the lower end of the first sling SW1. It should be noted that the determination of whether or not hook 7 is directly above the center of gravity position of the suspended load SL can be made using a machine learning technique.

[0061] In this case, the client-side controller AU2 can change the display position of the operation support image SG (a graphic image that shows the position of the center of gravity of the attached load SL), which is superimposed on the image of the attached load SL, based on the newly identified center of gravity position.

[0062] The operator of the work machine 100 stops the lifting of the hook 7 in response to an instruction from the rigging process support system SYS or the first rigger WK1 and then lowers the hook 7 to the original target position.

[0063] The first rigger WK1 can then instruct the operator of the work machine 100, via the communication device TD or similar, to move the hook 7 to a position directly above the newly identified center of gravity. It should be noted that the rigging support system SYS can automatically issue essentially the same instruction to the operator of the work machine 100, regardless of an instruction from the first rigger WK1. Alternatively, the first rigger WK1 can adjust the length of each of the four rigging ropes SW based on the newly identified center of gravity.

[0064] This new identification of the center of gravity position allows the SYS rigging support system, even after the center of gravity of the suspended load SL has been identified in the ready state, to determine whether hook 7 is directly above the center of gravity of the suspended load SL when lifting of hook 7 is initiated. Therefore, even after lifting of hook 7 has started, the SYS rigging support system can notify the first rigger WK1 that hook 7 is not directly above the center of gravity of the suspended load SL. The SYS rigging support system can also display the newly identified center of gravity position to the first rigger WK1 in an easily understandable manner.Therefore, the SYS lifting support system can prevent the suspended load SL from being lifted if the hook 7 is not directly above the center of gravity of the suspended load SL, and thus suppress swinging of the suspended load SL after lifting from the ground.

[0065] Next, another example of the processing for the new identification of the focus position will be given with reference to Fig. 9 described. Fig. 9 is a flowchart that illustrates another example of a processing sequence for the new identification of the center of gravity position. The one in Fig. The illustrated process for re-identifying the main focus position differs from the one in Fig. Figure 8 illustrated the process for re-identifying the center of gravity position by stating that the center of gravity position of the suspended load SL is re-identified based on the movement of the suspended load SL when a portion of the suspended load SL is detached from the installation surface IS before the suspended load SL is lifted from the ground. Otherwise, this corresponds to the process described in Figure 8. Fig. 9 illustrated procedures for the new identification of the main position in the Fig. Section 8 illustrated the procedure for the new identification of the center of gravity position. Therefore, redundant descriptions are omitted below, and only the difference is described in detail. It should be noted that "when part of the suspended load SL is separated from the installation area IS" means, in other words, that "another part of the suspended load SL is not yet separated from the installation area IS".

[0066] First, the client-side controller AU2 captures a series of images of a state where the attached load SL is raised (step ST21), as in the Fig. Figure 8 illustrated the focus position re-identification processing.

[0067] The client-side controller AU2 then detects the movement of the suspended load SL when a portion of the suspended load SL is detached from the installation surface IS (step ST22). The movement of the suspended load SL includes tilting, translation, rotation, or the like. For example, based on the image series acquired by the second image acquisition device CM2, the client-side controller AU2 can detect that a lower left front corner section of the suspended load SL, which has a substantially rectangular parallelepiped shape, is being lifted and that the suspended load SL is rotating counterclockwise about a vertical axis, with a lower right rear corner section of the suspended load SL serving as a pivot point.

[0068] The client-side controller AU2 then reidentifies the center of gravity position of the attached load SL based on the movement of the attached load SL (step ST23). For example, if the client-side controller AU2 determines that the attached load SL has rotated counterclockwise around a vertical axis, with the lower right rear corner section of the attached load SL acting as a pivot point, the client-side controller AU2 determines that hook 7 is not directly above the center of gravity position of the attached load SL. Specifically, the client-side controller AU2 determines that the correct center of gravity position, relative to the current incorrect center of gravity position, is on the left rear side. In this case, the client-side controller AU2 reidentifies the center of gravity position by shifting the current incorrect center of gravity position further toward the left rear side.

[0069] In this case, the client-side controller AU2 can change the display position of the operation support image SG (a graphic image that shows the position of the center of gravity of the attached load SL), which is superimposed on the image of the attached load SL, based on the newly identified center of gravity position.

[0070] The operator of the work machine 100 stops the lifting of the hook 7 in response to an instruction from the rigging process support system SYS or the first rigger WK1 and then lowers the hook 7 to the original target position.

[0071] The first rigger WK1 can then instruct the operator of the work machine 100, via the communication device TD or similar, to move the hook 7 to a position directly above the newly identified center of gravity. It should be noted that the rigging process support system SYS can automatically give the operator of the work machine 100 essentially the same instruction, independent of an instruction from the first rigger WK1.

[0072] Through this center of gravity position re-identification processing, the rigging support system SYS can inform the first rigger WK1 that hook 7 is not directly above the center of gravity of the suspended load SL. The rigging support system SYS can also present the newly identified center of gravity position to the first rigger WK1 in an easily understandable way. Therefore, the rigging support system SYS can prevent the suspended load SL from being lifted if hook 7 is not directly above the center of gravity of the suspended load SL, thus suppressing swinging of the suspended load SL after it has been lifted from the ground.

[0073] At the in Fig. In the illustrated example 9, the client-side controller AU2 is configured to detect the movement of the attached load SL based on the image series captured by the second image acquisition device CM2. However, if a detection device that detects the position and orientation of the hook 7 is provided on the hook 7, the client-side controller AU2 can be configured to detect the movement of the attached load SL based on an output from the detection device. In particular, the client-side controller AU2 can receive data output by the detection device of the working machine 100 via the communication device TD or the like, and estimate the movement of the attached load SL from the movement of the hook 7 detected based on the received data.It should be noted that the detection device that detects the position and orientation of hook 7 is, for example, a GNSS (Global Navigation Satellite System) compass, a gyroscope, an accelerometer, or a combination thereof.

[0074] Through such center of gravity position re-identification processing, the lifting support system SYS, even after the center of gravity position of the suspended load SL has been identified in the ready state, can determine whether hook 7 is directly above the center of gravity position of the suspended load SL when part of the suspended load SL is detached from the installation surface. Therefore, even after lifting of the suspended load SL has started, the lifting support system SYS can notify the first rigger WK1 that hook 7 is not directly above the center of gravity position of the suspended load SL. The lifting support system SYS can also display the newly identified center of gravity position to the first rigger WK1 in an easily understandable manner.Therefore, the SYS lifting support system can prevent the suspended load SL from being lifted if the hook 7 is not directly above the center of gravity of the suspended load SL, and thus suppress swinging of the suspended load SL after lifting from the ground.

[0075] As described above, the rigging support system SYS, according to the embodiments of the present disclosure, comprises the image acquisition device CM, which captures an image of the suspended load SL; the calculating device AU, which identifies the center of gravity position of the suspended load SL based on the image captured by the image acquisition device CM; and the output device, which notifies the rigger WK of the center of gravity position of the suspended load SL determined by the calculating device AU, or the position of the hook for lifting the suspended load SL, wherein the position of the hook 7 is calculated based on the center of gravity position of the suspended load SL (see, for example, the hook 7T in Fig. 7) It should be noted that the computing device AU is implemented, for example, by at least one of the machine-side controller AU1, the client-side controller AU2, or the server-side controller AU3. In the illustrated examples, the computing device AU is a client-side controller AU2. The image acquisition device CM, which captures an image of the attached load SL, is, for example, at least one of the first image acquisition device CM1, which is attached to the machine 100, the second image acquisition device CM2, which is mounted on the support device 200, the image acquisition device, which is attached to a structure such as a steel tower at a construction site, or the image acquisition device, which is attached to an aircraft such as a multicopter, which is flying above the attached load SL.In the illustrated examples, the image acquisition device CM is the second image acquisition device CM2. The output device can be a laser emission device, a projection device, a projector, or the like, attached to the working machine 100 or to a structure such as a steel tower at a construction site. In this case, for example, the laser emission device, serving as the display device DS, can indicate to the rigger WK the position of the center of gravity of the suspended load SL by emitting a laser beam to one or more positions on the surface of the suspended load SL corresponding to the position of the center of gravity of the suspended load SL.The output device can be an audio output device, such as a loudspeaker, that acoustically notifies the operator WK of various types of information, or it can be a vibration generation device that tactilely notifies the operator WK of various types of information. The lifting operation support system SYS can be configured to generate an image, sound, or vibration according to a distance between the position of hook 7 and the position of the center of gravity of the suspended load SL (a distance on an imaginary plane parallel to the installation surface). For example, the lifting operation support system SYS can emit an intermittent sound with a shorter output interval as the distance between the position of hook 7 and the position of the center of gravity of the suspended load SL decreases.

[0076] This configuration can assist the rigger WK in the rigging operation by providing WK ​​with information regarding the position of the center of gravity of the suspended load SL. This allows WK to attach the sling SW directly above the center of gravity of the suspended load SL to hook 7. Therefore, this configuration effectively prevents the suspended load SL from swinging or falling during lifting from the ground, as well as from falling during transport. In other words, this configuration enables the rigging operation to be carried out efficiently without relying on the experience, skills, know-how, intuition, or similar attributes of a seasoned rigger.Furthermore, the rigger WK can eliminate a complicated process in which he repeatedly has to make fine adjustments to the position of hook 7 relative to the suspended load SL while repeatedly lifting the suspended load SL by small amounts using the work machine 100. Therefore, this configuration reduces the workload for the rigger WK. This configuration also reduces the time required for a rigging operation performed by the rigger WK.

[0077] The output device can be the display device DS, which displays the operation support screen SG, showing the center of gravity position of the attached load SL identified by the computing device AU, or the position of hook 7 (see hook 7T in Fig. 7) for lifting the suspended load SL, whereby the position of hook 7 is calculated based on the center of gravity position of the suspended load SL. In this case, the operation support screen SG is displayed at a position visible to the rigger.

[0078] This configuration provides an effect of notifying the rigger WK in an easily understandable way about the position of the center of gravity of the suspended load SL, compared to a case where the rigger WK is notified acoustically or tactilely about the position of the center of gravity of the suspended load SL.

[0079] As in Fig.As illustrated in Figure 4, the operation support image SG can be superimposed on and displayed alongside the image captured by the image acquisition device CM. However, the operation support image SG can also be an image displayed independently in a different area than the area where the image of the attached load SL is displayed. In this case, the operation support image SG can be a combination of a computer graphics (CG) image representing the shape of the attached load SL and an image representing the center of gravity position, which is superimposed on and displayed alongside the CG image. Alternatively, the operation support image SG can be a text message, such as, “The center of gravity is located 30 cm to the right, 50 cm below, and 40 cm behind the apex on the upper left front side of the attached load SL.”

[0080] The configuration in which the operation support image SG is superimposed and displayed on the captured image taken by the image acquisition device CM provides an effect of notifying the rigger WK in an easily understandable way about the position of the center of gravity of the attached load SL, compared to the configuration in which the operation support image is displayed in a different area than the area where the captured image is displayed.

[0081] The calculating device AU is typically configured to identify the center of gravity position of the suspended load SL before the lifting rope SW is attached to the hook 7 for lifting the suspended load SL.

[0082] This configuration provides an advantage by efficiently supporting the rigging operation performed by the rigger WK, compared to a case where the center of gravity of the suspended load SL is identified after the rigging rope SW has been attached to hook 7. This is because the frequency of reattaching the rigging rope SW to hook 7 can be reduced.

[0083] The image acquisition device CM can capture an image of a state in which hook 7 is being raised while the multiple lifting slings SW are attached to hook 7 for lifting the suspended load SL. In this case, the calculation device AU can detect movement of at least one of the multiple lifting slings SW based on the image captured by the image acquisition device CM in a state in which hook 7 is being raised while the multiple lifting slings SW are attached to hook 7. The calculation device AU can then be configured to re-identify the center of gravity position of the suspended load SL based on the detected movement.

[0084] Therefore, this configuration provides an effect that more reliably suppresses swinging or falling of the suspended load SL when lifting it from the ground, as well as falling of the suspended load SL during transport. This is because it is possible to determine immediately before lifting the suspended load SL from the ground whether the hook 7 is directly above the center of gravity of the suspended load SL, and to notify the rigger WK that the hook 7 is not directly above the center of gravity of the suspended load SL.

[0085] The image acquisition device CM can capture an image of a state in which part of the suspended load SL is detached from the installation surface IS. In this case, the computing device AU can be configured to detect movement of the suspended load SL based on the image captured by the image acquisition device CM at a time when part of the suspended load SL is detached from the installation surface IS, and to re-identify the center of gravity position of the suspended load SL based on this movement. It should be noted that this re-identification of the center of gravity position is preferably performed before the suspended load SL is lifted from the ground, that is, before the suspended load SL is detached from the installation surface IS.If the calculating device AU determines, based on the movement, that hook 7 is not directly above the center of gravity of the suspended load SL, the calculating device AU can stop the lifting of hook 7 and request a responsible person (the slinger WK or the operator of the work machine 100) to lower the suspended load SL onto the installation surface IS. The calculating device AU can also request the responsible person to change the position of hook 7.

[0086] Therefore, this configuration provides an effect that more reliably prevents swinging or falling of the suspended load SL during lifting from the ground, as well as falling of the suspended load SL during transport. This is because it is possible to determine whether the hook 7 is directly above the center of gravity of the suspended load SL after part of the suspended load SL has been lifted, but before the suspended load SL is lifted from the ground, and to notify the rigger WK that the hook 7 is not directly above the center of gravity of the suspended load SL.

[0087] The rigging support method for supporting a rigging operation for the suspended load SL according to an embodiment of the present disclosure comprises a step of capturing an image of the suspended load SL by the image capture device CM, a step of identifying the center of gravity position of the suspended load SL by the computing device AU based on the image captured by the image capture device CM, and a step of notifying the rigger WK by the output device of the center of gravity position of the suspended load SL identified by the computing device AU or the position of the hook 7 for lifting the suspended load SL, wherein the position of the hook 7 is calculated based on the center of gravity position of the suspended load SL.

[0088] This method provides an effect of suppressing oscillation or falling of the suspended load SL during lifting from the ground as well as falling of the suspended load SL during transport.

[0089] The rigging support program according to an embodiment of the present disclosure is a program that causes a computer to execute a method for supporting a rigging operation for the suspended load SL, and the program causes the computer to perform: a step of capturing an image of the suspended load SL by the image capture device CM, a step of identifying the center of gravity position of the suspended load SL by the computing device AU based on the image captured by the image capture device CM, and a step of notifying the rigger WK by the output device of the center of gravity position of the suspended load SL identified by the computing device AU or the position of the hook 7 for lifting the suspended load SL, wherein the position of the hook 7 is calculated based on the center of gravity position of the suspended load SL.

[0090] This program provides an effect of suppressing oscillation or falling of the suspended load SL during lifting from the ground as well as falling of the suspended load SL during transport.

[0091] In the embodiment described above, the working machine 100 is a mobile crane, but it can also be a stationary crane. Alternatively, the working machine 100 can be an excavator with a crane function. That is, the working machine 100 can be an excavator equipped with a hook located on the rear of the bucket. The rigging support system SYS can exclude at least one of the working machine 100 or the management device 300. For example, the rigging support system SYS can be configured by a combination of the support device 200 and the management device 300, by a combination of the working machine 100 and the support device 200, or by the support device 200 alone.

[0092] In the embodiment described above, the sling support system SYS is configured to support a sling operation in which the suspended load SL is lifted by the hook 7 of the working machine 100. However, the sling support system SYS can also be configured to support a sling operation in which the suspended load SL is lifted using a chain hoist, such as a manual chain hoist or an electric chain hoist.

[0093] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the embodiments described above, nor is it limited to the embodiments described below. Various modifications, substitutions, and the like can be applied to the embodiments described above or below without departing from the scope of the present invention. Furthermore, the separately described features can be combined, provided no technical inconsistencies arise. REFERENCE MARK LIST 1 Undercarriage 2 swivel mechanism 3. Superstructure 4 outriggers 5 wire rope 6 hook holder 7.7 t hook 8 cabins 100 working machines 200 support device 300 Administrative device AU calculating device AU1 Machine-side control AU2 Client-side control AU3 Server-side control CM image acquisition device CM1 First Image Acquisition Device CM2 Second image acquisition device CM3 Third Image Acquisition Device DS display device DS1 First Display Device DS2 Second Display Device DS3 Third Display Device GC Combined Focus GC1 First focus GC2 Second Focus ID input device ID1 First input device ID2 Second Input Device ID3 Third Input Device SL Attached Load SL1 First attached load SL2 Second attached load SW mooring rope SW1 First anchor rope SW Second mooring rope SW3 Third mooring rope SW4 Fourth mooring rope SYS stop action support system TD communication device TD1 First Communication Device TD2 Second Communication Device TD3 Third Communication Device WK sniper WWI First Assault WWII Second Assault WK3 Third Assault QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2019-214464

[0002]

Claims

[1] An attachment support system that assists in the attachment of a suspended load, the system comprising: an image recording device that captures an image of the suspended load; a computing device that identifies a center of gravity position of the suspended load based on the image captured by the image-taking device; and an output device that notifies a rigger of the center of gravity position of the suspended load identified by the computing device or of the position of a hook for lifting the suspended load, wherein the position of the hook is calculated based on the center of gravity position of the suspended load. [2] Lifting support system according to claim 1, wherein the output device is a display device that displays a work support image indicating the center of gravity position of the suspended load or the position of the hook for lifting the suspended load. [3] A stop process support system according to claim 2, wherein the process support image is superimposed on and displayed over the image captured by the image acquisition device. [4] Lifting process support system according to claim 1, wherein the calculating device is configured to identify the center of gravity position of the suspended load before a lifting rope is attached to the hook for lifting the suspended load. [5] Attack process support system according to claim 1, wherein the image recording device captures an image of a state in which the hook is being raised while several lifting ropes for the suspended load are attached to the hook, and The computing device is configured to re-identify the center of gravity position of the attached load based on the image captured by the image acquisition device. [6] Attack process support system according to claim 1, wherein the image recording device captures an image of a condition in which part of the suspended load is separated from an installation surface, and The computing device is configured to re-identify the center of gravity position of the attached load based on the image captured by the image acquisition device. [7] An attachment support system that supports an attachment process for a suspended load, wherein the procedure comprises the steps: Taking an image of the suspended load using an image capture device; Identifying the center of gravity position of the suspended load using a computing device based on the image captured by the image acquisition device; and Notifying a rigger operator by means of an output device of the center of gravity position of the suspended load identified by the calculating device, or of a position of a hook for lifting the suspended load, wherein the position of the hook is calculated on the basis of the center of gravity position of the suspended load. [8] Program that causes a computer to execute a procedure to assist in a lifting operation for an attached load, the procedure comprising the steps: Taking an image of the suspended load using an image capture device; Identifying the center of gravity position of the suspended load using a computing device based on the image captured by the image acquisition device; and Notifying a rigger by means of an output device of the center of gravity position of the suspended load identified by the computing device or of a position of a hook for lifting the suspended load, wherein the position of the hook is calculated on the basis of the center of gravity position of the suspended load.

Images