Swing state determination device, construction machine, swing state determination system, swing state determination method, and program

Abstract

This swing state determination device (50) is a device for a construction machine (100) comprising: machine bodies (1, 2); an attachment (3) supported by the machine bodies (1, 2); and a suspension rope (20) suspended from a distal end portion (33) of the attachment (3). The swing state determination device (50) comprises a controller (60) that acquires detection data regarding the suspension rope (20) detected by an object information detector (40) attached to the distal end portion (33) of the attachment (3), identifies the position of an end portion (Pt) of the suspension rope (20) on the basis of the detection data, and determines the swing state of the suspension rope (20) on the basis of the position of the end portion (Pt).

Description

Vibration state determination device, construction machine, vibration state determination system, vibration state determination method, and program

[0001] The present disclosure relates to a technology for construction machines such as cranes.

[0002] Patent Document 1 discloses a suspension rope vibration determination method for determining the vibration state of a suspension rope hanging from the tip of a crane boom. This suspension rope vibration determination method includes, as steps executed by a data processing device, a detection data acquisition step of acquiring detection data including data on the three-dimensional detection position of an object detected by an object detection sensor, and a derivation step of deriving the vibration direction and vibration angle of the suspension rope based on the data on the detection position.

[0003] The object detection sensor in the suspension rope determination method described in Patent Document 1 is fixed to a portion between the tip of the boom and the base of the boom. Therefore, if there is an obstacle between an attachment such as a boom and the suspension rope, the object detection sensor may not be able to detect the suspension rope.

[0004] Japanese Unexamined Patent Application Publication No. 2021-004119

[0005] An object of the present disclosure is to provide a technology capable of determining the vibration state of a suspension rope even when there is an obstacle between an attachment and the suspension rope.

[0006] A vibration state determination device according to an aspect of the present disclosure is a vibration state determination device for a construction machine including a machine body, an attachment supported by the machine body, a suspension rope hanging from the tip of the attachment, and an object information detector attached to the tip of the attachment, the controller being configured to acquire detection data on the suspension rope detected by the object information detector, specify the position of the terminal end of the suspension rope based on the detection data, and determine the vibration state of the suspension rope based on the position of the terminal end.

[0007] This is a side view showing a construction machine according to an embodiment. This is a perspective view showing the upper part of an attachment of a construction machine according to an embodiment and a suspension rope hanging from the tip of the attachment. This is a perspective view showing the upper part of an attachment of a construction machine according to an embodiment and a suspension rope hanging from the tip of the attachment. This is a block diagram showing the components of a construction machine equipped with a swing state determination device according to an embodiment. This is a diagram showing an example of training data in which the positions of the starting end and the ending end of a suspension rope are annotated in image data captured by a camera. This is a diagram showing an example of training data in which the positions of the starting end and the ending end of a suspension rope are annotated in image data captured by a camera. This is a diagram for explaining the identification of the positions of the starting end and the ending end of a suspension rope by the swing state determination device and the determination of the swing state of the suspension rope by the swing state determination device. This is a diagram for explaining the identification of the positions of the starting end and the ending end of a suspension rope by the swing state determination device and the determination of the swing state of the suspension rope by the swing state determination device. This is a diagram for explaining the flow of how an image recognition model (AI model) is constructed by machine learning using training data. This figure illustrates the flow of the controller of the swing state determination device, which acquires the image data, identifies the position of the suspension rope, and determines the swing state. This is a flowchart illustrating an example of the calculation process performed by the controller. This is a block diagram showing the components of a swing state determination system equipped with a swing state determination device according to a modified embodiment.

[0008] Embodiments of the present disclosure will be described with reference to the drawings. Figure 1 is a side view showing a construction machine 100 according to this embodiment. The construction machine 100 comprises a lower traveling body 1, an upper slewing body 2, and a boom 3.

[0009] Note that the directions "front," "rear," "left," and "right" indicated in the diagram are directions based on the orientation of the upper slewing body 2. Specifically, when the construction machine 100 is viewed from above, i.e., in a plan view, the direction of the horizontal component in the direction in which the boom 3 extends from the upper slewing body 2 is defined as the front, and the opposite direction is defined as the rear. The left-right direction is a horizontal direction perpendicular to the front-rear direction.

[0010] The lower traveling body 1 is equipped with a crawler-type traveling device. In other words, the construction machine 100 is a self-propelled crane.

[0011] The upper slewing body 2 is supported by the lower traveling body 1 so as to be able to rotate around a pivot axis that extends in the vertical direction. The upper slewing body 2 comprises a slewing frame 2a that forms the base portion of the upper slewing body 2 and is supported by the lower traveling body 1, and a cab 2b that is supported by the slewing frame 2a. The cab 2b is located at the front of the slewing frame 2a. Inside the cab 2b are a seat for the operator, an operating device 81 (see Figure 4) that receives operations from the operator, and the like. The operating device 81 may include an operating lever, an operating pedal, an operating switch, etc. A counterweight 4 is located at the rear of the upper slewing body 2.

[0012] The construction machine 100 is equipped with a slewing motor 91 for slewing the upper slewing body 2 relative to the lower traveling body 1. The slewing motor 91 may be a hydraulic motor that is operated by the supply of hydraulic fluid from a hydraulic pump (not shown), or it may be an electric motor that is operated by electricity.

[0013] The boom 3 is supported by the upper slewing body 2 in a luffable manner. The boom 3 is an example of an attachment in this disclosure. In the upright position as shown in Figure 1, the boom 3 has a shape that extends upward in one direction from the upper slewing body 2. The boom 3 includes a lower boom member 31, an intermediate boom member 32, and an upper boom member 33. The lower boom member 31, the intermediate boom member 32, and the upper boom member 33 are connected in this order along the longitudinal direction of the boom 3.

[0014] The lower boom member 31 is a member located at the bottom of the boom 3 in the upright position. One end of the lower boom member 31 is rotatably attached to the slewing frame 2a of the upper slewing body 2. The upper boom member 33 is a member located at the top of the boom 3 in the upright position. The intermediate boom member 32 is a member located between the lower boom member 31 and the upper boom member 33. The intermediate boom member 32 may include at least one lattice structure. In the specific example shown in Figure 1, the intermediate boom member 32 includes a plurality of lattice structures 32a, 32b, 32c, and 32d, and these lattice structures 32a, 32b, 32c, and 32d are connected in this order along the longitudinal direction.

[0015] The upper boom member 33 constitutes the tip of the boom 3. The upper boom member 33 is an example of the tip of an attachment in this disclosure.

[0016] Specifically, the upper boom member 33, which serves as the tip of the boom 3, includes a boom member body 33a connected to the upper end of the intermediate boom member 32 (for example, a lattice structure 32d), and a sheave 33b rotatably supported by the boom member body 33a. The upper boom member 33 may be, for example, a member called a boom head, a member called a tower cap, or any other member. In the upright position, the sheave 33b may be supported, for example, at the front of the boom member body 33a. The upper boom member 33 may further include a sheave 33c rotatably supported by the boom member body 33a behind the sheave 33b in the upright position.

[0017] The construction machine 100 includes a gantry 5, a boom luffing rope 8, a lower spreader 6, an upper spreader 7, a boom guy line 9, a boom luffing winch 11, a hook lifting winch 12, a lifting rope 20, and a hook device 23.

[0018] The gantry 5 is attached to the upper slewing body 2 at the rear of the boom 3. The gantry 5 functions as a support during the luffing motion of the boom 3. The gantry 5, boom luffing rope 8, lower spreader 6, and upper spreader 7 support the boom 3 via boom guy lines 9.

[0019] The gantry 5 comprises a compression member 5a and a tension member 5b. The compression member 5a is mounted on the upper slewing body 2 in a way that allows it to be raised and lowered. When crane operations are performed to lift a load 29, the compression member 5a is positioned to extend rearward and upward from its lower end to its upper end, as shown in Figure 1. The tension member 5b connects the upper end of the compression member 5a to the upper slewing body 2.

[0020] The boom luffing rope 8 is a wire rope used to luff the boom 3. The boom luffing winch 11 winds up and unwinds the boom luffing rope 8. The boom luffing winch 11 includes a winch drum around which the boom luffing rope 8 is wound, and a winch motor 92 that rotates the winch drum.

[0021] The suspension rope 20 is a wire rope used to raise and lower the hook device 23 while supporting it. The hook lifting winch 12 winds up and unwinds the suspension rope 20. The hook lifting winch 12 includes a winch drum around which the suspension rope 20 is wound, and a winch motor 93 that rotates the winch drum.

[0022] Each of the winch motors 92 and 93 may be a hydraulic motor that operates by being supplied with hydraulic fluid from a hydraulic pump (not shown), or it may be an electric motor that operates using electricity.

[0023] In the specific example shown in Figure 1, the boom luffing winch 11 and the hook lifting winch 12 are supported by the upper slewing body 2, but at least one of these winches may be supported by the boom 3, for example.

[0024] The lower spreader 6 may be connected to the tip (upper end) of the gantry 5. Each of the lower spreader 6 and the upper spreader 7 has multiple sheaves. The multiple sheaves may be arranged, for example, in the width direction (left-right direction). The boom luffing rope 8 is placed over a guide sheave located at the tip of the gantry 5, and then looped around the multiple sheaves of the lower spreader 6 and the multiple sheaves of the upper spreader 7. The end of the boom luffing rope 8 is fixed to a predetermined location (for example, the gantry 5, the lower spreader 6, or the upper spreader 7).

[0025] One end of the boom guy line 9 is connected to the upper spreader 7, and the other end of the boom guy line 9 is connected to the upper part of the boom 3 (for example, the upper boom member 33). When the boom luffing winch 11 winds in or unwinds the boom luffing rope 8, the distance between the lower spreader 6 and the upper spreader 7 changes, causing the boom 3 to luff relative to the upper slewing body 2.

[0026] The suspension rope 20 hangs down from the upper boom member 33. Specifically, the suspension rope 20 hangs down from the sheave 33b of the upper boom member 33. The suspension rope 20 is unfurled from the hook lifting winch 12, extends upward along the boom 3 behind the boom 3, is hooked onto the sheave 33c of the upper boom member 33, extends forward, is hooked onto the sheave 33b of the upper boom member 33, and extends downward from this sheave 33b. The hook device 23 is supported on the portion of the suspension rope 20 that hangs down from the upper boom member 33. When the hook lifting winch 12 retracts or unfurls the suspension rope 20, the hook device 23 moves up and down.

[0027] The hook device 23 supports the suspended load 29, for example, via a wire rope for lifting. As shown in Figures 1 and 2, the hook device 23 may have a hook device body 23a, a hook portion 23b supported by the hook device body 23a, and a hook sheave 23c, which is a sheave rotatably supported by the hook device body 23a. The suspension rope 20 extending downward from the sheave 33b of the upper boom member 33 is hooked onto the hook sheave 23c of the hook device 23 and is arranged to extend upward from the hook sheave 23c, and one end of the suspension rope 20 may be fixed to the upper part of the boom 3 (for example, the upper boom member 33). In this case, as shown in Figures 1 and 2, the suspension rope 20 includes a plurality of rope portions (for example, a first rope portion 21 and a second rope portion 22) that extend downward from the upper boom member 33 of the boom 3. The first rope portion 21 and the second rope portion 22 extend downward from the upper boom member 33 to the hook device 23 in a position where they are spaced apart horizontally between the upper boom member 33 and the hook device 23.

[0028] Furthermore, as shown in Figure 3, the portion of the suspension rope 20 extending from the upper boom member 33 to the hook device 23 may be composed of a single rope section. In this case, the hook device 23 has a hook device body 23a and a hook portion 23b, but does not have a hook sheave 23c. In this case, the lower end of the suspension rope 20 hanging down from the upper boom member 33 is fixed to the hook device body 23a of the hook device 23.

[0029] Figure 4 is a block diagram showing the components of the construction machine 100. The construction machine 100 includes an operation control unit 70. The operation control unit 70 has a computer including a processing unit such as a CPU, MPU, GPU, and memory. The operation control unit 70 controls various operations of the construction machine 100. The operation control unit 70 may control, for example, the operation of the boom luffing winch 11, the operation of the hook lifting winch 12, and the slewing operation of the upper slewing body 2. The operation control unit 70 may also include a slewing control unit 71 and a winch control unit 72. The operation control unit 70 realizes the respective functions of the slewing control unit 71 and the winch control unit 72 by having the processing unit execute a program stored in the memory.

[0030] The construction machine 100 includes a flow regulator 91a for adjusting the direction and flow rate of hydraulic fluid supplied to a slewing motor 91 (an example of an actuator), a flow regulator 92a for adjusting the direction and flow rate of hydraulic fluid supplied to a winch motor 92 (an example of an actuator) of a boom luffing winch 11, and a flow regulator 93a for adjusting the direction and flow rate of hydraulic fluid supplied to a winch motor 93 (an example of an actuator) of a hook lifting winch 12.

[0031] The slewing control unit 71 outputs a control command to the flow regulator 91a in response to the operator's operation (slewing operation) received by the actuator 81. As a result, the slewing motor 91 performs the operation corresponding to the slewing operation. The winch control unit 72 outputs a control command to the flow regulator 92a in response to the operator's operation (luffing winch operation) received by the actuator 81. As a result, the winch motor 92 of the boom luffing winch 11 performs the operation corresponding to the luffing winch operation. The winch control unit 72 outputs a control command to the flow regulator 93a in response to the operator's operation (lifting winch operation) received by the actuator 81. As a result, the winch motor 93 of the hook lifting winch 12 performs the operation corresponding to the lifting winch operation.

[0032] Each of the flow regulators 91a, 92a, and 93a changes its output according to a control command (e.g., current value) input from the operation control unit 70. For example, each of the flow regulators 91a, 92a, and 93a may include a spool for adjusting the direction and flow rate of hydraulic fluid supplied to the actuator corresponding to the flow regulator, a pair of pilot ports, and a pair of electromagnetic proportional valves. Each electromagnetic proportional valve is located in an oil passage connecting the pilot port of the spool corresponding to the electromagnetic proportional valve to a pilot pump (not shown) and adjusts the pilot pressure input to the pilot port. Each spool operates when pilot pressure is input to the pilot port corresponding to the spool, allowing hydraulic fluid to be supplied to the actuator corresponding to the spool. As a result, the swing motor 91, winch motor 92, and winch motor 93 each operate according to the control command output by the operation control unit 70.

[0033] The construction machine 100 is equipped with a payout amount detector 82 for detecting a payout amount correlation value that correlates with the length of the lifting rope 20 being paid out from the hook lifting winch 12. The payout amount detector 82 may be, for example, a sensor that detects the rotation of the winch drum of the hook lifting winch 12 (e.g., a rotary encoder), or it may be another sensor capable of detecting the payout amount correlation value. The payout amount detector 82 inputs the detection result to at least one of the operation control unit 70 and the controller 60, which will be described later.

[0034] The construction machine 100 may be equipped with an input device 83. The input device 83 may be located, for example, inside the cab 2b of the upper slewing body 2. The input device 83 receives input operations from workers such as operators. The input operation may be, for example, an input operation relating to the number of ropes in the rope section. The workers perform an input operation to the input device 83 corresponding to the number of ropes extending from the upper boom member 33 to the hook device 23. For example, as shown in Figure 2, if there are two ropes in the rope section, the workers perform an input operation to the input device 83 corresponding to two ropes, and for example, as shown in Figure 3, if there is one rope section, the workers perform an input operation to the input device 83 corresponding to one rope. The input device 83 inputs information corresponding to the input operation to at least one of the operation control unit 70 and the controller 60, and the operation control unit 70 or the controller 60 sets the set value for the number of ropes in the rope section to a value corresponding to the input operation.

[0035] The construction machine 100 may be equipped with a display 84. The display 84 may include a monitor (display), a device for projecting images onto an object, or a device using AR (Augmented Reality) technology. The display 84 may be located, for example, inside the cab 2b of the upper rotating body 2. The display 84 displays various data output from at least one of the motion control unit 70 and the controller 60. Operators and other personnel involved in the work can grasp the information displayed on the display 84.

[0036] As shown in Figure 4, the construction machine 100 includes an object information detector 40 and a vibration state determination device 50 according to this embodiment. The vibration state determination device 50 includes a controller 60.

[0037] As shown in Figures 1 to 3, the object information detector 40 is attached to the upper boom member 33 of the boom 3 (an example of the tip of the attachment). The object information detector 40 may also be attached to the boom member body 33a of the upper boom member 33. The object information detector 40 may also be attached to the side surface (right side, left side, lower side, or front side) of the boom member body 33a of the upper boom member 33. In the specific example shown in Figures 1 to 3, the object information detector 40 is attached to the right side surface of the boom member body 33a of the upper boom member 33.

[0038] The object information detector 40 is a detector capable of acquiring information (object information) about objects placed at the work site. The object information detector 40 acquires information about the lifting rope 20 and hook device 23 of the construction machine 100. The object information detector 40 outputs the detected object information (detection data), and at least one of the controller 60 and the operation control unit 70 acquires the object information (detection data).

[0039] In this embodiment, the object information detector 40 is a camera 40A (an example of an imaging device). The camera 40A is a device capable of capturing images of the suspension rope 20 and the hook device 23. However, the object information detector 40 is not limited to the camera 40A, and may be, for example, a distance measuring device 40B capable of detecting the three-dimensional position information of an object. Furthermore, the object information detector 40 may include both the camera 40A and the distance measuring device 40B. The distance measuring device 40B will be described later.

[0040] The camera 40A is mounted on the upper boom member 33 in a position such that its imaging range includes a portion of the suspension rope 20 and the hook device 23. In the specific examples shown in Figures 1 to 3, the camera 40A is positioned in a downward orientation such that its imaging range is located below the camera 40A. The camera 40A includes the space extending directly below it as its imaging range and acquires images of that space. The imaging range of the camera 40A includes a portion or all of the portion of the suspension rope 20 that extends from the upper boom member 33 to the hook device 23, and the hook device 23. The imaging range of the camera 40A includes, for example, the end portion Pt of the suspension rope 20 and the hook device 23, as shown in Figures 5 and 6.

[0041] The type of camera 40A is not particularly limited. Camera 40A has an image sensor such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). Camera 40A may be configured to acquire image data (an example of detection data) including a part of the suspension rope 20 and the hook device 23 at a predetermined frame rate, and to output the image data sequentially. The image data sequentially output from camera 40A is sequentially input to the vibration state determination device 50. As a result, the vibration state determination device 50 can acquire a series of videos (moving data) composed of a large number of image data captured at a predetermined frame rate.

[0042] The controller 60 has a computer that includes a processing unit such as a CPU, MPU, and GPU, and memory. The controller 60 includes a data acquisition unit 61, a rope identification unit 62, and a swing state determination unit 63. The controller 60 realizes the functions of the data acquisition unit 61, the rope identification unit 62, and the swing state determination unit 63 by having the processing unit execute a program stored in the memory.

[0043] The vibration state determination device 50 according to this embodiment has the following first feature.

[0044] [First Feature] In the swing state determination device 50, the data acquisition unit 61 of the controller 60 acquires the detection data of the suspension rope 20 detected by the object information detector 40. The rope identification unit 62 of the controller 60 identifies the position of the terminal end Pt of the suspension rope 20 based on the detection data, and the swing state determination unit 63 of the controller 60 determines the swing state of the suspension rope 20 based on the position of the terminal end Pt.

[0045] The determination of the swing state by the controller 60 may include determining whether the suspension rope 20 is swinging, may include calculating the swing amount (swing width) of the suspension rope 20, may include calculating the swing angle of the suspension rope 20, or may include calculating other parameters representing the swing of the suspension rope 20. The swing state determined by the controller 60 includes both the state where the suspension rope 20 is stationary and the state where the suspension rope 20 is swinging. The state where the suspension rope 20 is swinging is, for example, as shown by the arrow of the two-dot chain line in FIG. 1, the state of the suspension rope 20 when the hook device 23 supported by the suspension rope 20 moves along an arc trajectory with the upper end Pu of the suspension rope 20 (the portion corresponding to the upper boom member 33 of the boom 3 in the suspension rope 20) as the center. The swing state can be represented by parameters such as, for example, the swing amount (swing width) and swing angle of the suspension rope 20.

[0046] The terminal end Pt of the suspension rope 20 may be, for example, the portion located directly above the hook device 23 within the range between the upper end Pu of the suspension rope 20 and the hook device 23. The terminal end Pt of the suspension rope 20 may be the lowermost portion located at the lowest position within the range between the upper end Pu of the suspension rope 20 and the hook device 23, or may be a portion including the lowermost portion and its vicinity.

[0047] In the swing state determination device 50 having this first feature, since the object information detector 40 for detecting the suspension rope 20 hanging from the upper boom member 33 of the boom 3 is attached to the upper boom member 33 of the boom 3, even when an obstacle exists between the boom 3 and the suspension rope 20, the suspension rope 20 can be detected. Thereby, the controller 60 can acquire the detection data, specify the position of the terminal portion Pt in the suspension rope 20 based on the acquired detection data, and determine the swing state based on the position of the specified terminal portion Pt.

[0048] The respective shapes of the hook device 23 supported by the suspension rope 20 and the suspended load supported by the hook device 23 are various. On the other hand, since the suspension rope 20 has a characteristic shape that extends linearly from the tip of the attachment to the hook device 23 regardless of the type, it is preferable as a detection target for determining the swing state compared to the hook device 23 and the suspended load. Further, the terminal portion Pt of the suspension rope 20 is a portion where the displacement amount when the suspension rope 20 swings is larger compared to, for example, the intermediate portion between the upper end portion Pu and the terminal portion Pt of the suspension rope 20. Since the controller 60 determines the swing state based on the position of the terminal portion Pt where the displacement amount is relatively large, even when the swing of the hook device 23 is relatively small, the swing state of the suspension rope 20 can be appropriately determined.

[0049] The swing state determination device 50 according to the present embodiment further includes the following second to ninth features.

[0050] [Second Feature] In the swing state determination device 50, the object information detector 40 is a camera 40A (an example of an imaging device), the detection data is image data captured by the camera 40A, and the controller 60 may be an image recognition controller that identifies the position of the end portion Pt by performing image recognition using the image data. In this case, the controller 60 can identify the position of the end portion Pt of the suspension rope 20 by image recognition, regardless of the shape, color, and other characteristics of the hook device 23 and the suspended load 29. The image recognition may be performed, for example, by known image recognition processing such as pattern matching.

[0051] The image recognition method for extracting the position of the end portion Pt of the suspension rope 20 from image data captured by the camera 40A is not particularly limited, but it is preferably a method relating to the following third feature.

[0052] [Third Feature] In the swing state determination device 50, the controller 60 uses an image recognition model obtained by machine learning and the image data to determine the position of the end portion Pt. The controller 60 of the swing state determination device 50, which has the third feature, can more appropriately determine the position of the end portion Pt of the suspension rope 20 using an image recognition model obtained by machine learning. Furthermore, since the determination of the swing state can be achieved by adding equipment for image recognition to an existing crane system, it is advantageous in terms of introduction costs.

[0053] The controller 60 of the swing state determination device 50 uses the image recognition model, which takes the image data captured by the camera 40A as input and outputs the position of the end portion Pt of the suspension rope 20, to acquire the position of the end portion Pt of the suspension rope 20.

[0054] [Fourth feature] In the swing state determination device 50, the image recognition model is a model obtained by machine learning using training data to which annotations are added to the positions of the starting end Ps and the ending end Pt of the suspension rope.

[0055] In this embodiment, the controller 60 may not only determine the position of the end portion Pt using the image recognition model and the image data, but also determine the position of the starting end portion Ps of the suspension rope 20. As shown in Figure 5, the starting end portion Ps of the suspension rope 20 is the end portion of the suspension rope 20 included in the image data, and is the end portion located on the opposite side of the end portion Pt of the suspension rope 20. Therefore, the starting end portion Ps of the suspension rope 20 does not necessarily coincide with the upper end portion Pu of the suspension rope 20.

[0056] The image recognition model is generated using a known machine learning algorithm such as a neural network. The training data to be input into the machine learning algorithm can be created, for example, by using a camera 40A attached to the upper boom member 33 of the boom 3 to image the suspension rope 20 and the hook device 23 under various conditions, and by adding annotations to the position of the end portion Pt in each of the various image data obtained, that is, by linking the position of the end portion Pt in each of the various image data.

[0057] Figure 5 shows an example of training data in which the positions of the starting end Ps and ending end Pt of the suspension rope 20 are annotated in the image data captured by the camera 40A. Figure 5 shows the case where there is one rope extending from the upper boom member 33 to the hook device 23, and Figure 6 shows the case where there are two ropes extending from the upper boom member 33 to the hook device 23. The image data shown in Figures 5 and 6 each include several obstacles in addition to the suspension rope 20 and the hook device 23. In the image data shown in Figures 5 and 6 each, the positions of the starting end Ps and ending end Pt of the suspension rope 20 are annotated. That is, in each image data, the positions of the starting end Ps and ending end Pt of the suspension rope 20 are labeled.

[0058] Figure 7 is a diagram illustrating the determination of the swing state of the suspension rope 20 by the swing state determination device 50, specifically the identification of the starting end Ps and ending end Pt positions of the suspension rope 20, and the determination of the swing state of the suspension rope 20 by the swing state determination device 50. Figure 7 shows a case where there is only one rope extending from the upper boom member 33 to the hook device 23. In Figure 7, the suspension rope 20 shown by a solid line indicates, for example, the position of the suspension rope 20 included in image data (first image data) acquired by the controller 60 at a certain point in time, and the suspension rope 20 shown by a dashed line indicates, for example, the position of the suspension rope 20 included in image data (second image data) acquired by the controller 60 after the acquisition of the first image. The first image data and the second image data are at least two image data sets acquired by the controller 60 for the purpose of determining the swing state. The second image data may be, for example, image data acquired a predetermined time after the acquisition of the first image data, or image data acquired a predetermined number of frames later than the first image data.

[0059] When the controller 60 acquires the first image data captured by the camera 40A, it uses the acquired first image data as input and the trained image recognition model, which outputs the position of the starting end Ps and the ending end Pt of the suspension rope 20, to acquire the positions of the starting end Ps and the ending end Pt of the suspension rope 20.

[0060] [Fifth Feature] In the swing state determination device 50, the controller 60 calculates the difference between the stationary coordinates, which are the coordinates of the end Pt when the suspension rope 20 is stationary, and the coordinates of the end Pt (actual coordinates) when the suspension rope 20 is swinging, and determines the swing state based on the calculated difference. In this case, the controller 60 can determine the swing state by a relatively simple calculation that includes calculating the difference between the stationary coordinates, which are set in advance or acquired in advance, and the actual coordinates (actual coordinates) of the end Pt during work using the construction machine 100. The controller 60 may also calculate the difference between the stationary coordinates and the actual coordinates as the amount of swing of the suspension rope 20.

[0061] Specifically, the controller 60, for example, before crane operation begins, uses a trained image recognition model that outputs the positions of the starting end Ps and ending end Pt of the lifting rope 20 as input to image data input from the camera 40A while the lifting rope 20 and hook device 23 are stationary, to acquire the positions of the starting end Ps and ending end Pt of the lifting rope 20. The acquired positions of the starting end Ps and ending end Pt may be represented by coordinates in a camera coordinate system, which is a coordinate system based on the camera 40A. This camera coordinate system may be a two-dimensional Cartesian coordinate system represented by the X and Y axes, for example, as shown by the dashed lines in Figure 7.

[0062] The controller 60 may convert the coordinates of the end portion Pt in the camera coordinate system (stationary coordinates) to an end portion coordinate system, for example, as shown by the solid line in Figure 7. The end portion coordinate system is a coordinate system based on the position of the end portion Pt when the suspension rope 20 is stationary, and may be a two-dimensional orthogonal coordinate system represented by the x and y axes, for example, as shown in Figure 7. The controller 60 has pre-stored specification information such as information regarding the relative position of the camera 40A with respect to the upper boom member 33 of the boom 3, and information regarding the boom 3, such as the length of the boom 3. The controller 60 can also obtain attitude information regarding the attitude of the boom 3 from an attitude detector 85 (see Figure 4) that detects the attitude of the boom 3. Therefore, the controller 60 can use the specification information and the attitude information to convert the coordinates of the end portion Pt in the camera coordinate system (stationary coordinates) to coordinates in the end portion coordinate system (stationary coordinates).

[0063] The attitude detector 85 may include, for example, an inertial measuring unit (IMU) attached to the boom 3, or it may be a sensor for detecting a payout correlation value that correlates with the length of the boom luffing rope 8 being paid out from the boom luffing winch 11. The attitude detector 85 inputs the detection result to at least one of the controller 60 and the motion control unit 70.

[0064] When the crane operation is started, the controller 60 acquires the actual coordinates of the starting end Ps and the actual coordinates of the ending end Pt of the lifting rope 20 during the operation, using the image data captured by the camera 40A and the image recognition model. The controller 60 then calculates the difference between the stationary coordinates and the actual coordinates and determines the swing state based on the calculated difference. Specifically, the controller 60 may calculate the difference between the stationary coordinates and the actual coordinates as the amount of swing of the lifting rope 20. Alternatively, the controller 60 may determine the direction of the swing of the stationary lifting rope 20 from the difference between the stationary coordinates and the actual coordinates. If the stationary coordinates and the actual coordinates are expressed in the end coordinate system, the controller 60 may determine the direction of the swing based on the sign of the difference.

[0065] [Sixth Feature] As shown in Figure 8, if the suspension rope 20 includes two rope sections 21 and 22 that extend downward from the upper boom member 33 of the boom 3, the controller 60 may identify the coordinates of the respective end portions Pt of the two rope sections 21 and 22 based on the image data, calculate the average value of the coordinates of the end portions Pt of the two rope sections 21 and 22, and determine the swing state based on the change in the average value. In this case, the controller 60 can determine the swing state of the suspension rope 20 regardless of the number of rope sections 21 and 22.

[0066] Specifically, the controller 60, for example, before crane operation begins, uses a trained image recognition model that outputs the position of the starting end Ps and ending end Pt of the lifting rope 20 as input from image data input from the camera 40A while the lifting rope 20 and hook device 23 are stationary, to acquire the positions of the starting end Ps and ending end Pt of the rope portion 21 of the lifting rope 20, and the positions of the starting end Ps and ending end Pt of the rope portion 22 of the lifting rope 20. The positions of each starting end Ps and each ending end Pt may be represented by coordinates in the camera coordinate system (stationary coordinates), or by coordinates in the ending end coordinate system (stationary coordinates).

[0067] Next, the controller 60 calculates the average static coordinate, which is the average of the static coordinates of the end Pt of the rope section 21 and the end Pt of the rope section 22.

[0068] When the crane operation is started, the controller 60 acquires the actual coordinates of the starting end Ps and ending end Pt of the rope portion 21 of the lifting rope 20, and the actual coordinates of the starting end Ps and ending end Pt of the rope portion 22 of the lifting rope 20, using image data captured by the camera 40A and the image recognition model. The controller 60 calculates the actual average value, which is the average of the actual coordinates of the ending end Pt of the rope portion 21 and the actual coordinates of the ending end Pt of the rope portion 22. The controller 60 then calculates the difference between the static average value and the actual average value and determines the swing state based on the calculated difference. Specifically, the controller 60 may calculate the difference between the static average value and the actual average value as the amount of swing of the lifting rope 20. The controller 60 may also determine the direction of the swing relative to the suspension rope 20 when stationary based on the difference between the stationary average value and the actual average value.

[0069] [Seventh Feature] In the vibration state determination device 50, the camera 40A may have a zoom function, in which case the controller 60 may determine the vibration state using the change in the position of the end portion Pt of the suspension rope 20 and the zoom magnification of the camera 40A. In this case, the controller 60 determines the vibration state taking the zoom magnification into consideration, so the vibration state can be appropriately determined regardless of the zoom magnification.

[0070] [Feature #8] In the swing state determination device 50, the controller 60 may calculate the swing angle of the suspension rope 20 using the change in the position of the end portion Pt of the suspension rope 20 and the detection result input from the payout amount detector 82 which detects the amount of the suspension rope 20 that is paid out.

[0071] In this case, if the swing angle is greater than or equal to a predetermined threshold, the controller 60 may, for example, perform control to suppress the swing of the suspension rope 20, or perform control to limit the operation of the construction machine. The controller 60 may also output information regarding the calculated swing angle, and the display may display the outputted information. The display may be, for example, a display 84 provided on the construction machine, or a display provided on an external information terminal located away from the construction machine. This allows operators, work managers, and other work personnel to recognize the swing angle of the suspension rope 20 calculated by the controller 60.

[0072] [Ninth Feature] In the swing state determination device 50, if the suspension rope 20 includes two rope sections 21 and 22 extending downward from the upper boom member 33 of the boom 3, the controller 60 may determine the number of rope sections 21 and 22 based on the image data, compare the determined number with the set value for the number of rope sections 21 and 22 set based on the input to the input device 83 by the work personnel, and output the comparison result to a display device such as a display unit 84. In this case, the work personnel, such as the operator and work manager, can recognize whether the number of rope sections 21 and 22 they input matches the actual number by looking at the comparison result output to the display unit 84, for example.

[0073] Figure 9 illustrates another specific example of the process by which an image recognition model (image recognition AI model) is constructed using machine learning with training data. In the following explanation, the case in which the controller 60 of the vibration state determination device 50 constructs the image recognition model is used as an example, but the image recognition model may be constructed using a computer separate from the vibration state determination device 50.

[0074] In step (1) shown in Figure 9, image data necessary for machine learning is collected. Specifically, the controller 60 acquires a large number of image data, including the suspension rope 20 and hook device 23, captured by the camera 40A, under various conditions. From the viewpoint of improving the accuracy of machine learning, it is preferable that these various conditions include conditions where, for example, the time of day, weather, and background are all different from each other.

[0075] In step (2) shown in Figure 9, the image data is preprocessed. The controller 60 creates a large number of training data by annotating the position of the start end Ps and the position of the end end Pt for each of the large number of image data acquired in step (1). Specifically, the controller 60 creates a large number of training data for each of the large number of image data in which the coordinates of the start end Ps and the coordinates of the end end Pt are registered. In other words, the controller 60 creates a large number of training data for each of the large number of image data in which the coordinates of the start end Ps and the coordinates of the end end Pt are associated with the image data.

[0076] In step (3) of Figure 9, an image recognition model is generated by machine learning using a large amount of training data and stored in a storage medium such as the memory of the swing state determination device 50. That is, when image data including the suspension rope 20 and the hook device 23 is input to the controller 60, the controller 60 trains an image recognition model that outputs the position (e.g., coordinates) of the end part Pt of the suspension rope 20 using the large amount of training data.

[0077] In step (4) of Figure 9, the accuracy of the image recognition model generated in step (3) is evaluated. Specifically, for example, the controller 60 makes the image recognition model recognize image data different from the numerous training data (image data including the suspension rope 20 and the hook device 23) (by inputting the image data into the image recognition model) and obtains output from the image recognition model (for example, the coordinate values ​​of the end part). The controller 60 then determines whether the accuracy of the output from the image recognition model (for example, the detection rate) is equal to or greater than a predetermined accuracy threshold. If the accuracy of the output is less than the accuracy threshold, the controller 60 repeats the processing in steps (1) to (4). If the accuracy of the output is equal to or greater than the accuracy threshold, the controller 60 stores the image recognition model in the storage medium (step (5) of Figure 9) and terminates the image recognition model construction process.

[0078] Figure 10 is a diagram illustrating an example of the process performed by the controller 60 of the swing state determination device 50, including acquiring image data, identifying the position of the suspension rope 20, and determining the swing state. However, the processing flow performed by the controller 60 is not limited to the specific example shown in Figure 10.

[0079] As shown in Figure 10, once the controller 60 (data acquisition unit 61) acquires the image data, the controller 60 (rope identification unit 62) performs preprocessing on the image data. The controller 60 may perform preprocessing such as noise reduction, image enhancement, and distortion correction.

[0080] Next, the controller 60 (rope identification unit 62) performs feature extraction in the image data. The extracted features may include, for example, brightness distribution, color occurrence rate, color distribution, color type, positional relationship of objects, and edges (boundaries where colors change). The controller 60 may use known algorithms such as SIFT (Scale-Invariant Feature Transform) and HOG (Histograms of Oriented Gradients) as algorithms for deriving feature quantities for image recognition.

[0081] Next, the controller 60 (rope identification unit 62) uses the image recognition model described with reference to Figure 9, for example, and the image data from which the features have been extracted, to identify the position of the end portion Pt of the suspension rope 20. The controller 60 (image recognition controller) may implement a known machine learning algorithm as an image recognition algorithm, such as a convolutional neural network (CNN). Through image recognition by the controller 60 with such an image recognition algorithm implemented, the end portion Pt of the suspension rope 20 is recognized, and the position (coordinates) of the end portion Pt is identified.

[0082] Next, the controller 60 (swing state determination unit 63) determines the swing state of the suspension rope 20 based on the position of the end part Pt. The controller 60 may determine whether or not the suspension rope 20 is swinging, determine (calculate) the amount of swing (swing amplitude) of the suspension rope 20, determine (calculate) the swing angle of the suspension rope 20, determine (calculate) the direction of swing of the suspension rope 20, or determine (calculate) other parameters that represent the swing of the suspension rope 20.

[0083] Next, the controller 60 (vibration state determination unit 63) outputs the determination result, which may be displayed on, for example, the display unit 84.

[0084] Figure 11 is a flowchart showing an example of the calculation process performed by the controller 60. However, the calculation process performed by the controller 60 is not limited to the specific example shown in Figure 11.

[0085] The controller 60 of the vibration state determination device 50 determines whether or not image data (camera video) has been input from the camera 40A (step S11). If no image data has been input to the controller 60 (NO in step S11), the controller 60 repeats the process in step S11.

[0086] When image data is input to the controller 60 (YES in step S11), the controller 60 performs image processing (preprocessing) of the image data (step S12) and performs feature extraction on the preprocessed image data (step S13). These preprocessing and feature extraction steps may be the same as those described above with reference to Figure 10.

[0087] Next, the controller 60 uses the trained image recognition model and the image data from which the features have been extracted to determine the position of the end portion Pt of the suspension rope 20 (step S14).

[0088] Next, the controller 60 determines whether the portion of the suspension rope 20 extending from the upper boom member 33 to the hook device 23 is a single rope portion, that is, whether there is one rope portion (step S15).

[0089] If the number of rope sections is one (YES in step S15), the controller 60 determines the swing state based on the difference between the stationary coordinates and the actual coordinates that have been stored in advance (step S16). On the other hand, if the number of rope sections is not one but, for example, two (NO in step S15), the controller 60 determines the swing state based on the difference between the stationary average value and the actual average value that have been stored in advance (step S17).

[0090] [Modifications] Although embodiments of the present disclosure have been described above, the present disclosure is not limited to the embodiments described above and includes, for example, the following modifications.

[0091] (A) A method for determining the swing state, a modified embodiment of the swing state determination method, is a determination method for a construction machine 100 comprising a machine body, a boom 3 supported by the machine body, a suspension rope 20 hanging from an upper boom member 33 of the boom 3, and an object information detector 40 (e.g., a camera 40A) attached to the upper boom member 33. This swing state determination method includes: the controller 60 acquiring detection data for the suspension rope 20 detected by the object information detector 40; the controller 60 identifying the position of the end portion Pt of the suspension rope 20 based on the detection data; and the controller 60 determining the swing state of the suspension rope 20 based on the position of the end portion Pt.

[0092] (B) Regarding the program, the modified program of the embodiment is a program for a construction machine 100 comprising a machine body, a boom 3 supported by the machine body, a suspension rope 20 hanging from an upper boom member 33 of the boom 3, and an object information detector 40 (e.g., a camera 40A) attached to the upper boom member 33. This program causes a computer to perform the following processes: acquiring detection data of the suspension rope 20 detected by the object information detector 40; identifying the position of the end portion Pt of the suspension rope 20 based on the detection data; and determining the swing state of the suspension rope 20 based on the position of the end portion Pt. Furthermore, the storage medium according to the modified embodiment may store the program. The storage medium may be a non-temporary computer-readable storage medium that stores the program. The swing state determination method is realized when the program is executed by a computer processor (e.g., a controller 60).

[0093] (C) Regarding the swing state determination device, the swing state determination device 50 according to the above embodiment is provided on the construction machine 100, but it may also be provided on a device (external information terminal) located at a distance from the construction machine 100. Alternatively, some of the functions of the controller 60 of the swing state determination device 50 (data acquisition unit 61, rope identification unit 62, and swing state determination unit 63) may be provided on the construction machine 100, and the remaining parts of these functions may be provided on the external information terminal.

[0094] (D) Regarding the swing state determination system, Figure 12 is a block diagram showing the components of a swing state determination system 300 equipped with a swing state determination device 50 according to a modified embodiment. This swing state determination system 300 comprises a swing state determination device 50 and a construction machine 100. The swing state determination system 300 may further include an external information terminal 200. The swing state determination device 50 is equipped with a communication device 64, the construction machine 100 is equipped with a communication device 86, and the external information terminal 200 is equipped with a communication device not shown in the figure. As a result, the swing state determination device 50, the construction machine 100 and the external information terminal 200 can communicate with each other via a network 400. The network 400 is an information communication network such as the Internet, telephone network, mobile phone network, satellite communication network, WAN (Wide Area Network), LAN (Local Area Network), or dedicated line.

[0095] In this vibration state determination system 300, detection data (e.g., image data) detected by an object information detector 40 such as a camera 40A is transmitted from the construction machine 100 to the vibration state determination device 50 via the network 400. The vibration state determination result by the controller 60 of the vibration state determination device 50 may be transmitted to the construction machine 100 via the network 400 and displayed on the display unit 84, or it may be transmitted to an external information terminal 200 via the network 400 and displayed on the screen of the external information terminal 200.

[0096] The external information terminal 200 may be, for example, an information terminal such as a tablet computer, smartphone, laptop personal computer, or desktop personal computer.

[0097] (E) Regarding automatic operation or remote control, the construction machine 100 (e.g., a crane) may be configured to operate in response to operations given to the control device 81 by an operator riding in the cab 2b of the construction machine 100, but it may also be operated automatically. When the construction machine 100 is operated automatically, the motion control unit 70 of the construction machine 100 may control the automatic operation of the construction machine 100 based on automatic operation data stored in the storage medium of the motion control unit 70 or the storage medium of the external information terminal 200. The automatic operation data may be generated, for example, by a known teaching method using the construction machine 100.

[0098] Furthermore, the construction machine 100 may be remotely controlled by a remote control device located at a distance from the construction machine 100. The remote control device may be an external information terminal 200, or a cockpit as shown in the figure. In this case, the vibration state determination device 50 may be mounted on the construction machine 100, on the remote control device, or on a device other than the construction machine 100 and the remote control device (for example, an external information terminal 200).

[0099] (F) Regarding the attachment, in the above embodiment the attachment is composed of the boom, but the construction machine in this disclosure may also include the boom (first attachment) and a jib (second attachment) not shown, supported at the tip of the boom. In this case the construction machine in this disclosure may also include at least one of a suspension rope (first suspension rope) hanging down from the tip of the first attachment and a suspension rope (second suspension rope) hanging down from the tip of the second attachment. The controller of the swing state determination device in this disclosure may be configured to acquire detection data (first detection data) for the first suspension rope detected by an object information detector (first object information detector) attached to the tip of the first attachment, to identify the position of the end of the first suspension rope based on the first detection data, and to determine the swing state of the first suspension rope based on the position of the end. Furthermore, the controller of the swing state determination device in this disclosure may be configured to acquire detection data (second detection data) for the second suspension rope detected by an object information detector (second object information detector) attached to the tip of the second attachment, to identify the position of the end of the second suspension rope based on the second detection data, and to determine the swing state of the second suspension rope based on the position of the end.

[0100] The boom in the above embodiment includes a lattice structure, but the attachment in this disclosure may be, for example, an extendable telescopic boom.

[0101] (G) Object Information Detector In the above embodiment, the object information detector 40 is a camera 40A (imaging device), but the object information detector in this disclosure is not limited to an imaging device, and may be, for example, a distance measuring device 40B capable of detecting the three-dimensional position information of an object. In this case, the distance measuring device 40B acquires the three-dimensional position information of the suspension rope 20 and the hook device 23. The distance measuring device 40B may be a LiDAR (Light Detection and Ranging) capable of acquiring point cloud data (an example of three-dimensional position information) of the suspension rope 20 and the hook device 23. The distance measuring device 40B may also be a stereo camera, an ultrasonic sensor, a total station, or any other sensor capable of acquiring three-dimensional data. Furthermore, the object information detector 40 may be a combination of two or more of these devices.

[0102] Furthermore, the object information detector in this disclosure may include both the camera 40A and the distance measuring device 40B.

[0103] (H) Regarding the machine itself

[0104] The machine body in the embodiment shown in Figure 1 includes a self-propelled lower traveling body 1 and an upper slewing body 2 that is rotatably supported on the lower traveling body 1. However, the machine body in this disclosure is not limited to the specific example shown in Figure 1, and may have a structure including, for example, a non-self-propelled base portion (not shown) and an upper slewing body (not shown) that is rotatably supported on this base portion. In this case, the construction machine may be a fixed crane. The lower traveling body 1 in the embodiment is equipped with a crawler-type traveling device, but the traveling device of the lower traveling body may, for example, have wheels.

[0105] (I) Regarding the vibration state determination device and the motion control unit, in the above embodiment, the vibration state determination device 50 (controller 60) and the motion control unit 70 are configured as separate information processing devices. However, for example, the controller 60 of the vibration state determination device 50 may also have the functions of the motion control unit 70. In this case, the motion control unit 70 shown in Figures 4 and 12 can be omitted.

[0106] (J) Regarding the position of the starting end of the suspension rope, in the above embodiment, the controller 60 of the swing state determination device 50 not only determines the position of the terminal Pt using the image recognition model and the image data, but also determines the position of the starting end Ps of the suspension rope 20. However, the controller 60 does not need to determine the position of the starting end Ps of the suspension rope 20 while determining the position of the terminal Pt using the image recognition model and the image data.

[0107] As described above, this disclosure provides a technology that can determine the swing state of a suspension rope even when an obstacle exists between the attachment and the suspension rope.

[0108] A swing state determination device according to the first embodiment is a swing state determination device for a construction machine comprising a machine body, an attachment supported by the machine body, a suspension rope hanging from the tip of the attachment, and an object information detector attached to the tip of the attachment, comprising a controller configured to acquire detection data for the suspension rope detected by the object information detector, to identify the position of the end of the suspension rope based on the detection data, and to determine the swing state of the suspension rope based on the position of the end.

[0109] The controller's determination of the swing state may include determining whether the suspension rope is swinging or not, calculating the amount (amplitude) of the suspension rope's swing, calculating the angle of the suspension rope's swing, or calculating other parameters that represent the swing of the suspension rope. The swing state determined by the controller includes both the state in which the suspension rope is stationary and the state in which the suspension rope is swinging. The state in which the suspension rope is swinging is the state of the suspension rope when the suspension rope swings as the hook device supported by the suspension rope moves along an arc trajectory centered on the upper end of the suspension rope (the part of the suspension rope corresponding to the tip of the attachment). The swing state can be represented by parameters such as the amount (amplitude) and angle of the suspension rope's swing.

[0110] The end portion of the suspension rope may, for example, be the portion located directly above the hook device within the range between the upper end of the suspension rope and the hook device. The end portion of the suspension rope may also be the lowest point within the range between the upper end of the suspension rope and the hook device, or it may include the lowest point and its vicinity.

[0111] In this first embodiment, the object information detector for detecting the suspension rope hanging from the tip of the attachment is attached to the tip of the attachment, so that the suspension rope can be detected even if there is an obstacle between the attachment and the suspension rope. As a result, the controller can acquire the detection data, determine the position of the end of the suspension rope based on the acquired detection data, and determine the swing state based on the determined position of the end.

[0112] The shapes of the hook device supported by the suspension rope and the suspended load supported by the hook device vary. On the other hand, the suspension rope, regardless of its type, has a characteristic shape that extends linearly from the tip of the attachment to the hook device, making it a more preferable target for detection compared to the hook device and the suspended load. Furthermore, the end portion of the suspension rope is a part where the amount of displacement is greater when the suspension rope is swinging, compared, for example, the intermediate portion between the upper end and the end portion of the suspension rope. In the first embodiment, the controller determines the swing state based on the position of the end portion where the amount of displacement is relatively large, so that the swing state of the suspension rope can be appropriately determined even when the swing of the hook device is relatively small.

[0113] The vibration state determination device according to the second embodiment preferably further comprises the following configuration in addition to the vibration state determination device according to the first embodiment. That is, in the vibration state determination device according to the second embodiment, the object information detector is an imaging device, the detected data is image data captured by the imaging device, and the controller may determine the position of the end portion by performing image recognition using the image data.

[0114] The vibration state determination device according to the third embodiment preferably comprises the following further configuration in addition to the vibration state determination device according to the second embodiment. That is, in the vibration state determination device according to the third embodiment, the controller preferably determines the position of the terminal using an image recognition model obtained by machine learning and the image data. In this third embodiment, the position of the terminal can be determined more appropriately using the image recognition model.

[0115] The swing state determination device according to the fourth embodiment preferably further comprises the following configuration in addition to the swing state determination device according to the third embodiment. That is, in the swing state determination device according to the fourth embodiment, the image recognition model is preferably a model obtained by machine learning using training data to which the positions of the starting end and the ending end of the suspension rope are annotated. In this fourth embodiment, the controller may not only determine the position of the ending end using the image recognition model and the image data, but also determine the position of the starting end of the suspension rope.

[0116] The swing state determination device according to the fifth embodiment preferably further comprises the following configuration in addition to the swing state determination device according to any one of the first to fourth embodiments. That is, in the swing state determination device according to the fifth embodiment, the controller preferably calculates the difference between the stationary coordinates, which are the coordinates of the end of the suspension rope when it is stationary, and the coordinates of the end of the suspension rope when it is swinging (actual coordinates), and determines the swing state based on the calculated difference. In this fifth embodiment, the controller can determine the swing state by a relatively simple calculation that includes calculating the difference between the stationary coordinates, which are set in advance or acquired in advance, and the actual coordinates of the end of the suspension rope during work using the construction machine (actual coordinates). In this fifth embodiment, the controller may calculate the difference between the stationary coordinates and the actual coordinates as the amount of swing of the suspension rope.

[0117] The swing state determination device according to the sixth embodiment preferably further comprises the following configuration in addition to the swing state determination device according to any one of the first to fifth embodiments. That is, in the swing state determination device according to the sixth embodiment, the suspension rope includes a plurality of rope portions that each extend downward from the tip of the attachment, and the controller may identify the coordinates of the end of each of the plurality of rope portions based on the image data, calculate the average value of the coordinates of the end of the plurality of rope portions, and determine the swing state based on the change in the average value. In this sixth embodiment, the controller can determine the swing state of the suspension rope regardless of the number of rope portions.

[0118] The seventh embodiment of the swing state determination device preferably further comprises the following configuration in addition to the swing state determination device according to any one of the first to sixth embodiments. That is, in the swing state determination device according to the seventh embodiment, the imaging device preferably has a zoom function, and the controller preferably calculates the amount of swing of the suspension rope using the change in the position of the end portion and the zoom magnification of the imaging device. In this seventh embodiment, since the controller calculates the amount of swing considering the zoom magnification, the swing state can be appropriately determined regardless of the zoom magnification.

[0119] The swing state determination device according to the eighth embodiment preferably further comprises the following configuration in addition to the swing state determination device according to any one of the first to seventh embodiments. That is, in the swing state determination device according to the eighth embodiment, the controller preferably calculates the swing angle of the suspension rope using the change in the position of the end portion of the suspension rope and the detection result input from the payout amount detector that detects the amount of the suspension rope paid out.

[0120] In this eighth aspect, if the swing angle is greater than or equal to a predetermined threshold, the controller may, for example, perform control to suppress the swing of the suspension rope, or perform control to limit the operation of the construction machine. In this eighth aspect, it is more preferable that the controller outputs information regarding the calculated swing angle to a display, and the display displays the input information. The display may be provided, for example, on the construction machine, or on an external information terminal located away from the construction machine. This allows operators, work managers, and other work personnel to recognize the swing angle of the suspension rope calculated by the controller.

[0121] The 9th embodiment of the swing state determination device preferably further comprises the following configuration in addition to the swing state determination device according to any one of the first to eighth embodiments. That is, in the swing state determination device according to the 9th embodiment, the suspension rope includes a plurality of rope portions extending downward from the tip of the attachment, and the controller may identify the number of rope portions based on the image data, compare the identified number with a set value for the number of rope portions set based on input from the work personnel, and output the comparison result. In this 9th embodiment, work personnel such as operators and work managers can recognize whether the number of rope portions they input matches the actual number by, for example, looking at the comparison result output on the display.

[0122] The construction machine according to the tenth embodiment comprises a machine body, an attachment supported by the machine body, a suspension rope hanging from the tip of the attachment, an object information detector attached to the tip of the attachment, and a sway state determination device according to any one of the first to ninth embodiments. As in this tenth embodiment, the sway state determination device may be provided in the construction machine.

[0123] The 11th embodiment of the swing state determination system comprises a construction machine equipped with a machine body, an attachment supported by the machine body, a suspension rope hanging from the tip of the attachment, and an object information detector attached to the tip of the attachment, and a swing state determination device according to any one of the first to 9th embodiments.

[0124] A method for determining the swing state according to the twelfth embodiment is a method for determining the swing state for a construction machine comprising a machine body, an attachment supported by the machine body, a suspension rope hanging from the tip of the attachment, and an object information detector attached to the tip of the attachment, comprising: a controller acquiring detection data for the suspension rope detected by the object information detector; the controller identifying the position of the end of the suspension rope based on the detection data; and the controller determining the swing state of the suspension rope based on the position of the end.

[0125] The program according to the 13th embodiment is a program for a construction machine comprising a machine body, an attachment supported by the machine body, a suspension rope hanging from the tip of the attachment, and an object information detector attached to the tip of the attachment, wherein the program causes a computer to perform the following processes: acquiring detection data of the suspension rope detected by the object information detector; identifying the position of the end of the suspension rope based on the detection data; and determining the swing state of the suspension rope based on the position of the end.

[0126] Furthermore, the storage medium according to the modified version of the thirteenth embodiment may store the program. The storage medium may be a non-temporary computer-readable storage medium that stores the program. The vibration state determination method is realized when the program is executed by a computer processor.

Claims

1. A swing state determination device for a construction machine comprising a machine body, an attachment supported by the machine body, a suspension rope hanging from the tip of the attachment, and an object information detector attached to the tip of the attachment, the swing state determination device comprising a controller configured to acquire detection data for the suspension rope detected by the object information detector, to identify the position of the end of the suspension rope based on the detection data, and to determine the swing state of the suspension rope based on the position of the end.

2. The shake state determination device according to claim 1, wherein the object information detector is an imaging device, the detected data is image data captured by the imaging device, and the controller determines the position of the terminal by performing image recognition using the image data.

3. The vibration state determination device according to claim 2, wherein the controller determines the position of the terminal portion using an image recognition model obtained by machine learning and the image data.

4. The swing state determination device according to claim 3, wherein the image recognition model is a model obtained by machine learning using training data on which annotations are added to the position of the starting end and the position of the ending end of the suspension rope.

5. The swing state determination device according to any one of claims 1 to 4, wherein the controller calculates the difference between the stationary coordinates, which are the coordinates of the end of the suspension rope when it is stationary, and the coordinates of the end of the suspension rope when it is swinging, and determines the swing state based on the calculated difference.

6. The suspension rope includes a plurality of rope portions extending downward from the tip of the attachment, the controller identifies the coordinates of the end of each of the plurality of rope portions based on the detection data, calculates the average value of the coordinates of the end of the plurality of rope portions, and determines the swing state based on the change in the average value, the swing state determination device according to any one of claims 1 to 5.

7. The swing state determination device according to any one of claims 2 to 4, wherein the imaging device has a zoom function, and the controller calculates the amount of swing of the suspension rope using the change in the position of the end portion and the zoom magnification of the imaging device.

8. The swing state determination device according to any one of claims 1 to 7, wherein the controller calculates the swing angle of the suspension rope using the change in position of the end portion of the suspension rope and the detection result input from the payout amount detector that detects the amount of the suspension rope paid out.

9. The swing state determination device according to any one of claims 1 to 8, wherein the suspension rope includes a plurality of rope portions extending downward from the tip of the attachment, the controller identifies the number of rope portions based on the detection data, compares the identified number with a set value for the number of rope portions set based on input from the workers, and outputs the comparison result.

10. A construction machine comprising: a machine body; an attachment supported by the machine body; a suspension rope hanging from the tip of the attachment; an object information detector attached to the tip of the attachment; and a swing state determination device according to any one of claims 1 to 9.

11. A construction machine comprising a machine body, an attachment supported by the machine body, a suspension rope hanging from the tip of the attachment, and an object information detector attached to the tip of the attachment; and a swing state determination device according to any one of claims 1 to 9.

12. A method for determining the swing state of a construction machine comprising a machine body, an attachment supported by the machine body, a suspension rope hanging from the tip of the attachment, and an object information detector attached to the tip of the attachment, the method comprising: a controller acquiring detection data for the suspension rope detected by the object information detector; the controller identifying the position of the end of the suspension rope based on the detection data; and the controller determining the swing state of the suspension rope based on the position of the end.

13. A program for a construction machine comprising a machine body, an attachment supported by the machine body, a suspension rope hanging from the tip of the attachment, and an object information detector attached to the tip of the attachment, the program causing a computer to perform the following processes: acquiring detection data for the suspension rope detected by the object information detector; identifying the position of the end of the suspension rope based on the detection data; and determining the swing state of the suspension rope based on the position of the end.

Images