Rear turning interference monitoring device

Abstract

To display whether an interfered article can be moved by detecting interference of a rear part of an upper turning body with the article present in the surroundings when a mobile crane is installed.SOLUTION: A rear-turning interference monitoring device 10 comprises: an image acquisition unit 11 that acquires an image including the sweeping range of a rear part of an upper turning body of a mobile crane by rotation of the upper turning body; an interference detection unit 12 that detects an article that can be interfered with by the rear part through the rotation of the upper turning body; a portable determination unit 14 that determines whether the interfered article can be moved from the image when the interference detection unit 12 detects the interfered article; and a display unit 15 that displays whether the interfered article is present and whether it can be moved.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] This relates to rear swing interference monitoring, which monitors whether the rear of the upper swing body of a mobile crane interferes with other objects due to swing. [Background technology]

[0002] A driving assistance technology that warns of interference with other objects when reversing or parking a vehicle has been put to practical use. For example, Patent Document 1 describes a technology that enables parking according to a desired parking method. The information processing device of Patent Document 1 calculates the time required for automatic parking to be completed for each mode according to the parking method of automatic parking that automatically parks a vehicle, and controls the display of the required time according to the mode, thereby enabling desired parking to be performed according to the time required for automatic parking to be completed. The technology of Patent Document 1 is touted as being applicable, for example, to an automatic parking system incorporated as a part of an automatic driving system equipped in a vehicle capable of automatic driving. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-140890 A Summary of the Invention [Problem to be solved by the invention]

[0004] When applying driving assistance technology to a mobile crane, it is possible to apply it not only to check for interference when moving and setting up the crane, but also to monitor interference at the rear caused by the rotation of the upper rotating body during crane operation. However, driving assistance technology does not know what kind of object is interfering with the vehicle. When applying vehicle driving technology to a mobile crane, the operator needs to determine whether the interfering object can be removed before setting up the crane.

[0005] An object of the present invention is to detect, when a mobile crane is being set up, whether the rear part of the upper rotating body interferes with a nearby object due to the rotation of the upper rotating body, and to display whether the interfering object can be moved. [Means for solving the problem]

[0006] A rear turning interference monitoring device according to an aspect of the present invention comprises: an image acquisition unit that acquires an image including a range swept by a rear portion of an upper rotating body due to rotation of the upper rotating body of the mobile crane; an interference detection unit that detects an interference object that may interfere with the rear portion due to the rotation of the upper rotating body; a portability determination unit that determines whether or not the interfering object can be moved from the image when the interference detection unit detects the interfering object; a display unit that displays the presence or absence of the interfering object and whether or not the interfering object can be moved; Equipped with.

[0007] Preferably, the portability determination unit includes a trained model of a neural network that classifies an area of ​​the interfering object in the image into a case where the interfering object can be moved and a case where the interfering object cannot be moved.

[0008] Preferably, the portability determination unit includes a trained model of a neural network that classifies the type of the interfering object from the area of ​​the interfering object in the image.

[0009] Preferably, the interference detection unit detects the interfering object from information on the position and size of objects around the mobile crane obtained from at least one of an image recognition device, a radar, a laser scanner, and an ultrasonic detector.

[0010] Preferably, when the portability determination unit determines that the interfering object cannot be moved, a movement information acquisition unit acquires information on whether the mobile crane has been moved before starting crane work by the mobile crane; A storage unit that stores the determination result of the portability determination unit and information on whether the mobile crane was moved before starting crane work in association with each other; Equipped with. Effect of the Invention

[0011] The rear swing interference monitoring device of the present invention makes it possible to check whether an object that the rear of the upper rotating body interferes with can be moved when deciding the installation position of the mobile crane. If an object that cannot be moved is found to be interfering, the installation position of the mobile crane can be changed before the mobile crane is placed in the working position. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is an external view showing an example of a mobile crane equipped with a rear swing interference monitoring device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a functional block diagram of a rear turning interference monitoring device according to the first embodiment of the present invention; [Diagram 3] Block diagram of a portability determination unit according to the first embodiment. [Figure 4] FIG. 1 is a diagram showing an example of a neural network according to a first embodiment; [Diagram 5] Another example of a neural network classifying types of interference. [Figure 6] A flowchart showing an example of an operation of monitoring rear turning interference according to the first embodiment. [Figure 7] Functional block diagram of a rear turning interference monitoring device according to a second embodiment [Figure 8] A flowchart showing an example of an operation of monitoring rear turning interference according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings, in which the same or equivalent parts are designated by the same reference numerals.

[0014] Embodiment 1 FIG. 1 is an external view showing an example of a mobile crane equipped with a rear swing interference monitoring device according to an embodiment of the present invention. The crane 100 shown in FIG. 1 is a rough terrain crane. The lower running body 20 of the crane 100 is equipped with an engine housed in an engine room 23 arranged at the rear of a lower frame 24. The lower running body 20 is equipped with steerable wheels 27 and wheels 28 that can be driven by the engine, and can travel on the ground. The lower running body 20 is equipped with outriggers 25, 26 at the front and rear of the lower frame 24, respectively. The outriggers 25, 26 can be extended to the left and right of the lower running body 20, extended downward and placed on the ground, and the lower frame 24 can be supported by the outriggers 25, 26.

[0015] An upper rotating body 30 is rotatably mounted on the center of the lower frame 24 via a swivel bearing 29. The upper rotating body 30 includes a swivel frame 31, a telescopic boom 32, a hoisting cylinder 33, a hoisting device 34, a hook 35, and a wire rope 36.

[0016] The base end 38 of the telescopic boom 32 is supported on the swivel frame 31 so that it can be raised and lowered around a fulcrum pin 39. A hoisting cylinder 33 interposed between the swivel frame 31 and the telescopic boom 32 drives the telescopic boom 32 to raise and lower. Figure 1 shows the telescopic boom 32 in a lowered state with the outriggers 25, 26 folded in. The crane 100 has a cab 22, which is used both for traveling and for crane operation, located on the swivel frame 31 to the right of the telescopic boom 32.

[0017] The crane 100 is equipped with a hook 35 capable of suspending a load. Fig. 1 shows the hook 35 housed in the lower frame 24. A load can be suspended by the hook 35 when the telescopic boom 32 is raised and extended, the hook 35 is removed from the lower frame 24, and the hook 35 is suspended by a wire rope 36 from a top sheave attached to the boom tip of the telescopic boom 32. When suspending a load, the top sheave is positioned above the hook 35.

[0018] A hoisting device 34 for winding up a wire rope 36 is provided behind the revolving frame 31. The wire rope 36 for suspending a hook 35 is wound around a top sheave, then around a slide sheave disposed behind a base end 38, and wound up by the hoisting device 34.

[0019] In order to keep the crane 100 stable during operation, a counterweight is attached to the rear part 37 of the rotating frame 31. When the rotating frame 31 rotates around the central axis of the rotating bearing 29, the rear part 37 of the rotating frame 31 including the counterweight swings out of the range of the lower running body 20. If there is an object within the rotating range of the rotating frame 31 beside the lower running body 20, the rear part 37 will interfere with the object when the upper rotating body 30 rotates. The crane 100 is equipped with a rear swing interference monitoring device that monitors whether the rear part 37 interferes with other objects when swinging.

[0020] 2 is a functional block diagram of the rear swing interference monitoring device according to the first embodiment of the present invention. An object detection sensor 18 that detects objects around the crane 100 and a camera 19 that captures images of the surroundings of the upper swing body 30 are connected to the rear swing interference monitoring device 10 (hereinafter sometimes referred to as the monitoring device 10). The object detection sensor 18 and the camera 19 are disposed, for example, one on each side of the upper swing body 30. The object detection sensor 18 detects objects in the spaces on the right and left sides of the upper swing body 30. The camera 19 captures images of the spaces on the right and left sides of the upper swing body 30, including the range swept by the rear part 37 due to the swing of the upper swing body 30.

[0021] The monitoring device 10 includes an image acquisition unit 11, an interference detection unit 12, a recognition range setting unit 13, a portability determination unit 14, and a display unit 15. The image acquisition unit 11 acquires an image including a range swept by the rear part 37 due to the rotation of the upper rotating body 30 from a camera 19. The interference detection unit 12 acquires information indicating the positions and sizes of objects around the crane 100 from an object detection sensor 18.

[0022] The object detection sensor 18 is, for example, an image recognition device, a radar, a laser scanner, or an ultrasonic detector. The interference detection unit 12 detects, as an interfering object, an object in a range swept by the rear part 37 due to the rotation of the upper rotating body 30, based on information indicating the position and size of the object around the crane 100 relative to the crane 100. When the object detection sensor 18 is an image recognition device, for example, the camera 19 may be a stereo camera, and surrounding objects may be detected from images of the camera 19. In that case, the object detection sensor 18 can be included in the monitoring device 10.

[0023] When an interfering object is detected by interference detection unit 12, recognition range setting unit 13 sets the recognition range as a range including the interfering object in the image acquired by image acquisition unit 11. The position and imaging range of camera 19 are set so as to capture an image of a space including the range swept by rear portion 37, so the image of camera 19 includes an image of the interfering object. Recognition range setting unit 13 sets the recognition range so as to include the image of the interfering object.

[0024] If there is an interfering object, the portability determination unit 14 determines whether or not the interfering object can be moved from the recognition range of the image acquired by the image acquisition unit 11. The portability determination unit 14 determines that the interfering object can be moved if it is, for example, a ladder or stepladder, a temporary fence, a vehicle, or a temporary toilet. The portability determination unit 14 determines that the interfering object cannot be moved if it is a structure fixed to the ground, for example, a fixed fence, a fence or wall, an embedded pillar, or a standing tree.

[0025] The display unit 15 displays the presence or absence of an interfering object detected by the interference detection unit 12, and the result of the determination by the portability determination unit 14 as to whether the interfering object can be moved. To display the presence of an interfering object, for example, a buzzer may sound or a warning sound may be generated from a speaker. A lamp may also be turned on, or a warning icon or image may be displayed on a screen monitored by the operator. The warning may be a combination of an audio display and a visual display. The result of the determination as to whether the interfering object can be moved may be displayed, for example, as characters or icons together with an image of the interfering object on a screen monitored by the operator. The display unit 15 may display whether the interfering object can be moved by distinguishing the type, color, flashing or brightness of the character string or icon, or a combination thereof.

[0026] 3 is a block diagram of a portability determination unit according to embodiment 1. In embodiment 1, the portability determination unit 14 includes a trained model 41 of a neural network and a movement possibility calculation unit 42. The trained model 41 includes a neural network that classifies an interfering object into two states, for example, a state where the interfering object can be moved and a state where the interfering object cannot be moved, based on the recognition range of the image.

[0027] FIG. 4 is a diagram showing an example of a neural network according to the first embodiment. The neural network 43 is composed of an input layer, an intermediate layer, and an output layer, each including a node composed of an artificial neuron, and edges connecting the nodes between adjacent layers. The intermediate layer includes one or more n layers. Each pixel value xi in the recognition range of the image is input to each node i (i=1...k) of the input layer. In the intermediate layer, the output of each previous layer is combined and the result calculated by the activation function is transmitted to the subsequent layer and finally output to the output layer. In the intermediate layer, the convolution layer and the pooling layer are repeated to classify the image data in the recognition range into a case where the interfering object can move and a case where the interfering object cannot move. The output layer has nodes for a case where the interfering object can move and a case where the interfering object cannot move, and each node j (j=1, 2) outputs the probability pj that the image data is classified into state j.

[0028] A plurality of image data classified into cases where the interfering object can be moved and cases where the interfering object cannot be moved are input to the neural network 43, and machine learning is performed by adjusting each parameter through backpropagation of the difference between the state in which the image data is classified and the output of the neural network 43. The image data to be machine-learned includes at least image data in which the interfering object can be moved and image data in which the interfering object cannot be moved.

[0029] The movement possibility calculation unit 42 in Fig. 3 calculates whether or not the interfering object can be moved based on the probability of when the interfering object can be moved and when it cannot be moved, which is the output of the neural network 43. For example, if the difference between the probability of when the interfering object can be moved minus the probability of when it cannot be moved is greater than a set threshold, it calculates that the interfering object can be moved, and if it is equal to or less than the threshold, it calculates that the interfering object cannot be moved. The portability determination unit 14 outputs the calculation result of the movement possibility calculation unit 42 to the display unit 15 as a determination result of whether or not the interfering object can be moved.

[0030] The rear swing interference monitoring device 10 includes an I / O interface, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU is, for example, a microprocessor, and is a central processing unit that executes various processes and calculations. The monitoring device 10 reads out a control program stored in the ROM, and executes the control program with the CPU while using the RAM as a work memory, thereby performing the operations of the image acquisition unit 11, the interference detection unit 12, the recognition range setting unit 13, the portability determination unit 14, and the display unit 15. The monitoring device 10 may be incorporated into a crane control device (not shown). The monitoring device 10 may also be realized as a function of the crane control device.

[0031] Fig. 5 is a diagram showing another example of a neural network that classifies the type of interference. Neural network 44 in Fig. 5 shows a different example of trained model 41 in Fig. 3. Neural network 44 in Fig. 5 has a similar structure to neural network 43 in Fig. 4. In neural network 44 that classifies the type of interference, an output layer is composed of nodes p1, p2, p3, ...pM, the number of which is equal to the number of types to be classified.

[0032] The neural network 44 classifies the type of interference object from the image data included in the recognition range into, for example, a ladder or stepladder, a temporary fence, a vehicle, a temporary toilet, a fixed fence, a wall or fence, an embedded pillar, and a standing tree.

[0033] A plurality of image data classified into types of interference objects are input to the neural network 44, and the neural network 44 is trained to carry out machine learning by backpropagating the difference between the types of interference objects for which the image data is classified and the output of the neural network 44 and adjusting each parameter.

[0034] When the trained model 41 is the neural network 44 in Fig. 5, the movement possibility calculation unit 42 in Fig. 3 calculates whether or not the interfering object can be moved based on the probability of each type of interfering object output from the neural network 44. For example, the movement possibility calculation unit 42 calculates whether or not the interfering object can be moved according to whether or not the node with the highest probability output from the neural network 44, i.e., the most likely type of interfering object, can be moved. Alternatively, for example, the types of output nodes of the neural network 44 are divided into types that can be moved and types that cannot be moved, and the larger total value of the probability of each type is adopted to calculate whether or not the interfering object can be moved.

[0035] Neural network 44 in Fig. 5 may be a part of neural network 43 in Fig. 4. That is, the neural network 43 in Fig. 4 may be configured by connecting a layer that classifies and outputs cases in which an interfering object can be moved and cases in which an interfering object cannot be moved based on the type of interfering object to the output layer of neural network 44 in Fig. 5.

[0036] 6 is a flowchart showing an example of the operation of rear swing interference monitoring according to the first embodiment. The rear swing interference monitoring is started by an operation of the operator of the crane 100. For example, the rear swing interference monitoring is started when a command is issued to extend the outriggers 25, 26. Alternatively, the rear swing interference monitoring is started when the swing lock of the upper swing body is released, or when the fixing lock of the boom to the stowed position is released.

[0037] In a narrow work site, there may be a situation where materials are placed there and it is impossible to pass through the site in the traveling position. In such a case, the boom may be raised and the upper rotating body may be rotated without extending the outriggers 25, 26 to move around the site. In such a case, rear swing interference monitoring is also required. In addition, in a crawler crane that does not have outriggers, the rear swing interference monitoring is activated when the swing lock of the upper rotating body is released or when the lock for fixing the boom to the stowed position is released.

[0038] The image acquisition unit 11 of the rear swing interference monitoring device 10 acquires from the camera 19 an image including the range swept by the rear section 37 due to the swing of the upper swing body 30 (step S10). The interference detection unit 12 acquires information indicating the positions and sizes of objects around the crane 100 from the object detection sensor 18, and detects objects within the range swept by the rear section 37 due to the swing of the upper swing body 30 as interfering objects (step S11). If there is no interfering object that the rear section 37 will interfere with due to the swing of the upper swing body 30 (step S12; N), the interference monitoring operation ends.

[0039] If there is an interfering object (step S12; Y), the display unit 15 displays a warning that there is an interfering object (step S13). Next, the recognition range setting unit 13 sets a recognition range as a range including the interfering object in the image acquired by the image acquisition unit 11, and the portability determination unit 14 determines whether the interfering object can be moved based on the image data of the set recognition range (step S14).

[0040] If it is determined that the interfering object can be moved (step S15; Y), the display unit 15 displays that the interfering object can be moved (step S16). If it is determined that the interfering object cannot be moved (step S15; N), the display unit 15 displays that the interfering object cannot be moved (step S17). After displaying whether or not the interfering object can be moved, the interference monitoring operation ends.

[0041] According to the rear rotation interference monitoring device 10 of embodiment 1, an object within the range swept by the rear section 37 of the upper rotating body 30 due to the rotation of the upper rotating body 30 is detected as an interfering object, and further, whether or not the interfering object can be moved is determined and displayed. Therefore, when determining the installation position of the crane 100, it is possible to confirm whether or not the object interfering with the rear section 37 of the upper rotating body 30 can be moved.

[0042] The monitoring device 10 of the first embodiment can be applied to mobile cranes such as truck cranes, all-terrain cranes, crawler cranes, and railway cranes in addition to rough terrain cranes.

[0043] Embodiment 2 7 is a functional block diagram of a rear turning interference monitoring device according to embodiment 2. The rear turning interference monitoring device 10 of embodiment 2 further includes a movement information acquisition unit 16 and a storage unit 17. The other configurations are the same as those of embodiment 1.

[0044] The movement information acquisition unit 16 acquires information as to whether the crane 100 has been moved or not before starting the crane operation of the crane 100 after the portability determination unit 14 has determined whether the interfering object can be moved or not. For example, the monitoring device 10 can obtain information on the movement of the crane 100 and the state of the crane operation from a crane control device (not shown).

[0045] The movement of the crane 100 may be recognized by a change in the image acquired by the image acquisition unit 11 from the camera 19. The start of crane operation can be recognized, for example, by the extension of the outriggers 25, 26, the raising or lowering of the telescopic boom 32, the rotation of the upper rotating body 30, or the operation of the hoisting device 34.

[0046] The memory unit 17 stores the determination result of the portability determination unit 14 and information on whether the crane 100 was moved before starting the crane work in association with each other. It is possible to record the reason why the installation location of the crane 100 had to be changed, and to collect data that is useful for planning the crane work.

[0047] The information stored in the memory unit 17 can be of four types: (1) the interfering object can be moved; the crane does not move, (2) the interfering object can be moved; the crane moves, (3) the interfering object cannot be moved; the crane does not move, and (4) the interfering object cannot be moved; the crane moves. The memory unit 17 may further have the operator input whether or not the judgment result of the portability judgment unit 14 differs from the actual situation, and may store the input information in association with the judgment result. These data and image data can be fed back as learning data for the portability judgment unit 14.

[0048] Fig. 8 is a flowchart showing an example of the operation of rear swing interference monitoring according to embodiment 2. Steps S10 to S17 in the flowchart of Fig. 8 are the same as steps S10 to S17 in Fig. 6. If it is determined that the interfering object cannot be moved (step S15; N), the display unit 15 displays that the interfering object cannot be moved (step S17), and the movement information acquisition unit 16 acquires information as to whether or not the crane 100 has been moved before starting the crane operation (step S18).

[0049] After indicating that the interfering object can be moved (step S16), and after obtaining information as to whether or not the crane 100 has been moved (step S18), the memory unit 17 stores the determination result as to whether or not the interfering object can be moved in association with information as to whether or not the crane 100 has been moved before starting the crane operation (step S19), and terminates the interference monitoring operation.

[0050] In FIG. 8, when it is determined that the interfering object cannot be moved (step S15; N, step S17), information as to whether or not the crane 100 has been moved is obtained (step S18). However, even when it is determined that the interfering object can be moved (step S15; Y, step S16), information as to whether or not the crane 100 has been moved may be obtained before starting the crane operation.

[0051] According to the rear swing interference monitoring device 10 of the second embodiment, the reason why the installation location of the crane 100 had to be changed can be recorded, and data useful for planning crane operations can be collected. [Explanation of symbols]

[0052] 10 Rear turning interference monitoring device 11 Image acquisition section 12 Interference detection unit 13 Recognition range setting section 14 Portability determination section 15 Display 16 Movement information acquisition unit 17 Memory section 18 Object detection sensor 19 Camera 20 Undercarriage 22 Cab 23 Engine Room 24 Lower Frame 25,26 Outrigger 27,28 wheels 29 Swing Bearing 30 Upper rotating body 31 Swivel Frame 32 Telescopic boom 33 Dinghy cylinder 34 Winding device 35 Hook 36 Wire Rope 37 Rear 38 Proximal end 39 Fulcrum pin 41 Trained Models 42 Mobility possibility calculation unit 43,44 Neural Networks 100 Crane

Claims

1. an image acquisition unit that acquires an image including a range swept by a rear portion of an upper rotating body due to rotation of the upper rotating body of the mobile crane; an interference detection unit that detects an interference object that may interfere with the rear portion due to the rotation of the upper rotating body; a portability determination unit that determines whether or not the interfering object can be moved from the image when the interference detection unit detects the interfering object; a display unit that displays the presence or absence of the interfering object and whether or not the interfering object can be moved; A rear turning interference monitoring device comprising:

2. The rear turning interference monitoring device according to claim 1 , wherein the portability determination unit includes a trained model of a neural network that classifies an area of ​​the interfering object in the image into a case where the interfering object can be moved and a case where the interfering object cannot be moved.

3. The rear turning interference monitoring device according to claim 1 or 2, wherein the portable determination unit includes a trained model of a neural network that classifies the type of the interfering object from the area of ​​the interfering object in the image.

4. 4. The rear swing interference monitoring device according to claim 1, wherein the interference detection unit detects the interfering object from information on the positions and sizes of objects around the mobile crane obtained from at least one of an image recognition device, a radar, a laser scanner, and an ultrasonic detector.

5. a movement information acquisition unit that acquires information on whether the mobile crane has been moved before starting crane work using the mobile crane when the portability determination unit determines that the interfering object cannot be moved; A storage unit that stores the determination result of the portability determination unit and information on whether the mobile crane was moved before starting crane work in association with each other; The rear turning interference monitoring device according to any one of claims 1 to 4, comprising:

Images