Information processing device and information processing method

Abstract

To provide an information processing device and the like capable of appropriately grasping a situation around a suspended load.SOLUTION: This information processing device comprises: a position calculation unit that calculates the position of a suspended load suspended by a work machine on the basis of attitude information indicating the attitude of the work machine; a photographing range control unit that controls the photographing range of a camera in accordance with the position of the suspended load calculated by the position calculation unit; and a camera selection unit that selects one of a plurality of cameras installed at a plurality of different places in accordance with the position of the suspended load calculated by the position calculation unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and an information processing method. [Background technology]

[0002] Patent Document 1 discloses a lifting support system that, based on information about the hanging position of a suspended load acquired from a drive control unit, identifies the size of the suspended load at a position corresponding to the position information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-110436 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology disclosed in Patent Document 1, images of the downward direction are acquired by a camera attached to the tip of the jib (near the hook), which makes it impossible to properly grasp the situation around the suspended load, including, for example, the area below the suspended load where the view is obstructed by the load.

[0005] Therefore, in one aspect, the present invention aims to provide an information processing device and the like that can appropriately grasp the situation around a suspended load. [Means for solving the problem]

[0006] In one embodiment, a position calculation unit that calculates the position of a load suspended by a work machine based on attitude information that indicates the attitude of the work machine; a photographing range control unit that controls a photographing range of the camera in accordance with the position calculated by the position calculation unit; An information processing device comprising: [Effects of the Invention]

[0007] According to one aspect, the present invention makes it possible to properly grasp the situation around the suspended load. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating a configuration of an information processing apparatus according to an embodiment of the present invention. [Figure 2] 10 is a flowchart showing a process in the information processing device. [Figure 2A] FIG. 1 is a diagram showing a lifting operation using a tower crane as a work machine. [Figure 3] FIG. 2 is a front view showing the installation position of the camera. [Figure 4] FIG. 2 is a top view showing the installation position of the camera. DETAILED DESCRIPTION OF THE INVENTION

[0009] FIG. 1 is a diagram showing the configuration of an information processing apparatus according to this embodiment.

[0010] 1, the information processing device 10 of this embodiment includes a position calculation unit 11 that calculates the position of a load suspended from a work machine based on attitude information that indicates the attitude of the work machine, and an imaging range control unit 12 that controls the imaging range of a camera 20 in accordance with the position calculated by the position calculation unit 11. The information processing device 10 also includes a display unit 13 that displays an image captured by the camera 20.

[0011] The information processing device 10 also includes a camera selection unit 14 that selects one of a plurality of cameras 20 installed at a plurality of different locations according to the position of the suspended load calculated by the position calculation unit 11.

[0012] The information processing device 10 also includes an output unit 15 that outputs images captured by the camera 20 to the automatic driving system of the work machine. The automatic driving system of the work machine will be described later.

[0013] The position calculation unit 11 calculates the position of the suspended load based on attitude information indicating the attitude of the work machine (for example, data such as the jib angle).

[0014] 1, the camera 20 and the driving device 21 that controls the direction of the camera 20 are controlled by the imaging range control unit 12. The imaging range control unit 12 controls the driving device 21 or the camera 20 in accordance with the position of the suspended load calculated by the position calculation unit 11 so that the camera 20 captures an image of the area below.

[0015] The display unit 13 displays the images captured by the camera 20 in a manner that is visible to the operator of the work machine. For example, the display unit 13 displays the images captured by the camera 20 on a monitor screen installed in the driver's seat of the work machine.

[0016] All or part of the information processing device 10 of this embodiment can be configured as a device mounted on a work machine. Also, all or part of the information processing device 10 can be configured as an external device (for example, a server) capable of communicating with the work machine, camera 20, etc.

[0017] Next, the operation of the information processing device 10 will be described.

[0018] Fig. 2 is a flowchart showing the processing in the information processing device, and Fig. 2A is a diagram showing the state of lifting work using a tower crane as a work machine. The processing shown in Fig. 2 shows an example in which multiple cameras 20 are installed.

[0019] In step S102, the position calculation unit 11 acquires posture information indicating the posture of the tower crane 50, such as the angle of the jib 52, and calculates the position of the suspended load 61 based on the acquired posture information. Note that instead of directly acquiring the posture information as data, for example, the posture information displayed on the display device of the driver's seat 51 may be read by OCR and acquired as data.

[0020] In step S104, the camera selection unit 14 selects one of the multiple cameras 20 according to the position of the suspended load 61 calculated by the position calculation unit 11. Here, it is sufficient that a camera 20 that can view the surroundings of the suspended load 61, including the area 62 (FIG. 3) below the suspended load 61, is selected. For example, a camera 20 that is close to the position of the suspended load 61 may be selected, or a camera 20 that can capture the entire surroundings of the suspended load 61 may be selected. Furthermore, for example, if multiple images can be displayed on the driver's seat 51 (FIG. 3) of the tower crane 50, multiple cameras 20 may be selected.

[0021] In step S106, the shooting range control unit 12 operates the camera 20 selected by the camera selection unit 14 in step S104 or the corresponding driving device 21 to acquire an image captured by the camera 20, and the process proceeds to step S102.

[0022] In step S106, the photographing range control unit 12 acquires an image of the surroundings of the suspended load 61, including the area 62 below the position of the suspended load 61 calculated by the position calculation unit 11, as an image captured by the camera 20. Here, the photographing range control unit 12 controls the photographing range of the camera 20 in accordance with the position of the suspended load 61 calculated by the position calculation unit 11 so that the camera 20 can capture an image of the area below. For example, the photographing range control unit 12 can change the photographing direction of the camera 20 in the vertical and horizontal directions by operating the drive device 21, thereby causing the photographing range of the camera 20 to follow the suspended load 61. Furthermore, the photographing range control unit 12 may adjust the field of view size of the camera 20 as appropriate in accordance with the position of the suspended load 61.

[0023] In this way, in this embodiment, an image of the surroundings of the load 61, including the area 62 below the position calculated by the position calculation unit 11, is acquired and displayed so that the operator can see it. This allows the operator to properly recognize the situation around the load 61, including, for example, the area below the load 61. In contrast, when using an image from a camera that captures an image below from a position near the crane hook, such as the tip 52a of the jib 52, it can be difficult to confirm the situation below the load 61, where the view is blocked by the load 61, depending on the size and shape of the load 61.

[0024] Next, an example of installation of the camera 20 will be described.

[0025] FIG. 3 is a front view showing the installation position of the camera, and FIG. 4 is a top view showing the installation position of the camera.

[0026] In the examples of FIGS. 3 and 4 , cameras 20 are installed at position P0 on or near the operator's seat 51 ( FIG. 3 ) of the tower crane 50, and at positions P1, P2, and P3 around the tower crane 50. This allows the camera 20 at positions P1, P2, and P3 to be selected depending on the horizontal position of the jib 52 of the tower crane 50, for example, when a suspended load 61 is in the blind spot of the camera 20 installed at position P0. For example, cameras 20 at positions P1, P2, and P3 close to the suspended load 61 or positions P1, P2, and P3 that are not blocked by obstacles (including the main body of the tower crane 50) may be selected. Furthermore, for example, camera 20 at position P0 may be selected regardless of the position of the suspended load 61, and images from cameras 20 at positions P1, P2, and P3 may be added depending on the horizontal position of the jib 52. Note that positions P1, P2, and P3 may be selected from high-altitude locations with an open view, such as a temporary fence or platform close to the site boundary 70. The number of cameras 20 is arbitrary, but for example, cameras 20 can be installed at two to four locations around the tower crane 50 so as to capture all directions of the tower crane 50. Furthermore, for example, when multiple tower cranes 50 are used, cameras 20 attached to other tower cranes 50 can also be selected.

[0027] Next, an example in which the information processing device 10 is applied to an automatic driving system for a work machine will be described.

[0028] BACKGROUND ART Known automatic operation systems for work machines automatically control the work machine so that a suspended load is transported along a preset route during a lifting operation.

[0029] The information processing device 10 can provide the captured image by the camera 20 via the output unit 15 to such an automatic driving system.

[0030] For example, in an automatic driving system, image recognition processing can be performed on images captured by the camera 20, and the results of the image recognition can be reflected in automatic driving.

[0031] For example, when lifting a suspended load that is not registered in the automated driving system in advance, the size and shape of the suspended load can be determined based on images captured by camera 20 and reflected in the system. The shape of the suspended load, etc., recognized from images captured by camera 20 can be modeled and reproduced, allowing the system to recognize the shape of the suspended load, etc. In this case, by reproducing the orientation of the suspended load (e.g., the posture of the steel frame, the longitudinal direction), the operation during automated driving can be optimized. For example, the shape and orientation of the suspended load at the loading location can be recognized based on images captured by camera 20 and reflected in the operation during loading. Furthermore, the situation at the unloading location and the shape and orientation of the suspended load during transport can be recognized based on images captured by camera 20 and reflected in the operation during unloading.

[0032] It is also possible to recognize the surrounding situation based on images captured by the camera 20 and reflect this in the operation during automatic driving. For example, the presence or absence of people at the loading and unloading locations can be recognized and reflected in the operation. Furthermore, automatic winding and lowering can also be triggered by the movement of a person near the loading and unloading locations (for example, raising their hand).

[0033] Furthermore, it is possible to provide captured images showing not only the surroundings of the suspended load but also the situation on the pre-set route. This allows the automated driving system to grasp the situation on the planned route, and for example, makes it possible to re-set the route if an obstacle enters the planned route. In this way, by reflecting the results of image recognition on the images captured by the camera 20 in the automated driving system, safety and productivity can be improved.

[0034] Furthermore, the system can automatically generate a detection area for the suspended load based on the recognition results of the suspended load shape, etc., using the image captured by the camera 20. By limiting the detection area, it is possible to reduce the processing load.

[0035] Furthermore, the automatic operation system calculates and simulates load sway during lifting in advance. However, if load sway exceeds the prediction during actual lifting and is detected based on the image captured by camera 20, this can be reflected in the system. Specifically, the lifting operation can be controlled to suppress load sway. Furthermore, if the load sway exceeds the prediction, an emergency stop may be performed or automatic operation may be stopped. Furthermore, if the load sway exceeds the prediction during actual lifting, machine learning may be performed again using the data from that time to improve the accuracy of the next prediction.

[0036] Furthermore, images captured by camera 20 can be provided to a supervisor during autonomous driving. For example, when the autonomous driving system is started and operated from outside the site, the work cannot be directly observed with the naked eye. However, by viewing the images captured by camera 20 provided via output unit 15, the supervisor can carry out autonomous driving while confirming safety. Furthermore, by pointing camera 20 at a location specified via the user interface of the autonomous driving system, the supervisor can check the site conditions, such as the loading and unloading locations. Furthermore, the shape of the suspended load recognized from the images captured by camera 20 may be modeled and reproduced and presented to the supervisor. In this case, the necessary information can be provided to the supervisor by reproducing the load orientation (e.g., the posture and longitudinal direction of the steel frame).

[0037] As described above, in this embodiment, the position of a load suspended by a work machine is calculated based on attitude information that indicates the attitude of the work machine, and the imaging range of the camera 20 is controlled in accordance with the calculated position of the load. This makes it possible to properly grasp the situation around the load regardless of the position of the load.

[0038] Although the embodiments have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications and changes are possible within the scope of the claims. In addition, it is also possible to combine all or a plurality of components of the above-described embodiments. [Explanation of symbols]

[0039] 10 Information Processing Systems 11 Position calculation section 12 Shooting range control section 13 Display section 14 Camera selection section 15 Output section

Claims

1. a position calculation unit that calculates the position of a load suspended by a work machine based on attitude information that indicates the attitude of the work machine; a photographing range control unit that controls a photographing range of the camera in accordance with the position calculated by the position calculation unit; An information processing device comprising:

2. The information processing apparatus according to claim 1 , further comprising a camera selection unit that selects one of a plurality of cameras installed at a plurality of different locations as the camera, in accordance with the position calculated by the position calculation unit.

3. The information processing apparatus according to claim 1 , further comprising a display unit that displays an image captured by the camera.

4. The information processing device according to claim 1 , further comprising an output unit that outputs the image captured by the camera to an automatic driving system that controls the work machine so as to transport the suspended load along a predetermined route.

5. a position calculation step of calculating the position of a load suspended by a work machine based on attitude information indicating the attitude of the work machine; a photographing range control step of controlling a photographing range of the camera in accordance with the position calculated in the position calculation step; An information processing method comprising:

Images