Crane-mounted imaging system and program

Abstract

To provide a photographing system and a program for a crane capable of photographing the crane or / and a periphery of the crane while inhibiting an article that is not desired to be photographed from entering an image.SOLUTION: A photographing system for a crane includes: a movable body (10) having a photographing unit (21) for photographing the crane or / and a periphery of the crane; a storage unit (31) for storing first information indicating a target that is permitted to be displayed or is not permitted to be displayed; and an image creation unit (32) for creating a display image in which the target that is not permitted to be displayed is excluded from photographing data of the photographing unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] ,

[0006] , , , ,

[0005] , , , , , ,

[0001] The present invention relates to a crane imaging system and a program.

Background Art

[0002] Patent Document 1 discloses a training explanation system that enables trainees to grasp the overall situation of a site by photographing the site where a crane is working from above.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When photographing a crane from above as in Patent Document 1, the surrounding area of the site gets into the video. Therefore, there is a problem that if there is something that is not desired to be photographed for privacy protection or the like in the surrounding area of the site, it will get into the video.

[0005] An object of the present invention is to provide a crane imaging system and a program that can photograph a crane, the periphery of the crane, or both while suppressing the inclusion of objects that are not desired to be photographed in the video.

Means for Solving the Problems

[0006] A crane imaging system according to one aspect of the present invention includes a moving body having an imaging unit that images a crane, the periphery of the crane, or both, where the moving body is an aircraft capable of flight, Furthermore, a storage unit that stores first information indicating an object of display permission or display non-permission, An image creation unit that creates a display image from which the items that are not permitted to be displayed have been excluded from the imaging data of the aforementioned imaging unit, A distance detection sensor that detects the distance to the object to be imaged, Equipped with, The aforementioned imaging unit photographs the part of the crane to be inspected, The aforementioned imaging unit photographs the inspection target from multiple directions. death, The image creation unit uses the detection results from the distance detection sensor to exclude items that are not permitted to be displayed from the captured data. do.

[0009] One of the present inventions is a photographic system for cranes, A storage unit that stores first information indicating whether display is permitted or not, An image creation unit creates a display image from image data obtained by an imaging unit that photographs the inspection target part of a crane, and by an aircraft capable of flying while the imaging unit photographs the inspection target part from multiple directions, capturing images of the crane, its surroundings, or both, and excluding items that are not permitted to be displayed. A distance detection sensor for detecting the distance from the imaging unit to the object to be imaged, Equipped with 、 The image creation unit uses the detection results of the distance detection sensor to exclude items that are not permitted to be displayed from the image data. ru.

[0012] A program according to one of the present invention is: Computers, A storage unit that stores first information indicating whether display is permitted or not, A flyable aircraft takes photographs, and the image data obtained by photographing the inspection target part of the crane from multiple directions is used. Using the detection result of the distance detection sensor that detects the distance from the imaging unit to the object being imaged, Image creation unit that creates display images from which the items that are not permitted to be displayed have been excluded. To make it function as such. [Effects of the Invention]

[0014] According to the present invention, it is possible to photograph the crane, its surroundings, or both, while suppressing the inclusion of unwanted elements in the video. [Brief explanation of the drawing]

[0015] [Figure 1]It is a block diagram showing a crane imaging system according to Embodiment 1 of the present invention. [Figure 2] It is a diagram showing an example of a site where a crane is arranged. [Figure 3] It is a diagram for explaining the operating principle of the distance detection unit. [Figure 4] It is a diagram (A) for explaining the imaging process of the crane by the moving body and an enlarged view (B) of the moving body. [Figure 5] It is a diagram showing the visual field range of the imaging unit and the visual field range of the distance detection unit. [Figure 6] It is a flowchart showing an example of the image creation process executed by the image creation unit. [Figure 7] It is a diagram showing an example of the mask process using the detected distance information. [Figure 8] It is a block diagram showing a crane imaging system according to Embodiment 2 of the present invention. [Figure 9] It is a diagram for explaining an example of the imaging process by the crane imaging system according to Embodiment 2. [Figure 10] It is a block diagram showing a crane imaging system according to Embodiment 3 of the present invention.

Mode for Carrying Out the Invention

[0016] Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings.

[0017] (Embodiment 1) FIG. 1 is a block diagram showing a crane imaging system according to Embodiment 1 of the present invention. FIG. 2 is a diagram showing an example of a site where a crane is arranged. The crane imaging system of the present embodiment is a system that performs imaging on a part or all of the crane T and diagnoses the crane T based on the captured image. Hereinafter, an example in which the boom B is the diagnosis target will be described, but the diagnosis target may be extended to various parts such as the jib, wire, cab, slewing frame, traveling body, each connecting part, various electrical components, or the entire crane T.

[0018] As shown in Figure 2, the area surrounding the site where the crane T is positioned contains objects that are not to be captured in the images (such as buildings N1, residential areas N2 and N3, and roads used by people). Therefore, the crane imaging system of this embodiment is configured to allow images to be displayed, to disallow images, or both, and imaging is performed so that only images that are permitted to be displayed are included in the captured images.

[0019] The crane photography system according to Embodiment 1 comprises a mobile body 10 and a computer 80, as shown in Figure 1. The mobile body 10 is a so-called drone that can fly in the air and perform movements in all directions, such as up, down, forward, backward, left, and right, as well as forward and reverse rotation. The mobile body 10 comprises a plurality of propellers 11, a drive unit 12 that drives the plurality of propellers 11, a positioning unit 13 that performs positioning, and a movement control unit 14 that controls the drive unit 12 to move and rotate the mobile body 10.

[0020] The positioning unit 13 measures the position and orientation of the moving object 10 by performing positioning using, for example, GNSS (Global Navigation Satellite System), beacons, or both, and detecting direction using a gyro sensor or the like. The position and orientation information measured by the positioning unit 13 is supplied to the movement control unit 14.

[0021] The mobile unit 10 further includes an imaging unit 21 that photographs the crane T, the area around the crane T, or both; a distance detection unit 25 that can detect the distance of each part within a two-dimensional field of view; a setting information storage unit 31 that stores setting information (corresponding to first information) indicating objects that are permitted to be displayed, objects that are not permitted to be displayed, or both; an image creation unit 32 that creates a composite image by removing the parts that are not permitted to be displayed from the imaging data obtained by imaging by the imaging unit 21; an image storage unit 33 that stores the created composite image; a data processing unit 34 that discards the original imaging data of the composite image; a communication unit 36 ​​that can transmit the image from the image storage unit 33 to an external computer 80 or the like; and a control unit 38 that performs overall control of movement and imaging. The communication unit 36 ​​may also be used to receive surveying information of the crane T and its movable elements (boom B, jib, etc.) from a total station.

[0022] The imaging unit 21 includes an optical system and an image sensor for capturing images within a predetermined field of view. The imaging unit 21 is, for example, a monocular camera.

[0023] Figure 3 shows the operating principle of the distance detection unit. The distance detection unit 25 detects the distance to objects in each detection area of ​​the field of view W1, which has a horizontal field of view angle α and a vertical field of view angle β, centered on the sensor origin O1. For example, a Time of Flight (TOF) scanning sensor using an infrared laser can be applied as the distance detection unit 25. By scanning each detection area of ​​the field of view W1 with the scanning sensor, the distance to objects in each part of the field of view W1 can be detected. Note that the distance detection unit 25 is not limited to the above method; various configurations, such as a stereo camera, may be applied as long as they can detect the distance to objects in each part of a predetermined field of view.

[0024] Figure 4 shows a diagram (A) illustrating the process of photographing a crane by a mobile unit, and an enlarged view of the mobile unit (B). Figure 5 shows the field of view of the distance detection unit and the field of view of the photography unit. Here, we assume a scenario in which the mobile unit 10 photographs the boom B from a position at a predetermined distance of X1, Y1, and Z1 in the X, Y, and Z directions from the origin S0 of the crane T. In this case, as shown in Figure 5, a part of the boom B becomes the target of photography by the photography unit 21 and the target of detection by the distance detection unit 25.

[0025] As shown in Figure 5, the field of view W1 of the distance detection unit 25 is set to include the field of view W2 of the imaging unit 21. Including the field of view means that when the object to be photographed and the object to be distance detected are placed on the same plane, the area on the plane photographed by the imaging unit 21 is included in the area on the plane where the distance is detected by the distance detection unit 25. The field of view W1 and W2 can also be referred to as the angle of view.

[0026] The setting information storage unit 31 is a storage device that stores setting information, and for which display permission is granted, it stores position information representing the position of each part of the crane T. This position information is position information in which the position of each part of the crane T is expressed in relative coordinates. For movable elements such as the boom B or jib, one relative coordinate is set for each movable element, and the setting information storage unit 31 may store position information using the relative coordinate for each movable element. Furthermore, the position information may be represented by 3D model data, or by polygon mesh data that shows the relative position of each part surface. Once the stopping position and orientation of the crane T are determined, and the orientation of each movable element is determined, the absolute position of each part of the crane T and each part of each movable element can be determined from the above relative position information. The stopping position, orientation of the crane T and the orientation of each movable element can be determined, for example, by measuring the positions of multiple points on the crane T and multiple points on each movable element using surveying equipment such as a total station. The setting information storage unit 31 may also have a setting input function that allows setting information to be input from outside the device.

[0027] The configuration information storage unit 31 may also include location information representing the absolute position of the site E (see Figure 2) as a subject for display permission.

[0028] The central control unit 38, image creation unit 32, setting information storage unit 31, and data processing unit 34 are software modules implemented, for example, by the execution of a program by the CPU (Central Processing Unit) of a microcomputer. The microcomputer is equipped with a CPU, ROM (Read Only Memory), and RAM (Random Access Memory) that provides a working memory area, and is installed in a mobile device. The ROM stores a program that causes the microcomputer to function as the central control unit 38, image creation unit 32, setting information storage unit 31, and data processing unit 34.

[0029] The central control unit 38 holds information indicating multiple locations to be photographed set on the crane T, as well as control information indicating multiple shooting positions and shooting directions associated with the multiple locations to be photographed. The central control unit 38 outputs commands to the movement control unit 14 to move the moving body 10 to each shooting position and to rotate the moving body 10 so that it faces each shooting direction. Furthermore, after moving to each shooting position and rotating, the central control unit 38 outputs commands to the distance detection unit 25 and the shooting unit 21, respectively, for distance detection and shooting. For example, if the object to be inspected is boom B, and images taken from multiple directions from the bottom to the top of boom B are required, the central control unit 38 stores information indicating multiple locations to be photographed set on boom B, and multiple shooting positions and shooting directions associated with the multiple locations to be photographed. Figure 4(A) shows a location to be photographed P1, shooting positions (X1, Y1, Z1), and shooting direction Z, which is one example. If a seamless image is required from the bottom to the top of boom B, two adjacent shooting locations may be set to partially overlap.

[0030] The image creation unit 32 performs image creation processing to create a composite image from which the parts that are not permitted to be displayed have been removed from the captured data.

[0031] During the image creation process, the data processing unit 34 deletes the original imaging data of the composite image after the image creation unit 32 has created the composite image, in order to prevent imaging data that may contain parts that are not permitted to be displayed from being leaked to the outside.

[0032] <Image creation process> Next, we will explain the image creation process performed by the image creation unit. Figure 6 is a flowchart showing the steps of the image creation process.

[0033] The image creation process is performed with the crane T stationary in a predetermined position. When the image creation process starts, the image creation unit 32 first reads the position information of the crane T (relative position information, for example, 3D model information) stored in the setting information storage unit 31 (step S1), and obtains absolute position information of multiple points of the crane T received from the total station (step S2). From the information in step S2, the image creation unit 32 can determine the absolute position and orientation of the crane T, as well as the orientation of each movable element (boom B, etc.), and by combining this with the position information in step S1, it can determine the absolute position of each part of the crane T.

[0034] Meanwhile, the central control unit 38 similarly acquires the information from steps S1 and S2, and from this information obtains the absolute position and orientation of the crane T, the posture of each movable element, and the shooting position and shooting direction information expressed in absolute position. Furthermore, the central control unit 38 moves the mobile body 10 to the shooting position and rotates the mobile body 10 in the shooting direction. Then, the central control unit 38 first drives the distance detection unit 25 to detect the distance of objects present in each detection section of its field of view W1. The distance information of each detected detection section is sent to the image creation unit 32 (step S3).

[0035] When the image creation unit 32 acquires distance information for each detection area, it corrects the recognized position of which component of the crane T is located in which detection area of ​​the field of view W1 based on the distance information and the position information of the crane T stored in the setting information storage unit 31 (step S4). The process in step S4 corresponds to an example of the process of the "object detection unit that detects the object to be displayed" according to the present invention. Furthermore, the absolute position and orientation information of the mobile body 10, the absolute position and orientation information of the crane T, and the orientation information of each movable element can be used to calculate which part of the crane T is located in which area of ​​the field of view W1 of the distance detection unit 25. In addition, the distance between each component of the crane T and the mobile body 10 can be calculated. However, there are errors in the absolute position information of the mobile body 10 and the crane T. Therefore, there is a discrepancy between the calculation result and the actual detection result. In step S3, the image creation unit 32 compares the detection result and calculation result of the field of view W1 to determine the above-mentioned error and reflects the determined error in the position information, thereby accurately determining which component of the crane T is located in which detection area of ​​the field of view W1.

[0036] Next, the image creation unit 32 determines the area to be masked (step S5) such that, within the field of view W1 of the distance detection unit 25, an image is extracted from the detection area (white area in Figure 7) where a part of the crane T (for example, a component of the boom B) that is set as an object to be displayed is located, and the image is removed from the other detection areas (shaded area in Figure 7). Note that a margin may be included between the edge of the component of the crane T in the field of view W1 and the area to be masked. The area to be masked corresponds to the area in which an object that is not permitted to be displayed (for example, a building adjacent to the site) may be captured in the image when captured by the shooting unit 21.

[0037] In parallel with the processing in steps S4 and S5, the control unit 38 causes the imaging unit 21 to perform the imaging process. The image creation unit 32 then receives the imaging data from the imaging unit 21 (step S6). As mentioned above, the field of view W2 of the imaging unit 21 is included in the field of view W1 of the distance detection unit 25. Therefore, each pixel of the imaging data is included in either the section from which to extract the image determined in step S5 or the section from which to remove the image. The image creation unit 32 performs a masking process on the imaging data according to the determination result in step S5 and creates a masked composite image (step S7). The processing in step S7 corresponds to an example of the processing of the "masking processing unit" according to the present invention.

[0038] In steps S4 to S7, a display image is created in which the items that are not permitted to be displayed are excluded, based on the information indicating the items that are permitted to be displayed stored in the setting information storage unit 31 and the detection results of the distance detection unit 25.

[0039] Next, the image creation unit 32 stores a composite image of one of the areas being photographed in the image storage unit 33 (step S8). When storing in step S8, the image creation unit 32 may store the composite image in association with tag information that identifies the shooting position and shooting direction from which the composite image was taken. Once the composite image is created, the data processing unit 34 deletes the original imaging data of the composite image (step S9).

[0040] Once one composite image is created and stored, the image creation unit 32 determines whether the shooting of all preset target locations has been completed (step S10). If not, it returns to step S3 to process the next shooting location. The central control unit 38 moves and rotates the moving body 10 in the shooting position and direction, and repeats the process of causing the distance detection unit 25 and the shooting unit 21 to perform distance detection and image capture until shooting from all of the preset shooting locations is completed. Furthermore, in conjunction with these processes, the image creation unit 32 and the data processing unit 34 execute the processes in steps S3 to S8 for the image data captured at each shooting location. As a result, a composite image of all the target locations of boom B is stored in the image storage unit 33. Then, if the determination process in step S10 determines that the shooting of all target locations has been completed, the image creation unit 32 terminates the image creation process.

[0041] <Image display processing> As shown in Figure 1, the computer 80 includes a display unit 81 for displaying images, a functional module 82 for creating a three-dimensional image from multiple composite images stored in the image storage unit 33 of the mobile body 10, a diagnostic unit 83 for diagnosing the crane T based on the composite image supplied from the mobile body 10, and a communication unit 84 for communicating with the mobile body 10 wirelessly or via wired connection. The functional module 82 and the diagnostic unit 83 are software modules implemented, for example, by the CPU executing a program. The computer 80 includes a CPU, a storage device, and RAM. The storage device stores programs that implement the functional module 82 and the diagnostic unit 83.

[0042] The operator can use the computer 80 to read multiple composite images stored in the image storage unit 33 of the mobile unit 10 wirelessly or via a wired connection, and by inputting a display command, the composite image can be displayed on the display unit 81.

[0043] The operator can create a 3D image from multiple read composite images by operating the function module 82 of the computer 80, and display it via the display unit 81. The 3D image display function includes the ability to rotate, translate, enlarge, or reduce the 3D image in three dimensions. The function module 82 for creating 3D images has 3D model data of the inspection target of the crane T (e.g., boom B) stored in advance, and creates a 3D image in which the corresponding parts of the composite image are attached to the surface of each part of the 3D model from the tag information linked to the composite image and the 3D model data. The tag information indicates which part of the subject the composite image was photographed from, at which shooting position and from which shooting direction. Alternatively, instead of the function module 82 of the computer 80 creating the 3D image, the image creation unit 32 may create the 3D image on the moving body 10.

[0044] After loading multiple composite images, the operator can perform a diagnostic process on the crane T by activating the diagnostic unit 83 of the computer 80. The diagnostic unit 83 diagnoses the crane T using the composite images, 3D images, or both. For example, the diagnostic unit 83 is configured to compare the details, parts, and whole of the composite image or 3D model data with a reference image, diagnose any differences as abnormalities, and output the diagnostic results. Alternatively, the diagnostic unit 83 may be omitted, and an inspector may examine the crane T for abnormalities by looking at the composite images or 3D images.

[0045] As described above, the crane imaging system of Embodiment 1 includes a setting information storage unit 31 that stores information indicating objects that are permitted or not permitted to be displayed, and an image creation unit 32 that creates a composite image from which objects that are not permitted to be displayed have been excluded from the imaging data of the imaging unit 21 of the mobile body 10. Therefore, a crane T placed on site can be targeted, the imaging unit 21 of the mobile body 10 can be used to photograph the target, and a composite image can be created from the imaging data by excluding objects in the surrounding area that are not desired to be photographed. The created composite image can then be used for diagnosing the crane T, etc. When diagnosing the crane T by visual inspection by an inspector, it is necessary to lower the boom B, but even if there is no space to lower the boom B, the crane T can be diagnosed based on the composite image by using the crane imaging system of this embodiment.

[0046] Furthermore, according to the crane imaging system of Embodiment 1, the image creation unit 32 is provided on the moving body 10. Therefore, the possibility of images containing unwanted objects being intercepted by a third party can be greatly reduced.

[0047] Furthermore, according to the crane imaging system of Embodiment 1, the image creation unit 32 detects the object to be displayed (for example, step S4 in Figure 6) and removes parts other than the object to be displayed from the captured image (for example, steps S5 and S7 in Figure 6). Therefore, even if, for example, an object that is not to be displayed is included in the imaging data between the constituent materials of the boom B, a composite image can be easily created by leaving the necessary parts such as the constituent materials of the boom B and excluding the parts that are not to be displayed.

[0048] Furthermore, according to the crane imaging system of Embodiment 1, once a composite image is created from which the object that is not permitted to be displayed has been excluded, the data processing unit 34 deletes the original imaging data. Therefore, the possibility of images containing unwanted images being intercepted by a third party can be greatly reduced.

[0049] Furthermore, the crane imaging system of Embodiment 1 includes a distance detection unit 25 for detecting the distance to the object to be photographed, and the image creation unit 32 further uses the detection results of the distance detection unit 25 to exclude objects that are not permitted to be displayed from the captured image. Therefore, the image creation unit 32 can more accurately distinguish between objects that are permitted to be displayed and objects that are not permitted to be displayed.

[0050] Furthermore, according to the crane imaging system of Embodiment 1, the field of view W1 of the distance detection unit 25 includes the field of view W2 of the imaging unit 21. With this configuration, the distance of objects captured in the entire range photographed by the imaging unit 21 can be detected, and masking can be performed by simple information processing based on the distance information. Also, since the field of view W1 of the distance detection unit 25 is larger than the field of view W2 of the imaging unit 21, the distance of everything captured in the field of view W2 can be detected. Therefore, there are no parts remaining in the captured image where it is impossible to determine whether an object is permitted to be displayed or not due to the inability to detect the distance.

[0051] Furthermore, according to the crane imaging system of Embodiment 1, the information of the crane T (position information of each part) is included as the subject of display permission, and the image creation unit 32 creates a composite image from which the crane T is extracted. With this configuration, a composite image suitable for diagnosing whether there are any abnormalities in each part of the crane T can be obtained.

[0052] Furthermore, the crane imaging system of Embodiment 1 can obtain a three-dimensional image of the crane T. Therefore, inspectors can easily grasp the overall image of the crane T or boom B from the three-dimensional image, and can easily determine which part of the whole the detailed image of each part corresponds to.

[0053] Furthermore, according to the crane imaging system of Embodiment 1, the diagnostic unit 83 diagnoses the crane T based on a composite image or a three-dimensional image, so that the diagnostic results of the crane T can be obtained without requiring an inspector.

[0054] In Embodiment 1, the image creation unit 32 was configured to determine the range in which the crane T was captured using the detection results of the distance detection unit 25. However, the image creation unit may determine the range in which the crane T was captured in the captured data from the absolute position information of each part of the crane T and the absolute position and orientation information of the moving body 10, without using the distance detection results. Alternatively, the image creation unit 32 may be configured to determine the range in which the crane T was captured in the captured data by performing image recognition instead of distance detection. In that case, the image creation unit may perform image recognition by comparing the image with pre-stored three-dimensional model data of the crane T, or it may perform image recognition using a pre-stored recognition image pattern without using the three-dimensional model data. Furthermore, the image creation unit 32 may have a function to improve the accuracy of recognizing each part of the crane T through a learning function.

[0055] Furthermore, Embodiment 1 showed an example in which the location information (such as 3D model data) of the crane T, which is an object that is permitted to be displayed, is stored in the setting information storage unit 31. However, the setting information storage unit 31 may also store location information of objects that are not permitted to be displayed, such as buildings around the site. In this case, the image creation unit 32 may be configured to determine the range in the image data that includes objects that are not permitted to be displayed, based on the location information and distance detection results of the moving object 10, or based on the results of image recognition, and to mask this range.

[0056] Furthermore, Embodiment 1 showed an example in which information indicating the relative positions of each part of an object, such as 3D model data, is used as information indicating the object for which display is permitted. However, if the object for which display is permitted and the object for which display is not permitted are color-coded, color information may be used; if they can be distinguished by the reflection of light of a specific wavelength, that identification information may be used; and if they can be distinguished by temperature, temperature information may be used.

[0057] Furthermore, in Embodiment 1, an example was shown in which one of the following components is provided on the mobile body 10 and the other on the computer 80: a setting information storage unit 31 that stores setting information indicating the target of display permission, an image creation unit 32, a data processing unit 34 that deletes imaging data, and a function module 82 for creating a 3D image. However, each of these components may be provided on either the mobile body 10 or the computer 80, or on a management terminal separate from the computer 80, a server device, a computer provided on the crane, etc. As long as each component can cooperate and function through communication, etc., their arrangement is not limited. The management terminal may be a portable management terminal for site supervisors. Also, a display unit capable of displaying images may be provided on the mobile body. Moreover, the crane photography system according to the present invention may not include a configuration for displaying images (for example, the computer 80), but may be configured to perform only the creation of composite images or 3D images.

[0058] Furthermore, Embodiment 1 showed an example in which the crane imaging system was used to acquire images for the diagnosis of crane T. However, the crane imaging system according to the present invention may be used to perform various tasks or functions that can utilize images of crane T or the area around crane T, such as managing the operation of crane T, managing the load during transport, managing the transport route of the load, safety management around crane T, and recording the operating status of crane T. In this case, the information subject to display permission may include information indicating the load. In addition, information on the posture of crane T may be provided to the sequential measurement and image creation unit 32, and the composite image, with the range of non-display permission excluded, may be sent wirelessly to a computer 80 or the like in real time and displayed. In this case, the composite image may be sent to a management server via a communication network and viewed by an administrator at a location away from the site, or made viewable by a site supervisor via a tablet, or made viewable by the operator in the cab.

[0059] Furthermore, in Embodiment 1, the crane imaging system was described as having a base configuration of the mobile body 10 (body, propeller 11, drive unit 12, positioning unit 13, and movement control unit 14) and a configuration for imaging the crane T (imaging unit 21, distance detection unit 25, setting information storage unit 31, image creation unit 32, image storage unit 33, data processing unit 34, communication unit 36, and overall control unit 38). However, the crane imaging system may not have a base configuration of the mobile body 10 (body, propeller 11, drive unit 12, positioning unit 13, and movement control unit 14), and may be used by mounting the configuration for imaging the crane T on a general-purpose mobile body. In this case, if the overall control unit 38 is connected to the movement control unit 14 of the mobile body 10 and can send movement commands to the movement control unit 14, the operation of Embodiment 1 can be realized. Furthermore, the mobile unit 10 does not fly, but rather moves along rails or ropes provided near the crane T. Any method of movement is acceptable as long as it can move to a position where it can photograph the area of ​​the crane T that needs to be photographed.

[0060] (Embodiment 2) Figure 8 is a block diagram illustrating a crane imaging system according to Embodiment 2 of the present invention. In Embodiment 2, components similar to those in Embodiment 1 are denoted by the same reference numerals and detailed descriptions are omitted. Figure 9 shows an example of imaging performed by the crane imaging system of Embodiment 2. The crane imaging system of Embodiment 2 comprises a mobile body 10A and a computer 80. The mobile body 10A is a so-called drone and comprises a plurality of propellers 11, a plurality of drive units 12, a positioning unit 13, a movement control unit 14, an imaging unit 21, a setting information storage unit 31, an image storage unit 33, a communication unit 36, and a general control unit 38A.

[0061] The control unit 38A and the configuration information storage unit 31 are software modules implemented, for example, by the CPU of a microcomputer executing a program. The microcomputer comprises a CPU, ROM (Read Only Memory), and RAM (Random Access Memory) that provides a working memory area, and is installed in the mobile unit 10A. The ROM stores a program that causes the microcomputer to function as the control unit 38A and the configuration information storage unit 31.

[0062] The configuration information storage unit 31 stores information indicating what is permitted to be displayed, such as the location information of the crane T (e.g., 3D model data), the location information of the site E1 where the crane T is installed, and the location information of the site enclosure E2. The configuration information storage unit 31 may also store information indicating what is not permitted to be displayed, such as the location information of other buildings in the vicinity of the site.

[0063] The control unit 38A is pre-configured with information about the area to be photographed (the area to be photographed), which may be the crane T, the area around the crane T, or both. The control unit 38A calculates the shooting position and direction of the area to be photographed so that no unauthorized objects appear in the background when the area to be photographed is photographed. The control unit 38A then moves the mobile body 10A to the calculated shooting position and directs the shooting unit 21 in the calculated shooting direction to perform the photograph.

[0064] For example, as shown in Figure 9, when photographing a target location P2 set on boom B, the control unit 38A calculates multiple shooting positions Q1 to Q3 and shooting directions R1 to R3 where the background F1 to F3 is empty and the site is either the site area E1 or the enclosure E2. The control unit 38A then moves the mobile body 10A so that the shooting unit 21 is positioned at these shooting positions Q1 to Q3 and shooting directions R1 to R3, controls the orientation of the shooting unit 21, and causes the shooting unit 21 to perform the shooting process. The shooting unit 21 has a function to narrow the field of view and a function to change the shooting direction, and the control unit 38A may control the shooting direction and field of view of the shooting unit 21.

[0065] This imaging process yields images of the area being photographed from multiple directions, without including any objects that are not permitted to be displayed. Based on these images, the crane T can be diagnosed, and a 3D image of the object to be diagnosed can be created. While visually inspecting the crane T requires lowering the boom B, even in sites where there is no space to lower the boom B, the crane T can be photographed, and the crane T can be diagnosed based on the captured images.

[0066] The captured images can also be used for managing cargo during transport, managing the cargo transport route, safety management around the crane T, and recording the operating status of the crane T. In these cases, the central control unit 38A calculates the shooting position and direction such that the background of the area to be photographed is the site premises or enclosure. The central control unit 38A then moves the mobile body 10 so that the shooting unit 21 is positioned at the calculated shooting position and direction, controls the orientation of the shooting unit 21, and causes the shooting unit 21 to perform the shooting process. The captured images are then distributed in real time to a computer in the control room, a tablet of the site supervisor, a computer in the cab, etc., allowing for the above-mentioned management or recording to be performed.

[0067] As described above, the crane photography system of Embodiment 2 includes a setting information storage unit 31 that stores information indicating objects that are permitted to be displayed or objects that are not permitted to be displayed, and a control unit 38 that, based on this information, causes the crane T, its surroundings, or both to be photographed in a way that prevents objects that are not permitted to be displayed from being captured. Therefore, even if there are objects in the surroundings that are not to be photographed, it is possible to take photographs that capture the necessary parts without including those parts.

[0068] In Embodiment 2, the crane imaging system was described as comprising a base configuration of a mobile body 10 and a configuration for imaging the crane T (imaging unit 21, setting information storage unit 31, image storage unit 33, and overall control unit 38A). However, the crane imaging system may not have a base configuration of a mobile body 10, and may be used by mounting the configuration for imaging the crane T onto a general-purpose mobile body. In that case, the above-described operation can be achieved if the overall control unit 38A is connected to the mobile control unit 14 of the mobile body 10 so as to be able to input and output commands and responses.

[0069] (Embodiment 3) Figure 10 is a block diagram showing a crane imaging system according to Embodiment 3 of the present invention. In Embodiment 3, components similar to those in Embodiment 1 are denoted by the same reference numerals and detailed descriptions are omitted. The crane imaging system of Embodiment 3 comprises a mobile body 10B and a computer 80. The mobile body 10B is a so-called drone and comprises a propeller 11, a drive unit 12, a positioning unit 13, a movement control unit 14, an imaging unit 21B, an image storage unit 33, and a general control unit 38B.

[0070] The imaging unit 21B is configured to image an object using light other than visible light (such as infrared light) or electromagnetic waves, and includes an optical system that captures light other than visible light or electromagnetic waves and forms an image, and an imaging unit that receives the formed light or electromagnetic waves and converts them into an electrical signal. The imaging unit 21B may also have a light source that outputs light other than visible light or electromagnetic waves. With this configuration, when the imaging unit 21B approaches each part of the crane T, each part of the crane T is captured in the image, while content that should not be displayed, such as distant buildings, roads, or information that could identify individuals in those places, is not captured in the image.

[0071] The control unit 38B is pre-configured with information about the area to be photographed (the area to be photographed), which may be the crane T, the area around the crane T, or both. The control unit 38B moves the mobile body 10B to the vicinity of the area to be photographed, rotates the mobile body 10 so that the photographing unit 21B faces the area to be photographed, and then outputs a photographing command to the photographing unit 21B to perform the photographing.

[0072] As described above, the crane imaging system of Embodiment 3 includes an imaging unit 21B that takes images by receiving light other than visible light or electromagnetic waves. Therefore, even if there are objects in the vicinity of the site that are not to be photographed, those parts will not be captured in the image, and only the necessary parts will be photographed. This image can then be used to perform diagnostics of each part of the crane T, to manage its operation, or to record its operation.

[0073] In Embodiment 3, the crane imaging system was described as comprising a base configuration of a mobile body 10 and a configuration for imaging the crane T (imaging unit 21B, image storage unit 33, and overall control unit 38B). However, the crane imaging system may not have a base configuration of a mobile body 10, and may be used by mounting the configuration for imaging the crane T onto a general-purpose mobile body. In that case, the above-described operation can be realized if the overall control unit 38B is connected to the mobile control unit 14 of the mobile body 10 so that signals can be input and output.

[0074] The embodiments of the present invention have been described above. However, the present invention is not limited to the embodiments described above. For example, the moving body may be a combination of two or three of the embodiments 1 to 3. Furthermore, the type of crane that the crane imaging system according to the present invention targets is not particularly limited and may be a tower crane, wheel crane, truck crane, gantry crane, jib crane, etc.

[0075] Furthermore, in the above embodiment, an example was described in which the ROM of a microcomputer mounted on the mobile units 10 and 10A was used as the computer-readable medium for storing the program according to the present invention, but the invention is not limited to this. As the medium for storing the program, a storage device of another computer such as a management terminal and a server device may be used, or a portable recording medium such as an HDD (Hard Disk Drive), CD-ROM, or DVD disc may be used. In addition, a carrier wave may be used as the medium for providing the data of the program according to the present invention via a communication line. Furthermore, in the above embodiment, a configuration was shown in which the program according to the present invention is executed by a microcomputer mounted on the mobile units 10 and 10A, and the microcomputer functions as each software module according to the program. However, the program according to the present invention may be executed by a computer located other than the mobile unit, such as a computer equipped on a crane, a management terminal, or a server device, and the computer may function as each software module according to the program.

[0076] Furthermore, although the above embodiment shows an example where the mobile body 10 is a so-called drone, the mobile body may be a device that moves along rails or ropes provided near the crane, or any other method of movement that can move it to a position where it can photograph the area of ​​the crane that needs to be photographed.

[0077] 10 Mobile Units 11 propellers 12 Drive unit 13 Positioning Unit 14. Movement Control Unit 21 Photography Department 21B Non-Visible Light Imaging Section 25 Distance detection unit 31. Configuration information storage unit (storage unit) 32 Image Creation Section 33 Image storage unit 34 Data Processing Unit 36 Communications Department 38, 38B General Control Unit 38A General Control Unit (Control Unit) 80 Computers 81 Display section 82 Function Modules 83 Diagnostic Department 84 Communications Department T Crane B boom Site E, E1 E2 Enclosure F1~F3 Background P1, P2 Filmed locations R1~R3 ​​Shooting direction Q1-Q3 Shooting location W1 Distance detection unit field of view W2 field of view of the shooting unit

Claims

1. The system includes a mobile body having a camera unit that photographs the crane, the area around the crane, or both. The aforementioned mobile body is a flying aircraft, Furthermore, A storage unit that stores first information indicating whether display is permitted or not, An image creation unit that creates a display image from which the items that are not permitted to be displayed have been excluded from the imaging data of the aforementioned imaging unit, A distance detection sensor that detects the distance to the object to be imaged, Equipped with, The aforementioned imaging unit photographs the part of the crane to be inspected, The aforementioned imaging unit photographs the inspection target from multiple directions, The image creation unit is a crane shooting system that uses the detection results of the distance detection sensor to exclude objects that are not permitted to be displayed from the image data.

2. The image creation unit is provided on the moving body, The photographic system for a crane according to claim 1.

3. The aforementioned image creation unit, A target detection unit detects the object to be displayed from the imaging data of the imaging unit based on the first information, A mask processing unit that removes portions other than the target detected by the target detection unit from the imaging data, A photographic system for a crane according to claim 1 or claim 2, including the following:

4. The system further includes a data processing unit that discards the original imaging data of the display image created by the image creation unit. A photographic system for a crane according to any one of claims 1 to 3.

5. The distance detection sensor is capable of detecting the distance of each part within a predetermined field of view. The field of view of the imaging unit is included within the field of view of the distance detection sensor. The distance detection sensor also detects areas outside the field of view of the imaging unit. The photographic system for a crane according to claim 1.

6. The aforementioned first information includes information about cranes as the subject of display permission. The image creation unit creates a display image in which the crane portion is extracted. A crane photography system according to any one of claims 1 to 5.

7. The crane photography system according to claim 6, wherein the displayed image is a three-dimensional image of the crane.

8. A storage unit that stores first information indicating whether display is permitted or not, An image creation unit creates a display image from image data obtained by an imaging unit that photographs the inspection target part of a crane, and by an aircraft capable of flying while the imaging unit photographs the inspection target part from multiple directions, capturing images of the crane, its surroundings, or both, and excluding items that are not permitted to be displayed. A distance detection sensor for detecting the distance from the imaging unit to the object to be imaged, Equipped with, The image creation unit is a crane shooting system that uses the detection results of the distance detection sensor to exclude objects that are not permitted to be displayed from the image data.

9. The system further includes a diagnostic unit that performs a diagnosis of the crane using the images captured by the aforementioned imaging unit. A photographic system for a crane according to any one of claims 1 to 8.

10. Computers, A storage unit that stores first information indicating whether display is permitted or not, An image creation unit creates a display image from which items that are not permitted to be displayed have been excluded, using the detection results of a distance detection sensor that detects the distance from the shooting unit to the object being photographed, based on image data taken by an airborne aircraft and the inspection target part of the crane from multiple directions. A program that makes it function as such.

Images