Photographing system and program for crane

The crane photography system effectively excludes unwanted surroundings from crane images using a mobile object and image processing, ensuring accurate and privacy-compliant image capture for diagnosis and management.

JP2024169573APending Publication Date: 2024-12-05SUMITOMO HEAVY IND CONSTR CRANES CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2024161138
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-09-18
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

When photographing a crane from above, surrounding areas that are not desired to be captured, such as for privacy reasons, are often included in the image.

Method used

A crane photography system equipped with a mobile object, storage unit, and image creation section that excludes unwanted objects from the image based on stored permission information, using distance detection and image processing to create a composite image.

Benefits of technology

Enables the capture of crane images without including undesired surroundings, facilitating accurate diagnosis and management of the crane's condition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024169573000001_ABST
    Figure 2024169573000001_ABST
Patent Text Reader

Abstract

To provide a photographing system and a program for a crane capable of photographing the crane or / and a periphery of the crane while inhibiting an article that is not desired to be photographed from entering an image.SOLUTION: A photographing system for a crane includes: a movable body (10) having a photographing unit (21) for photographing the crane or / and a periphery of the crane; a storage unit (31) for storing first information indicating a target that is permitted to be displayed or is not permitted to be displayed; and an image creation unit (32) for creating a display image in which the target that is not permitted to be displayed is excluded from photographing data of the photographing unit.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a crane photography system and a program. [Background technology]

[0002] Patent Document 1 discloses a tour explanation system that allows visitors to understand the entire site by photographing the site where a crane is working from above. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-215849 A Summary of the Invention [Problem to be solved by the invention]

[0004] As in Patent Document 1, when a crane is photographed from above, the surrounding area of ​​the site is included in the image. Therefore, if there are things around the site that you do not want to be photographed for privacy reasons, etc., there is a problem that these things will be included in the image.

[0005] The present invention aims to provide a crane photography system and program that can photograph a crane, the area around the crane, or both, while preventing unwanted objects from being included in the footage. [Means for solving the problem]

[0006] A crane photography system according to one aspect of the present invention comprises: A moving body having an imaging unit that images a crane, the surroundings of the crane, or both of them; a storage unit in which first information indicating an object that is permitted or not permitted to be displayed is stored; The image generating unit generates a display image from the image data captured by the imaging unit, the display image being obtained by excluding objects that are not permitted to be displayed.

[0007] A crane photography system according to one aspect of the present invention comprises: A moving body having an imaging unit that images a crane, the surroundings of the crane, or both of them; a storage unit in which first information indicating an object that is permitted or not permitted to be displayed is stored; a control unit that causes the photographing unit to capture an image so as not to capture an object that is not permitted to be displayed based on the first information; Equipped with.

[0008] A crane photography system according to one aspect of the present invention comprises: A moving body having an imaging unit that images a crane, the surroundings of the crane, or both of them, The photographing section photographs by receiving light other than visible light or electromagnetic waves.

[0009] A crane photography system according to one aspect of the present invention comprises: a storage unit in which first information indicating an object that is permitted or not permitted to be displayed is stored; an image creation unit that creates a display image from image data captured by an image capture unit that captures an image of the crane, the area around the crane, or both of the crane and the surrounding area, from which objects that are not permitted to be displayed are removed; Equipped with.

[0010] A crane photography system according to one aspect of the present invention comprises: a storage unit in which first information indicating an object that is permitted or not permitted to be displayed is stored; a control unit that causes an imaging unit to capture an image of the crane, the surroundings of the crane, or both of them so as not to capture an object that is not permitted to be displayed based on the first information; Equipped with.

[0011] A crane photography system according to one aspect of the present invention comprises: The vehicle can be mounted on a moving body and is provided with a photographing unit for photographing the crane, the surroundings of the crane, or both; The photographing section photographs by receiving light other than visible light or electromagnetic waves.

[0012] A program according to one aspect of the present invention comprises: Computer, a storage unit in which first information indicating an object that is permitted or not permitted to be displayed is stored; and an image creation unit that creates a display image from imaging data including the crane, the vicinity of the crane, or both, excluding objects that are not permitted to be displayed; Function as.

[0013] A program according to one aspect of the present invention comprises: Computer, a storage unit in which first information indicating an object that is permitted or not permitted to be displayed is stored; and a control unit that causes an imaging unit to capture an image of the crane, the vicinity of the crane, or both of them so as not to capture an object that is not permitted to be displayed based on the first information; Function as. Effect of the Invention

[0014] According to the present invention, it is possible to obtain an effect of being able to capture an image of a crane, the surroundings of the crane, or both, while preventing unwanted objects from being captured in the image. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram showing a crane imaging system according to a first embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram showing an example of a site where a crane is placed. [Diagram 3] 4A to 4C are diagrams illustrating the operation principle of a distance detection unit. [Figure 4] FIG. 1A is a diagram for explaining the imaging process of a crane by a moving object, and FIG. [Figure 6] 4 is a diagram showing the field of view of an imaging unit and the field of view of a distance detection unit; FIG. [Diagram 5] 11 is a flowchart showing an example of an image creation process executed by an image creation unit. [Figure 7] FIG. 13 is a diagram illustrating an example of a mask process using detected distance information. [Figure 8]FIG. 11 is a block diagram showing a crane imaging system according to a second embodiment of the present invention. [Figure 9] 13A to 13C are diagrams illustrating an example of an imaging process performed by the crane imaging system of the second embodiment. [Figure 10] FIG. 11 is a block diagram showing a crane imaging system according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0017] (Embodiment 1) Fig. 1 is a block diagram showing a crane photography system according to a first embodiment of the present invention. Fig. 2 is a diagram showing an example of a site where a crane is installed. The crane photography system according to this embodiment is a system that photographs a part or all of a crane T and diagnoses the crane T based on the photographed images. In the following, an example in which a boom B is the diagnosis target will be described, but the diagnosis target may be expanded to various parts such as a jib, wire, cab, rotating frame, traveling body, each connecting part, various electrical components, or the entire crane T.

[0018] 2, around the site where the crane T is placed, there are things that do not want to appear in the captured image (such as a building N1, residential areas N2 and N3, roads on which people pass, etc.). Therefore, in the crane photography system of this embodiment, objects that are permitted to be displayed, objects that are not permitted to be displayed, or both are set, and photography is performed so that only objects that are permitted to be displayed are included in the captured image.

[0019] The crane photography system according to the first embodiment includes a moving body 10 and a computer 80, as shown in Fig. 1. The moving body 10 is, for example, a so-called drone that can fly in the air and move up and down, forward and backward, left and right, and turn forward and backward. The moving body 10 includes multiple propellers 11, a drive unit 12 that drives the multiple propellers 11, a positioning unit 13 that performs positioning, and a movement control unit 14 that controls the drive unit 12 to move and turn the moving body 10.

[0020] The positioning unit 13 measures the position and orientation of the mobile object 10 by, for example, performing positioning using a Global Navigation Satellite System (GNSS), a beacon, or both, and detecting the orientation using a gyro sensor or the like. The information on the position and orientation measured by the positioning unit 13 is supplied to the mobile control unit 14.

[0021] The moving body 10 further includes an imaging unit 21 that captures the crane T, the periphery of the crane T, or both of them, a distance detection unit 25 that can detect the distance of each part in the two-dimensional field of view, a setting information storage unit 31 that stores setting information (corresponding to first information) indicating an object permitted to be displayed, an object not permitted to be displayed, or both of them, an image creation unit 32 that creates a composite image in which the image data obtained by the imaging unit 21 has been removed from the image data that is not permitted to be displayed, an image storage unit 33 that stores the created composite image, a data processing unit 34 that discards the original image data of the composite image, a communication unit 36 ​​that can transmit the image in the image storage unit 33 to an external computer 80, etc., and an overall control unit 38 that performs overall control of movement and imaging. The communication unit 36 ​​may also be used to receive survey information of the crane T and its movable elements (boom B, jib, etc.) from a total station.

[0022] The photographing unit 21 includes an optical system and an image sensor for photographing an image within a predetermined field of view. The photographing unit 21 is, for example, a monocular camera.

[0023] FIG. 3 is a diagram showing the operation principle of the distance detection unit. The distance detection unit 25 detects the distance to an object present in each detection section of the visual field range W1 having a horizontal angle of view α and a vertical angle of view β centered on the sensor origin O1. For example, a TOF (Time of Flight) scanning sensor using an infrared laser can be applied as the distance detection unit 25. By scanning each detection section of the visual field range W1 with the scanning sensor, the distance to an object present in each part of the visual field range W1 can be detected. Note that the distance detection unit 25 is not limited to the above type, and various configurations such as a stereo camera can be applied as long as the distance to an object present in each part of a predetermined visual field range can be detected.

[0024] Fig. 4 is a diagram (A) explaining the photographing process of the crane by the moving body, and an enlarged view (B) of the moving body. Fig. 5 is a diagram showing the field of view of the distance detection unit and the field of view of the photographing unit. Here, a scene is assumed in which the moving body 10 photographs the boom B from a position that is a predetermined length X1, Y1, Z1 away from the origin S0 of the crane T in the three directions X, Y, and Z, respectively. In this case, as shown in Fig. 5, a part of the boom B becomes the subject of photographing by the photographing unit 21 and also becomes the subject of detection by the distance detection unit 25.

[0025] 5, field of view range W1 of distance detection unit 25 is set so as to include field of view range W2 of imaging unit 21. Including the field of view range means that when a subject to be photographed and a subject to distance detection are placed on the same plane, the range on the plane photographed by imaging unit 21 is included in the range on the plane in which the distance is detected by distance detection unit 25. Field of view ranges W1 and W2 may also be referred to as angles of view.

[0026] The setting information storage unit 31 is a storage device in which setting information is stored, and position information representing the position of each part of the crane T, for example, is stored as a display permission target. This position information is position information in which the position of each part of the crane T is expressed by relative coordinates. For movable elements such as the boom B or the jib, one relative coordinate is set for each movable element, and the setting information storage unit 31 may store position information using the relative coordinates for each movable element. In addition, the position information may be represented by three-dimensional model data, or may be represented by polygon mesh data indicating the relative position of the surface of each part. When the stopping position and orientation of the crane T are determined and the posture of each movable element is determined, the absolute positions of each part of the crane T and each part of each movable element can be obtained from the above-mentioned relative position information. The stopping position and orientation of the crane T and the posture of each movable element can be obtained by measuring the positions of multiple points of the crane T and multiple points of each movable element using a surveying instrument such as a total station. The function as the setting information storage unit 31 may have a setting input function for inputting setting information from outside the device.

[0027] The setting information storage unit 31 may further include location information indicating the absolute location of the site E (see FIG. 2) as an object of display permission.

[0028] The overall control unit 38, the image creation unit 32, the setting information storage unit 31, and the data processing unit 34 are software modules that are realized, for example, by a central processing unit (CPU) of a microcomputer executing a program. The microcomputer includes a CPU, a read only memory (ROM), and a random access memory (RAM) that provides a working memory area, and is installed in a mobile object. The ROM stores programs that cause the microcomputer to function as the overall control unit 38, the image creation unit 32, the setting information storage unit 31, and the data processing unit 34.

[0029] The general control unit 38 has information indicating a plurality of locations to be photographed set on the crane T, and control information indicating a plurality of photographing positions and photographing directions associated with the plurality of locations to be photographed. The general control unit 38 outputs commands to the movement control unit 14 to move each photographing position and the moving body 10 and rotate the moving body 10 so that it faces each photographing direction. Furthermore, the general control unit 38 outputs a distance detection command and a photographing command to the distance detection unit 25 and the photographing unit 21, respectively, after moving to each photographing position and rotating. For example, if the inspection object is a boom B and images photographed from multiple directions from the bottom end to the top end of the boom B are required, the general control unit 38 stores information indicating a plurality of locations to be photographed set on the boom B and a plurality of photographing positions and photographing directions associated with the plurality of locations to be photographed. The photographed location P1, the photographing position (X1, Y1, Z1), and the photographing direction Z shown in FIG. 4(A) are one example. If an uninterrupted image is required from the bottom end to the top end of the boom B, two adjacent shooting locations may be set so that they partially overlap each other.

[0030] The image creation unit 32 executes image creation processing to create a composite image in which the portions not permitted to be displayed have been removed from the captured image data.

[0031] During the image creation process, the data processing unit 34 deletes the original imaging data of the composite image after the image creation unit 32 creates the composite image, so that the imaging data, which may include parts that are not permitted to be displayed, is not leaked to the outside.

[0032] <Image creation process> Next, the image creation process executed by the image creation unit will be described with reference to the flowchart of FIG 6.

[0033] The image creation process is executed with the crane T stationary in a predetermined posture. When the image creation process is started, the image creation unit 32 first reads the position information (relative position information, for example, 3D model information) of the crane T stored in the setting information storage unit 31 (step S1), and acquires information on the absolute positions of multiple points of the crane T received from the total station (step S2). From the information in step S2, the image creation unit 32 can determine the absolute position and orientation of the crane T, as well as the posture of each movable element (boom B, etc.), and by combining the position information in step S1, the absolute position of each part of the crane T can be determined.

[0034] Meanwhile, the overall control unit 38 similarly acquires the information of steps S1 and S2, and from this information, determines the absolute position and orientation of the crane T, the posture of each movable element, and information on the shooting position and shooting direction expressed by absolute positions. Furthermore, the overall control unit 38 moves the moving body 10 to the shooting position and rotates the moving body 10 in the shooting direction. Then, the overall control unit 38 first drives the distance detection unit 25 to detect the distance of objects present in each detection section in the field of view W1. Information on the detected distance of each detection section is sent to the image creation unit 32 (step S3).

[0035] When the image creation unit 32 acquires the distance information of each detection section, it corrects the recognized position of which component of the crane T is located in which detection section of the field of view W1 based on the distance information and the position information of the crane T stored in the setting information storage unit 31 (step S4). The processing of step S4 corresponds to an example of the processing of the "object detection unit that detects the object of display permission" according to the present invention. It is possible to calculate which part of the crane T is located in which section of the field of view W1 of the distance detection unit 25 from the information on the absolute position and orientation of the moving body 10, the information on the absolute position and orientation of the crane T, and the information on the posture of each movable element. Furthermore, it is possible to calculate the distance between each component of the crane T and the moving body 10. However, the information on the absolute positions of the moving body 10 and the crane T includes an error. For this reason, a deviation occurs between the calculation result and the actual detection result. In step S3, the image creation unit 32 compares the detection result of the field of view W1 with the calculation result to determine the deviation due to the above-mentioned error, and by reflecting the determined deviation in the position information, it is possible to accurately determine which component part of the crane T is located in which detection section of the field of view W1.

[0036] Next, the image creation unit 32 determines a mask processing area (step S5) so that an image is extracted for a detection area (white part in FIG. 7) in which a part of the crane T set as an object permitted to be displayed (e.g., a component of the boom B) is located within the field of view W1 of the distance detection unit 25, and an image is removed for other detection areas (shaded part in FIG. 7). Note that a margin may be included between the edge of the component of the crane T in the field of view W1 and the part to be masked. The area to be masked corresponds to a range in which an object not permitted to be displayed (e.g., a building adjacent to the work site) may appear in the captured image when photographed by the photographing unit 21.

[0037] In parallel with the processing of steps S4 and S5, the general control unit 38 causes the photographing unit 21 to execute a photographing process. Then, the image creation unit 32 receives the imaging data from the photographing unit 21 (step S6). As described above, the field of view W2 of the photographing unit 21 is included in the field of view W1 of the distance detection unit 25. Therefore, each pixel of the imaging data is included in either the section from which the image is extracted or the section from which the image is removed, as determined in step S5. The image creation unit 32 performs a mask process on the imaging data according to the determination result of step S5, and creates a masked composite image (step S7). The process of step S7 corresponds to an example of the process of the "mask processing unit" according to the present invention.

[0038] Through steps S4 to S7, based on the information indicating the objects permitted to be displayed stored in the setting information storage unit 31 and the detection result of the distance detection unit 25, a display image is created from which the objects not permitted to be displayed have been removed.

[0039] Next, the image creation unit 32 stores the composite image of one photographed location in the image storage unit 33 (step S8). When storing in step S8, the image creation unit 32 may store the composite image by linking it to tag information that can identify the photographing position and the photographing direction from which the composite image was photographed. Once the composite image is created, the data processing unit 34 deletes the original imaging data of the composite image (step S9).

[0040] When one composite image is created and stored, the image creation unit 32 judges whether or not the photographing of all the previously set photographing locations has been completed (step S10), and if not, returns the process to step S3 to perform processing at the next photographing position. The general control unit 38 repeats the process of moving and rotating the moving body 10 to the photographing position and photographing direction and making the distance detection unit 25 and the photographing unit 21 detect the distance and take an image until the photographing from all the previously set photographing positions is completed. Furthermore, in conjunction with these processes, the image creation unit 32 and the data processing unit 34 execute the processes of steps S3 to S8 for the image data photographed at each photographing position. As a result, the composite image of all the photographing locations of the boom B is stored in the image storage unit 33. Then, when it is judged in the judgment process of step S10 that the photographing of all the photographing locations has been completed, the image creation unit 32 ends the image creation process.

[0041] <Image display processing> 1, the computer 80 has a display unit 81 that displays an image, a functional module 82 that creates a three-dimensional image from a plurality of composite images stored in the image storage unit 33 of the mobile body 10, a diagnosis unit 83 that diagnoses the crane T based on the composite image supplied from the mobile body 10, and a communication unit 84 that communicates with the mobile body 10 wirelessly or via wire. The functional module 82 and the diagnosis unit 83 are software modules that are realized, for example, by a CPU executing a program. The computer 80 includes a CPU, a storage device, and a RAM. The storage device stores programs that realize the functional module 82 and the diagnosis unit 83.

[0042] An operator can use the computer 80 to read multiple composite images stored in the image storage unit 33 of the moving object 10 via wireless or wired connection, and input a display command to output and display the composite images on the display unit 81.

[0043] An operator can create a three-dimensional image from the multiple composite images read out by operating the functional module 82 of the computer 80, and display and output the image via the display unit 81. The display function of the three-dimensional image includes a function of rotating, translating, enlarging or reducing the three-dimensional image three-dimensionally and displaying it. The functional module 82 for creating a three-dimensional image stores three-dimensional model data of an inspection target (e.g., the boom B) of the crane T in advance, and creates a three-dimensional image in which a corresponding part of the composite image is attached to the surface of each part of the three-dimensional model based on tag information linked to the composite image and the three-dimensional model data. The tag information is information indicating which photographed part was photographed from which photographing position and photographing direction in which the composite image was taken. Note that the image creation unit 32 may create the three-dimensional image in the moving body 10, instead of the functional module 82 of the computer 80 creating the three-dimensional image.

[0044] After reading in the multiple composite images, the operator can execute a diagnostic process for the crane T by operating the diagnostic unit 83 of the computer 80. The diagnostic unit 83 diagnoses the crane T using the composite image, the three-dimensional image, or both. For example, the diagnostic unit 83 is configured to compare the details, parts, and the whole of the composite image or the three-dimensional model data with a reference image, diagnose any areas with differences as abnormal, and output the diagnostic results. Note that the diagnostic unit 83 may be omitted, and an inspector may inspect the crane T for abnormalities by looking at the composite image or the three-dimensional image.

[0045] As described above, the crane photography system of the first embodiment includes the setting information storage unit 31 in which information indicating objects permitted or prohibited from display is stored, and the image creation unit 32 that creates a composite image from the imaging data of the imaging unit 21 of the moving body 10 in which the objects prohibited from display are excluded. Therefore, the crane T placed at the site is photographed by the imaging unit 21 of the moving body 10, and a composite image can be created from the imaging data in which objects around the site that are not desired to be photographed are excluded. The created composite image can be used for diagnosing the crane T, etc. When diagnosing the crane T visually by an inspector, it is necessary to lower the boom B, but even if there is no space to lower the boom B, the crane T can be diagnosed based on the composite image by using the crane photography system of this embodiment.

[0046] Furthermore, according to the crane photography system of the first embodiment, the image creation unit 32 is provided in the moving body 10. Therefore, it is possible to greatly reduce the possibility that an image showing something that is not desired to be photographed will be intercepted by a third party.

[0047] Furthermore, according to the crane photography system of the first embodiment, the image creation unit 32 detects objects permitted to be displayed (for example, step S4 in FIG. 6) and removes parts other than the objects permitted to be displayed from the photographed image (for example, steps S5 and S7 in FIG. 6). Therefore, for example, even if an object not permitted to be displayed is included in the imaging data among the constituent materials of the boom B, a composite image can be created by simple processing in which necessary parts such as the constituent materials of the boom B are left and the parts not permitted to be displayed are removed.

[0048] Furthermore, according to the crane photography system of the first embodiment, once a composite image from which objects not permitted to be displayed have been removed is created, the data processing unit 34 deletes the original captured image data. This makes it possible to greatly reduce the possibility that a captured image showing objects that should not be captured will be intercepted by a third party.

[0049] Furthermore, the crane photography system of the first embodiment has a distance detection unit 25 for detecting the distance to the photography target, and the image creation unit 32 further uses the detection result of the distance detection unit 25 to remove targets that are not permitted to be displayed from the photographed image. Therefore, the image creation unit 32 can more accurately distinguish between targets that are permitted to be displayed and targets that are not permitted to be displayed.

[0050] Furthermore, according to the crane photography system of the first embodiment, the field of view W1 of the distance detection unit 25 includes the field of view W2 of the photography unit 21. With this configuration, the distance of an object captured in the entire range photographed by the photography unit 21 can be detected, and masking can be performed by simple information processing based on the distance information. Also, since the field of view W1 of the distance detection unit 25 is larger than the field of view W2 of the photography unit 21, the distance of everything photographed in the field of view W2 can be detected. Therefore, there is no case where a part cannot be detected in the photographed image and it is not possible to determine whether it is an object permitted to be displayed or not permitted to be displayed.

[0051] Furthermore, according to the crane photography system of the first embodiment, information about the crane T (positional information about each part) is included as an object permitted to be displayed, and the image creation unit 32 creates a composite image in which the crane T is extracted. With this configuration, a composite image suitable for diagnosing whether each part of the crane T has an abnormality can be obtained.

[0052] Furthermore, according to the crane photography system of the first embodiment, a three-dimensional image of the crane T can be obtained. Therefore, an inspector can easily grasp the overall image of the crane T or the boom B from the three-dimensional image, and can easily grasp which part of the whole the detailed image of each part corresponds to.

[0053] Furthermore, according to the crane photography system of the first embodiment, the diagnosis unit 83 diagnoses the crane T based on the composite image or the three-dimensional image, so that the diagnosis result of the crane T can be obtained without the need for an inspector.

[0054] In the first embodiment, the image creation unit 32 uses the detection result of the distance detection unit 25 to determine the range in which the crane T is captured. However, the image creation unit may determine the range in which the crane T is captured in the captured data from information on the absolute positions of each part of the crane T and information on the absolute position and orientation of the moving body 10 without using the distance detection result. The image creation unit 32 may also be configured to perform image recognition instead of distance detection to determine the range in which the crane T is captured in the captured data. In this case, the image creation unit may perform image recognition by comparing the image with three-dimensional model data of the crane T stored in advance, or may perform image recognition using an image pattern for recognition stored in advance without using three-dimensional model data. The image creation unit 32 may also have a function of improving the accuracy of recognizing each part of the crane T by a learning function.

[0055] Furthermore, in the first embodiment, an example has been shown in which the setting information storage unit 31 stores position information (three-dimensional model data, etc.) of the crane T, which is an object permitted to be displayed. However, the setting information storage unit 31 may store position information of an object not permitted to be displayed, for example, a building in the vicinity of a work site. In this case, the image creation unit 32 may be configured to determine a range including an object not permitted to be displayed in the captured image data from the detection results of the position information and distance of the mobile object 10 or from the result of image recognition, and to perform mask processing on this range.

[0056] In the first embodiment, information indicating the relative positions of each part of an object, such as three-dimensional model data, is used as information indicating an object permitted to be displayed. However, if the object permitted to be displayed and the object not permitted to be displayed are color-coded, color information may be used, if the object can be identified from the reflection of a specific wavelength of light, the identification information may be used, and if the object can be identified from the temperature, temperature information may be used.

[0057] In the first embodiment, the setting information storage unit 31 in which setting information indicating the object of display permission is stored, the image creation unit 32, the data processing unit 34 for deleting captured data, and the functional module 82 for creating a three-dimensional image are provided in the moving body 10, and the computer 80, but these elements may be provided in either the moving body 10 or the computer 80, or may be provided in a management terminal other than the computer 80, a server device, a computer provided in the crane, or the like. As long as the elements can function in cooperation with each other through communication or the like, their arrangement is not limited. The management terminal may be a management mobile terminal for site supervision. The moving body may be provided with a display unit capable of displaying images. The crane photography system according to the present invention may not include a configuration for displaying images (for example, the computer 80), and may be configured to perform up to creating a composite image or a three-dimensional image.

[0058] Furthermore, in the first embodiment, an example was shown in which the crane photography system was used for the purpose of acquiring diagnostic images of the crane T. However, the crane photography system according to the present invention may be used to perform various tasks or functions that can utilize photographed images of the crane T or the surroundings of the crane T, such as managing the operation of the crane T, managing loads during transportation, managing the transport route of the load, managing safety around the crane T, and recording the operating status of the crane T. In this case, the information that is permitted to be displayed may include information indicating the load. Also, the information on the posture of the crane T may be provided to the sequential measurement and image creation unit 32, and the composite image from which the range not permitted to be displayed is removed may be sent to the computer 80 or the like in real time via wireless and displayed. In this case, the composite image may be sent to a management server via a communication network, and may be viewed by a manager at a location away from the site, may be viewed by a supervisor at the site via a tablet, or may be viewed by a driver in the cab.

[0059] In the first embodiment, the crane photography system has been described as having the base configuration of the moving body 10 (the machine body, the propeller 11, the drive unit 12, the positioning unit 13, and the movement control unit 14) and the configuration for photographing the crane T (the photography unit 21, the distance detection unit 25, the setting information storage unit 31, the image creation unit 32, the image storage unit 33, the data processing unit 34, the communication unit 36, and the general control unit 38). However, the crane photography system may not have the base configuration of the moving body 10 (the machine body, the propeller 11, the drive unit 12, the positioning unit 13, and the movement control unit 14), and may be configured to be used by mounting the configuration for photographing the crane T on a general-purpose moving body. In this case, if the general control unit 38 is connected to the movement control unit 14 of the moving body 10 and can send a movement command to the movement control unit 14, the operation of the first embodiment can be realized. Furthermore, the moving body 10 does not fly, but may be a device that moves along rails or ropes prepared near the crane T, and any method of movement may be used as long as it can move to a position where it can photograph the part of the crane T that needs to be photographed.

[0060] (Embodiment 2) FIG. 8 is a block diagram showing a crane photography system according to a second embodiment of the present invention. In the second embodiment, the same components as those in the first embodiment are given the same reference numerals and detailed description is omitted. FIG. 9 shows a diagram for explaining an example of photography by the crane photography system of the second embodiment. The crane photography system of the second embodiment includes a moving body 10A and a computer 80. The moving body 10A is a so-called drone, and includes a plurality of propellers 11, a plurality of drive units 12, a positioning unit 13, a movement control unit 14, a photography unit 21, a setting information storage unit 31, an image storage unit 33, a communication unit 36, and a general control unit 38A.

[0061] The overall control unit 38A and the setting information storage unit 31 are software modules that are realized, for example, by a CPU of a microcomputer executing a program. The microcomputer includes a CPU, a ROM (Read Only Memory), and a RAM (Random Access Memory) that provides a working memory area, and is provided in the mobile object 10A. The ROM stores a program that causes the microcomputer to function as the overall control unit 38A and the setting information storage unit 31.

[0062] The setting information storage unit 31 stores, as information indicating objects permitted to be displayed, position information of the crane T (e.g., three-dimensional model data), position information of the site E1 where the crane T is to be installed, position information of the site enclosure E2, etc. The setting information storage unit 31 may also store, as information indicating objects not permitted to be displayed, position information of other buildings around the site, etc.

[0063] Information on the location to be photographed (location to be photographed) among the crane T, the periphery of the crane T, or both is set in advance in the general control unit 38A. When photographing the location to be photographed, the general control unit 38A calculates the photographing position and photographing direction of the location to be photographed so that an object that is not permitted to be displayed does not appear in the background. Then, the general control unit 38A moves the moving body 10A to the calculated photographing position, points the photographing unit 21 in the calculated photographing direction, and executes photographing.

[0064] For example, as shown in Fig. 9, when photographing a photographed location P2 set on a boom B, the general control unit 38A calculates a plurality of photographing positions Q1-Q3 and photographing directions R1-R3 where the backgrounds F1-F3 are the sky and the site E1 or the enclosure E2. Then, the general control unit 38A moves the moving body 10A so that the photographing unit 21 is disposed at these photographing positions Q1-Q3 and photographing directions R1-R3, controls the orientation of the photographing unit 21, and causes the photographing unit 21 to perform photographing processing. The photographing unit 21 has a function of narrowing the angle of view and a function of changing the photographing direction, and the general control unit 38A may control the photographing direction and angle of view of the photographing unit 21.

[0065] According to this type of photography processing, images are obtained in which the photographed location is viewed from multiple directions and do not include objects that are not permitted to be displayed, and the crane T can be diagnosed based on these images, or a 3D image of the object to be diagnosed can be created. In order for an inspector to visually diagnose the crane T, it is necessary to lower the boom B, but even at a site where there is no space to lower the boom B, the crane T can be photographed and diagnosed based on the photographed images.

[0066] The captured images can also be used for managing loads during transport, managing the transport route of the load, managing safety around the crane T, recording the operating status of the crane T, and the like. In these cases, the general control unit 38A calculates the shooting position and shooting direction in which the background of the location to be photographed is the site or enclosure. Then, the general control unit 38A moves the moving body 10 so that the shooting unit 21 is positioned at the calculated shooting position and shooting direction, controls the orientation of the shooting unit 21, and causes the shooting unit 21 to perform shooting processing. Then, the captured images can be distributed in real time to a computer in the control room, the tablet of the on-site supervisor, a computer in the cab, or the like, and the above-mentioned management or recording can be performed.

[0067] As described above, the crane photography system of the second embodiment includes the setting information storage unit 31 that stores information indicating objects permitted to be displayed or objects not permitted to be displayed, and the general control unit 38 that photographs the crane T, its surroundings, or both, based on this information, so that objects not permitted to be displayed are not captured in the image. Therefore, even if there are objects in the surrounding area that should not be photographed, the objects can be photographed without capturing those parts and capturing the necessary parts.

[0068] In the second embodiment, the crane photography system has been described as including a base configuration of the moving body 10 and a configuration for photographing the crane T (photographing unit 21, setting information storage unit 31, image storage unit 33, and general control unit 38A). However, the crane photography system may be configured not to include a base configuration of the moving body 10, but to be used by mounting a configuration for photographing the crane T on a general-purpose moving body. In that case, the general control unit 38A can be connected to the movement control unit 14 of the moving body 10 so as to be able to input and output commands and responses thereto, thereby realizing the above-mentioned operation.

[0069] (Embodiment 3) 10 is a block diagram showing a crane photography system according to a third embodiment of the present invention. In the third embodiment, the same components as those in the first embodiment are given the same reference numerals and detailed description thereof will be omitted. The crane photography system of the third embodiment includes a moving body 10B and a computer 80. The moving body 10B is a so-called drone, and includes a propeller 11, a drive unit 12, a positioning unit 13, a movement control unit 14, a photography unit 21B, an image storage unit 33, and a general control unit 38B.

[0070] The photographing unit 21B is configured to capture an image of a target using light other than visible light (such as infrared light) or electromagnetic waves, and has an optical system that captures light or electromagnetic waves other than visible light and forms an image, and an imaging unit that receives the formed light or electromagnetic waves and converts them into an electrical signal. The photographing unit 21B may have a light source that outputs light or electromagnetic waves other than visible light. With this configuration, when the photographing unit 21B approaches each part of the crane T, the various parts of the crane T are captured in the image, while content that should not be displayed, such as distant buildings, roads, or information that can identify the individuals present there, is not captured in the image.

[0071] Information on the location to be photographed (location to be photographed) among the crane T, the periphery of the crane T, or both is set in advance in the general control unit 38B. The general control unit 38B moves the moving body 10B to the vicinity of the location to be photographed, and after rotating the moving body 10B so that the photographing unit 21B faces the location to be photographed, outputs a photographing command to the photographing unit 21B to perform photographing.

[0072] As described above, the crane photography system of the third embodiment is provided with the photography unit 21B that receives light other than visible light or electromagnetic waves to photograph. Therefore, even if there are things around the site that you do not want to photograph, it is possible to photograph the necessary parts without capturing those parts. Then, with this image, it is possible to diagnose each part of the crane T, manage operation, record operation, and the like.

[0073] In the third embodiment, the crane photography system has been described as including a base configuration of the moving body 10 and a configuration for photographing the crane T (photographing unit 21B, image storage unit 33, and general control unit 38B). However, the crane photography system may be configured not to include a base configuration of the moving body 10, but to be used by mounting a configuration for photographing the crane T on a general-purpose moving body. In that case, the above-mentioned operation is realized when the general control unit 38B is connected to the movement control unit 14 of the moving body 10 so as to be able to input and output signals therebetween.

[0074] Each embodiment of the present invention has been described above. However, the present invention is not limited to the above-mentioned embodiments. For example, the moving body may be configured by combining two or three of the first to third embodiments. In addition, the type of crane that is the subject of photography by the crane photography system according to the present invention is not particularly limited, and may be, for example, a tower crane, a wheel crane, a truck crane, a gantry crane, a jib crane, or the like.

[0075] In the above embodiment, the ROM of the microcomputer mounted on the moving body 10, 10A is applied as a computer-readable medium for storing the program according to the present invention, but the present invention is not limited to this. As a medium for storing the program, a storage device of another computer such as a management terminal or a server device may be applied, or a portable recording medium such as a HDD (Hard Disk Drive), a CD-ROM, or a DVD disk may be applied. In addition, a carrier wave may be applied as a medium for providing data of the program according to the present invention via a communication line. Furthermore, in the above embodiment, the program according to the present invention is executed by a microcomputer mounted on the moving body 10, 10A, and the microcomputer functions as each software module by the program. However, the program according to the present invention may be executed by a computer arranged other than the moving body, such as a computer provided on a crane, a management terminal, or a server device, and the computer may function as each software module by the program.

[0076] In the above embodiment, the moving body 10 is a so-called drone, but the moving body may be a device that employs any moving method as long as it can move to a position where it can photograph the part of the crane that needs to be photographed, such as a device that moves along a rail or rope prepared near the crane. Other details shown in the embodiment may be modified as appropriate without departing from the spirit of the invention. [Explanation of symbols]

[0077] 10 Mobile 11 Propeller 12 Drive unit 13 Positioning unit 14 Movement control section 21 Photography Department 21B Non-visible light imaging section 25 Distance detection unit 31 Setting information storage section (storage section) 32 Image Creation Department 33 Image storage section 34 Data Processing Section 36 Communications Department 38, 38B General control unit 38A General control unit (control unit) 80 Computer 81 Display section 82 Functional Modules 83 Diagnostic Department 84 Communications Department T-Crane B Boom Site E, E1 E2 Enclosure F1~F3 Background P1, P2 Photographed area R1~R3 ​​Shooting direction Q1~Q3 Shooting positions W1 Field of view of distance detector W2 Field of view of the imaging unit

Claims

1. A moving body having an imaging unit that images a crane, the surroundings of the crane, or both of them; a storage unit in which first information indicating an object of display permission or display non-permission is stored; an image creation unit that creates a display image from the image data captured by the imaging unit by excluding objects that are not permitted to be displayed; A crane photography system comprising:

2. The image creation unit is provided on the moving body. The crane imaging system according to claim 1.

3. The image creation unit includes: an object detection unit that detects an object permitted to be displayed from the image data captured by the imaging unit based on the first information; a mask processing unit that removes from the imaging data portions other than the object detected by the object detection unit; The crane photography system according to claim 1 or 2, further comprising:

4. The display device further includes a data processing unit that discards the original captured image data of the display image created by the image creation unit. The crane photography system according to any one of claims 1 to 3.

5. the moving object further includes a distance detection unit for detecting a distance to an imaging target, the image creation unit uses a detection result of the distance detection unit to exclude from the imaging data an object that is not permitted to be displayed. The crane photography system according to any one of claims 1 to 4.

6. The distance detection unit is capable of detecting the distance of each part within a predetermined field of view, The field of view of the photographing unit is included in the field of view of the distance detection unit. The crane photography system according to claim 5.

7. The first information includes information about a crane as a subject of display permission, The image creation unit creates a display image in which a portion of the crane is extracted. The crane photography system according to any one of claims 1 to 6.

8. 8. The crane photography system according to claim 7, wherein the displayed image is a three-dimensional image of the crane.

9. A moving body having an imaging unit that images a crane, the surroundings of the crane, or both of them; a storage unit in which first information indicating an object of display permission or display non-permission is stored; a control unit that causes the photographing unit to photograph an object that is not permitted to be displayed based on the first information so as not to be photographed; A crane photography system comprising:

10. A moving body having an imaging unit that images a crane, the surroundings of the crane, or both of them, The photographing unit photographs by receiving light or electromagnetic waves other than visible light. Crane photography system.

11. a storage unit in which first information indicating an object of display permission or display non-permission is stored; an image creation unit that creates a display image from image data captured by an image capture unit that captures an image of the crane, the area around the crane, or both of the crane and the surrounding area, from which objects that are not permitted to be displayed are removed; A crane photography system comprising:

12. a storage unit in which first information indicating an object of display permission or display non-permission is stored; a control unit that causes an imaging unit to capture an image of the crane, the surroundings of the crane, or both of them so as not to capture an object that is not permitted to be displayed based on the first information; A crane photography system comprising:

13. The photography system for a crane according to claim 11 or 12, further comprising the photography unit that can be mounted on a moving body and is capable of photographing the crane, the surroundings of the crane, or both.

14. The vehicle can be mounted on a moving body and is provided with a photographing unit for photographing the crane, the surroundings of the crane, or both; The photographing unit photographs by receiving light or electromagnetic waves other than visible light. Crane photography system.

15. Further comprising a diagnosis unit that diagnoses the crane using the image captured by the imaging unit. The crane photography system according to any one of claims 1 to 14.

16. Computer, a storage unit in which first information indicating an object that is permitted or not permitted to be displayed is stored; and an image creation unit that creates a display image from imaging data including the crane, the vicinity of the crane, or both, excluding objects that are not permitted to be displayed; A program that functions as a

17. Computer, a storage unit in which first information indicating an object that is permitted or not permitted to be displayed is stored; and a control unit that causes an imaging unit to capture an image of the crane, the surroundings of the crane, or both of them so as not to capture an object that is not permitted to be displayed based on the first information; A program that functions as a

Citation Information

Patent Citations

  • Observation explanation system

    JP2017215849A