Image generation device, diagnosis apparatus using image generation device, data storage system, image generation method and image generation program

Abstract

To provide an image generation device capable of implementing information processing equal to or more than information, which can be visually obtained by a person, by putting a movable camera into a dangerous space such as a narrow and dark space, a diagnosis apparatus using the image generation device, a data storage system, an image generation method and an image generation program.SOLUTION: An image generation program 8 comprises: a contour setting part P1 which identifies a contour 10L of an observation target range of an observation target 10u on the basis of data which are detected while moving under the observation target; a component identification part P2 which identifies a component by extracting feature points indicated by the component on the basis of image data; a map allocation processing part P3 for allocating a map 10M consisting of an array of a plurality of sections 10S inside of the identified contour of the observation target range; a map setting processing part P4 by which the image data are allocated to the sections of the map and integrated to a bottom face image 100 of the observation target; and a component inspection processing part P8 for determining and inspecting an abnormal location of each component identified by the component identification part P2.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an image generating device capable of processing image data of difficult-to-identify locations such as the undersides of vehicles, buildings, and other man-made objects, a diagnostic device using an image generating device, a data storage system, an image generating method, and an image generating program. [Background technology]

[0002] When calculating an estimate for regular car inspections, etc., it is necessary to visually check the car body and each part to confirm whether maintenance or replacement is required. In particular, when checking the underside of the car body, the vehicle must be moved onto a jack lift or to an underground pit, and after ensuring safety, the mechanic must get under the car body and visually check each part to confirm whether maintenance or replacement is required. Similar inspection work is also required when checking for defects in the car and arranging parts. The same is true when assessing used car purchases and setting sales prices.

[0003] In recent years, not only gasoline engines, diesel engines, and hybrid vehicles, but also hydrogen engines, fuel cell vehicles, and other various parts are installed in each vehicle model, and the number of parts is increasing, and the structure is becoming more complex and diverse. In particular, for new vehicles that have been delivered only a short time ago and vehicles with low mileage, the number of inspection items is increasing, even though the probability of breakdown is relatively low, and the inspection workload of mechanics is increasing.

[0004] In addition, with improvements in the working environment, the working hours of mechanics have been reduced. Furthermore, with the decline in the working population, there are concerns about a shortage of mechanics. Similar issues exist not only in the maintenance of automobiles, but also in the inspection of facilities under floors, ceilings, or above the ceiling of buildings.

[0005] For example, Patent Document 1 (JP Patent Publication 2005-200844A) shows a self-propelled cart for working inside a tunnel, which has a work deck arranged at the top inside the road tunnel, a plurality of legs each provided on both sides of the width of the work deck, each with wheels at the bottom, and a drive unit for supplying power to the wheels, and which is arranged across both sides of the width of the road tunnel, and in an area other than the vehicle traffic area inside the road tunnel, so that vehicles can pass inside it, and work can be done without blocking vehicle traffic even when the road tunnel is in use. In addition, because it is self-propelled, there is no need for cables or the like for supplying power. In addition, a self-propelled cart for working inside a tunnel is shown, which improves workability because work is done on the work deck.

[0006] For example, the self-propelled camera for pets shown in Patent Document 2 (JP 2022-15161 A) is equipped with a housing that runs on an installation surface on multiple wheels, a camera housed in the housing, a chassis that supports the camera fixed to the housing so that it can rotate freely, a wheel drive source that drives the wheels, a camera rotation drive source that rotates the camera, suction cups that adhere to the installation surface to prevent the housing from tipping over, an opening / closing valve that opens and closes an air hole drilled in the suction cup, and an opening / closing mechanism that drives the opening and closing of the opening / closing valve, and has the potential to be applied as a technology for moving freely in narrow spaces such as under vehicles or under the floors of buildings and taking pictures. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2005-200844 A [Patent Document 2] Patent Publication No. 2022-15161 Summary of the Invention [Problem to be solved by the invention]

[0008] The self-propelled cart for working inside tunnels described in Patent Document 1 is effective for inspecting and working on the underside of high places such as the ceiling of a tunnel, but is not effective for inspecting the underside located at the top in narrow spaces such as under a vehicle, under the floor of a building, or in the attic (inside a suspended ceiling).

[0009] The self-propelled pet camera of Patent Document 2 is capable of photographing the underside of an object located above while moving in narrow spaces such as under a vehicle, under the floor of a building, or above the ceiling (inside a suspended ceiling), etc. However, the subject of the photograph is a moving pet, and the camera cannot be used as is to inspect the underside of a vehicle or building.

[0010] In view of the above circumstances, an object of the present invention is to provide an image generating device capable of generating a bottom image of a specified object, a diagnostic device using the image generating device, a data storage system, an image generating method, and an image generating program.

[0011] Another object of the present invention is to provide an image generating device that can generate an image of the underside of a specified object even in places where a person would need large-scale facilities such as a jack lift or underground pit to enter, or where a worker would have to risk crawling into a dark, narrow space, as well as a diagnostic device, a data storage system, an image generating method, and an image generating program that use the image generating device. [Means for solving the problem]

[0012] In order to solve at least one of the above problems, the present invention provides an image generating device that can acquire image data with high accuracy by identifying contours, such as underside images of the undercarriage of a vehicle or underside images of a room space such as a ceiling surface, as well as a diagnostic device, a data storage system, an image generating method, and an image generating program that use the same. That is, the present invention provides an image generating device that generates an underside image of an object to be observed, the image generating device comprising: a contour setting unit that identifies the contour of the observation range of the object to be observed based on image data captured by an imaging device moving below the object to set the area within the contour as a contour area; and an image output unit that outputs an image acquisition area within the contour area where image data has been acquired and an image non-acquired area where image data has not been acquired, and the image generating device generates an underside image from image data acquired within the contour area.

[0013] The contour setting unit identifies the contour of the observation range at an early stage when the execution of the image generation program is started. The contour setting unit can identify the contour of the observation range based on distance measurement data measured by a distance measurement unit of the imaging device. The contour setting unit can also identify the contour of the observation range from differences in color, brightness, and the like based on image data captured by the camera of the imaging device.

[0014] The image generating device can include a part feature extraction unit that extracts feature points indicated by parts that constitute the observation target and are present in the observation target range based on the image data within the contour area, and a part identification unit that identifies parts from the extracted feature points.

[0015] The part identification unit identifies parts on the captured image of the observation target, and can extract feature points indicated by the observation target part based on the image data, and identify the part from the feature points. The feature points indicated by the part are also called feature information, and can extract and identify characteristic elements on the image that are effective for identifying individual parts, such as the basic shape of each part common to various vehicles and the part dimensions that differ from vehicle to vehicle. For example, in the case of an exhaust pipe, the overall shape of the part can be determined from the pipe shape whose basic shape is a curved line from the image data, and the part dimensions can be read from the distance between the flange parts connected by bolts and nuts. In the case of a suspension, the suspension method can be identified from the image data, and each of its components, such as coil springs, shock absorbers, lower arms, upper arms, bushes, boots, stabilizers, axles, oil pans, catalysts, mufflers, silencers, bolts and nuts, can be identified.

[0016] The image generating device may also include a map allocation processing unit that allocates a map in which a plurality of sections are arranged within a contour area set by the contour setting unit, a map setting processing unit that sets the image data in each section of the map to generate an underside image of the object to be observed, and a part inspection processing unit that inspects at least one of the shape, color, size and position of each part identified by the part identification unit.

[0017] The map allocation processing unit allocates a map consisting of an array of a plurality of sections within the contour of the observation target range. The sections of the map may be a two-dimensional map arranged on a plane. The sections of the map may be, for example, rectangular squares, or may be equilateral triangles, parallelograms, regular hexagons, etc. The sections of the map may be a three-dimensional map arranged in a three-dimensional space. The sections of the map may be, for example, three-dimensional shapes such as cubes, equilateral triangular prisms, square prisms, regular hexagonal prisms, etc.

[0018] The map setting processing unit assigns the image data to each section of the map and integrates it into the bottom image of the observation target. The integrated bottom image can assign a two-dimensional image to the two-dimensional map and a three-dimensional image to the three-dimensional map. Also, the two-dimensional image can be image-processed and assigned to the three-dimensional map.

[0019] The part inspection processing unit inspects each of the identified parts, and based on image information such as the shape, dimensions, and color of each part, compares it with image samples of various fault conditions (part conditions) such as deformation, scratches, wear, cracks, breakage, deterioration, oil leakage, liquid leakage, burnout, etc., and if an image pattern matching any of the image samples is found, it can be determined that the part is abnormal.

[0020] The contour setting unit extracts, from the imaging device, the positions of the observation target that can be imaged within a certain distance range set for each observation target as characteristic points of the contour, and connects the characteristic points in a linear fashion to identify the contour of the observation target range.

[0021] When the characteristic points of the contour of the observation range are set, the contour setting unit can determine the contour of the observation range. The contour setting unit can identify the contour of the observation range of the observation target based on distance measurement data measured by a distance measurement unit mounted on the imaging device while the imaging device moves under the observation target. The contour setting unit determines the difference in distance measurement data between the inside and outside of the contour of the observation range of the observation target based on the distance measurement data from the distance measurement unit, and extracts it as a characteristic point of the contour. The characteristic points indicating the contour of the observation range are set as point-like positions where the difference in distance between long and short distances in the distance measurement data changes greatly, and a series of these point-like positions can be recognized as the contour of the observation range.

[0022] The contour setting unit can determine the contour of the observation target range based on the distance range input to the distance range input unit. The distance range input unit sets and inputs the distance range from the imaging device to the observation target. The distance range input unit can be provided in a data storage system. The distance range input unit can be the imaging device, a remote control for the imaging device, or an input field displayed on the display of the fault location identification terminal. The distance range input unit can input the shortest distance and the longest distance from the imaging device to the observation target. In this case, anything existing between the input shortest distance and longest distance is recognized as the observation target.

[0023] The contour setting unit can determine the difference between the inside and outside of the contour of the observation range on the basis of image data of a still image or video captured by the camera, and extract the difference as a feature point of the contour. The feature points indicating the contour of the observation range are taken as point-like positions where the brightness or color tone of the image data changes significantly, and a series of these point-like positions can be recognized as the contour of the observation range.

[0024] The present invention also provides a diagnostic device using the image generating device. That is, the present invention provides a diagnostic device using the image generation device, which includes a part information acquisition processing unit that acquires a standard image of the part identified by the part identification unit, a part comparison processing unit that compares the image of the part identified by the part identification unit with the standard image of the part acquired by the part information acquisition processing unit, and a judgment processing unit that judges the state of the part identified by the part identification unit from the comparison result in the part comparison processing unit.

[0025] Furthermore, when a bottom surface information acquisition processing unit that acquires a standard image of the bottom surface image of the observation target is provided, a defect state (a defect state that is an example of a part state) such as a change (displacement) in the mounting position of each part or deformation can be determined by comparing the bottom surface image created by the image generating device with the standard image of the bottom surface image acquired by the bottom surface information acquisition processing unit. Of course, a normal state (a normal state that is an example of a part state) in which defects such as a change (displacement) in the mounting position of each part or deformation do not occur, or a normal use state (a normal use state that is an example of a part state) in which such defects are not recognized, can be determined.

[0026] The diagnostic device may further include a part inspection processing unit that inspects each part identified by the part identification unit, and an image generation processing unit that generates a superimposed image of the underside image of the observation target integrated by the map setting processing unit and the part image to which the standard information of the part acquired by the part information acquisition processing unit is linked. The judgment processing unit can acquire and display or save image inspection data including the underside image of the observation target, comparison information of each part, and the result of the inspection.

[0027] The part information acquisition processing unit acquires standard information of the identified part, and the bottom information acquisition processing unit acquires standard information of the bottom image of the inspection target. The standard information of the part is information that serves as an inspection standard (comparison target) for the part, and can be information such as the numerical value, shape, and color of each part of the part in a factory shipment state (new). The standard information of the bottom image can be a bottom image of the inspection target in a new state, or a bottom image formed by combining common parts of bottom images created for the same type of inspection target. These standard information can be acquired, for example, from a file stored in the failure location identification terminal, or from another device connected via a network. Specifically, the standard information can be acquired from a recording means in which the standard information is recorded in association with information identifying the inspection target, and can be acquired by searching using information such as a model number identifying the inspection target as a search key. If the standard information cannot be obtained, the deterioration state such as damage, cracks, and wear can be determined from the captured image.

[0028] The image generation processing unit generates a superimposed image of the integrated bottom image of the observation target and a standard image of the part as standard information of the part. At least one of the generated superimposed images is made semi-transparent, and the superimposed images can be compared to visually check for misalignment or deformation of each part. It is desirable that the orientation and scale ratio of the standard image of each part as the standard information of the part is adjusted so that the orientation and outer shape of the part match the corresponding part in the bottom image.

[0029] The component comparison processing unit compares the image of the identified component with the acquired standard image of the component and determines the different parts. The component comparison processing unit can distinguish the difference between the two image data. The component comparison processing unit can determine whether the placement position of the component in the bottom image matches the placement information of the acquired correctly placed component (whether it is placed in the correct position). The part comparison processing unit can determine whether the shape and color of the part in the bottom image match the shape and color of the acquired normal (for example, new) part, or the extent to which defects such as deformation, wear, cracks, fading, dirt, etc. have progressed, and identify the degree of deterioration or damage of each part. In other words, it can determine the damage state (part condition) of the part when it is used. The part comparison processing unit can, for example, constantly or temporarily store sample images (which can be images linked with information such as an explanation of the deterioration state, the need for adjustment or replacement, etc.) of each deterioration stage to be compared for each part, and select a sample image that is closest to the part in the bottom image, thereby determining the deterioration state of the part. The sample images of each deterioration stage may be stored in a file in the failure location identification program, or may be obtained from, for example, a server, a website on the Internet, a cloud, etc.

[0030] The part inspection processing unit inspects each part based on the result of comparing the part images. Specifically, the part inspection processing unit can determine whether each part needs replacement or maintenance based on differences in the position, shape, size, color, etc. of each part. For example, the part inspection processing unit can convert the position, shape, size, color, etc. of each part in the bottom image into numerical values, convert the position, shape, size, color, etc. from the acquired standard information of each part, calculate the numerical difference, and determine whether maintenance is necessary or not based on a standard value (for example, in mm) for determining pass / fail that is preset for each part. For example, the part inspection processing unit can compare the image data of the part in the bottom image with the standard image data of the part acquired by the part information acquisition processing unit using RGB values ​​in pixel units, perform correction processing so that subtle differences in color are not detected as differences, and control the display of parts with large differences in size by flashing red, or by surrounding parts with large differences in size of parts with a flashing red circle.

[0031] The information such as the underside image of the observation target acquired by the judgment processing unit, the comparison information of each part, and the image inspection data including the inspection result can be displayed on the display of the fault location identification terminal. In addition, the various information acquired by the judgment processing unit can be stored in the imaging device, the fault location identification terminal, or an external storage device, another fault location identification terminal, a server, a website on the Internet, a cloud, or the like connected thereto.

[0032] The comparative information of a part may be information obtained by superimposing and comparing image data of the captured part with image data of the standard information of the corresponding part, or by comparing a numerical value extracted from the image data of the captured part with a corresponding numerical value in the standard information of the corresponding part.

[0033] The diagnostic device may further include a document creation processing unit that creates and stores at least one of an estimate and a work chart based on the stored data in the determination processing unit.

[0034] The document creation processing unit includes a document creation processing unit that creates at least one of an estimate regarding the cost of updating the part status and a work chart regarding the updating work, based on the part status of the part determined by the judgment processing unit. The document creation processing unit can output the created documents such as the estimate and work chart to the display of the failure point identification terminal, or output them as printed matter to a printer. The document creation processing unit may electronically transmit digital information (files) of the created documents such as the estimate and work chart.

[0035] The present invention also provides a data storage system that utilizes the image generating device. That is, a data storage system is provided which comprises an image generating device and an image storage device which stores an underside image generated by the image generating device, the image generating device comprising a transmission unit for transmitting the underside image, and the image storage device comprising a storage unit which acquires the underside image and retains it in association with specific information which identifies the observation subject from which the underside image was generated.

[0036] The multiple image generating devices can be used by various people in various places, regardless of the user or location. For example, the image generating devices may be installed in automobile repair shops nationwide and used by mechanics, sales representatives, etc. Also, the image generating devices may be used by maintenance workers of buildings and structures nationwide, sales representatives of house manufacturers, etc. A data storage system can be configured using such multiple image generating devices, and the multiple image generating devices can be connected to a server and various types of data can be stored in big data managed by the server, for example.

[0037] An image generation program can be installed in a plurality of image generation devices. Such an image generation program can be widely provided so that anyone can easily use it. For example, the image generation program can be made public and provided for free or for a fee via a distribution service on an Internet website, and various data acquired by the image generation program can be configured to be transmitted to and stored in a specific server via the Internet.

[0038] The server can receive and store image inspection data including bottom images of the object to be observed, comparison information for each part, and inspection results from the multiple image generating devices. The server can store various data received from the multiple image generating devices as big data. The big data stored in the server can be widely used for various purposes, such as final inspection before delivery of new products, estimates for maintenance of used products, improving the efficiency of inspection and maintenance work, and price evaluation of used products.

[0039] The present invention also provides an imaging device for photographing the underside of an object to be observed, which can be used to obtain an underside image with the image generating device. That is, an imaging device is provided that is equipped with a camera that photographs an object to be observed, a moving body having a running drive source or a floating moving source, and a movement control unit that controls the movement of the moving body so that the camera photographs the observation range of the object to be observed. Such an imaging device has a holder section to which a mobile terminal can be removably attached, and a mobile terminal equipped with application software as a movement control section and a LIDAR (Light Detection and Ranging) can replace the camera and communication device or removable media, and can also be removably mounted on the holder section. Have

[0040] The imaging device can be configured to capture images while moving under an object to be observed, for example, by having a travel drive source so that it can travel on the ground. The imaging device can also be configured to have a levitation source so that it can travel in gas or liquid. Furthermore, the imaging device can be configured to be moved manually or by another drive source without having a drive source. The imaging device can also be equipped with an image processing device having an image generation program and a fault location identification program.

[0041] The moving body can be formed by integrating the main components constituting the imaging device and having various outer shapes such as a skeleton frame, a housing, a capsule, etc. The moving body may have a track laid so as to travel thoroughly under the observation subject, and a traveling drive source for traveling on the track.

[0042] The traveling drive source enables the imaging device to travel on the ground, and can be, for example, a drive source such as a motor or an engine, and wheels, caterpillar tracks, or a combination thereof connected to the drive source. The traveling drive source may have a steering mechanism that steers under control. The levitation movement source enables the imaging device to levitate in gas or liquid and move, and can be, for example, a drive source such as a motor or an engine, and a fixed wing, rotary wing, or hovercraft structure that generates lift by utilizing the drive force of the drive source. The levitation movement source may have a steering mechanism that steers under control.

[0043] The camera is used to capture an image of an object to be observed, and may be built into or integrally mounted on a moving body, or may be detachably mounted on the moving body. Also, a camera built into a mobile terminal detachably mounted on the moving body may be used.

[0044] The movement control unit can be configured to control the movement of the imaging device, for example, to guide the imaging device to a route that allows efficient image capture (for example, to prioritize non-overlapping routes). The movement control unit can be a control unit for a travel drive source or levitation movement source built into, for example, a moving body of the imaging device, or a program installed in the control unit. The movement control unit can also be a program installed in the image generating device, and can remotely control the travel drive source or levitation movement source of the imaging device. The movement control unit can be configured with a receiving unit installed in the imaging device, and a controller that is installed separately from the imaging device and transmits a movement control signal to the receiving unit wirelessly or via wire to remotely control the imaging device. The controller can be operated by artificial intelligence or by a person.

[0045] When the contour setting unit of the fault location identification program (image generation program) identifies the contour of the observation range of the observation target, the movement control unit can guide the mobile object to the shortest route along the contour of the observation range. For example, when the movement control unit detects a characteristic point indicating the contour of the observation range, the movement control unit can guide the mobile object to go around the contour of the observation range by tracing the characteristic points linearly connected to the characteristic point.

[0046] The communication device enables communication between the imaging device and the fault location identification terminal, and can communicate based on various wireless communication standards, such as Bluetooth (registered trademark, omitted hereafter), wireless LAN, infrared communication, etc. The removable media temporarily stores image data captured by the imaging device and various data acquired during the shooting process, and makes the stored data removable from the imaging device and portable. Such removable media can be mounted on the imaging device by being attached to a slot or tray, etc., or connected to a connection terminal, and stores image data captured by the camera and various data acquired during the shooting process in real time.

[0047] The mobile terminal can be mounted on the imaging device to perform the functions of a camera, a communication device, etc., and is preferably small, thin, and lightweight enough to be mounted on the imaging device, and has a dustproof or liquid-proof seal structure. Such a mobile terminal can download an application software that serves as a movement control unit and control the imaging device to run or float via wireless communication. It can also download an image generation program and a fault location identification program using the image generation program and operate according to the programs. The mobile terminal can be mounted with the image generation program and the fault location identification program using the image generation program and operate under the control of the image generation program.

[0048] The observation object is the object of observation by the image generating device, diagnostic device, and data storage system using the same of the present invention. The observation object can be a downward surface or an object arranged to be exposed to the downward surface, and can exist in the air or in a liquid. Such observation objects can be various objects such as the undercarriage of a vehicle, under the floor of a building, the attic, the ceiling wall of a tunnel or cavern, the lower surface of the ceiling wall of a storage tank, etc.

[0049] The present invention also provides an image generating method for solving at least any one of the above problems. That is, the present invention provides an image generation method that is executed by a computer and generates an underside image of an object to be observed, comprising: a contour setting step that identifies the contour of the observation range of the object to be observed based on image data captured by moving around the underside of the object to be observed and sets the area within the contour as a contour area; a component feature extraction step that extracts feature points exhibited by parts that constitute the object to be observed and that are present in the observation range based on the image data within the contour area; and a component identification step that identifies parts from the feature points.

[0050] The present invention provides an image generating method for solving at least any one of the above problems. That is, the present invention provides an image generation program for generating an underside image of an object to be observed, the image generation program comprising: a contour setting step for identifying the contour of the observation range of the object to be observed based on image data captured by moving under the object to set the area within the contour as a contour area; a component feature extraction step for extracting feature points indicated by components constituting the object to be observed that are present in the observation range based on the image data within the contour area; and a component identification step for identifying the components from the feature points. Effect of the Invention

[0051] According to the image generating device of the present invention, the contour of the observation range of the observation target is identified, and after identifying the observation range, a bottom image is generated based on the captured image data, and further the parts of the observation target are identified, so that the parts of the observation target can be identified more efficiently. By allocating the captured image data to a map in which a plurality of sections are arranged within the contour of the observation range and integrating it into the bottom image, each part of the observation target and its arrangement can be identified more accurately.

[0052] According to the image generating device of the present invention, by grasping the contour of the observation range of the observation target in advance, the subsequent photographing of the observation target can be performed more efficiently. When the contour setting unit extracts the boundary between the observation target located within a certain distance range from the imaging device and the outside of the observation target as a characteristic point of the contour and identifies the contour of the observation target range by connecting the characteristic points in a line, for example, the contour of the observation target range can be more efficiently and accurately identified by providing a distance measuring unit in the imaging device. Also, when the boundary between the light and dark color caused by the corners or edges of the observation target is extracted as a characteristic point of the contour and identifies the contour of the observation target range by connecting the characteristic points in a line, for example, the contour of the observation target range can be more efficiently and accurately identified by a camera mounted on the imaging device.

[0053] According to the diagnostic device using the image generating device of the present invention, since a superimposed image of the underside image and the standard information of the parts acquired separately from the captured image data can be generated, each part of the observation target can be visually confirmed together with the standard information of the parts acquired. Furthermore, the part image in the underside image can be superimposed and directly compared with the standard image of the parts acquired. The image of the part identified by the part identification unit is compared with the standard image of the part acquired by the part information acquisition processing unit, and based on the result of the image matching, the underside image of the observation target, the comparison information of each part, and image inspection data including the inspection result can be more efficiently acquired. Furthermore, the various data can be quickly displayed on a display as image data that is visually easy to see and intuitively easy to understand, and can be stored in a storage medium. Necessary inspections can be performed more efficiently. The burden of the inspection work can be reduced.

[0054] According to the diagnostic device of the present invention, quotations and work sheets can be quickly created and stored based on the data stored in the judgment processing unit, and the amount of work required for administrative tasks such as quality control, delivery, quotations, maintenance and inspection, and evaluation of used goods can be significantly reduced.

[0055] According to the data storage system of the present invention, various data generated by multiple image generating devices or fault location identifying terminals can be stored in a server more efficiently. The various data stored in the server can be used in a wide range of applications, such as being used as big data at each stage of the product distribution process, or for parts quality control and product development.

[0056] According to the data storage system of the present invention, the image capture device is provided with a holder section to which a mobile terminal can be detachably attached, and the mobile terminal is detachably mounted on the holder section, so that the mobile terminal replaces the camera and communication device or removable media. When the mobile terminal is equipped with application software for controlling the image capture and the movement control section, the mobile terminal controls the image capture and movement of the image capture device. Furthermore, the LIDAR mounted on the mobile terminal can measure the distance from the image capture device to the observation target, and can identify the outline of the observation target range. For example, the labor required for lifting up a vehicle for an estimate for vehicle inspection can be reduced, reducing the number of work steps, and an estimate with higher accuracy than visual inspection can be made. [Brief description of the drawings]

[0057] [Figure 1] 6 is a flowchart of a failure location identification program using an image generation program. [Diagram 2] 13 is a flowchart of a recognition process for the undercarriage and the component 101. [Diagram 3] 11 is a flowchart of a process for recognizing the entire undercarriage. [Figure 4] 13 is a flowchart of a process for recognizing the undercarriage and components 101 after grasping the entire undercarriage. [Diagram 5] FIG. 1 is a conceptual diagram showing a data accumulation system for the undercarriage of an automobile 10a. [Figure 6] A conceptual diagram showing a data storage system in the attic of building 10b. [Figure 7] (a) A photograph showing an underside image 100 of the underside portion 10u of the object to be observed (automobile) integrated by the map setting processing unit P4. (b) A plan view showing the contour 10L of the underside portion 10u of the object to be observed (automobile) identified by the contour setting unit P1. (c) A two-dimensional map 10M assigned to the contour 10L of the underside portion 10u of the object to be observed (automobile) by the map assignment processing unit P3. (d) A two-dimensional map 10M in which the image generation processing unit superimposes the underside image 100 of the object to be observed and the acquired standard information 102 of the component 101. [Figure 8](a) A perspective view of an imaging device 4 having tracks 7a on which a mobile terminal 2b is mounted. (b) A perspective view of an imaging device 4 having tracks 7a on which a camera 51 is mounted. (c) A three-sided view of a three-wheeled (7a, 7b) imaging device 4 on which a camera 51 is mounted. (d) A perspective view of an imaging device 4 having multiple rotors 71. [Figure 9] (a) is a perspective view showing a manually moved imaging device 4. (b) is a front view and a side view showing a manually moved imaging device 4. (c) is a front view showing an imaging device 4 equipped with a plurality of cameras 51. [Figure 10] (a) A bottom view showing the underside portion 10u of the roof of the building 10b. (b) A two-dimensional map 10M assigned to the contour 10L of the underside portion 10u of the roof of the building 10b. (c) An overlay image of the two-dimensional map 10M and the underside image 100 of the underside portion 10u of the roof of the building 10b. [Figure 11] Hardware configuration diagram. EMBODIMENT 1 FOR CARRYING OUT THEINVENTION

[0058] Hereinafter, an image generating device, a diagnostic device using the image generating device, a data storage system, an image generating method, and an image generating program according to one embodiment of the present invention will be specifically described with reference to the drawings. In particular, in this embodiment, an automobile or a building is the object of observation, but other man-made objects or natural objects, etc. can also be the object of observation.

[0059] As shown in Fig. 5 and Fig. 8(c), the data storage system 1 of this embodiment includes a failure location identification terminal 2 and a server 3. The data storage system 1 includes a moving object 5, a camera 51, a mobile control unit 52, and an imaging device 4 having a communication device 53 or a removable media 84. The imaging device 4 is set to have outer dimensions (particularly height) that allow it to enter between the lower surface of a floor panel 10u of an automobile 10a to be observed and the tire contact surface (ground surface) as shown in Fig. 5, or into an underfloor space of a building 10u or an upper (attic) space of a suspended ceiling of a building as shown in Fig. 6, and move freely in the horizontal direction.

[0060] The imaging device 4 has a disk-shaped moving body 5, as shown in Fig. 5 and Fig. 8(c), for example. The moving body 5 is provided with a battery, a drive motor, left and right drive wheels 7a driven by the drive motor, and a steering wheel 7b steered by a servo motor. The drive motor and the servo motor can be controlled by a movement control unit 52 mounted in the moving body 5.

[0061] As shown in Figures 8(a) and (b), the driving wheels 7a of the imaging device 4 can be, for example, caterpillar wheels. The driving wheels 7a are provided with three or more caterpillar wheels, each with an independent suspension mechanism, and can be steered by being driven and controlled. The driving wheels 7a can be provided with, for example, four to eight caterpillar wheels, each with an independent suspension mechanism.

[0062] 8(d), the imaging device 4 may have a levitation moving source 7. The levitation moving source 7 is a rotor 71 provided at each tip of a plurality of pillars extending in the centrifugal direction on the same horizontal plane from the moving body 5, and for example, three to ten of them may be provided. The moving body 5 having the rotor 71 may be provided with one or more ground wheels 7c.

[0063] As shown in Fig. 5, Fig. 8(b) and (c), the camera 51 can be mounted, for example, on the top wall of the moving body 5, with its optical axis facing vertically upward. The camera 51 has a recording control unit 51c capable of taking still or moving images, and can be, for example, a CCD (Charge Coupled Devices), a CMOS (Complementary Metal Oxide Semiconductor), or the like. The moving body 5 can be provided with one or more lights 50b that illuminate the observation target 10u that is the subject, and an illumination control unit that adjusts the brightness of the lights 50b.

[0064] The imaging device 4 desirably includes a distance measurement unit 50c and a distance measurement control unit for controlling the distance measurement unit 50c in the moving object 5. The distance measurement unit 50c may be, for example, a LIDAR, a SPAD (Single Photon Avalanche Diode), an optical sensor, a radio wave sensor, or an ultrasonic sensor.

[0065] The communication device 53 can, for example, wirelessly communicate with the fault location identifying terminal 2, and can enable wireless communication between the movement control unit 52 and the recording control unit 51c and the fault location identifying terminal 2. The communication device 53 can perform communication in accordance with communication standards such as wireless LAN (Wi-Fi) and Bluetooth, infrared communication, and other wireless communication.

[0066] 8(c), the imaging device 4 can be equipped with a hardware interface 54h for a removable medium (flash memory) 54. The hardware interface 54h can be, for example, a card slot 54s for an SD card, a USB connection terminal 54t for a USB memory stick, or the like.

[0067] The movement control unit 52 receives a control signal output by the fault location identification terminal 2 through the communication device 53 and controls the drive wheels 7a and the steering wheels 7b. For example, the fault location identification terminal 2 has a route calculation processing unit that receives image data captured by the camera 51 of the imaging device 4, stored in the recording control unit 51c, and transmitted through the communication device 53, and calculates a movement route based on the received image data. The route calculation processing unit transmits the calculated movement route data to the movement control unit 52 of the imaging device 4 through the communication device 53.

[0068] The movement control unit 52 may also have a route calculation processing unit that performs calculations on a travel route that allows for the most efficient shooting based on distance data measured by the distance measurement unit 50c and stored in the distance measurement control unit. In this case, the route calculation processing unit enables the movement control unit 52 of the imaging device 4 to perform autonomous driving, acceleration, deceleration, stopping, and steering. Furthermore, the movement control unit 52 may have a route calculation processing unit that performs calculations on a travel route that allows for the most efficient shooting based on image data captured by the camera 51 and stored in the recording control unit 51c. In this case, the route calculation processing unit enables the movement control unit 52 of the imaging device 4 to perform autonomous driving, acceleration, deceleration, stopping, and steering.

[0069] The fault location identification terminal 2 can be configured by mounting an image generation device 8 and a diagnosis device 9 using the image generation device 8. The image generation device 8 has, for example, a contour setting unit P1, a part identification unit P2, a map allocation processing unit P3, a map setting processing unit P4, a part information acquisition processing unit P5, and an image generation processing unit P6. The diagnosis device 9 has, for example, an image generation device 8, a part comparison processing unit P7, a part inspection processing unit P8, and a judgment processing unit P9. Furthermore, the diagnosis device 9 may have a document creation processing unit P10. The fault location identification terminal 2 can use a computer 2a as shown in FIG. 5, and the fault location identification terminal 2 can be replaced with a mobile terminal 2b as shown in FIG. 6.

[0070] 8(a), the imaging device 4 is provided with a holder unit 50 to which a mobile terminal 6 can be detachably attached on the top wall of a moving object 5. The mobile terminal 6 including at least one of a camera 51, a light 50b, a LIDAR (distance measuring unit) 50c, a movement control unit 52, a communication device 53, and a removable media 54 can be detachably mounted on the holder unit 50.

[0071] In this case, the mobile terminal 6 can be used as an image generating device that has a contour setting unit that identifies the contour of the observation range of the observation target (frame information of the lower side of the observation target) based on image data captured by the moving body 5 while moving below the observation target, and sets the area within the contour as a contour area, and an image output unit that outputs an image acquisition area within the contour area where image data has been acquired and an image non-acquired area where image data has not been acquired, and can further be used as an image generating device that generates an underside image from image data acquired within the contour area.

[0072] The mobile terminal 6 can also be used as a diagnostic device if it is equipped with a part identification unit that identifies parts from feature points based on the image data within the contour area, a part information acquisition processing unit that acquires a standard image of the part identified by the part identification unit, a part comparison processing unit that compares the image of the part identified by the part identification unit with the standard image of the part acquired by the part information acquisition processing unit, and a judgment processing unit that judges the part state of the part identified by the part identification unit from the comparison result in the part comparison processing unit.

[0073] The imaging device 4 may be a manual type as shown in Fig. 9(a), (b) and (c). The imaging device 4 is configured, for example, by providing swivel casters 7c at the left and right ends of the bottom of a rectangular mobile body 5 that is elongated from side to side. An operating rod 50a that is inclined backward extends from a position halfway between the left and right ends of the top wall of the mobile body 5. The operating rod 50a may have a telescopic structure. Four cameras 51 are provided on the front wall of the mobile body 5, which are arranged at equal intervals between the left and right ends and are directed diagonally upward and forward. As shown in Fig. 9(c), two or more cameras 51, for example, three, for example, three to eight, can be provided on the mobile body 5. The manual imaging device 4 is configured by providing at least one of a light 50b, a distance measuring unit 50c, a communication device 53 and a removable media on the mobile body 5.

[0074] The manual imaging device 4 shown in FIG. 9 is operated by gripping the operating rod 50a, and the moving body 5 is moved to the underside of the observation target 10u shown in FIG. 5 and FIG. 6. The cameras 51 simultaneously capture images while the swivel casters 7c are placed on the ground and rolled, so that a wide range can be captured more efficiently. The multiple cameras 51 provided on the moving body 5 are preferably set to the same focal length and arranged so that the distances from the observation target 10u, which is the subject, to the cameras 51 are the same. In addition, the multiple cameras 51 can be set to long and short focal lengths and arranged so that the distances from the observation target 10u, which is the subject, to the cameras 51 match the respective focal lengths. This makes it possible to capture wide-angle images and enlarged images at the same time. Furthermore, the multiple cameras 51 are provided so that the optical axes are oriented in different directions, so that a three-dimensional image of higher quality can be captured more quickly.

[0075] Furthermore, in a manual imaging device, for example, a holder part to which a mobile terminal can be detachably attached is provided at the tip of the operating rod, and the mobile terminal can be detachably attached to the holder part. The manual imaging device can be used to take pictures by holding the operating rod like a selfie stick and bringing the mobile terminal close to the underside of a ceiling of a building or the underside of a car, etc., as described above. The mobile terminal can be used as an image generating device or a diagnostic device. Second Embodiment of the Invention

[0076] (Automobile inspection process) In the following, with reference to FIGS. 1 to 5 and 7 to 9, the image generating device 8 and the diagnostic device 9 used in the automobile data accumulation system 1 will be described according to the flow of the inspection work of an automobile 10a by the data accumulation system 1.

[0077] When the fault location identification terminal 2 is started, an input screen for information identifying the automobile 10a, which is the observation target 10, is displayed on the display of the computer 2a serving as the fault location identification terminal 2. On this screen, information effective for identifying the vehicle, such as the name, chassis number, model, engine model, type, and body shape of the automobile 10a, as well as vehicle inspection certificate information, etc., are input (S1).

[0078] When the imaging device 4 is placed on the ground below the underside 10u of the automobile 10a, which includes the floor panel, chassis frame, suspension, etc. to be observed, the imaging device 4 determines its position relative to the underside 10u of the automobile 10a based on image data captured by the mounted camera 51 while moving slowly (for example, 10 to 30 cm / s) in a random direction, and stores and recognizes the imaging start position (S2).

[0079] The imaging device 4 uses a sensor to determine whether it is located under the underside portion 10u of the automobile 10a (S3). The sensor can be a distance measuring unit 50c made up of a LIDAR, an optical sensor (e.g., an infrared sensor), a radio wave sensor (e.g., a transmitting / receiving antenna), an ultrasonic sensor, or the like. For example, it can be set to determine that the imaging device 4 is located under the underside portion 10u when the distance from the imaging device 4 to the underside portion 10u is within 1 m or within 50 cm. In addition, it can detect the brightness and color shading of the image captured by the camera 51, and determine that the imaging device 4 is located under the underside portion 10u when the image is darker or darker than the outside of the underside portion 10u.

[0080] If it is not recognized that it is located under the lower surface portion 10u for a certain period of time (for example, 10 to 30 seconds) or more, the measurement is terminated, and the robot returns to the starting position (where it was originally placed) and stops (A) (S15). In this case, the robot changes course while continuing detection, and if it is recognized that it is located under the lower surface portion 10u during the time it returns to the starting position, it can be controlled to proceed to the next step (S5).

[0081] If it is recognized that the object is located below the lower surface portion 10u within a certain period of time (for example, 10 to 30 seconds), then image capture of a moving image (or still image) is started (S5). Image capture can be performed at a maximum speed of, for example, 30 to 500 cm / s while the vehicle is traveling with deceleration, such as stopping, making U-turns, rotating, etc.

[0082] Then, it is determined whether or not "undercarriage recognition mode 2" is set (S6). If "undercarriage recognition mode 2" is set, recognition processing of the components 101 within the contour 10L of the observation target 10u is performed (S10).

[0083] (Contour setting section P1) On the other hand, if the "undercarriage recognition mode 2" is not set (S6), it is determined whether or not the edge part of the undercarriage is imaged (S7). If the edge part of the undercarriage is not imaged, a recognition process of the undercarriage and the components 101 (S8) is performed.

[0084] (Undercarriage recognition mode 1) The recognition process (S8) of the undercarriage and components 101 sets "undercarriage recognition mode 1" (S8a). Undercarriage recognition mode 1 can be a recognition mode that, for example, identifies the outer periphery of the observation target 10 as the contour 10L of the observation target range, and then recognizes the components 101 (floor panel, chassis frame, suspension, bumper, side sill, fender, etc.) of the underside of the observation target. By recognizing the contour 10L of the observation target range in advance, it is possible to anticipate the arrangement of parts and further improve the detection accuracy and inspection accuracy of components such as parts.

[0085] When the "undercarriage recognition mode 1" is set (S8a), for example, the camera 51 (recording control unit 51c), the light 50b (illumination control unit), and the driving source 7 (movement control unit 52) ​​can be set to conditions that make it easy to recognize the outer periphery of the component 101, and each of them can be controlled. When the camera 51 (recording control unit 51c) captures a characteristic point that indicates the contour 10L of the underside portion 10u, the driving source 7 can be controlled to slow down and run slowly along the characteristic point, giving priority to identifying the contour 10L (S8b). The tires, the components 101 of the undercarriage, etc. are identified, and the driving source 7 moves so as not to come into contact with them.

[0086] (Part Identification Section P2) Furthermore, based on the (two-dimensional or three-dimensional) image data of the camera 51, the components 101 present on the underside of the observation target 10 are recognized while identifying the contour 10L of the observation target range (S8c). The components 101 are recognized by identifying the feature points of each component 101, and information on the identified components 101 can be stored (S8d). The feature points of each component 101 can be determined and extracted based on, for example, the different shapes, sizes, colors, arrangements, etc. of each component 101. The components 101 can be, for example, exhaust pipes, catalysts, mufflers, suspensions, axles, floor panels, bolts and nuts that connect them, and other components 101 that are visibly exposed on the underside of the vehicle body.

[0087] (Map allocation processing unit P3) When the imaging of the edge portion of the underside is completed (S7), the entire contour 10L of the observation target range is recognized (S9). The contour 10L (edge ​​of the underside) portion of the observation target range is stored together with the position data of the characteristic points thereof (S9a). Based on the stored position data of the characteristic points of the contour 10L, a "underside frame map 10M" is created (S9b) in which a two-dimensional map (or a three-dimensional map) consisting of a two-dimensional array (or a three-dimensional array) of a plurality of planar sections 10S (or three-dimensional sections) is assigned within the contour 10L of the observation target range.

[0088] The "underside frame map 10M" can be, for example, map data in which a plurality of arranged sections 10S are assigned to image data that has been generated and processed to be integrated into an underside image 100 based on imaging data of the object of observation 10.

[0089] (Undercarriage recognition mode 2) It is determined whether or not the creation of the "undercarriage frame map 10M" has been completed (S9c), and if it is incomplete, the process returns to the step of determining whether or not it is located under the underside portion 10u of the automobile 10a (S3). On the screen of the fault location identification terminal 2, in order to easily visually distinguish between the sections 10F for which photographing has been completed and the sections 10N for which photographing has not yet been completed, the sections 10N for which photographing has not yet been completed are displayed in color, and the sections 10F for which photographing has been completed can be displayed in white or black, or by pasting an image of the corresponding section 10S that has been photographed.

[0090] (Map setting processing section P4) When the creation of the "undercarriage frame map 10M" is completed, the inside of the "undercarriage frame map 10M" is set as the imaging range (S9d), and "undercarriage recognition mode 1" is set (S9e). The "undercarriage recognition mode 2" can be, for example, a recognition mode that identifies the underside portion 10u and each of its components 101 (floor panel, chassis frame, suspension, etc.), and enables inspection of each component 101 on the image data.

[0091] (Part information acquisition processing unit P5) Based on information input (S1) on an input screen for information identifying the automobile 10a, which is the observation target 10, standard information 102 of the constituent elements 101 under the automobile 10a is acquired (S9f). The standard information 102 of the constituent elements 101 can be part names, part numbers, assembly numbers, blueprints, service manuals, parts lists, and associated information (dimensions, materials), colors, shapes, two-dimensional (or three-dimensional) images, etc. By acquiring the standard information 102 in advance, it becomes possible to identify the constituent elements 101 with greater accuracy based on that information.

[0092] The standard information 102 of the constituent element 101 can be acquired from information stored in, for example, the external storage device 3c of the server 3, or from data stored on the Internet 3a, the cloud 3b, etc. Acquisition of the standard information 102 of the constituent element 101 can be controlled, for example, by extracting an image of the constituent element 101 from a captured image, and based on the image of the constituent element 101, searching the stored data (2, 3, 3c) or searching the Internet 3a (3b) to acquire the standard information 102 of the constituent element 101.

[0093] The "standard information 102 of the underbody constituent element 101" is superimposed on the "underbody frame map 10M", and the generated image data is saved (stored) (S9g). The generated image data can be displayed by superimposing an image of a translucent part 102 to which the standard information 102 of the underbody constituent element 101 is linked on the underbody frame map 10M, and when the mouse pointer is placed over a part 102 displayed translucently on the display of the computer 2a, the part 102 changes to a blinking display, and a small screen displaying the detailed standard information 102 of the part 102, such as numerical data and text information, is expanded and displayed, and when the mouse pointer is removed from the part 102, the display returns to the original display. A similar operation is possible even when the computer 2a is replaced with a mobile terminal 2b.

[0094] Based on the generated image data, it is determined whether or not standard information 102 of the positions and shapes of all components 101 identified and stored in "undercarriage recognition mode 1" is available (S9h). If standard information 102 of the positions and shapes of all components 101 is available, the position, shape and color data of each component 101 is compared and collated with the acquired (S9f) standard information 102 of the position, shape and color (S9j). The collation results of each component 101 (102) are temporarily stored in, for example, the imaging device 4 (51c, 54) or the fault location identification terminal 2 (S9k).

[0095] When there is no standard information 102 of the position, shape, and color of some or all of the components 101 (S9h), or after the collation results of all the components 101 are temporarily stored (S9k), the camera 51 (recording control unit 51c), the illumination intensity of the light 50b (illumination control unit), and the traveling speed of the traveling drive source 7 (movement control unit 52) ​​can be set and controlled to conditions that allow each of the components 101 of the lower surface portion 10u to be photographed in detail. When the camera 51 (recording control unit 51c) captures characteristic points that indicate each of the components 101 of the lower surface portion 10u, the speed of the traveling drive source 7 may be reduced and the traveling drive source 7 may travel slowly while processing the characteristic points, so that the identification of each of the components 101 is prioritized (S9m).

[0096] (Image generation processing unit P6) After grasping the entire undercarriage (within the contour 10L of the underside portion 10u of the automobile 10a), a recognition process is performed on the undercarriage (within the contour 10L) and the components 101 within the contour 10L of the observation target 10u (S10). The captured image is compared with an overlaid image to match each component 101 (S10a), and as a result of the matching, it is determined whether all components 101 have been recognized (S10b). The positions, shapes, and colors of all components 101 are permanently stored (S10c) in, for example, the imaging device 4 (51c, 54) or the fault location identification terminal 2. The permanently stored position information of the components 101 is more accurate than the provisionally stored position information.

[0097] Then, it is determined whether imaging of the entire imaging range has been completed (S10d). For example, as shown in Fig. 7(d), when the section 10F for which imaging has been completed is displayed over the entire undercarriage frame map 10M displayed on the display of the computer 2a, and the section 10N for which imaging has not yet been completed is not displayed, it is possible to determine whether imaging of the entire imaging range has been completed.

[0098] (Part comparison processing section P7) It is determined (S10e) whether all the temporarily stored components 101 match the position, shape, and color of the temporarily stored (S9k) comparison result of each component 101. Furthermore, it is possible to check whether the parts 102 of the standard information obtained before creating the vehicle underbody frame map 10M are included in the components 101 recognized after creating the underbody frame map 10M, thereby making it possible to search for all components 101 without omission.

[0099] If any of the components 101 do not match the position, shape, or color temporarily stored (S9k), the positions, shapes, and colors of all the components 101 stored in actual storage (S10c) are superimposed on the positions, shapes, and colors of all the components 101 temporarily stored (S9k) and compared on the display (S10f). The user instructions based on the comparison result are reflected in the actual storage, and the duplicated temporary storage is deleted (S10g).

[0100] (Parts inspection processing section P8) The components 101 are investigated (S11). The components 101 can be investigated by comparing the components 101 on the superimposed images, such as misalignment of the axle, deformation or cracks in the suspension, oil leaks, etc. The RGB values ​​of each component 101 are compared on a pixel-by-pixel basis, and components 101 with large differences are investigated. (Decision processing unit P9) The overall image of the undercarriage, the comparison information of the components 101, and the inspection results are displayed on the display (S12).

[0101] (Document creation processing section P10) Based on the inspection results, an estimate for the cost of updating the part status of the parts and a work chart for the updating work are prepared and stored in the storage device of the computer 2a or the external storage device 3c of the server 3 (S13). The recognition mode, which is set to either "undercarriage recognition mode 1" or "undercarriage recognition mode 2", is reset (S14). All processes are completed (S15). Updating work is work to update the status of parts, such as work to modify, replace, or repair parts.

[0102] The overall image of the undercarriage, the comparison information (101, 102) of the components 101, and the inspection results stored (S13) by the computer 2a as the fault location identification terminal 2 can be stored, for example, in an external storage device 3c of the server 3. The server 3 receives information transmitted from a plurality of fault location identification terminals 2, and stores it together with information on each automobile manufacturer, vehicle model, type, year, registration year, registration prefecture, mileage, maintenance history, etc., and can manage the stored data as big data that can be statistically processed. MODE FOR CARRYING OUT THE PRESENTINVENTION 3

[0103] (Building inspection process) 6 and 10, the image generating device 8 and the diagnostic device 9 used in the data accumulation system 1 of the building 10b will be shown according to the flow of the inspection work of the data accumulation system 1 for the walls, hanging fittings, air conditioning equipment, various piping, wiring, etc. of the underside portion 10u of the roof in the space above the suspended ceiling 10h of the building 10b.

[0104] When the fault location identifying terminal 2 and the imaging device 4 are started, an input screen for inputting information for identifying the building 10b, which is the observation target 10, is displayed on the display of the mobile terminal 2b acting as the fault location identifying terminal 2. On this screen, information such as the type of architectural structure of the building 10b (wooden, aluminum, light steel frame, heavy steel frame, reinforced concrete, steel reinforced concrete, concrete filled steel pipe, concrete block), the name of the construction company or house manufacturer, model, year, year of registration, age, prefecture of registration, address, GPS (Global Positioning System) data, blueprints, maintenance history, etc. are input (S1).

[0105] The imaging device 4 is placed on the suspended ceiling 10h below the underside of the underside portion 10u of the roof or upper floor of the building 10b to be observed. The imaging device 4 may be any of the imaging devices 4 having a traveling drive source 7 shown in Fig. 8(a) to Fig. 8(c), or the imaging device 4 having a levitating moving source 7 shown in Fig. 8(d). The imaging device 4 identifies the position of the imaging device 4 relative to the underside portion 10u based on the captured image data of the mounted camera 51 while traveling slowly (for example, 10 to 30 cm / s) in a random direction, and stores and recognizes the imaging start position (S2).

[0106] The imaging device 4 judges whether it is located under the lower surface portion 10u by the distance measuring unit 50c (S3). For example, it can be set to judge that it is located under the lower surface portion 10u when the distance from the imaging device 4 to the lower surface portion 10u is within 3 m or within 1 m. It can also sense the brightness and color shading of the image captured by the camera 51, and distinguish the brightness and color difference between the outer vertical wall of the lower surface portion 10u, thereby judging that it is located under the lower surface portion 10u.

[0107] If it is not recognized that it is located under the lower surface portion 10u for a certain period of time (e.g., 30 seconds to 1 minute) or more, the measurement is terminated, and the robot returns to the starting position (where it was originally placed) and stops (S15). In this case, the robot changes course while continuing detection, and if it is recognized that it is located under the lower surface portion 10u before returning to the starting position, the robot can be controlled to proceed to the next step (S5).

[0108] If it is recognized that the object is located below the lower surface portion 10u within a certain period of time (e.g., 30 seconds to 1 minute), then it starts capturing moving images (or still images) (S5). The capturing of images can be performed at a maximum speed of, for example, 30 to 500 cm / s, while moving with stops, U-turns, rotations, deceleration, etc.

[0109] It is determined whether or not "undercarriage recognition mode 2" is set (S6). If "undercarriage recognition mode 2" is set, recognition processing is performed for the components 101 within the contour 10L of the observation target 10u (S10).

[0110] (Contour setting section P1) If the "undercarriage recognition mode 2" is not set (S6), it is determined whether or not an image of an inner corner of the undercarriage has been captured (S7). If an image of an inner corner of the undercarriage has not been captured, a recognition process of the undercarriage and the components 101 (S8) is performed.

[0111] (Undercarriage recognition mode 1) The recognition process (S8) of the undercarriage and components 101 sets "undercarriage recognition mode 1" (S8a). The undercarriage recognition mode 1 can be, for example, a recognition mode that identifies the outer periphery of the underside portion 10u as the contour 10L of the observation target range.

[0112] When the "undercarriage recognition mode 1" is set, for example, the camera 51 (recording control unit 51c), the light 50b (illumination control unit), and the traveling drive source 7 or the levitation movement source 71 (movement control unit 52) ​​can be set to conditions that make it easy to recognize the outer periphery of the underside portion 10u, and each can be controlled. When the camera 51 (recording control unit 51c) captures a characteristic point that indicates the outline 10L of the underside portion 10u, the traveling drive source 7 or the levitation movement source 71 can be controlled to slow down the movement speed and move slowly along the characteristic point, prioritizing the identification of the outline 10L (S8b). The surrounding pillars, walls, hanging fixtures, air conditioning equipment, piping, wiring, etc. are identified, and the movement can be made so as not to come into contact with them.

[0113] (Part Identification Section P2) Furthermore, based on the (two-dimensional or three-dimensional) image data of the camera 51, the components 101 of the observation target 10u are recognized while identifying the contour 10L of the observation target range (S8c). The components 101 are recognized by identifying the feature points of each component 101, and storing information on the identified components 101 (S8d). The feature points of each component 101 can be determined and extracted based on, for example, the shape, size, color, arrangement, etc., which differ for each component 101. The components 101 can be visible components 101, such as pipes, ducts, wiring, air conditioners, hanging fixtures, pillars, beams, walls, etc.

[0114] (Map allocation processing unit P3) When the imaging of the edge portion of the undercarriage is completed (S7), the entire contour 10L of the observation target range is recognized (S9). The contour 10L of the observation target range (corners of the undercarriage) is stored together with the position data of the characteristic points (S9a). Based on the stored position data of the characteristic points of the contour 10L, a "undercarriage frame map 10M" is created (S9b) in which a two-dimensional map (or a three-dimensional map) consisting of a two-dimensional array (or a three-dimensional array) of a plurality of planar sections 10S (or three-dimensional sections) is assigned to the contour 10L of the observation target range.

[0115] As shown in Figs. 10(a) and 10(b), the sections 10F for which photographing has been completed and sections 10N for which photographing has not yet been completed can be displayed in different colors on the display of the mobile terminal 2b. For example, the imaging device 4 can photograph along the characteristic points indicating the outer periphery of the lower surface portion 10u, and photograph so as to go around the contour 10L of the observation target range. When the contour 10L of the observation target range is completed, the sections 10F for which photographing has been completed are displayed in a line around the displayed lower frame map 10M on the display of the mobile terminal 2b, and the sections 10N for which photographing has not yet been completed are displayed in a colored filled-in image in the center, so that the contour 10L of the observation target range can be identified before detailed photographing of each component 101. After identifying the contour 10L of the observation target range, photographing of the sections 10N for which photographing has not yet been completed can be performed, and finally, as shown in Fig. 10(c), control can be performed so that all sections 10S in the "lower frame map 10M" are displayed.

[0116] The "underside frame map 10M" can be, for example, map data in which a plurality of arranged sections 10S are assigned to image data that has been generated and processed to be integrated into an underside image 100 based on imaging data of the observation object 10u.

[0117] (Undercarriage recognition mode 2) It is determined whether or not the creation of the "underbody frame map 10M" has been completed (S9c), and if it is incomplete, the process returns to the step of determining whether or not it is located below the underside portion 10u (S3).

[0118] (Map setting processing section P4) When the creation of the "undercarriage frame map 10M" is completed, the inside of the "undercarriage frame map 10M" is set as the imaging range (S9d), and "undercarriage recognition mode 1" is set (S9e). The "undercarriage recognition mode 2" can be a recognition mode that distinguishes, for example, the underside portion 10u from each of the components 101, such as pillars, walls, hanging fixtures, air conditioning equipment, piping, and wiring, and enables inspection of each of the components 101 on the image data.

[0119] (Part information acquisition processing unit P5) Based on information input (S1) on an input screen for information identifying the building 10b, which is the observation target 10, standard information 102 of the components 101 under the observation target 10 is acquired (S9f). The standard information 102 of the components 101 can be part names, part numbers, assembly numbers, blueprints, service manuals, parts lists, and associated information (dimensions, materials, parts list information), colors, shapes, two-dimensional (or three-dimensional) images, etc. The standard information 102 of the components 101 can be acquired from information stored in, for example, the external storage device 3c of the server 3, as well as data stored on the Internet 3a, the cloud 3b, etc.

[0120] 10(c), the "standard information 102 of the underbody components 101" is overlaid on the "underbody frame map 10M" and the generated image data is saved (S9g). The generated image data is displayed by overlaying an image of a semi-transparent component 102 to which the standard information 102 of the underbody components 101 is linked on the underbody frame map 10M, and when the mouse pointer is placed over a component 102 displayed semi-transparently on the screen, the component 102 changes to a blinking display, and a small screen displaying the detailed numerical data and text information of the standard information 102 of the relevant part 102 is expanded and displayed, and when the mouse pointer is removed from the component 102, the display returns to the original display.

[0121] Based on the generated image data, it is determined whether or not there is standard information 102 of the positions and shapes of all components 101 (underside portions 10u) identified and stored in "undercarriage recognition mode 1" (S9h). If there is standard information 102 of the positions and shapes of all components 101, the position, shape and color data (underside portions 10u) of each component 101 is compared and collated with the acquired (S9f) standard information 102 of the positions, shapes and colors (S9j). The collation results for each part are temporarily stored (S9k).

[0122] When there is no standard information 102 of the position, shape and color of some or all of the components 101 (lower surface portion 10u) (S9h), or after the collation results of all the components 101 are temporarily stored (S9k), the camera 51 (recording control unit 51c), the illumination intensity of the light 50b (illumination control unit), and the traveling speed of the traveling drive source 7 (movement control unit 52) ​​or the moving speed of the levitating moving source 71 can be set and controlled to conditions that allow each of the components 101 of the observation target 10u to be photographed in detail. When the camera 51 (recording control unit 51c) captures a characteristic point indicating each of the components 101 of the lower surface portion 10u, the speed of the traveling drive source 7 or the levitating moving source 71 can be controlled to be reduced and moved slowly while processing the characteristic point, so that the identification of each of the components 101 is prioritized (S9m).

[0123] (Image generation processing unit P6) After grasping the entire undercarriage (within the contour 10L of the underside portion 10u), a recognition process is performed for the undercarriage and the components 101 within the contour 10L of the underside portion 10u (S10). The captured image is compared with an overlaid image to match each component 101 (S10a), and as a result of the matching, it is determined whether all the components 101 have been recognized (S10b). The positions, shapes, and colors of all the components 101 are permanently stored (S10c). The permanently stored position information of the components 101 is more accurate than the provisionally stored position information.

[0124] It is determined whether imaging of the entire imaging range has been completed (S10d). For example, as shown in Fig. 6 and Fig. 10(c), when the section 10F for which imaging has been completed is displayed over the entire undercarriage frame map 10M displayed on the display of the mobile terminal 2b, and the section 10N for which imaging has not yet been completed is not displayed, it is possible to determine whether imaging of the entire imaging range has been completed.

[0125] (Part comparison processing section P7) It is determined (S10e) whether all the temporarily stored components 101 match the position, shape, and color of the temporarily stored (S9k) result of the comparison of each component 101. If any of the components 101 do not match the temporarily stored (S9k) position, shape, or color, all the components 101 actually stored (S10c) are superimposed on the positions, shapes, and colors of all the components 101 temporarily stored (S9k) and compared and displayed on the display 2b (S10f). The user's instructions based on the comparison result are reflected in the actual storage, and the duplicated temporary storage is deleted (S10g).

[0126] (Parts inspection processing section P8) The components 101 are investigated (S11). The components 101 can be investigated by comparing the components on the superimposed image 100 (101, 102) to determine whether there is a misalignment of pipes, deformation or cracks in walls or beams, water leaks, etc. The RGB values ​​of each component are compared on a pixel-by-pixel basis, and components 101 with large differences can be investigated.

[0127] It is possible to distinguish materials from images. Materials such as wood and insulation can be distinguished, and the information obtained can be used as an index for real estate appraisal. It can also be used to detect pests such as termites. (Decision processing unit P9) An overall image 100 of the undercarriage, comparison information (101, 102) of a component 101, and the inspection results are displayed on a display (S12).

[0128] (Document creation processing section P10) Based on the inspection results, at least one of an inspection estimate and a work chart is created and stored in the storage device of the mobile terminal 2b or the external storage device 3c of the server 3 (S13). The recognition mode, which is set to either "undercarriage recognition mode 1" or "undercarriage recognition mode 2", is reset (S14). All processes are completed (S15).

[0129] The overall image of the undercarriage, the comparison information (101, 102) of the components 101, and the inspection results stored (S13) by the mobile terminal 2b as the fault location identifying terminal 2 can be accumulated, for example, in the external storage device 3c of the server 3. The server 3 receives information transmitted from a plurality of fault location identifying terminals 2, accumulates the information together with information such as the type of architectural structure, the name of the construction company or house manufacturer, the model, the year of registration, the age of construction, the prefecture of registration, and the maintenance history, and can manage the accumulated data as big data that can be statistically processed.

[0130] 11 is a hardware configuration diagram showing an example of the image generating device 8, the diagnosis device 9 using the image generating device, and the fault location identifying terminal 2 and data storage system equipped with these. These devices include an input device 11, an output device 12, a drive device 13, an auxiliary storage device 14, a main storage device 15, an arithmetic processing device 16, and an interface device 17, which are connected to each other by a bus B.

[0131] The input device 11 may be a keyboard, a mouse, etc., and is used to input various signals. The output device 12 may be a display device, etc., and is used to display various windows, data, etc. The interface device 17 may be a modem, a LAN card, etc., and is used to connect to a network.

[0132] In this embodiment, the image generating device 8, the diagnostic device 9 using the image generating device, the fault location identifying terminal 2 equipped with these, and the programs for executing each process as the data storage system are at least a part of various programs that control the processing in each device. The various programs are provided, for example, by distributing a recording medium 18 or by downloading from a network. The recording medium 18 on which such programs are recorded can be of various types, including recording media that optically, electrically, or magnetically record information, such as CD-ROMs, flexible disks, and magneto-optical disks, and semiconductor memories that electrically record information, such as ROMs and flash memories.

[0133] Programs for executing various processes are installed in the auxiliary storage device 14. The auxiliary storage device 14 stores the installed programs as well as necessary files, data, etc. The main storage device 15 reads out the programs that define the processes from the auxiliary storage device 14 and stores them. The arithmetic processing device 16 then realizes various processes according to the programs stored in the main storage device 15. [Industrial Applicability]

[0134] The image generating device, diagnostic device, and data storage system using the same according to the present embodiment can be used to check the underside of man-made objects or natural objects such as mountains, caves, etc. In particular, they can be used to check and inspect the underside of industrial products such as automobiles, ships, and aircraft, and man-made objects such as buildings, and to collect the obtained information. [Explanation of symbols]

[0135] 1. Data storage system 2. Fault location identification terminal 2a Same computer 2b Same as above (Mobile device) 3 Server 3a Same Internet 3b Same cloud 3c Same external storage device 4 Camera 5. Mobile 50 Same holder, 50a same operation lever 50b Same Light (Lighting control unit) 50c Distance measurement unit (distance measurement control unit) 51 Same Camera 51c Same recording control section 52 Same movement control unit (route calculation processing unit) 53 Communication device 54 Same Removable Media 54h Same Hardware Interface 54s same slot 54t Same connection terminal 6. Mobile devices 7. Traveling drive source (levitation movement source) 7a Same drive wheel 7b Same steering wheel 7c same swivel caster (ground wheel) 71 Rotary wing (fixed wing or hovercraft structure, etc.) 8 Image Generator P1 Same as Contour setting section P2 Parts Specifications Department P3 Map allocation processing unit P4 Same map setting processing section P5 Part information acquisition processing unit P6 Image generation processing unit 9. Fault Location Identification Program P7 Part comparison processing section P8 Parts Inspection Processing Section P9 Judgment processing unit P10 Document Creation Processing Unit 10 Observation Object 10a Same Automobile 10b Same building 10u Same bottom part 10h Same Suspended ceiling 10L Same Contour of observation range 10M Undercarriage Map 10S Section 10F Completed photoshoot area 10N Areas where photography has not yet been completed 100 Bottom image 101 Information on the photographed part 102 Standard Information Parts Information

Claims

1. An image generating device for generating a bottom image of an object to be observed, a contour setting unit that identifies a contour of an observation range of the observation target based on image data captured by an imaging device moving below the observation target, and sets an area within the contour as a contour area; The image generating device includes an image output unit that outputs, within the contour area, an image acquisition region for which image data has been acquired and an image non-acquisition region for which image data has not been acquired.

2. a part feature point extraction unit that extracts feature points indicated by parts constituting the observation target that are present in an observation target range based on the image data within the contour area; The image generating apparatus according to claim 1 , further comprising a part identifying unit that identifies a part from the extracted feature points.

3. a map allocation processing unit that allocates a map in which a plurality of partitions are arranged within the outline area set by the outline setting unit; a map setting processing unit that sets the image data in each section of the map to generate an underside image of the observation target; 3. The image generating apparatus according to claim 2, further comprising a part inspection processing unit for inspecting at least one of a shape, a color, a size and a position of each part identified by said part identification unit.

4. The contour setting unit is 2. The image generating device according to claim 1, wherein positions of the observation object that can be imaged within a certain distance range set for each observation object are extracted as feature points of a contour, and the feature points are connected linearly to identify the contour of the observation object range.

5. a part information acquisition processing unit that acquires a standard image of the part identified by the part identification unit; a part comparison processing unit that compares an image of the part identified by the part identification unit with a standard image of the part acquired by the part information acquisition processing unit; 3. The diagnostic device using an image generating device according to claim 2, further comprising a determination processing section that determines a part state of the part identified by said part identification section based on a comparison result in said part comparison processing section.

6. 6. The diagnostic device according to claim 5, further comprising a document creation processing unit that creates at least one of an estimate for costs required for updating the part status and a work sheet for the updating work, based on the part status of the part determined by the determination processing unit.

7. The image generating device according to claim 1 ; an image storage device that stores the bottom image generated by the image generating device; Equipped with The image generating device includes a bottom image transmitting unit, The image storage device is a data storage system having a memory unit that acquires a bottom surface image and stores the bottom surface image in association with specific information that identifies the observation subject from which the bottom surface image was generated.

8. Furthermore, A moving body equipped with a camera for photographing an object to be observed and having a driving source for traveling or a levitating moving source; 8. The data storage system according to claim 7, further comprising: a movement control unit that controls the movement of the moving body so that the camera photographs the observation range of the observation target.

9. 1. An image generating method for generating a bottom image of an object to be observed, the method comprising: a contour setting step of identifying a contour of an observation range of the observation target based on image data captured by moving below the observation target and setting an area within the contour as a contour area; a part feature point extraction step of extracting feature points indicated by parts constituting the observation target, which are present in an observation target range, based on the image data within the contour area; and a part identification step of identifying a part from the feature points.

10. An image generating program for generating a bottom image of an object to be observed, a contour setting step of identifying a contour of an observation range of the observation target based on image data captured by moving below the observation target and setting an area within the contour as a contour area; a part feature point extraction step of extracting feature points indicated by parts constituting the observation target, which are present in an observation target range, based on the image data within the contour area; and a part identification step of identifying a part from the feature points.

Images