Image generating device, diagnostic device using image generating device, data storage system, image generating method, and image generating program

Abstract

To provide an image generation device capable of implementing information processing equal to or more than information, which can be visually obtained by a person, by putting a movable camera into a dangerous space such as a narrow and dark space, a diagnosis apparatus using the image generation device, a data storage system, an image generation method and an image generation program.SOLUTION: An image generation program 8 comprises: a contour setting part P1 which identifies a contour 10L of an observation target range of an observation target 10u on the basis of data which are detected while moving under the observation target; a component identification part P2 which identifies a component by extracting feature points indicated by the component on the basis of image data; a map allocation processing part P3 for allocating a map 10M consisting of an array of a plurality of sections 10S inside of the identified contour of the observation target range; a map setting processing part P4 by which the image data are allocated to the sections of the map and integrated to a bottom face image 100 of the observation target; and a component inspection processing part P8 for determining and inspecting an abnormal location of each component identified by the component identification part P2.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an image generation device that can process image data of difficult-to-identify locations such as the undersides of vehicles, buildings, and other man-made objects, a diagnostic device that uses the image generation device, a data storage system, an image generation method, and an image generation program. [Background technology]

[0002] When calculating an estimated price for a regular automobile inspection, it is necessary to visually inspect the vehicle and each part to determine which parts need maintenance or replacement. In particular, when inspecting the underside of the vehicle, the vehicle must be moved onto a jack lift or into an underground pit, and after ensuring safety, a mechanic must get under the vehicle and visually inspect each part to determine which parts need maintenance or replacement. Similar inspection work is required when checking for defects in the vehicle and arranging for parts. The same is true when assessing the purchase price of a used car and setting a selling price.

[0003] In recent years, not only gasoline engines, diesel engines, and hybrid vehicles, but also hydrogen engines, fuel cell vehicles, etc. are being installed in each vehicle model, and the number of parts is increasing, and the structure is becoming more complex and diverse. In particular, new vehicles that have not been delivered for a long time or have a low mileage have an increasing number of inspection items, even though the probability of failure is relatively low, and the inspection workload of mechanics is increasing.

[0004] In addition, improvements in the working environment have led to shorter working hours for mechanics. Furthermore, with the decline in the working population, there are concerns about a shortage of mechanics. Similar issues exist not only in automobile maintenance, but also in the inspection of equipment under floors, ceilings, and above the ceiling of buildings.

[0005] For example, Patent Document 1 (JP 2005-200844 A) discloses a self-propelled cart for tunnel work, which includes a work deck located at the top of a road tunnel, multiple legs with wheels at the bottom, and a drive unit for supplying power to the wheels. The cart straddles both sides of the road tunnel in the width direction and is located in an area of ​​the road tunnel excluding the vehicle traffic area, allowing vehicles to pass inside. Even when the road tunnel is in use, work can be done without impeding vehicle traffic. Furthermore, because the cart is self-propelled, there is no need for cables or the like to supply power. Furthermore, a self-propelled cart for tunnel work is disclosed, which allows work to be done on the work deck, improving workability.

[0006] Furthermore, for example, the self-propelled pet camera shown in Patent Document 2 (JP 2022-15161 A) comprises a housing that moves on an installation surface on multiple wheels, a camera housed in the housing, a chassis that supports the camera fixed to the housing so that it can rotate freely, a wheel drive source that drives the wheels, a camera rotation drive source that rotates the camera, suction cups that adhere to the installation surface to prevent the housing from tipping over, an opening / closing valve that opens and closes an air vent drilled in the suction cup, and an opening / closing mechanism that drives the opening / closing valve, and has the potential to be applied as a technology for moving freely in narrow spaces such as under vehicles or under the floors of buildings and taking photographs. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-200844 [Patent Document 2] Japanese Patent Publication No. 2022-15161 Summary of the Invention [Problem to be solved by the invention]

[0008] The self-propelled cart for working inside tunnels described in Patent Document 1 is effective for inspecting and working on the underside of high places such as the ceiling of a tunnel, but is not effective for inspecting the underside of high places in narrow spaces such as under a vehicle, under the floor of a building, or in the attic (inside a suspended ceiling).

[0009] Furthermore, the self-propelled pet camera of Patent Document 2 is capable of photographing the underside of an object located above while moving within narrow spaces such as under a vehicle, under the floor of a building, or in the attic (inside a suspended ceiling), etc. However, the subject of the photograph is a moving pet, and the camera cannot be used as is to inspect the underside of a vehicle or building.

[0010] In view of the above circumstances, the present invention aims to provide an image generating device capable of generating a bottom image of a specified object, a diagnostic device using the image generating device, a data storage system, an image generating method, and an image generating program.

[0011] Another object of the present invention is to provide an image generation device, a diagnostic device using an image generation device, a data storage system, an image generation method, and an image generation program that can generate an image of the underside of a specified object even in places where a person would need large-scale facilities such as a jack lift or an underground pit to enter, or where a worker would have to risk crawling into a dark, narrow space. [Means for solving the problem]

[0012] In order to solve at least one of the above problems, the present invention provides an image generation device that can acquire image data with high accuracy by identifying contours, such as underside images of the undercarriage of a vehicle or underside images of a room space such as a ceiling surface, as well as a diagnostic device, a data storage system, an image generation method, and an image generation program that use the same. That is, the present invention provides an image generating device that generates an underside image of an object to be observed, and includes a contour setting unit that identifies the contour of the observation range of the object to be observed based on image data captured by an imaging device moving below the object to set the area within the contour as a contour area, and an image output unit that outputs, within the contour area, an image acquisition area for which image data has been acquired and an image non-acquisition area for which image data has not been acquired, and generates an underside image from image data acquired within the contour area.

[0013] The contour setting unit identifies the contour of the observation range at an early stage when the execution of the image generation program starts. The contour setting unit can identify the contour of the observation range based on distance measurement data measured by the distance measurement unit of the imaging device. The contour setting unit can also identify the contour of the observation range from differences in color, brightness, etc. based on image data captured by the camera of the imaging device.

[0014] The image generating device can include a component feature point extraction unit that extracts feature points indicated by components that constitute the object of observation and are present in the object of observation range based on the image data within the contour area, and a component identification unit that identifies the components from the extracted feature points.

[0015] The part identification unit identifies parts in the captured image of the object of observation by extracting feature points from the image data and identifying the part from those feature points. The feature points of a part, also referred to as feature information, can be extracted and identified by extracting characteristic elements from the image that are useful for identifying individual parts, such as the basic shape of each part common to various vehicles and the part dimensions that vary from vehicle to vehicle. For example, in the case of an exhaust pipe, the overall shape of the part can be determined from the image data based on the basic curved pipe shape, and the part dimensions can be read from the distance between the flanges connected by bolts and nuts. In the case of a suspension, the suspension system can be identified from the image data, and each component, such as coil springs, shock absorbers, lower arms, upper arms, bushings, boots, stabilizers, axles, oil pans, catalysts, mufflers, silencers, bolts and nuts, etc., can be identified.

[0016] The image generating device may also include a map allocation processing unit that allocates a map in which a plurality of sections are arranged within the contour area set by the contour setting unit, a map setting processing unit that sets the image data in each section of the map to generate an underside image of the object to be observed, and a part inspection processing unit that inspects at least one of the shape, color, size, and position of each part identified by the part identification unit.

[0017] The map allocation processing unit allocates a map consisting of an array of multiple sections within the contour of the observation range. The map sections may be a two-dimensional map arranged on a plane. The map sections may be, for example, rectangular squares, or may be equilateral triangles, parallelograms, regular hexagons, etc. The map sections may be a three-dimensional map arranged in a three-dimensional space. The map sections may be, for example, three-dimensional shapes such as cubes, equilateral triangular prisms, square prisms, regular hexagonal prisms, etc.

[0018] The map setting processing unit assigns the image data to each section of the map and integrates them into the bottom image of the observation object. The integrated bottom image can be assigned as a two-dimensional image to the two-dimensional map, or a three-dimensional image to the three-dimensional map. Also, the two-dimensional image can be processed and then assigned to the three-dimensional map.

[0019] The part inspection processing unit inspects each identified part, and compares image information such as the shape, dimensions, and color of each part with image samples of various fault conditions (part conditions) such as deformation, scratches, wear, cracks, breakage, deterioration, oil leaks, liquid leaks, and burnout, and if an image pattern matching any of the image samples is found, it can determine that the part is abnormal.

[0020] The contour setting unit extracts the positions of the observation objects that can be imaged within a certain distance range set for each observation object from the imaging device as characteristic points of the contour, and can identify the contour of the observation object range by connecting the characteristic points in a line.

[0021] Once the feature points of the contour of the observation range are set, the contour setting unit can determine the contour of the observation range. The contour setting unit can identify the contour of the observation range of the observation target based on distance measurement data measured by a distance measurement unit mounted on the imaging device while the imaging device moves under the observation target. The contour setting unit determines the difference in distance measurement data between the inside and outside of the contour of the observation range of the observation target based on the distance measurement data from the distance measurement unit, and extracts it as a feature point of the contour. The feature points indicating the contour of the observation range are point-like positions where the difference in distance between long and short distances in the distance measurement data changes greatly, and a series of these point-like positions can be recognized as the contour of the observation range.

[0022] The contour setting unit can determine the contour of the observation target range based on the distance range input to the distance range input unit. The distance range input unit sets and inputs the distance range from the imaging device to the observation target. The distance range input unit can be provided in a data storage system. The distance range input unit can be the imaging device, a remote control for the imaging device, or an input field displayed on the display of a fault location identification terminal. The distance range input unit can input the shortest distance and the longest distance from the imaging device to the observation target. In this case, anything between the input shortest distance and longest distance is recognized as the observation target.

[0023] The contour setting unit can determine the difference between the inside and outside of the contour of the observation range on the basis of image data of a still image or video captured by the camera, and extract the difference as a feature point of the contour. The feature points that indicate the contour of the observation range are defined as point-like positions where the brightness or color density in the image data changes significantly, and a series of these point-like positions can be recognized as the contour of the observation range.

[0024] The present invention also provides a diagnostic device using the image generating device. That is, the present invention provides a diagnostic device using the image generation device, which includes a part information acquisition processing unit that acquires a standard image of the part identified by the part identification unit, a part comparison processing unit that compares the image of the part identified by the part identification unit with the standard image of the part acquired by the part information acquisition processing unit, and a judgment processing unit that judges the state of the part identified by the part identification unit from the comparison result in the part comparison processing unit.

[0025] Furthermore, when a bottom surface information acquisition processing unit is provided that acquires a standard image of the bottom surface image of the observation target, it is possible to determine a defect state (a defect state that is an example of a component state) such as a change (misalignment) in the mounting position of each component or deformation by comparing the bottom surface image created by the image generation device with the standard bottom surface image acquired by the bottom surface information acquisition processing unit. Of course, it is possible to determine a normal state (a normal state that is an example of a component state) in which defects such as a change (misalignment) in the mounting position of each component or deformation do not occur, or a normal use state (a normal use state that is an example of a component state) in which these defects are not recognized.

[0026] The diagnostic device may also include a part inspection processing unit that inspects each part identified by the part identification unit, and an image generation processing unit that generates a superimposed image of the underside image of the observation target integrated by the map setting processing unit and the part image associated with the standard information of the part acquired by the part information acquisition processing unit. The judgment processing unit can also acquire, display, or save image inspection data including the underside image of the observation target, comparison information for each part, and inspection results.

[0027] The component information acquisition processor acquires standard information for the identified component, and the underside information acquisition processor acquires standard information for the underside image of the inspection target. The component standard information serves as the component inspection standard (comparison target) and can be information such as the numerical values, shape, and color of each component in the component's factory-shipped state (brand new). The underside image standard information can be a underside image of the component in its brand new state, or a underside image formed by combining common parts from underside images created for the same type of inspection target. This standard information can be acquired, for example, from a file stored in the fault location identification terminal or from another device connected via a network. Specifically, it can be acquired from a recording device that stores standard information associated with information identifying the inspection target, or by searching using information identifying the inspection target, such as a model number, as a search key. If standard information is not available, the state of deterioration, such as damage, cracks, or wear, can be determined from the captured image.

[0028] The image generation processing unit generates an overlaid image of the integrated bottom image of the observation target and a component standard image as standard information of the component. At least one of the generated overlaid images is made semi-transparent, allowing the overlaid images to be compared and visually confirmed for misalignment or deformation of each part. It is desirable that the orientation and scale ratio of the component standard image as standard information of the component be adjusted so that the orientation and outer shape of the component match the corresponding component in the bottom image.

[0029] The component comparison processing unit compares the image of the identified component with the acquired standard image of the component and determines the differences. The component comparison processing unit can distinguish the differences between the two image data. The component comparison processing unit can determine whether the placement position of the component in the bottom image matches the placement information of the acquired correctly placed component (whether the component is placed in the correct position). The part comparison processing unit can determine whether the shape and color of the part in the bottom image match the shape and color of the acquired normal (for example, new) part, or determine the extent of defects such as deformation, wear, cracks, fading, dirt, etc., and identify the degree of deterioration or damage of each part. In other words, it can determine the damage state (part condition) related to use of the part. The component comparison processing unit can, for example, permanently or temporarily store sample images (which can be images linked with information such as a description of the deterioration state, whether adjustment or replacement is necessary, etc.) of each component for each stage of deterioration to be compared, and select the sample image that is closest to the component in the bottom image, thereby determining the degree of deterioration of the component. The sample images for each stage of deterioration may be stored in a file in the fault location identification program, or may be obtained from, for example, a server, a website on the Internet, the cloud, etc.

[0030] The part inspection processing unit inspects each part based on the results of the part image comparison. Specifically, it can determine whether each part needs replacement or maintenance based on differences in its position, shape, dimensions, color, etc., to determine its condition. For example, the part inspection processing unit digitizes the position, shape, dimensions, color, etc. of each part in the bottom image, digitizes the position, shape, dimensions, color, etc. from the acquired standard information for each part, calculates the numerical difference, and determines whether maintenance is necessary based on a predetermined pass / fail standard value (e.g., in millimeters) for each part. For example, the part inspection processing unit compares the image data of the part in the bottom image with the standard image data of the part acquired by the part information acquisition processing unit using RGB values ​​in pixel units, performs correction processing to prevent subtle color differences from being detected as differences, and can control the display of parts with large differences in color by flashing red, or by encircling parts with large dimensional differences in a flashing red circle.

[0031] The information acquired by the judgment processing unit, such as the underside image of the observation target, the comparison information of each part, and the image inspection data including the inspection results, can be displayed on the display of the fault location identification terminal. Also, the various information acquired by the judgment processing unit can be saved in the imaging device, the fault location identification terminal, or an external storage device, another fault location identification terminal, a server, a website on the Internet, the cloud, or the like connected thereto.

[0032] The comparison information for a part can be information obtained by superimposing and comparing image data of the captured part with image data of the standard information for the corresponding part, or by comparing a numerical value extracted from the image data of the captured part with a corresponding numerical value in the standard information for the corresponding part.

[0033] The diagnostic device may also include a document creation processing unit that creates and stores at least one of an estimate and a work sheet based on the data stored in the determination processing unit.

[0034] The document creation processing unit includes a document creation processing unit that creates at least one of an estimate regarding the cost of updating the component status and a work sheet regarding the update work, based on the component status determined by the judgment processing unit. The document creation processing unit can output the created documents, such as the estimate or work sheet, to the display of the fault location identification terminal or output them as printed matter to a printer. The document creation processing unit may also electronically transmit digital information (files) of the created documents, such as the estimate or work sheet.

[0035] The present invention also provides a data storage system that utilizes the image generating device. That is, a data storage system is provided which comprises an image generating device and an image storage device which stores the bottom image generated by the image generating device, the image generating device comprising a transmission unit for the bottom image, and the image storage device comprising a storage unit which acquires the bottom image and stores it in association with specific information which identifies the observation object from which the bottom image was generated.

[0036] The multiple image generation devices can be used by various people in various places, regardless of the user or location. For example, the image generation devices may be installed in automobile repair shops nationwide and used by mechanics, sales representatives, etc. The image generation devices may also be used by maintenance workers at buildings and structures nationwide, sales representatives at home builders, etc. A data storage system can be configured using such multiple image generation devices, and the multiple image generation devices can be connected to a server and store various types of data in big data managed by the server, for example.

[0037] An image generation program can be installed in multiple image generation devices. Such an image generation program can be widely provided so that anyone can easily use it. For example, the image generation program can be made public and provided for free or for a fee via a distribution service on an Internet website, and various data acquired by the image generation program can be configured to be transmitted to and stored on a specific server via the Internet.

[0038] The server can receive and store image inspection data, including bottom images of the object to be observed, comparison information for each part, and inspection results, from multiple image generation devices. The server can store various data received from multiple image generation devices as big data. The big data stored in the server can be widely used for a variety of purposes, such as final inspections before delivery of new products, estimates for maintenance of used products, improving the efficiency of inspection and maintenance work, and price evaluations of used products.

[0039] The present invention also provides an imaging device for photographing the underside of an object to be observed, which can be used to obtain an underside image with the image generating device. That is, an imaging device is provided that is equipped with a camera that photographs the object to be observed, a moving body that has a running drive source or a floating moving source, and a movement control unit that controls the movement of the moving body so that the camera can photograph the observation range of the object to be observed. Such an imaging device has a holder section to which a mobile terminal can be attached and detached, and a mobile terminal equipped with application software as a movement control section and LIDAR (Light Detection and Ranging) can replace the camera and communication device or removable media and can be detachably mounted on the holder section. have,

[0040] The imaging device can be configured to capture images while moving under the object of observation, for example, by having a travel drive source so that it can travel on the ground. The imaging device can also be configured to have a levitation drive source so that it can travel in gas or liquid. Furthermore, the imaging device can be configured without a drive source so that it can be moved manually or by another drive source. The imaging device can also be equipped with an image processing device that has an image generation program and a fault location identification program.

[0041] The moving body can be formed by integrating the main components that make up the imaging device and having various outer shapes such as a skeleton frame, a housing, a capsule, etc. The moving body may have a track laid so as to travel thoroughly under the object to be observed, and a driving source for traveling on the track.

[0042] The traveling drive source enables the imaging device to travel on the ground, and can be, for example, a drive source such as a motor or engine, and wheels, caterpillars, or a combination thereof connected to the drive source. The traveling drive source may have a steering mechanism that steers under control. The levitating movement source enables the imaging device to levitate and move in gas or liquid, and can be, for example, a drive source such as a motor or engine, and a fixed wing, rotary wing, or hovercraft structure that generates lift using the driving force of the drive source. The levitating movement source may have a steering mechanism that steers under control.

[0043] The camera is used to capture an image of an object to be observed, and can be built into or integrally mounted on a mobile body, or can be detachably mounted on the mobile body. Alternatively, a camera built into a mobile terminal detachably mounted on the mobile body can be used.

[0044] The movement control unit controls the movement of the imaging device and can be configured to, for example, guide the imaging device to a route that allows efficient image capture (e.g., prioritize non-overlapping routes). The movement control unit can be a control unit for a driving source or a levitation source built into, for example, a moving body of the imaging device, or a program installed in the control unit. The movement control unit can also be a program installed in the image generating device, and can remotely control the driving source or the levitation source of the imaging device. The movement control unit can be configured with a receiving unit installed in the imaging device and a controller installed separately from the imaging device that transmits a movement control signal to the receiving unit wirelessly or via wire to remotely control the imaging device. The controller can be operated by artificial intelligence or by a person.

[0045] When the contour setting unit of the failure location identification program (image generation program) identifies the contour of the observation range of the observation target, the movement control unit can guide the mobile object to the shortest route along the contour of the observation range. For example, when the movement control unit detects a characteristic point that indicates the contour of the observation range, it can guide the mobile object to go around the contour of the observation range by following the characteristic points that are linearly connected to the characteristic point.

[0046] The communication device enables communication between the imaging device and the fault location identification terminal, and can communicate based on various wireless communication standards, such as Bluetooth (registered trademark, hereafter omitted), wireless LAN, and infrared communication. The removable media temporarily stores image data captured by the imaging device and various data acquired during the image capture process, and allows the stored data to be removed from the imaging device and carried around. Such removable media can be installed in the imaging device by inserting it into a slot or tray, or by connecting it to a connection terminal, and stores image data captured by the camera and various data acquired during the image capture process in real time.

[0047] The mobile terminal can be mounted on the imaging device and perform the functions of a camera, communication device, etc. It is desirable that the mobile terminal be small, thin, and lightweight enough to be mounted on the imaging device, and have a dustproof or liquid-proof seal structure. Such a mobile terminal can download application software that serves as a movement control unit and control the imaging device to move or float via wireless communication. It can also download an image generation program and a fault location identification program using the image generation program and operate according to the program. The mobile terminal can be equipped with the image generation program and the fault location identification program using the image generation program and operate under the control of the program.

[0048] The observation object is the object of observation by the image generating device, diagnostic device, and data storage system using the same of this invention. The observation object can be a downward-facing surface or an object arranged so as to be exposed to the downward-facing surface, and can exist in the air or in a liquid. Such observation objects can be various objects, such as the undercarriage of a vehicle, the underfloor of a building, the attic, the ceiling wall of a tunnel or cavern, the underside of the ceiling wall of a storage tank, etc.

[0049] The present invention also provides an image generating method for solving at least one of the above problems. That is, the present invention provides an image generation method that is executed by a computer and generates an underside image of an object to be observed, the image generation method comprising: a contour setting step that identifies the contour of the observation range of the object to be observed based on image data captured by moving below the object to set the area within the contour as a contour area; a component feature point extraction step that extracts feature points indicated by components that constitute the object to be observed and that are present in the observation range based on the image data within the contour area; and a component identification step that identifies the components from the feature points.

[0050] The present invention provides an image generating method for solving at least one of the above problems. That is, the present invention provides an image generation program for generating an underside image of an object to be observed, the image generation program comprising: a contour setting step for identifying the contour of the observation range of the object to be observed based on image data captured by moving under the object to set the area within the contour as a contour area; a component feature point extraction step for extracting feature points indicated by components that constitute the object to be observed and that are present in the observation range based on the image data within the contour area; and a component identification step for identifying the components from the feature points. [Effects of the Invention]

[0051] According to the image generating device of the present invention, the contour of the observation range of the observation target is identified, and after identifying the observation range, a bottom image is generated based on the captured image data, and the parts of the observation target are further identified, so that the parts of the observation target can be identified more efficiently. By allocating the captured image data to a map in which multiple sections are arranged within the contour of the observation range and integrating it into the bottom image, each part of the observation target and its arrangement can be identified more accurately.

[0052] According to the image generating device of the present invention, by grasping the contour of the observation range of the observation target in advance, subsequent photographing of the observation target can be performed more efficiently. When the contour setting unit extracts the boundary between the observation target located within a certain distance range from the imaging device and outside the observation target as contour feature points and identifies the contour of the observation target range by connecting the feature points in a line, the contour of the observation target range can be identified more efficiently and accurately by, for example, providing a distance measuring unit in the imaging device. Furthermore, when the contour of the observation target range is identified by extracting the boundary between light and dark color shading or brightness due to corners or edges of the observation target as contour feature points and connecting the feature points in a line, the contour of the observation target range can be identified more efficiently and accurately by, for example, using a camera mounted on the imaging device.

[0053] A diagnostic device using the image generating device of the present invention can generate a superimposed image of the underside image and standard information of the component acquired separately from the captured image data, allowing each component under observation to be visually confirmed along with the acquired standard information. Furthermore, the component image in the underside image can be superimposed on the acquired standard image of the component for direct comparison. The component image identified by the component identification unit is compared with the standard image of the component acquired by the component information acquisition processing unit, and based on the image matching results, image inspection data including the underside image of the component under observation, comparison information for each component, and inspection results can be more efficiently acquired. Furthermore, these various data can be quickly displayed on a display and stored in a storage medium as image data that is visually easy to view and intuitively understand. Necessary inspections can be performed more efficiently. The burden of inspection work can be reduced.

[0054] According to the diagnostic device of the present invention, estimates and work sheets can be quickly created and stored based on the data stored in the judgment processing unit, and the number of administrative steps required for quality control, delivery, estimates, maintenance and inspection, evaluation of used products, etc. can be significantly reduced.

[0055] The data storage system of the present invention allows various data generated by multiple image generation devices or fault location identification terminals to be stored in a server more efficiently. The various data stored in the server can be used as big data at each stage of the product distribution process, and can be used in a wide range of applications, such as for parts quality control and product development.

[0056] According to the data storage system of the present invention, the imaging device is provided with a holder unit to which a mobile terminal can be detachably attached. The mobile terminal is detachably mounted on the holder unit, so that the mobile terminal can replace the camera and communication device or removable media. When application software for image capture control and movement control is installed on the mobile terminal, the mobile terminal controls the image capture and movement of the imaging device. Furthermore, the LIDAR installed on the mobile terminal can measure the distance from the imaging device to the observation target, making it possible to identify the outline of the observation target range. For example, this reduces the labor required to lift a vehicle for vehicle inspection estimates, thereby reducing work hours and enabling more accurate estimates than visual inspections. [Brief explanation of the drawings]

[0057] [Figure 1] 10 is a flowchart of a fault location identification program using an image generation program. [Figure 2] 10 is a flowchart of a recognition process for an undercarriage and a component 101. [Figure 3] 10 is a flowchart of a process for recognizing the entire undercarriage. [Figure 4] 10 is a flowchart of a process for recognizing the undercarriage and the components 101 after grasping the entire undercarriage. [Figure 5] FIG. 1 is a conceptual diagram showing a data accumulation system for the undercarriage of an automobile 10a. [Figure 6] A conceptual diagram showing a data storage system in the attic of building 10b. [Figure 7] (a) A photograph showing an underside image 100 of the underside portion 10u of the object to be observed (automobile) integrated by the map setting processing unit P4. (b) A plan view showing the contour 10L of the underside portion 10u of the object to be observed (automobile) identified by the contour setting unit P1. (c) A two-dimensional map 10M assigned to the contour 10L of the underside portion 10u of the object to be observed (automobile) by the map assignment processing unit P3. (d) A two-dimensional map 10M in which the image generation processing unit overlays the underside image 100 of the object to be observed with the acquired standard information 102 of the component 101. [Figure 8](a) A perspective view showing an imaging device 4 having tracks 7a on which a mobile terminal 2b is mounted. (b) A perspective view showing an imaging device 4 having tracks 7a on which a camera 51 is mounted. (c) A three-sided view showing an imaging device 4 with three wheels (7a, 7b) on which a camera 51 is mounted. (d) A perspective view showing an imaging device 4 with multiple rotors 71. [Figure 9] (a) A perspective view showing a manually moved imaging device 4. (b) Front and side views showing a manually moved imaging device 4. (c) A front view showing an imaging device 4 equipped with multiple cameras 51. [Figure 10] (a) A bottom view showing the underside portion 10u of the roof of the building 10b. (b) A two-dimensional map 10M assigned to the contour 10L of the underside portion 10u of the roof of the building 10b. (c) An overlaid image of the two-dimensional map 10M and the underside image 100 of the underside portion 10u of the roof of the building 10b. [Figure 11] Hardware configuration diagram. EMBODIMENT 1 FOR CARRYING OUT THE INVENTION

[0058] Hereinafter, an image generating device, a diagnostic device using the image generating device, a data storage system, an image generating method, and an image generating program according to one embodiment of the present invention will be specifically described with reference to the drawings. In particular, in this embodiment, an automobile or a building is the object of observation, but other man-made objects or natural objects, etc. can also be the object of observation.

[0059] As shown in Figures 5 and 8(c), the data storage system 1 of this embodiment includes a fault location identification terminal 2 and a server 3. The data storage system 1 is provided with a moving object 5, a camera 51, a movement control unit 52, and an imaging device 4 having a communication device 53 or removable media 84. The imaging device 4 is set to have external dimensions (particularly height) that enable it to enter the space between the underside of a floor panel 10u of an automobile 10a to be observed and the tire contact surface (ground surface) as shown in Figure 5, or the space under the floor of a building 10u or the space above a suspended ceiling (attic) of the building as shown in Figure 6, and move freely in the horizontal direction.

[0060] 5 and 8(c), the imaging device 4 has a disk-shaped mobile body 5. The mobile body 5 is provided with a battery, a drive motor, left and right drive wheels 7a driven by the drive motor, and a steering wheel 7b steered by a servo motor. The drive motor and servo motor can be controlled by a movement control unit 52 mounted inside the mobile body 5.

[0061] As shown in Figures 8(a) and 8(b), the drive wheels 7a of the imaging device 4 can be, for example, caterpillar wheels. The drive wheels 7a can be provided with three or more caterpillar wheels, each with an independent suspension mechanism, and can be steered by being driven and controlled. The drive wheels 7a can be provided with, for example, four to eight caterpillar wheels, each with an independent suspension mechanism.

[0062] 8(d), the imaging device 4 may also have a levitation moving source 7. The levitation moving source 7 is a rotor 71 provided at the tip of each of a plurality of supports extending in the centrifugal direction on the same horizontal plane from the moving body 5, and for example, three to ten rotors 71 may be provided. The moving body 5 having the rotor 71 may then be provided with one or more ground contact wheels 7c.

[0063] As shown in Figures 5, 8(b) and 8(c), the camera 51 can be mounted, for example, on the top wall of the mobile object 5, with its optical axis facing vertically upward. The camera 51 has a recording control unit 51c that can capture still images or videos, and can be, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The mobile object 5 can be provided with one or more lights 50b that illuminate the subject 10u to be observed, and an illumination control unit that adjusts the brightness of the lights 50b.

[0064] The imaging device 4 preferably includes a distance measurement unit 50c and a distance measurement control unit for controlling the distance measurement unit 50c in the moving object 5. The distance measurement unit 50c may be, for example, a LIDAR, a SPAD (Single Photon Avalanche Diode), an optical sensor, a radio wave sensor, or an ultrasonic sensor.

[0065] The communication device 53 can, for example, wirelessly communicate with the fault location identification terminal 2, and can enable wireless communication between the movement control unit 52 and the recording control unit 51c and the fault location identification terminal 2. The communication device 53 can perform communication in accordance with communication standards such as wireless LAN (Wi-Fi) and Bluetooth, infrared communication, and other wireless communication.

[0066] 8(c), the imaging device 4 may include a hardware interface 54h for removable media (flash memory) 54. The hardware interface 54h may be, for example, a card slot 54s for an SD card, a USB connection terminal 54t for a USB memory stick, or the like.

[0067] The movement control unit 52 receives control signals output by the fault location identification terminal 2 via the communication device 53 and controls the drive wheels 7a and steered wheels 7b. For example, the fault location identification terminal 2 has a route calculation processing unit that receives image data captured by the camera 51 of the imaging device 4, stored in the recording control unit 51c, and transmitted via the communication device 53, and calculates a movement route based on the received image data. The route calculation processing unit transmits the calculated movement route data to the movement control unit 52 of the imaging device 4 via the communication device 53.

[0068] The movement control unit 52 may also have a route calculation processing unit that calculates a travel route that will allow for the most efficient shooting based on distance data measured by the distance measurement unit 50c and stored in the distance measurement control unit. In this case, the route calculation processing unit enables the movement control unit 52 of the imaging device 4 to autonomously drive, accelerate, decelerate, stop, and steer. Furthermore, the movement control unit 52 may also have a route calculation processing unit that calculates a travel route that will allow for the most efficient shooting based on image data captured by the camera 51 and stored in the recording control unit 51c. In this case, the route calculation processing unit enables the movement control unit 52 of the imaging device 4 to autonomously drive, accelerate, decelerate, stop, and steer.

[0069] The fault location identification terminal 2 can be configured by incorporating an image generation device 8 and a diagnosis device 9 utilizing the same. The image generation device 8 has, for example, an outline setting unit P1, a component identification unit P2, a map allocation processing unit P3, a map setting processing unit P4, a component information acquisition processing unit P5, and an image generation processing unit P6. The diagnosis device 9 has, for example, the image generation device 8, a component comparison processing unit P7, a component inspection processing unit P8, and a judgment processing unit P9. The diagnosis device 9 may further include a document creation processing unit P10. The fault location identification terminal 2 can be a computer 2a as shown in FIG. 5, or can be replaced with a mobile terminal 2b as shown in FIG. 6.

[0070] 8(a), the imaging device 4 is provided with a holder unit 50 to which a mobile terminal 6 can be detachably attached on the top wall of a moving object 5. The holder unit 50 can detachably mount the mobile terminal 6 equipped with at least one of a camera 51, a light 50b, a LIDAR (distance measurement unit) 50c, a movement control unit 52, a communication device 53, and a removable media 54.

[0071] In this case, the mobile terminal 6 can be used as an image generating device that has a contour setting unit that identifies the contour of the observation range of the observation object (frame information of the lower side of the observation object) based on image data captured by the moving body 5 while moving below the observation object, and sets the area within the contour as a contour area, and an image output unit that outputs an image acquisition area within the contour area where image data has been acquired and an image non-acquisition area where image data has not been acquired, and can further be used as an image generating device that generates an underside image from image data acquired within the contour area.

[0072] Furthermore, the mobile terminal 6 can also be used as a diagnostic device if it is equipped with a part identification unit that identifies parts from feature points based on the image data within the contour area, a part information acquisition processing unit that acquires a standard image of the part identified by the part identification unit, a part comparison processing unit that compares the image of the part identified by the part identification unit with the standard image of the part acquired by the part information acquisition processing unit, and a judgment processing unit that judges the part state of the part identified by the part identification unit from the comparison result in the part comparison processing unit.

[0073] The imaging device 4 may be a manual type, as shown in FIGS. 9(a), (b), and (c). The imaging device 4 is configured, for example, by providing swivel casters 7c near the left and right ends of the bottom of a rectangular mobile body 5 that is elongated from side to side. A control rod 50a extends from a position halfway between the left and right ends of the top wall of the mobile body 5, tilting backward. The control rod 50a may have a telescopic structure. Four cameras 51 are provided on the front wall of the mobile body 5, spaced equally between the left and right ends and facing diagonally upward and forward. As shown in FIG. 9(c), two or more cameras 51, e.g., three, three to eight, can be provided on the mobile body 5. The manual imaging device 4 is configured by providing the mobile body 5 with at least one of a light 50b, a distance measuring unit 50c, a communication device 53, and a removable media.

[0074] The manual imaging device 4 shown in FIG. 9 is operated by grasping the operating rod 50a, and the mobile body 5 is moved to below the observation target 10u shown in FIGS. 5 and 6. By rolling the swivel casters 7c on the ground, each camera 51 simultaneously captures images, allowing for more efficient imaging of a wide area. The multiple cameras 51 mounted on the mobile body 5 are preferably set to the same focal length and positioned so that the distance from the observation target 10u to each camera 51 is the same. Alternatively, the multiple cameras 51 may be set to long and short focal lengths, and positioned so that the distance from the observation target 10u to each camera 51 matches the focal length. This allows for simultaneous capture of wide-angle and magnified images. Furthermore, the multiple cameras 51 are positioned so that their optical axes are oriented in different directions, enabling faster, higher-quality 3D images to be captured.

[0075] Furthermore, a manual imaging device may have, for example, a holder portion at the tip of the operating rod to which a mobile terminal can be detachably attached, and the mobile terminal can be detachably attached to the holder portion. The operating rod of the manual imaging device is held like a selfie stick, and the mobile terminal can be brought close to the underside of a ceiling of a building or the underside of a car, etc., to take a photograph. As mentioned above, the mobile terminal can be used as an image generating device or a diagnostic device. MODE FOR CARRYING OUT THE INVENTION 2

[0076] (Automobile inspection process) Below, with reference to Figures 1 to 5 and Figures 7 to 9, the image generation device 8 and the diagnosis device 9 used in the automobile data accumulation system 1 will be described according to the flow of the inspection work of an automobile 10a using the data accumulation system 1.

[0077] When the fault location identification terminal 2 is started, an input screen for information identifying the automobile 10a, which is the object of observation 10, is displayed on the display of the computer 2a serving as the fault location identification terminal 2. On this screen, information effective for identifying the vehicle, such as the name, chassis number, model, engine model, type, and body shape of the automobile 10a, as well as vehicle inspection certificate information, etc., are input (S1).

[0078] When the imaging device 4 is placed on the ground below the underside 10u of the automobile 10a, which includes the floor panel, chassis frame, suspension, etc. to be observed, the imaging device 4 moves slowly (for example, 10 to 30 cm / s) in a random direction, and based on the image data captured by the mounted camera 51, identifies the position of the imaging device 4 relative to the underside 10u of the automobile 10a, and stores and recognizes the imaging start position (S2).

[0079] The imaging device 4 uses a sensor to determine whether it is located below the underside portion 10u of the automobile 10a (S3). The sensor can be a distance measurement unit 50c including a LIDAR, an optical sensor (e.g., an infrared sensor), a radio wave sensor (e.g., a transmitting / receiving antenna), an ultrasonic sensor, or the like. For example, the imaging device 4 can be configured to determine that it is located below the underside portion 10u when the distance from the imaging device 4 to the underside portion 10u is within 1 meter or within 50 cm. The imaging device 4 can also detect the brightness and color density of the image captured by the camera 51, and determine that it is located below the underside portion 10u when the image is darker or darker than the outside of the underside portion 10u.

[0080] If it is not possible to determine that the robot is located below the lower surface portion 10u for a certain period of time (for example, 10 to 30 seconds), the measurement is terminated, and the robot returns to the starting position (where it was originally placed) and stops (A) (S15). In this case, the robot changes course while continuing detection, and if it is possible to determine that the robot is located below the lower surface portion 10u by the time it returns to the starting position, the robot can be controlled to proceed to the next step (S5).

[0081] If it is determined within a certain period of time (for example, 10 to 30 seconds) that the object is located below the lower surface portion 10u, then the camera starts capturing moving images (or still images) (S5). The image capturing can be performed at a maximum speed of, for example, 30 to 500 cm / s while the vehicle is traveling and decelerating, making stops, U-turns, rotations, etc.

[0082] Then, it is determined whether or not "undercarriage recognition mode 2" is set (S6). If "undercarriage recognition mode 2" is set, recognition processing of the components 101 within the contour 10L of the observation object 10u is performed (S10).

[0083] (Contour setting section P1) On the other hand, if the "undercarriage recognition mode 2" is not set (S6), it is determined whether or not the edge portion of the undercarriage has been imaged (S7). If the edge portion of the undercarriage has not been imaged, recognition processing of the undercarriage and the component 101 is performed (S8).

[0084] (Undercarriage recognition mode 1) The recognition process (S8) for the undercarriage and components 101 sets "undercarriage recognition mode 1" (S8a). Undercarriage recognition mode 1 can be a recognition mode that, for example, identifies the outer periphery of the observation target 10 as the outline 10L of the observation target range, and then recognizes the components 101 (floor panel, chassis frame, suspension, bumper, side sill, fender, etc.) on the underside of the observation target. By recognizing the outline 10L of the observation target range in advance, it is possible to anticipate the arrangement of parts and further improve the detection accuracy and inspection accuracy of components such as parts.

[0085] When "undercarriage recognition mode 1" is set (S8a), for example, the camera 51 (recording control unit 51c), light 50b (illumination control unit), and driving source 7 (movement control unit 52) ​​can be set to conditions that make it easy to recognize the outer periphery of the component 101, and each can be controlled accordingly. When the camera 51 (recording control unit 51c) captures a characteristic point that indicates the outline 10L of the underside portion 10u, the driving source 7 can be controlled to slow down and travel slowly along the characteristic point, prioritizing the identification of the outline 10L (S8b). The driving source 7 can identify the tires, undercarriage components 101, etc., and move so as not to come into contact with them.

[0086] (Part identification section P2) Furthermore, based on the (two-dimensional or three-dimensional) image data of the camera 51, the components 101 present on the underside of the observation target 10 are recognized while identifying the outline 10L of the observation target range (S8c). The components 101 are recognized by identifying the feature points of each component 101, and information on the identified components 101 can be stored (S8d). The feature points of each component 101 can be determined and extracted based on, for example, the shape, size, color, arrangement, etc., which differ for each component 101. The components 101 can be, for example, exhaust pipes, catalysts, silencers, suspensions, axles, floor panels, bolts and nuts that connect them, and other components 101 that are visibly exposed on the underside of the vehicle body.

[0087] (Map allocation processing unit P3) When the imaging of the lower edge portion is completed (S7), the entire outline 10L of the observation range is recognized (S9). The outline 10L (lower edge) portion of the observation range is stored together with the position data of its characteristic points (S9a). Based on the stored position data of the characteristic points of the outline 10L, an "underside frame map 10M" is created (S9b) in which a two-dimensional map (or a three-dimensional map) consisting of a two-dimensional array (or a three-dimensional array) of multiple planar sections 10S (or three-dimensional sections) is assigned to the outline 10L of the observation range.

[0088] The "underbody frame map 10M" can be, for example, map data in which multiple arranged sections 10S are assigned to image data that has been generated and processed to be integrated into an underside image 100 based on imaging data of the object of observation 10.

[0089] (Undercarriage recognition mode 2) It is determined whether or not the creation of the "undercarriage frame map 10M" has been completed (S9c), and if it is incomplete, the process returns to determining whether or not it is located under the underside portion 10u of the automobile 10a (S3). On the screen of the fault location identification terminal 2, the sections 10F for which photographing has been completed and the sections 10N for which photographing has not yet been completed can be easily distinguished by displaying the sections 10N for which photographing has not yet been completed in color, and the sections 10F for which photographing has been completed can be displayed in white or black, or by pasting an image of the corresponding section 10S that has been photographed.

[0090] (Map setting processing section P4) When the creation of the "undercarriage frame map 10M" is completed, the inside of the "undercarriage frame map 10M" is set as the imaging range (S9d), and "undercarriage recognition mode 1" is set (S9e). "Undercarriage recognition mode 2" can be a recognition mode that identifies, for example, the underside portion 10u and each of its components 101 (floor panel, chassis frame, suspension, etc.), and enables inspection of each component 101 on the image data.

[0091] (Part information acquisition processing unit P5) Based on information input (S1) on an input screen for information identifying the automobile 10a, which is the object of observation 10, standard information 102 of the constituent elements 101 under the automobile 10a is acquired (S9f). The standard information 102 of the constituent elements 101 can be the part name, part number, assembly number, blueprint, service manual, parts list, and any associated information (dimensions, material), color, shape, two-dimensional (or three-dimensional) image, etc. By acquiring the standard information 102 in advance, it becomes possible to more accurately identify the constituent elements 101 based on that information.

[0092] The standard information 102 of the constituent element 101 can be acquired from information stored in, for example, the external storage device 3c of the server 3, or from data stored on the Internet 3a, the cloud 3b, etc. Acquisition of the standard information 102 of the constituent element 101 can be controlled, for example, by extracting an image of the constituent element 101 from a captured image, and then searching the stored data (2, 3, 3c) or the Internet 3a (3b) based on the image of the constituent element 101 to acquire the standard information 102 of the constituent element 101.

[0093] The "standard information 102 of the undercarriage constituent element 101" is overlaid on the "undercarriage frame map 10M," and the generated image data is saved (stored) (S9g). The generated image data can be displayed by overlaying an image of a translucent component 102 associated with the standard information 102 of the undercarriage constituent element 101 on the undercarriage frame map 10M. When the mouse pointer is placed over a component 102 displayed translucently on the display of the computer 2a, the component 102 starts blinking, and a small screen is displayed displaying the detailed standard information 102 of the component 102, such as numerical data and text information. When the mouse pointer is moved away from the component 102, the display returns to its original state. Similar operations are possible when the computer 2a is replaced with a mobile terminal 2b.

[0094] Based on the generated image data, it is determined whether or not standard information 102 for the positions and shapes of all components 101 identified and stored in "undercarriage recognition mode 1" is available (S9h). If standard information 102 for the positions and shapes of all components 101 is available, the position, shape, and color data of each component 101 is compared and collated with the standard information 102 for position, shape, and color acquired (S9f) (S9j). The collation results for each component 101 (102) are temporarily stored in, for example, the image capture device 4 (51c, 54) or the fault location identification terminal 2 (S9k).

[0095] When the standard information 102 for the position, shape, and color of some or all of the components 101 is not available (S9h), or after the collation results for all of the components 101 have been temporarily stored (S9k), the lighting intensity of the camera 51 (recording control unit 51c), the light 50b (illumination control unit), and the traveling speed of the traveling drive source 7 (movement control unit 52) ​​can be set and controlled to conditions that allow detailed capture of each component 101 in the underside portion 10u. When the camera 51 (recording control unit 51c) captures characteristic points that indicate each component 101 in the underside portion 10u, the speed of the traveling drive source 7 may be reduced and the traveling drive source 7 may travel slowly while processing the characteristic points, giving priority to identifying each component 101 (S9m).

[0096] (Image generation processing unit P6) After grasping the entire undercarriage (within the outline 10L of the underside portion 10u of the automobile 10a), a recognition process is performed on the undercarriage (within the outline 10L) and the components 101 within the outline 10L of the object of observation 10u (S10). The captured image is compared with an overlaid image to match each component 101 (S10a), and as a result of the matching, it is determined whether all components 101 have been recognized (S10b). The positions, shapes, and colors of all components 101 are permanently stored (S10c) in, for example, the imaging device 4 (51c, 54) or the fault location identification terminal 2. The permanently stored position information of the components 101 is more accurate than the temporarily stored position information.

[0097] Then, it is determined whether imaging of the entire imaging range has been completed (S10d). For example, as shown in Fig. 7(d), when the entire undercarriage frame map 10M displayed on the display of the computer 2a shows the section 10F for which imaging has been completed and the section 10N for which imaging has not yet been completed is not displayed, it is possible to determine whether imaging of the entire imaging range has been completed.

[0098] (Part comparison processing section P7) It is determined (S10e) whether all temporarily stored components 101 match the position, shape, and color of the temporarily stored (S9k) collation results of each component 101. Furthermore, it is possible to check whether the parts 102 of the standard information obtained before creating the vehicle underbody frame map 10M are included in the components 101 recognized after creating the underbody frame map 10M, thereby enabling a search for all components 101 without omission.

[0099] If any of the components 101 do not match the position, shape, or color temporarily stored (S9k), the positions, shapes, and colors of all the components 101 stored permanently (S10c) are superimposed on the positions, shapes, and colors of all the components 101 temporarily stored (S9k) and are compared on the display (S10f).The user's instructions based on the comparison result are reflected in the permanent storage, and the overlapping temporary storage is deleted (S10g).

[0100] (Parts inspection processing section P8) The components 101 are investigated (S11). The components 101 can be investigated by comparing the components 101 on the superimposed images, such as axle misalignment, deformation or cracks in the suspension, and oil leaks. The RGB values ​​of each component 101 are compared on a pixel-by-pixel basis, and components 101 with large differences are investigated. (Decision processing unit P9) The overall image of the undercarriage, comparison information of the components 101, and the inspection results are displayed on the display (S12).

[0101] (Document Creation Processing Unit P10) Based on the inspection results, an estimate for the cost of updating the part status of the parts and a work sheet for the updating work are created and stored in the storage device of the computer 2a or the external storage device 3c of the server 3 (S13). The recognition mode, which is set to either "undercarriage recognition mode 1" or "undercarriage recognition mode 2," is reset (S14). All processes are completed (S15). Update work is work to update the status of parts, such as work to modify, replace, or repair parts.

[0102] The overall image of the undercarriage, the comparison information (101, 102) of the component 101, and the inspection results stored (S13) by the computer 2a as the fault location identification terminal 2 can be stored, for example, in an external storage device 3c of the server 3. The server 3 receives information sent from a plurality of fault location identification terminals 2, stores it together with information such as the automobile manufacturer, car model, type, year, registration year, registration prefecture, mileage, and maintenance history, and can manage the stored data as big data that can be statistically processed. MODE 3 FOR CARRYING OUT THE INVENTION

[0103] (Building inspection process) 6 and 10, the image generating device 8 and the diagnostic device 9 used in the data storage system 1 of the building 10b will be shown according to the flow of inspection work of the walls, hanging fittings, air conditioning equipment, various piping, wiring, etc. of the underside portion 10u of the roof in the space above the suspended ceiling 10h of the building 10b using the data storage system 1.

[0104] When the fault location identification terminal 2 and the camera 4 are started up, an input screen for information identifying the building 10b, which is the observation target 10, is displayed on the display of the mobile terminal 2b acting as the fault location identification terminal 2. On this screen, information such as the type of architectural structure of the building 10b (wood, aluminum, light steel frame, heavy steel frame, reinforced concrete, steel reinforced concrete, concrete filled steel pipe structure, concrete block), the name of the construction company or house manufacturer, model, year of manufacture, year of registration, age, registered prefecture, address, GPS (Global Positioning System) data, blueprints, maintenance history, etc. is input (S1).

[0105] The imaging device 4 is placed on a suspended ceiling 10h below the underside 10u of the roof or upper floor of the building 10b to be observed. The imaging device 4 can be either an imaging device 4 having a traveling drive source 7 as shown in any of Figures 8(a) to 8(c), or an imaging device 4 having a levitating moving source 7 as shown in Figure 8(d). The imaging device 4 identifies its position relative to the underside 10u based on image data captured by the mounted camera 51 while traveling slowly (for example, 10 to 30 cm / s) in a random direction, and stores and recognizes the imaging start position (S2).

[0106] The imaging device 4 determines whether it is located below the lower surface portion 10u using the distance measurement unit 50c (S3). For example, it can be set to determine that it is located below the lower surface portion 10u when the distance from the imaging device 4 to the lower surface portion 10u is within 3 m or within 1 m. In addition, it can detect the brightness and color density of the image captured by the camera 51, distinguish the brightness and color difference from the outer vertical wall of the lower surface portion 10u, and determine that it is located below the lower surface portion 10u.

[0107] If it cannot be determined that it is located below the lower surface portion 10u for a certain period of time (for example, 30 seconds to 1 minute) or more, the measurement is terminated, and the robot returns to the starting position (where it was originally placed) and stops (S15). In this case, the robot changes course while continuing detection, and if it is determined that it is located below the lower surface portion 10u before returning to the starting position, the robot can be controlled to proceed to the next step (S5).

[0108] If it is determined within a certain time (e.g., 30 seconds to 1 minute) that the robot is positioned below the lower surface portion 10u, it starts capturing video (or still images) (S5). The capturing of video (or still images) can be performed at a maximum speed of, for example, 30 to 500 cm / s, while the robot moves with stops, U-turns, rotations, deceleration, etc.

[0109] It is determined whether or not "undercarriage recognition mode 2" is set (S6). If "undercarriage recognition mode 2" is set, recognition processing of the components 101 within the contour 10L of the observation object 10u is performed (S10).

[0110] (Contour setting section P1) If the "undercarriage recognition mode 2" is not set (S6), it is determined whether an image of an inner corner of the undercarriage has been captured (S7). If an image of an inner corner of the undercarriage has not been captured, a recognition process of the undercarriage and the component 101 is performed (S8).

[0111] (Undercarriage recognition mode 1) The recognition process (S8) of the undercarriage and the components 101 sets "undercarriage recognition mode 1" (S8a). The undercarriage recognition mode 1 can be, for example, a recognition mode that identifies the outer periphery of the undersurface portion 10u as the outline 10L of the observation target range.

[0112] When "undercarriage recognition mode 1" is selected, for example, the camera 51 (recording control unit 51c), light 50b (illumination control unit), and travel drive source 7 or levitation movement source 71 (movement control unit 52) ​​can be set to conditions that make it easier to recognize the outer edge of the underside portion 10u, and each can be controlled accordingly. When the camera 51 (recording control unit 51c) captures characteristic points that indicate the outline 10L of the underside portion 10u, the travel speed of the travel drive source 7 or levitation movement source 71 can be controlled to slow down and move slowly along the characteristic points, prioritizing the identification of the outline 10L (S8b). The surrounding pillars, walls, hanging fixtures, air conditioning equipment, piping, wiring, etc. are identified, and the vehicle moves in a way that avoids contact with them.

[0113] (Part identification section P2) Furthermore, based on the (two-dimensional or three-dimensional) image data of the camera 51, the components 101 of the observation target 10u are recognized while identifying the outline 10L of the observation target range (S8c). The components 101 are recognized by identifying the feature points of each component 101 and storing information about the identified component 101 (S8d). The feature points of each component 101 can be determined and extracted based on, for example, the shape, size, color, arrangement, etc., which differ for each component 101. The components 101 can be visible components 101, such as pipes, ducts, wiring, air conditioners, hanging hardware, pillars, beams, walls, etc.

[0114] (Map allocation processing unit P3) When the imaging of the edge portion of the undercarriage is completed (S7), the entire outline 10L of the observation range is recognized (S9). The outline 10L of the observation range (the corners of the undercarriage) is stored together with the position data of their characteristic points (S9a). Based on the stored position data of the characteristic points of the outline 10L, an "undercarriage frame map 10M" is created (S9b) in which a two-dimensional map (or a three-dimensional map) consisting of a two-dimensional array (or a three-dimensional array) of multiple planar sections 10S (or three-dimensional sections) is assigned to the outline 10L of the observation range.

[0115] As shown in FIGS. 10(a) and 10(b), the sections 10F for which imaging has been completed and the sections 10N for which imaging has not yet been completed can be displayed in different colors on the display of the mobile terminal 2b. For example, the imaging device 4 can capture images along characteristic points indicating the outer periphery of the underside portion 10u, capturing images that circle the outline 10L of the observation area. After the outline 10L of the observation area is captured, the sections 10F for which imaging has been completed are displayed in a line around the periphery of the displayed underside frame map 10M on the display of the mobile terminal 2b, with the sections 10N for which imaging has not yet been completed displayed as solid color images in the center. This allows the outline 10L of the observation area to be identified before detailed imaging of each component 101. After identifying the outline 10L of the observation area, the sections 10N for which imaging has not yet been completed are captured, and control can be exercised to ultimately display all sections 10S within the "underside frame map 10M" as shown in FIG. 10(c).

[0116] The "underbody frame map 10M" can be, for example, map data in which a plurality of arranged sections 10S are assigned to image data that has been generated and processed to be integrated into the underside image 100 based on the imaging data of the object of observation 10u.

[0117] (Undercarriage recognition mode 2) It is determined whether or not the creation of the "underbody frame map 10M" has been completed (S9c), and if it is not completed, the process returns to determining whether or not it is located below the lower surface portion 10u (S3).

[0118] (Map setting processing section P4) When the creation of the "undercarriage frame map 10M" is completed, the inside of the "undercarriage frame map 10M" is set as the imaging range (S9d), and "undercarriage recognition mode 1" is set (S9e). "Undercarriage recognition mode 2" can be a recognition mode that distinguishes, for example, the underside portion 10u from each of the components 101, such as pillars, walls, hanging hardware, air conditioning equipment, piping, and wiring, and enables inspection of each component 101 on the image data.

[0119] (Part information acquisition processing unit P5) Based on the information input (S1) on an input screen for information identifying the building 10b, which is the observation target 10, standard information 102 of the components 101 under the observation target 10 is acquired (S9f). The standard information 102 of the components 101 can be part names, part numbers, assembly numbers, blueprints, service manuals, parts lists, and associated information (dimensions, materials, parts list information), color, shape, two-dimensional (or three-dimensional) images, etc. The standard information 102 of the components 101 can be acquired from information stored in, for example, an external storage device 3c of the server 3, or from data stored on the Internet 3a, cloud 3b, etc.

[0120] 10(c), the "standard information 102 of the underbody components 101" is overlaid on the "underbody frame map 10M," and the generated image data is saved (S9g). The generated image data is displayed by overlaying an image of a semi-transparent component 102 linked to the standard information 102 of the underbody components 101 on the underbody frame map 10M, and when the mouse pointer is placed over a component 102 displayed semi-transparently on the screen, the component 102 changes to a blinking display, and a small screen displaying the detailed standard information 102 of the component 102, such as numerical data and text information, is displayed, and when the mouse pointer is moved away from the component 102, the display returns to the original display.

[0121] Based on the generated image data, it is determined whether or not standard information 102 for the positions and shapes of all components 101 (underside portions 10u) identified and stored in "underbody recognition mode 1" exists (S9h). If standard information 102 for the positions and shapes of all components 101 exists, the position, shape, and color data (underside portions 10u) of each component 101 is compared and matched with the standard information 102 for position, shape, and color acquired (S9f) (S9j). The matching results for each component are temporarily stored (S9k).

[0122] If there is no standard information 102 for the position, shape, and color of some or all of the components 101 (underside portion 10u) (S9h), or after the matching results for all of the components 101 have been temporarily stored (S9k), the lighting intensity of the camera 51 (recording control unit 51c) and light 50b (illumination control unit), the traveling speed of the traveling drive source 7 (movement control unit 52), or the traveling speed of the levitating moving source 71 can be set and controlled to conditions that allow detailed capture of each component 101 of the observation target 10u. If the camera 51 (recording control unit 51c) captures characteristic points that indicate each component 101 of the underside portion 10u, the speed of the traveling drive source 7 or the levitating moving source 71 can be reduced and the traveling drive source 7 or the levitating moving source 71 can be controlled to move slowly while processing the characteristic points, giving priority to identifying each component 101 (S9m).

[0123] (Image generation processing unit P6) After grasping the entire undercarriage (within the outline 10L of the undersurface portion 10u), a recognition process is performed for the undercarriage and the components 101 within the outline 10L of the undersurface portion 10u (S10). The captured image is compared with an overlaid image to match each component 101 (S10a), and as a result of the matching, it is determined whether all components 101 have been recognized (S10b). The positions, shapes, and colors of all components 101 are permanently stored (S10c). The permanently stored position information of the components 101 is more accurate than the temporarily stored position information.

[0124] It is determined whether imaging of the entire imaging range has been completed (S10d). For example, as shown in Fig. 6 and Fig. 10(c), when the section 10F for which imaging has been completed is displayed over the entire undercarriage frame map 10M displayed on the display of the mobile terminal 2b, and the section 10N for which imaging has not yet been completed is not displayed, it is possible to determine whether imaging of the entire imaging range has been completed.

[0125] (Part comparison processing section P7) It is determined (S10e) whether all the temporarily stored components 101 match the position, shape, and color of the temporarily stored (S9k) result of the comparison of each component 101. If any of the components 101 do not match the temporarily stored (S9k) position, shape, or color, all the components 101 that have been actually stored (S10c) are superimposed on the positions, shapes, and colors of all the components 101 that have been temporarily stored (S9k) and displayed for comparison on the display 2b (S10f). The user's instructions based on the comparison result are reflected in the actual storage, and the overlapping temporary storage is deleted (S10g).

[0126] (Parts inspection processing section P8) The components 101 are investigated (S11). The components 101 can be investigated by comparing components on the superimposed image 100 (101, 102) to identify misaligned pipes, deformations or cracks in walls or beams, water leaks, etc. The RGB values ​​of each component are compared on a pixel-by-pixel basis, and components 101 with large differences can be investigated.

[0127] It is possible to identify materials from images. Materials such as wood and insulation can be identified, and the information obtained can be used as an indicator for real estate appraisal. It can also be used to detect pests such as termites. (Decision processing unit P9) The overall image 100 of the undercarriage, the comparison information (101, 102) of the component 101, and the inspection results are displayed on the display (S12).

[0128] (Document Creation Processing Unit P10) Based on the inspection results, at least one of an inspection estimate and a work sheet is created and stored in the storage device of the mobile terminal 2b or the external storage device 3c of the server 3 (S13). The recognition mode, which has been set to either "undercarriage recognition mode 1" or "undercarriage recognition mode 2," is reset (S14). All processes are completed (S15).

[0129] The overall image of the undercarriage, the comparison information (101, 102) of the component 101, and the inspection results that are stored (S13) by the mobile terminal 2b as the fault location identification terminal 2 can be stored, for example, in the external storage device 3c of the server 3. The server 3 receives information transmitted from a plurality of fault location identification terminals 2, and stores it together with information such as the type of building structure, the name of the construction company or house manufacturer, the model, the year of registration, the age of the building, the registered prefecture, and the maintenance history, and can manage the stored data as big data that can be statistically processed.

[0130] 11 is a hardware configuration diagram showing an example of the image generating device 8, the diagnostic device 9 using the image generating device, the fault location identification terminal 2 equipped with these, and the data storage system. These devices are an input device 11, an output device 12, a drive device 13, an auxiliary storage device 14, a main storage device 15, an arithmetic processing device 16, and an interface device 17, all of which are interconnected by a bus B.

[0131] The input device 11 may be a keyboard, a mouse, etc., and is used to input various signals. The output device 12 may be a display device, etc., and is used to display various windows, data, etc. The interface device 17 may be a modem, a LAN card, etc., and is used to connect to a network.

[0132] In this embodiment, the image generating device 8, the diagnostic device 9 using the image generating device, the fault location identification terminal 2 equipped with these, and the programs that execute each process as a data storage system are at least a part of various programs that control the processing of each device. The various programs are provided, for example, by distributing a recording medium 18 or by downloading from a network. The recording medium 18 on which such programs are recorded can be of various types, including recording media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks, and semiconductor memories that record information electrically, such as ROMs and flash memories.

[0133] The programs that execute various processes are installed in the auxiliary storage device 14. The auxiliary storage device 14 stores the installed programs as well as necessary files, data, etc. The main storage device 15 reads out the programs that define the processes from the auxiliary storage device 14 and stores them. The arithmetic processing device 16 then realizes various processes in accordance with the programs stored in the main storage device 15. [Industrial Applicability]

[0134] The image generating device, diagnostic device, and data storage system using the same according to this embodiment can be used to check the underside of man-made objects or natural objects such as mountains, caves, etc. In particular, they can be used to check and inspect the underside of industrial products such as automobiles, ships, and aircraft, and man-made objects such as buildings, and to collect the obtained information. [Explanation of symbols]

[0135] 1. Data storage system 2. Fault location identification terminal 2a Same computer 2b Same as above, mobile device 3 Server 3a Same Internet 3b Same cloud 3c Same external storage device 4 Camera 5. Mobile 50 Same holder, 50a same operation lever 50b Same Light (Lighting control unit) 50c Distance measurement unit (distance measurement control unit) 51 Same camera 51c Recording control unit 52 Same movement control unit (route calculation processing unit) 53 Same communication device 54 Removable Media 54h Hardware Interface 54s same slot 54t same connection terminal 6. Mobile devices 7. Traveling drive source (levitation movement source) 7a Same drive wheel 7b Same steering wheel 7c Swivel Caster (Ground Wheel) 71 Rotary wing (fixed wing or hovercraft structure, etc.) 8 Image Generation Program P1 Same as above Contour setting section P2 Parts Identification Department P3 Map allocation processing unit P4 Map setting processing section P5 Part information acquisition processing unit P6 Image generation processing unit 9. Fault location identification program P7 Part comparison processing section P8 Parts Inspection Department P9 Same judgment processing unit P10 Document Creation Processing Unit 10 Observation Object 10a Same automobile 10b Same building 10u Same bottom part 10h Same suspended ceiling 10L Same as above Contour of observation area 10M Undercarriage Frame Map 10S Section 10F: Area where photography was completed 10N Area where photography has not yet been completed 100 bottom image 101 Information on the photographed part 102 Standard Information Part Information

Claims

1. An image generating device for generating a bottom image of an object to be observed, a contour setting unit that identifies a series of point-like positions where the difference between long and short distances in the distance measurement data changes as the contour of the observation object range, based on distance measurement data measured by a distance measurement unit mounted on the imaging device while the imaging device is moving below the observation object, and sets an area within the contour as a contour area; an image acquisition unit that acquires image data of the contour area set by the contour setting unit; an image output unit that outputs an image acquisition region in which image data has been acquired and an image non-acquisition region in which image data has not been acquired, of the contour area; a component feature point extraction unit that extracts feature points indicated by components that constitute the observation target and that exist within the observation target range of the contour area, based on the image data of the contour area; a component identification unit that identifies a component based on the extracted feature points; An image generating device comprising:

2. 1. An image generation method that is executed by a computer and generates a bottom image of an object to be observed, comprising: a contour setting step of identifying a series of point-like positions where the difference between long and short distances in the distance measurement data changes as the contour of the observation object range based on distance measurement data measured by a distance measurement unit mounted on the imaging device while the imaging device is moving below the observation object, and setting the area within the contour as a contour area; an image acquisition step of acquiring image data of the contour area set in the contour setting step; an image output unit that outputs an image acquisition region in which image data has been acquired and an image non-acquisition region in which image data has not been acquired, of the contour area; a component feature point extraction step of extracting, based on the image data of the contour area, feature points indicated by components that constitute the observation target and that exist within the observation target range of the contour area; a part identification step of identifying a part from the extracted feature points; An image generation method comprising:

3. An image generation program that is executed by a computer and generates a bottom image of an object to be observed, a contour setting step of identifying a series of point-like positions where the difference between long and short distances in the distance measurement data changes as the contour of the observation object range based on distance measurement data measured by a distance measurement unit mounted on the imaging device while the imaging device is moving below the observation object, and setting the area within the contour as a contour area; an image acquisition step of acquiring image data of the contour area set in the contour setting step; an image output unit that outputs an image acquisition region in which image data has been acquired and an image non-acquisition region in which image data has not been acquired, of the contour area; a component feature point extraction step of extracting, based on the image data of the contour area, feature points indicated by components that constitute the observation target and that exist within the observation target range of the contour area; a part identification step of identifying a part from the extracted feature points; An image generation program comprising:

Images