Image forming apparatus, image forming method, image forming program and lower surface image pickup program

Abstract

To provide an image forming apparatus capable of implementing image information equal to or more than information, which can be obtained by a human visual observation, by putting a movable camera into a narrow and dark space, an image forming method, an image forming program and a lower surface image pickup program.SOLUTION: A lower surface image pickup program controls image pickup of an image forming apparatus 1 which forms a lower surface image of an observation target. The lower surface image pickup program includes an image pickup control step ST3 of selecting or setting a permission or refusal of pickup of the lower surface image in the image forming apparatus 1. The lower surface image pickup program also includes: a lower surface image acquisition step ST9 of acquiring the picked-up lower surface image; a vehicle diagnosis step ST9 of diagnosing a vehicle on the basis of the acquired lower surface image; an image pickup time identification step ST6 of identifying an image pickup possible time on the basis of a pickup situation (place, service, facility, etc.,) of the lower surface image; and a diagnosis content identification step ST8 of identifying a content of the diagnosis executed in the vehicle diagnosis step on the basis of the image pickup time.SELECTED DRAWING: Figure 12

Description

Technical Field

[0001] The present invention relates to an image forming apparatus, an image forming method, an image forming program, and an underside image capturing program that can process image data of the underside portion of a vehicle, a building, or other artifacts.

Background Art

[0002] When calculating the estimate price for regular inspections of automobiles, etc., it is necessary to visually check the vehicle body and each component to confirm the parts that need maintenance and replacement. In particular, when checking the underside of the vehicle body, the target vehicle needs to be moved onto a jack lift or into a pit, ensuring safety, and then a mechanic gets under the vehicle body to visually check each component and confirm the parts that need maintenance and replacement. The same confirmation work is required when checking for vehicle defects and arranging parts. The same applies when conducting used car purchase appraisals or setting selling prices.

[0003] In recent years, not only gasoline engines, diesel engines, and hybrid vehicles, but also various components are installed for each vehicle type such as hydrogen engines and fuel cell vehicles. The number of components has increased, the structure has become more complex and diversified. In particular, for vehicles with a short number of years since delivery as new cars or vehicles with a short mileage, although the failure probability is relatively low, the number of inspection items has increased, and the inspection workload of mechanics has increased.

[0004] In addition, with the improvement of the working environment, the working hours of mechanics have been shortened. Furthermore, with the decrease in the working population, a shortage of mechanic personnel is feared. The same problems exist not only in automobile maintenance but also in the inspection of facilities under the floor, in the ceiling space, or in the attic of buildings.

[0005] For example, Patent Document 1 (Japanese Patent Application Laid-Open No. 2005-200844) discloses a self-propelled tunnel working carriage having a working deck disposed at the upper part within a road tunnel, a plurality of legs each provided on both sides in the width direction of the working deck and having wheels at the lower part, and a driving unit for supplying power to the wheels. The carriage straddles both side portions in the width direction of the road tunnel and is disposed in a region excluding the vehicle passage area within the road tunnel, allowing vehicles to pass through the inside thereof. Even when the road tunnel is in use, work can be carried out without obstructing vehicle passage. Also, since it is self-propelled, there is no need for a cable or the like for supplying power. Further, since work is carried out on the working deck, workability can be improved.

[0006] Also, for example, the self-propelled camera for pets shown in Patent Document 2 (Japanese Patent Application Laid-Open No. 2022-15161) includes a housing that travels on an installation surface with a plurality of wheels, a camera housed in the housing, a chassis that rotatably supports the camera fixed to the housing, a wheel drive source for driving the wheels, a camera rotation drive source for rotating the camera, a suction cup that adsorbs to the installation surface to restrict the tipping of the housing, an opening / closing valve for opening and closing a ventilation hole formed in the suction cup, and an opening / closing mechanism for driving the opening / closing valve to open and close. It has the potential to be applied as a technology for freely moving and photographing in narrow spaces such as under vehicles or under the floors of buildings.

Prior Art Documents

Patent Documents

[0007]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0008] The self-propelled tunnel working carriage of Patent Document 1 is effective for inspection and work on the lower surface part at a high place such as the ceiling surface of the tunnel. However, it is not suitable for inspecting the lower surface part located above in a narrow space such as under a vehicle, under the floor of a building, or in the ceiling space (false ceiling).

[0009] In addition, the self-propelled camera for pets in Patent Document 2 is for pets that move around, and is not suitable for inspecting the underside of vehicles or buildings.

[0010] An object of the present invention is to provide an image forming apparatus, an image forming method, an image forming program, and an under-surface image capturing program that can process image data of the under-surface portions of vehicles, buildings, and other artifacts.

[0011] Another object of the present invention is to provide an image forming apparatus, an image forming method, an image forming program, and an under-surface image capturing program that can form an under-surface image of a predetermined object even in a place that requires a large-scale facility such as a jack lift or a pit, or a place where a worker has to risk entering a dark and narrow space.

[0012] Furthermore, an object of the present invention is to provide an image forming apparatus, an image forming method, an image forming program, and an under-surface image capturing program that can perform a diagnosis of the underside of a vehicle used by a user who has obtained prior permission, even at a gas station, a shopping mall, or other parking or charging spaces where vehicles regularly stop, such as an EV charging station.

Means for Solving the Problems

[0013] In order to solve at least one of the above problems, the present invention provides an image forming apparatus, an image forming method, an image forming program, and an under-surface image capturing program that can acquire image data with high accuracy by specifying the contour of an under-surface image such as the underside of a vehicle or an under-surface image of a room space such as a ceiling surface. That is, in the present invention, there is provided an image forming apparatus that forms a bottom surface image of an observation object. Based on image data captured by moving an imaging device below the observation object, a contour setting unit identifies the contour of an observation target range in the observation object and sets the area within the contour as a contour area. An image output unit outputs an image acquisition area where image data is acquired and an image non-acquisition area where image data is not acquired within the contour area. An image forming apparatus that forms a bottom surface image from the image data acquired within the contour area is provided.

[0014] The contour setting unit identifies the contour of the observation target range at an initial stage when starting the execution of the image forming program. The contour setting unit can identify the contour of the observation target range based on distance measurement data measured by the distance measurement unit of the imaging device. Further, the contour setting unit can identify the contour of the observation target range from differences in color, brightness, etc. based on image data captured by the camera of the imaging device.

[0015] The above image forming apparatus can include a component feature point extraction unit that extracts feature points indicated by components constituting the observation object existing in the observation target range based on the image data within the contour area, and a component identification unit that identifies components from the extracted feature points.

[0016] The component identification unit identifies components on the captured image of the observation target, and based on the image data, extracts the feature points indicated by the components of the observation target, and can identify the components from the feature points. The feature points indicated by the components are also referred to as feature information, and the characteristic elements on the image that are effective for identifying individual components, such as the basic shapes common to various vehicles of each component and the component dimensions that vary for each vehicle, can be extracted and identified. For example, in the case of an exhaust pipe, from the image data, the overall shape of the component can be determined from the pipe shape whose basic shape is continuous with a curve, and further, the component dimensions can be read from the distance between the flange portions connected by bolts and nuts. In the case of a suspension, the suspension system can be identified from the image data, and for each of its components, for example, coil springs, shock absorbers, lower arms, upper arms, bushings, boots, stabilizers, axles, oil pans, catalysts, mufflers, silencers, bolts and nuts, etc. can be discriminated.

[0017] The image forming apparatus may further include a map assignment processing unit that assigns a map in which a plurality of sections are arranged within the contour area set by the contour setting unit, a map setting processing unit that sets the image data in each section of the map to form a bottom surface image of the observation target, and a component inspection processing unit that performs at least one inspection of the shape, color, size, and position of each component identified by the component identification unit.

[0018] The map assignment processing unit assigns a map composed of an arrangement of a plurality of sections within the contour of the observation target range. The sections of the map can be a two-dimensional map arranged on a plane. The sections of the map can be, for example, rectangular grid-like, and can also be equilateral triangles, parallelograms, regular hexagons, etc. The sections of the map can be a three-dimensional map arranged in a three-dimensional space. The sections of the map can be, for example, three-dimensional shapes such as cubes, regular triangular prisms, square prisms, regular hexagonal prisms, etc.

[0019] The map setting processing unit assigns the image data to each section of the map and integrates it into the bottom surface image of the observation target. For the integrated bottom surface image, a two-dimensional image can be assigned to the two-dimensional map, and a three-dimensional image can be assigned to the three-dimensional map. Also, a two-dimensional image can be processed and assigned to the three-dimensional map through image processing.

[0020] The component inspection processing unit inspects each identified component. Based on the image information such as the shape, dimensions, and color of each component, it compares with the image samples of various failure states (component states) such as component deformation, scratches, wear, cracks, breakage, deterioration, oil leakage, liquid leakage, and burning, and when it discovers an image pattern that matches any of these image samples, it can determine that there is an abnormality in the component.

[0021] The contour setting unit extracts, from the imager, the position of the observation target that can be imaged within a certain distance range set for each observation target as the feature points of the contour, and can identify the contour of the observation target range by linearly connecting these feature points.

[0022] When the feature points of the contour of the observation target range are set, the contour setting unit can determine the contour of the observation target range. The contour setting unit can identify the contour of the observation target range of the observation target based on the distance measurement data measured by the distance measurement unit mounted on the imager while the imager moves under the observation target. The contour setting unit determines the difference in the distance measurement data between the inside and the outside of the contour of the observation target range based on the distance measurement data of the distance measurement unit, and extracts it as the feature points of the contour. The feature points indicating the contour of the observation target range are the dot-like positions where the difference in the length of the distance on the distance measurement data changes greatly, and the succession of these dot-like positions can be recognized as the contour of the observation target range.

[0023] The contour setting unit can determine the contour of the observation target range based on the distance range input by the distance range input unit. The distance range input unit sets and inputs the distance range from the imaging device to the observation target. The distance range input unit can be provided in the data storage system. The distance range input unit can be an input field or the like displayed on the imaging device, the remote control of the imaging device, or the display of the failure location identification terminal. The distance range input unit can input the shortest distance and the longest distance from the imaging device to the observation target. In this case, what exists between the input shortest distance and the longest distance is recognized as the observation target.

[0024] The contour setting unit can determine the difference in the image between the inside and the outside of the contour of the observation target range based on the still image or moving image data captured by the camera, and extract it as a feature point of the contour. The feature point indicating the contour of the observation target range is a dot position where the brightness or darkness, or the color shade, etc. on the image data changes greatly, and the continuous connection of these dot positions can be recognized as the contour of the observation target range.

[0025] And in the present invention, a diagnostic apparatus using the image forming apparatus is provided. That is, a diagnostic apparatus using the image forming apparatus is provided, which includes a component information acquisition processing unit that acquires a standard image of the component specified by the component specifying unit, a component comparison processing unit that compares the image of the component specified by the component specifying unit with the standard image of the component acquired by the component information acquisition processing unit, and a determination processing unit that determines the state of the component specified by the component specifying unit from the comparison result in the component comparison processing unit.

[0026] In the case of including a lower surface information acquisition processing unit that acquires a standard image of the lower surface image of the observation target, by comparing the lower surface image created by the image forming apparatus with the standard image of the lower surface image acquired by the lower surface information acquisition processing unit, it is possible to determine a defective state such as a change (shift) in the mounting position or deformation of each component (a defective state which is an example of the component state). Of course, it is possible to determine a normal state (a normal state which is an example of the component state) in which there are no changes (shifts) in the mounting positions or deformations of each component or a normal use state (a normal use state which is an example of the component state) in which these defects are not recognized.

[0027] In such a diagnostic apparatus, it is also possible to include a component inspection processing unit that inspects each component specified by the component specifying unit, and an image forming processing unit that forms a superimposed image of the lower surface image of the observation target integrated by the map setting processing unit and the component image in which the standard information of the component acquired by the component information acquisition processing unit is associated. Further, the determination processing unit can acquire image inspection data including the lower surface image of the observation target, comparison information of each component, and the result of the inspection, and display or save it.

[0028] The component information acquisition processing unit acquires the standard information of the specified component, and the lower surface information acquisition processing unit acquires the standard information of the lower surface image of the inspection target. The standard information of the component is information serving as a reference (comparison target) for inspecting the component, and can be information such as the numerical values, shapes, colors, etc. of each part of the component in the factory shipment state (new product). Further, the standard information of the lower surface image can be the lower surface image in the state where the inspection target is new, or can be a lower surface image formed by combining common parts in the lower surface images created for the same type of inspection target. These standard information can be acquired, for example, from a file stored in a failure location specifying terminal or from another device connected via a network. Specifically, it can be acquired from a recording means in which the standard information is recorded in association with information specifying the inspection target, and can be acquired by searching using information such as the model number specifying the inspection target as a search key. In addition, when the standard information cannot be obtained, it is also possible to determine a deterioration state such as breakage, crack, wear, etc. from the captured image.

[0029] The image formation processing unit forms a superimposed image of the integrated bottom surface image of the observation target and the component standard image as the standard information of the component. At least one of the formed superimposed images can be semi-transparent, and the misalignment and deformation of each part can be visually confirmed by comparing the superimposed images. It is desirable that the orientation and scale ratio of the standard image of each component are adjusted so that the orientation and outer shape of the component as the standard information of the component match the corresponding component in the bottom surface image.

[0030] The component comparison processing unit compares the image of the identified component with the acquired standard image of the component and determines the different parts. The component comparison processing unit can discriminate the difference between the two image data. The component comparison processing unit can determine whether the arrangement position of the component in the bottom surface image matches the acquired arrangement information of the correctly arranged component (whether it is arranged in the correct position). The component comparison processing unit determines whether the shape and color of the component in the bottom surface image match the shape and color of the acquired normal (e.g., new) component, or to what extent defects such as deformation, wear, cracks, fading, and dirt have progressed, and can identify the degree of deterioration and damage of each component. That is, the damaged state (component state) of the component due to use can be determined.

[0031] The component comparison processing unit can always or temporarily store, for example, sample images for each stage of deterioration to be compared for each component (which can be images associated with information such as the description of the deterioration state, the necessity of adjustment or replacement), and select the sample image closest to the component in the bottom surface image to determine the degree of deterioration of the component. The sample images for each stage of deterioration are stored in a file in the failure location identification program, and can also be obtained from, for example, a server, a website on the Internet, the cloud, etc.

[0032] The component inspection processing unit inspects each component based on the comparison result of the images of the components. Specifically, it can determine whether each component needs to be replaced or maintained based on differences in the placement position, shape, dimensions, color, etc. of each component. For example, the component inspection processing unit digitizes the placement position, shape, dimensions, color, etc. of each component in the bottom surface image, digitizes the placement position, shape, dimensions, color, etc. from the standard information of each acquired component, calculates the numerical difference, and can determine whether maintenance is required based on the preset pass / fail judgment reference value (e.g., in mm units) for each component. The component inspection processing unit, for example, compares the image data of the components in the bottom surface image with the standard image data of the components acquired by the component information acquisition processing unit in terms of the RBG values in pixel units, performs correction processing so as not to detect subtle color differences as differences, controls to display components with large differences as blinking red, or surrounds and displays parts with large dimensional differences of the components with a blinking red circle.

[0033] Information such as the bottom surface image of the observation target, the comparison information of each component, and the image inspection data including the inspection results acquired by the judgment processing unit can be displayed on the display of the failure location identification terminal. Also, various information acquired by the judgment processing unit can be stored in an imaging device, the failure location identification terminal, or an external storage device, another failure location identification terminal, a server, a website on the Internet, the cloud, etc. connected thereto.

[0034] The comparison information of the components can be the information obtained by overlapping and comparing the image data of the imaged components with the image data of the standard information of the corresponding components. Also, the comparison information of the components can be the information obtained by comparing the numerical values extracted from the image data of the imaged components with the corresponding numerical values of the standard information of the corresponding components.

[0035] Further, the diagnostic device can be provided with a document creation processing unit that creates and stores at least one of an estimate and a work order based on the stored data of the judgment processing unit.

[0036] The document creation processing unit includes a document creation processing unit that creates at least one of an estimate regarding the cost of the update work for the component state and a work ticket regarding the update work based on the component state of the component by the determination processing unit. The document creation processing unit can output the created documents such as estimates and work tickets to the display of the failure location specifying terminal or print them as printed materials by a printer. The document creation processing unit may also electronically transmit the digital information (files) of the created documents such as estimates and work tickets.

[0037] In addition, the present invention provides a data storage system using the image forming apparatus. That is, there is provided a data storage system including an image forming apparatus and an image storage apparatus that stores a bottom surface image formed by the image forming apparatus, the image forming apparatus including a transmission unit for the bottom surface image, and the image storage apparatus including a storage unit that acquires the bottom surface image and holds it in association with specific information for specifying an observation target for which the bottom surface image was formed.

[0038] The plurality of image forming apparatuses can be used by various people in various places regardless of the user or location. For example, the image forming apparatus may be installed in automobile repair factories nationwide and used by mechanics, sales representatives, etc. Also, the image forming apparatus may be used by maintenance workers of buildings and structures nationwide, sales representatives of house builders, etc. A data storage system can be configured using such a plurality of image forming apparatuses, and the plurality of image forming apparatuses are connected to a server. For example, various data can be stored in the big data managed by the server.

[0039] An image forming program can be installed in the plurality of image forming apparatuses. Such an image forming program can be widely provided so that anyone can easily use it. For example, the image forming program can be publicly provided for a fee or free via a distribution service on an Internet website, and various data acquired by the image forming program can be configured to be transmitted and stored in a specific server through the Internet.

[0040] The server can receive and store image inspection data including the bottom surface image of the observation target, comparison information of each component, and inspection results from a plurality of image forming apparatuses. The server can store various data received from a plurality of image forming apparatuses as big data. The big data stored in the server can be widely used for various purposes such as final inspection before delivery of new products, estimation of maintenance of used products, improvement of efficiency of inspection and maintenance work, and price evaluation of used products.

[0041] And the present invention also provides an imaging device for photographing the underside of the observation target. Such an imaging device can be used to acquire the bottom surface image by the image forming apparatus. That is, an imaging device is provided which includes a mobile body having a camera for photographing the observation target and a traveling drive source or a floating movement source, and a movement control unit for controlling the movement of the mobile body so that the camera photographs the observation target range of the observation target. Such an imaging device has a holder part to which a mobile terminal can be detachably attached, and an application software as a movement control unit and a mobile terminal equipped with LiDAR (Light Detection and Ranging) can replace the camera and a communication device or a removable medium and can be detachably mounted on the holder part.

[0042] The imaging device can be configured to photograph while moving under the observation target, for example, having a traveling drive source and traveling on the ground. Further, the imaging device can be configured to have a floating movement source and move in gas or liquid. Furthermore, the imaging device can be configured to be moved manually without a drive source or by other drive sources. And the imaging device can be equipped with an image processing device having an image forming program and a failure location specifying program.

[0043] The mobile body can integrate the main components constituting the imaging device and be formed in various outer shapes such as a skeletal frame, a housing, a capsule, etc. The mobile body may have a track laid without gaps under the observation target and a traveling drive source that travels on the track.

[0044] The traveling drive source enables the imaging device to travel on the ground. For example, it can be a drive source such as a motor or an engine, and wheels, an endless track, or a combination thereof connected to the drive source. The traveling drive source may have a steering mechanism that is steered under control. Also, the floating drive source enables the imaging device to float and move in the air or in a liquid. For example, it can use a drive source such as a motor or an engine, and a fixed wing, a rotary wing, or a hovercraft structure that generates lift using the driving force of the drive source. The floating drive source may have a steering mechanism that is steered under control.

[0045] The camera is for photographing the observation target. For example, it can be built-in or integrally mounted on the mobile body, or can be detachably mounted on the mobile body. Also, a camera built into a mobile terminal detachably mounted on the mobile body can be used.

[0046] The movement control unit controls the movement of the imaging device. For example, it can be configured to guide the imaging device to an efficient photographing path (e.g., giving priority to non-overlapping paths). The movement control unit can be a control unit of a traveling drive source or a floating drive source built into the mobile body of the imaging device, or a program installed on the control unit. The movement control unit can also be a program installed on an image forming device and can remotely operate the traveling drive source or the floating drive source of the imaging device. The movement control unit can be composed of a receiving unit provided on the imaging device and a controller provided separately from the imaging device that wirelessly or wiredly transmits a movement control signal to the receiving unit for remote operation. The controller can be operated by artificial intelligence or by a person.

[0047] When the contour setting unit of the failure location identification program (image formation program) identifies the contour of the observation target range of the observation target, the movement control unit can guide (movement control) the moving body along the shortest path along the contour of the observation target range. For example, when the movement control unit detects a feature point indicating the contour of the observation target range, it can trace the feature points linearly connected to the feature point and guide (movement control) the moving body to go around the contour of the observation target range.

[0048] The communication device enables communication between the imaging device and the failure location identification terminal, and can communicate based on various wireless communication standards such as Bluetooth (registered trademark, hereinafter omitted), wireless LAN, and infrared communication. The removable media temporarily stores the image data captured by the imaging device and various data acquired during the imaging process, and can take out and carry the stored data from the imaging device. Such removable media can be mounted by being attached to a slot or tray of the imaging device or connected to a connection terminal, and stores in real time the image data captured by the camera and various data acquired during the imaging process.

[0049] The mobile terminal can be mounted on the imaging device and undertake functions such as a camera and a communication device, and is preferably small, thin, and lightweight enough to be mounted on the imaging device, and has a dustproof seal structure or a liquidproof seal structure. Such a mobile terminal can also download application software (hereinafter also referred to as "app") serving as the movement control unit, and control the imaging device to travel or float via wireless communication. In addition, it can download an image formation program and a failure location identification program using the same, and operate according to those programs. The mobile terminal can mount an image formation program and a failure location identification program using the same, and operate under the control thereof.

[0050] The observation target is an object to be observed in the image forming apparatus, diagnostic apparatus, and data storage system using the same of the present invention. The observation target can be a downward-facing surface and an object arranged to be exposed on the downward-facing surface, and can exist in the air or in a liquid. Such observation targets can be, for example, various objects such as under the vehicle, under the floor of a building, in the attic, the ceiling in a tunnel or cave, the lower surface portion of the ceiling of a storage tank, etc.

[0051] Furthermore, the present invention provides an image forming apparatus for solving at least any one of the above problems. That is, in an image forming apparatus that forms a downward image of an observation target imaged by an imaging device, it can be connected to a moving means (mobile body) capable of moving in the space between the observation target and the ground surface in contact with the observation target, and an imaging setting unit capable of setting whether to permit imaging of the downward image, a state identification unit that identifies operation information regarding the operation state of the observation target, and when imaging is permitted by the imaging setting unit and the observation target is in a state where it can be imaged based on the operation information identified by the state identification unit, a movement control unit that controls the movement of the moving means (mobile body) in the space, and an imaging control unit that controls the imaging of the downward image by the imaging device when the movement control unit controls the movement of the moving means (mobile body). An image forming apparatus is provided.

[0052] The image forming apparatus forms, as an image capable of identifying and diagnosing, an overall image of the area under the observation target that can be used for diagnosing the area under the observation target, and each component appearing in the area under the observation target, based on the downward image of the observation target imaged by the imaging device. The image forming apparatus is, for example, a mobile terminal or computer equipped with the image forming program of the present invention and having acquired the downward image of the imaged observation target, or a server or computer that has received information on the downward image.

[0053] The imaging device captures the downward image of the observation target. The imaging device can capture at least one of a still image, a moving image, a 2D image, a 3D image, etc. The imaging device can be a mobile terminal, a tablet PC, a digital camera, etc. The imaging device can, for example, be wirelessly communicated with a device that forms part of an image forming device after an application or program software of the present invention is installed.

[0054] The imaging device can be connected to a moving means (mobile body) that can move in the space between the observation target and the ground surface of the observation target. The moving means (mobile body) is equipped with a driving source and a drive control unit, etc. that carry the imaging device and move in the space, or can be something like a selfie stick that carries the imaging device and moves in the space by the manual operation of the user.

[0055] The imaging setting unit can set whether to permit the imaging of the downward image. The imaging setting unit obtains permission from the user of the observation target for taking a downward image of the observation target. The imaging setting unit is incorporated into the image forming program of the present invention which is software. The image forming program can be installed and used on a mobile terminal used by the user of the observation target. When the image forming program is started for the first time, a screen for requesting permission for photographing the observation target is displayed. If the user sets a rejection, the image forming program ends or does not perform photographing of the observation target and utilization of the photographing information. If the user sets permission, the image forming program enables photographing of the observation target.

[0056] The state identification unit identifies operation information regarding the operation state of the observation target. The state identification unit can identify whether the observation target is stopped or moving based on a real-time captured image (video) of the observation target captured by the imaging device. The state identification unit communicates with the observation target to obtain information such as the speedometer, integrated mileage meter information, GPS information, and information that can determine the moving state such as a G-sensor of the observation target, and can identify the operation state of the observation target. The state identification unit obtains the position information of a mobile terminal or the like held by a user riding on the observation target (for example, a vehicle), refers to the moving speed (for example, 20 km / h or more) and the current position (for example, a lane or a parking lot), and can identify the operation state of the observation target.

[0057] The movement control unit controls the movement of the movement means (mobile body) in the space. The movement control unit functions when imaging is permitted to be set by the imaging setting unit and the observation target can be imaged based on the operation information identified by the state identification unit. The movement control unit does not function when imaging is not permitted to be set or is rejected by the imaging setting unit. The movement control unit does not function when the observation target cannot be imaged based on the operation information identified by the state identification unit. The movement control unit forms a map based on the information of the lower image of the observation target, associates the map with the position of the movement means (mobile body), and controls the movement means (mobile body) to move freely in the space corresponding to the map without contact with the observation target. The movement control unit is provided with various sensors such as an infrared sensor and a G-sensor on the movement means (mobile body), receives the information of the various sensors of the movement means (mobile body) in real time, and controls the movement means (mobile body) to move while maintaining a non-contact state with the observation target.

[0058] When the movement control unit controls the movement of the moving means (mobile body), the imaging control unit controls the imaging of the lower-round image by the imaging device. When the movement control unit is not controlling the movement of the moving means (mobile body), the imaging control unit stops controlling the imaging of the lower-round image. The imaging control unit cooperates with the movement control of the moving means (mobile body) by the movement control unit to perform shooting control. When the movement control unit is moving along the lower contour of the observation target, the imaging control unit preferentially captures an image of the lower contour of the observation target. When the movement control unit is controlling the movement, the imaging control unit grasps the imaged sections and the non-imaged sections, transmits the position information of the non-imaged sections to a position where they can be imaged to the movement control unit, and the movement control unit controls the movement of the moving means (mobile body) to the requested position to improve the efficiency of imaging.

[0059] Further, the present invention provides an image forming apparatus that calculates the time for imaging the underside of a vehicle. The image forming apparatus includes a time specifying unit that specifies the service time of a provided service provided in a state where the observation target can be imaged, and the movement control unit moves the moving means (mobile body) until the elapse of the service time specified by the time specifying unit after the state where the imaging is possible.

[0060] The time specifying unit specifies the service time. The time specifying unit specifies the service time of a provided service provided in a state where the observation target can be imaged. When the observation target is a vehicle, the time specifying unit specifies the service time provided for various services received by the observation target or the user for each type of facility, such as a gas station, an EV charging station, a shopping mall, a department store, a convenience store, a library, a store, a ferry (onboard), other parking or stopping spaces, a drive-through route where a vehicle temporarily stops such as a fast food restaurant, a car wash machine at a car wash, etc.

[0061] The time specifying unit is provided in a computer, a mobile terminal, or the like on which the image forming program of the present invention is installed. Information on the service time specified by the time specifying unit can be specified without requiring external information based on usage history information of various facilities recorded in a computer mounted on the observation target (vehicle) or a mobile terminal possessed by the user. Further, it can be specified from stay time information (external information) recorded by a web service (external information) associated with the observation target (vehicle) or the user's mobile terminal, based on the stay history of the observation target (vehicle) or the user at each facility. The external information can be acquired, for example, through a communication device of the observation target (vehicle) or the user's mobile terminal, and through a wireless LAN, the Internet, or the like.

[0062] The movement control unit controls the movement of the moving means (moving body). The movement control unit moves the moving means (moving body) from the state where imaging is possible until the service time specified by the time specifying unit has elapsed. The movement control unit can, for example, end the movement of the moving means (moving body) simultaneously with the end of the specified service time. The movement control unit can, for example, end the movement of the moving means (moving body) with a margin of time before the end of the specified service time. The movement control unit can end the movement of the moving means (moving body) 1 to 10 minutes (margin of time) before the end of the specified service time. Ending the movement of the moving means (moving body) with this margin of time can enhance safety.

[0063] The present invention also provides an image forming apparatus that estimates, using AI (that is, "artificial intelligence"; the same applies hereinafter), from history information the time for imaging the underside of a vehicle. That is, the time specifying unit includes a time estimating unit that estimates, from the content of the provided service, the time from the start of the provision of the provided service until the end of the provision of the provided service. The time estimating unit provides an image forming apparatus that estimates the time until the end of the provision using provision history information on the provided service provided so far.

[0064] The time estimation unit estimates the time (or time of day) until the end of the provided service. The time estimation unit estimates the time from the start of the provided service to the end of the provided service based on the content of the provided service. The time estimation unit is an AI installed in the observation target (vehicle) or the user's mobile terminal, or an AI such as a cloud or web service. The time estimation unit estimates the stay time based on various conditions such as the average stay time of various facilities, the user's usage conditions (average usage time), and the usage purpose on that day. In addition, the time estimation unit can obtain (external information) and specify from cloud or web services managed by each facility, such as the average stay time information or predicted stay time information of customers of each facility (predicted information taking into account conditions such as date, day of the week, holiday, weather, temperature, etc.). These external information can be obtained through the communication device of the observation target (vehicle) or the user's mobile terminal, and through wireless LAN, the Internet, etc.

[0065] The present invention also provides an image forming apparatus that obtains observation target information, diagnoses the observation target, and outputs work information such as repair and replacement based on the diagnosis. That is, it includes an acquisition unit that acquires observation target information regarding the observation target, a state diagnosis unit that diagnoses the state of the observation target based on the lower part image, and a diagnosis information output unit that outputs diagnosis information regarding the state diagnosis. The diagnosis information output unit outputs diagnosis information regarding the components constituting the observation target included in the lower part image, and work information for performing at least one of repair and replacement of the components based on the diagnosis information, and provides an image forming apparatus.

[0066] The acquisition unit acquires information on the object to be observed. The acquisition unit acquires information for identifying the manufacturer name, model, type name, etc. of the object to be observed, the shape, structure, dimensions, color, skeleton, frame, outer wall, panel, member, component, etc. attached to the underside of the object to be observed. The acquisition unit is a part of the computer installed with the image formation program of the present invention on the object to be observed or the user's mobile terminal, and the user can input information on the object to be observed into the input field displayed by the image formation program. The acquisition unit can acquire information on the object to be observed recorded in the storage unit of a computer installed on the object to be observed, or a server, cloud, web service, etc. managed by a manufacturer or maintenance factory through wired, wireless, Internet, etc. The acquisition unit can acquire the image of the underside of the object to be observed in a new state, the image of the components in a new state provided on the underside of the object to be observed, and detailed information such as their shape, dimensions, and materials. In addition, the acquisition unit can also acquire information on common components and related information on components whose overall shapes are similar but whose partial shapes, dimensions, materials, colors, etc. are different. Information on similar components can be used to prevent misrecognition of components and to use common components. Furthermore, the acquisition unit can acquire images, shapes, and dimension information of worn, aged, or damaged components of the object to be observed. Information on worn, aged, or damaged components serves as comparison information when the state diagnosis unit performs state diagnosis.

[0067] The state diagnosis unit performs state diagnosis on the object to be observed based on the image of the underside of the object to be observed. The state diagnosis unit performs state diagnosis by comparing the image of the underside of the object to be observed captured with the image information of a new product acquired by the acquisition unit. The state diagnosis unit performs state diagnosis by comparing the image information of the components of the object to be observed with the image information of new components. The state diagnosis unit performs state diagnosis by comparing the image information of the components of the object to be observed with the image information of worn, aged, or damaged components. The state diagnosis unit performs state diagnosis by referring to information such as the dimensions, shapes, and mounting postures of the components in addition to the image information of the components.

[0068] The diagnostic information output unit outputs diagnostic results based on the underbody image and information such as repairs and replacements of parts determined to be necessary based on the diagnostic results. The diagnostic information output unit outputs diagnostic information regarding the parts constituting the observation target included in the underbody image and work information for performing at least one of repair and replacement of the parts based on the diagnostic information. The diagnostic information output unit can transmit and output diagnostic information, repair and replacement estimates, etc. to a computer, mobile terminal, etc. mounted on the user's observation target. The diagnostic information output unit can output specific information of the observation target (such as vehicle inspection information), repair and replacement part information, etc. to a server that manages big data. The diagnostic information output unit can transmit and output to, for example, the server of a repair shop responsible for the maintenance of the observation target, the PCs and mobile terminals of sales staff and maintenance staff.

[0069] Furthermore, the present invention provides an image forming apparatus that images the underbody of a vehicle in multiple times. That is, the imaging control unit includes a determination unit that determines whether all the underbody images inside the contour of the observation target have been imaged by the imaging device. Before the imaging device images all the underbody images inside the contour, when the state discrimination unit determines that the observation target is not in a state where it can be imaged, the imaging is stopped and the captured partial underbody images are stored. In a state where the partial underbody images are stored, when the state discrimination unit determines that the observation target is in a state where it can be imaged, control is performed to image other partial underbody images different from the partial underbody images inside the contour, providing an image forming apparatus.

[0070] The determination unit determines whether all the underbody images inside the contour of the observation target have been imaged. The determination unit stores part or all of the underbody images in a mobile terminal possessed by the user or a server that can be wirelessly connected to the mobile terminal, and can read the stored underbody images, and overwrite, delete, edit, etc. the stored underbody images.

[0071] Furthermore, the present invention provides an image forming method in which the image forming apparatus obtains permission from a user to capture an image, and while ensuring safety according to the operating state of an observation target, realizes imaging under the observation target. That is, an image forming method realized in a computer which is an image forming apparatus that forms a downward image of an observation target captured by an imaging device, the computer being able to set whether to permit capturing of the downward image, identify operation information regarding the operating state of the observation target, when the capturing is permitted to be set and the observation target is in a state where it can be imaged based on the identified operation information, controlling the movement of a moving means (mobile body) capable of moving in the space between the observation target and the ground surface of the observation target in the space, and controlling the capturing of the downward image by the imaging device while controlling the movement of the moving means (mobile body). An image forming method is provided which executes the above.

[0072] The computer can set whether to permit capturing of a downward image, identify operation information regarding the operating state of the observation target, and only when the permission is set, the observation target can be imaged. The computer controls the movement of a moving means (mobile body) in the space under the observation target, and controls the capturing of the downward image of the observation target during the movement control of the moving means (mobile body). The computer can be a mobile terminal, a PC or a server remotely connected to an imaging device such as a camera. A mobile terminal equipped with the image forming program of this invention, a PC tablet, a PC with a camera, other terminals having an imaging function and equipped with the image forming program of this invention, or an external device such as a server equipped with the image forming program of this invention can communicate with each other, and can be an imaging device capable of capturing an image under the control of the external device.

[0073] Furthermore, the present invention provides an image forming program in which the image forming apparatus obtains permission from a user to capture an image, and while ensuring safety according to the operating state of an observation target, realizes imaging under the observation target. That is, a computer which is an image forming apparatus for forming an under-image of an observation target imaged by an imaging apparatus is caused to execute a setting step for making it possible to set whether or not to permit imaging of the under-image, an identifying step for identifying operation information regarding the operation state of the observation target, a movement control step for controlling the movement of a moving means (a moving body) capable of moving in the space between the observation target and the ground surface of the observation target when the imaging is permitted to be set and the observation target is in a state where it can be imaged based on the identified operation information, and an imaging control step for controlling imaging of the under-image by the imaging apparatus when controlling the movement of the moving means (the moving body). An image forming program is provided.

[0074] The setting step is a step for making it possible for a user of the observation target to set whether or not to permit imaging of the under-image. In the setting step, an input field for setting permission or non-permission can be displayed on the screen of terminals operated by the user, or a voice asking whether to permit the setting can be generated from a speaker. In the setting step, input operations by the user from various interfaces such as the touch panel, keyboard, and microphone of the terminals can be received, and it can be determined whether or not there is a setting of permission or non-permission. The setting of permission or non-permission of imaging confirms the intention of the user to reject imaging or the intention of the user to permit imaging. Furthermore, the setting of permission or non-permission of imaging may also be at least any one of the input settings of rejecting or permitting imaging by the user.

[0075] The identifying step identifies operation information regarding the operation state of the observation target. In the identifying step, based on the real-time imaging image, the current operation state of the observation target can be recognized, and it can be identified whether it is stopped or not. The identifying step can acquire real-time position information such as the GPS or G-sensor of the observation target, recognize the current operation state of the observation target, and identify whether it is stopped or not. The identifying step can acquire the position information of the observation target detected by monitoring cameras and various sensors installed in service facilities where the observation target has stopped or service facilities where it has parked, recognize the current operation state of the observation target, and identify whether it is stopped or not.

[0076] The movement control step is as follows: in the setting step, when the user sets the permission for imaging, and in the identification step, when it is identified that the observation target is in an imaging-enabled state, the movement means (mobile body) is controlled to move in the space between the observation target and the ground surface of the observation target. The movement control step can be performed, for example, when a movement control unit is provided in the movement means (mobile body) and a mobile terminal or server equipped with the image formation program of the present invention can receive an execution signal for the movement control step. The movement control unit that receives the execution signal identifies the contour under the observation target. Further, the movement control unit calculates a travel route that enables efficient imaging and controls the travel of the movement means (mobile body) according to the route.

[0077] The present invention also provides a bottom surface image capturing program that enables setting of permission for imaging and controls imaging according to the input of the permission setting. That is, a bottom surface image capturing program for controlling imaging in an image forming apparatus that forms a bottom surface image of an observation target, comprising at least an imaging control step of selecting or setting permission or rejection of imaging of the bottom surface image in the image forming apparatus.

[0078] The imaging control step is a process of obtaining permission from users, right holders, etc. of the observation target such as automobiles and buildings in advance when executing the image formation program. More specifically, it can be a process of obtaining permission for photographing the observation target in advance, a process of obtaining permission for using the image information under the photographed observation target, a process of obtaining permission for inspection based on the image information under the observation target, a process of obtaining permission for creating and transmitting an estimate based on the inspection data of the parts under the observation target, etc. Permission can also be obtained in detail regarding the shooting location, shooting timing, etc. For example, it can be permission for shooting in parking lots such as gas stations, EV charging stations, shopping malls, department stores, or in the drive-through routes where vehicles temporarily stop such as fast food restaurants and car washes. Also, a mobile terminal owned by a user who has installed the app of the bottom surface image capturing program can be mounted on the mobile body, and the area under the observation target can be scanned regardless of the location and time.

[0079] When the imaging control step executes an image formation program installed on a computer, mobile terminal, or the like, a screen for inputting various information of the observation target and the user (such as the manufacturer, model, vehicle inspection certificate information, etc. of the observation target) is displayed on the initial screen at the start of execution, and input to each input field can be requested. Furthermore, a cautionary message such as "When the user finishes inputting various information, the execution of the image formation program is permitted." can be displayed so that the input of various data can also serve as a permission operation. Also, on the initial screen at the start of execution of the image formation program, a check box is displayed for each permission item, and when the user checks it, a screen for inputting various information of the observation target and the user (such as the manufacturer, model, vehicle inspection certificate information, etc. of the observation target) can be displayed. The user is informed in advance of the location for scanning the underside of the vehicle. When the vehicle being observed is parked, stopped, or moving slowly at the scanning location, it can be regarded as the user having permitted the scan, and the scan can be performed. When the vehicle is parked, stopped, or moving slowly at the scanning location, permission for scanning from the user can be requested via an in-vehicle computer, mobile terminal, etc., and the scan can be started after permission is obtained.

[0080] The imaging control step selects or sets the permission or non-permission for imaging the bottom surface image. The imaging control step is provided in the bottom surface image imaging program. The imaging control step is a step of obtaining permission from the user to image the underside image of the observation target, and selects or sets the permission or rejection for imaging the bottom surface image. The imaging control step can display an input field or a setting field for inputting the selection of permission or rejection for imaging the bottom surface image on the screen of the terminal operated by the user. The imaging control step can generate a voice requesting the selection or setting of permission or rejection for imaging the bottom surface image from a speaker, earphone, headset, etc. of the terminal operated by the user, and request a voice response through the microphone. Voice input enables more secure response to the imaging control step even when the user is driving a vehicle or the like.

[0081] In addition, the present invention provides a bottom surface image capturing program that enables a user to specify a diagnosis menu according to the time when the vehicle to be observed receives another service other than vehicle diagnosis. That is, it includes a bottom surface image acquisition step of acquiring the captured bottom surface (underside) image, a vehicle diagnosis step of diagnosing the vehicle based on the acquired bottom surface image, an imaging time determination step of determining the imaging available time based on the imaging situation (location, service, facility, etc.) of the bottom surface (underside) image, and a diagnosis content determination step of determining the diagnosis content to be executed in the vehicle diagnosis step based on the imaging time, and provides a bottom surface image capturing program.

[0082] The bottom surface image acquisition step acquires the bottom surface (underside) image of the vehicle to be observed. The bottom surface image acquisition step is a step of moving a moving body equipped with a camera between the bottom surface of the vehicle and the ground contact surface of the vehicle, imaging the bottom surface of the vehicle, and acquiring the bottom surface (underside) image. The vehicle diagnosis step is based on the bottom surface (underside) image captured in the bottom surface image acquisition step, checks maintenance items such as part replacement and repair, and acquires information that can display and output a list of inspection items.

[0083] The imaging time determination step estimates or determines the time when the vehicle to be observed and its user stop or park the vehicle at the location where the bottom surface image acquisition step is executed. For example, the time when the vehicle to be observed and its user stop or park the vehicle can be estimated or determined from the average usage time, stay time, etc. of the visited facility. Also, the user can estimate or determine the time when the vehicle to be observed and its user stop or park the vehicle through an input field where the stay time can be input by the user himself, voice input, etc. The imaging time determination step can obtain information on the average stay time of general customers from a map service, etc. provided on the Internet for the facility visited by the vehicle to be observed and its user. The imaging time determination step accumulates the daily behavior history, etc. of the vehicle to be observed and its user, and based on the accumulated data, can estimate the stay time of the facility from the stay pattern and its statistics of the vehicle to be observed and its user.

[0084] The diagnostic content determination step determines the diagnostic content that can be executed within the residence time specified in the imaging time determination step based on the residence time. The diagnostic content determination step emphasizes safety and can prioritize diagnostic items for safety-critical components. For example, items such as around the brakes, hydraulic systems, suspension, fuel systems, electrical wiring, etc. can be prioritized, and items such as the frame and exhaust pipe can be given lower priority, and weighting can be performed according to the items. Also, when imaging and diagnosis are performed separately at multiple locations, when a certain period (1 week to 2 weeks) has passed since the first imaging, instead of only imaging and diagnosing the bottom (underside) images that have not been imaged or diagnosed, the area that was imaged initially can also be imaged with a coarser image quality, and while simply diagnosing for major damage and wear, the imaging and diagnosis of the bottom (underside) images that have not been imaged or diagnosed can be controlled. And when one to two months have passed since the first imaging, imaging and diagnosis can be started from the beginning again, and the image data and diagnostic data obtained for each imaging and diagnosis can be accumulated.

[0085] The location where the bottom image acquisition step of acquiring the bottom image can be a place where the vehicle is parked, stopped, or moving at a slow speed. For example, it can be a parking lot at a gas station, shopping mall, department store, etc., a traffic signal, a railroad crossing, etc., the route of a drive-through at a fast-food restaurant, or a congested area where one is waiting to enter a safari park. While the vehicle is parked, stopped, or moving slowly, a moving body equipped with a camera scans the underside of the vehicle. Also, the underside of the vehicle can be scanned at the timing of waiting for a traffic signal or a temporary stop at a railroad crossing. Furthermore, it can be scanned at the timing when the vehicle is simply parked or stopped, or when the vehicle is receiving various services such as refueling or charging.

[0086] The imaging time determination step is a process of grasping various situations such as the parking location of the vehicle, the services provided at that location, the purpose of using the parked facility, etc., and predicting the parking (staying) time. For location determination, various positioning technologies can be adopted, such as GPS (Global Positioning System), electromagnetic induction technology, 3D map data, Wi-Fi location information, RFID (Radio Frequency Identifier), beacon, geomagnetic positioning, acoustic wave positioning, UWB (Ultra Wide Band, ultra-wideband wireless), etc. And when parking at various facilities such as shopping malls, supermarkets, gas stations, libraries, coin parking lots, and other facilities, the staying time can be predicted based on the staying time information of the user recorded in the map application of the mobile terminal, etc. Also, when parking at an EV charging station, it is possible to communicate with the corresponding charging device to obtain the charging end time or obtain the charging time defined at the EV charging station. Additionally, the charging end time can be obtained from the computer installed in the vehicle.

[0087] The diagnosis content determination step is a process of determining the diagnosis items that can be performed within that time based on the parking (staying) time determined in the imaging time determination step. The diagnosis content determination step can select the diagnosis items that can be performed within that time according to their priorities based on the parking (staying) time. For example, it is possible to preferentially monitor important components for the safe driving of the vehicle, such as tires, axles, brakes, fuel tanks, engines, etc.

[0088] The present invention also provides a bottom surface image imaging program for performing diagnoses in multiple separate times. That is, the bottom surface image acquisition step and the vehicle diagnosis step are interlocked and executed multiple times, and a split holding step is provided for holding at least any one of the multiple executed bottom surface image acquisition steps and vehicle diagnosis steps in association with each other. A bottom surface image imaging program is provided.

[0089] If at least one of the lower surface image acquisition step and the vehicle diagnosis step cannot be completed by monitoring and diagnosing at one location, monitoring and diagnosis can be continued at multiple locations where the vehicle stops. When the vehicle moves to a shopping mall after refueling at a gas station, for the monitoring and diagnosis items that could not be completed at the gas station, based on the data stored in a holding means such as a storage device, monitoring and diagnosis can be continued at the shopping mall.

[0090] By performing the process related to the linkage between the underbody diagnosis and the services provided for each stop facility multiple times, a complete diagnosis process is carried out. The provisional diagnosis results up to a certain point are saved as a history for each user, and by aggregating these histories for each user, a complete diagnosis process is performed. More specifically, as a complete underbody diagnosis process, if there is a menu that lasts for a total of 15 minutes, a complete diagnosis process may be carried out by performing the linkage with the service provided 3 times (5 minutes × 3 times ⇒ 15 minutes) (the period from the first diagnosis to the last diagnosis is, for example, within 1 month). Of course, it may be divided like (3 minutes + 5 minutes + 10 minutes) where the working time is divided.

[0091] The present invention provides a lower surface image capturing device that enables a user of an observation target to select or reject permission for capturing a lower surface image of the observation target, or to set it, and further identifies the observation target and can hold and extract information of the observation target. That is, a lower surface image capturing device that controls imaging in an image forming device that forms a lower surface image of an observation target, comprising at least an imaging control unit that selects or sets permission or rejection of imaging of the lower surface image in the image forming device, a vehicle identification unit that identifies a vehicle that has captured a lower surface image, and a vehicle information holding unit that stores at least one of the lower surface image of each vehicle and the component information of the vehicle lower surface, and based on the vehicle identification information of the vehicle identified by the vehicle identification unit, a vehicle information extraction unit that extracts at least one of the lower surface image held by the vehicle information holding unit and the component information of the vehicle lower surface is provided.

[0092] The imaging control unit selects or sets the permission or rejection of imaging the bottom surface image in the image forming apparatus. Before starting the imaging of the bottom surface image, the imaging control unit can select or set the permission or rejection of imaging the observation target from a person who can permit imaging, such as the user, owner, or right holder of the observation target. At the stage of starting the use of the image forming apparatus, the imaging control unit can display a selection field for permission or rejection of imaging the observation target on a display connected to a mobile terminal or a computer. Also, an input field for setting the imaging of the observation target can be displayed at the same timing. When a person who can permit imaging of the observation target (for example, the user or owner of the observation target) selects permission or sets to permit imaging, control is performed to perform imaging based on this input. When a person who can permit imaging of the observation target selects rejection or sets to reject, control is performed not to perform imaging.

[0093] The vehicle identification unit sets identification information of the observation target for identifying the observation target for which the imaging control unit has obtained permission for imaging. The identification information of the observation target is information for identifying the vehicle or building of the observation target, such as the manufacturer name, model, and model year of the observation target. When the observation target is a vehicle, it is information effective for identifying the vehicle, such as its manufacturer name, model year, vehicle name, chassis number, model, engine model, type, and body shape, as well as vehicle inspection certificate information. Further, it can include information on parts outside the vehicle, such as non-genuine parts of the manufacturer incorporated in the vehicle of the observation target. The vehicle identification unit can be input and set by the vehicle user through an interface such as a mobile terminal or a computer. Also, the vehicle identification unit can communicate with a computer mounted on the vehicle of the observation target and obtain it.

[0094] The vehicle information storage unit stores at least one of the bottom surface image and the component information of the vehicle bottom surface for each vehicle identified by the vehicle identification unit. The vehicle information storage unit can be a storage unit mounted on the bottom surface image capturing device, or can be an external storage device capable of communicating via wireless communication or the like. The vehicle information extraction unit extracts at least one of the bottom surface image and the component information of the vehicle bottom surface held by the vehicle information storage unit. The vehicle information extraction unit can be configured using a control program for image processing or arithmetic processing mounted on the bottom surface image capturing device.

[0095] The present invention also provides a bottom surface image capturing device that handles information of so-called users such as users, owners, and right holders of an object to be observed. That is, further, a user information holding unit that holds user information of a user who owns or uses a vehicle, and a vehicle diagnosis unit that acquires the captured bottom surface image and diagnoses the vehicle based on the bottom surface image are provided, and a diagnosis result output unit that outputs the diagnosis result by the vehicle diagnosis unit to the user held by the user information holding unit is provided. A bottom surface image capturing device is provided.

[0096] The user information holding unit stores information for identifying an individual user. The user information holding unit can be a storage unit of the bottom surface image capturing device, or can be an external storage device capable of communication. The user information can be information input by the user himself / herself through an interface, or can be acquired from information stored in a customer list or the like recorded in a vehicle computer, the user's mobile terminal, or the storage device of an automobile repair shop where the user is registered.

[0097] The vehicle diagnosis unit acquires the captured bottom surface image and diagnoses the vehicle to be observed based on the bottom surface image. The vehicle diagnosis unit receives, for example, still image or moving image data captured by a camera, stores it in a storage unit inside and outside the bottom surface image capturing device, and diagnoses the vehicle based on the stored image data. The diagnosis result can be stored in a storage unit inside and outside the bottom surface image capturing device.

[0098] The diagnosis result output unit outputs the diagnosis result by the vehicle diagnosis unit to the user held by the user information holding unit. The diagnosis result output unit can notify the user by transmitting the diagnosis result stored in a storage unit provided inside the bottom surface image capturing device or an external storage unit to a mobile terminal, a computer, or an in-vehicle terminal of the user held by the user information holding unit. The in-vehicle terminal can transmit as character information, files, etc. through an e-mail or a browser displayed on a display provided with a touch panel.

Advantages of the Invention

[0099] According to the image forming apparatus of the present invention, after identifying the contour of the observation target range of the observation target and identifying the observation target range, a bottom surface image is formed based on the captured image data, and further the parts of the observation target are specified. Therefore, the parts of the observation target can be specified more efficiently. In addition, by assigning the captured image data to a map in which a plurality of sections are arranged within the contour of the observation target range and integrating it into the bottom surface image, each part of the observation target and its arrangement can be specified more accurately.

[0100] In addition, by enabling the imaging setting unit to set whether imaging under the observation target is permitted, imaging of the observation target that the user does not desire can be avoided. The state identification unit identifies the operation information of the observation target and confirms that the observation target is stopped, so that contact accidents and the like can be avoided and imaging can be performed more safely. In addition, diagnosis under the observation target can be performed at a location different from the maintenance factory. For example, for a vehicle as the observation target, high-precision diagnosis can be realized more simply and safely without performing a lift-up operation of lifting the vehicle to check the bottom surface. In addition, the time specifying unit specifies the service time, and the movement control unit moves the moving means (mobile body) until the service time elapses, so that imaging and diagnosis of the bottom surface image can be performed more efficiently. In addition, by providing the time specifying unit with a time estimation unit that estimates the stop time of the observation target using the history information of the provided services, the accuracy of service time estimation is improved, and more efficient diagnosis is realized. In addition, the acquisition unit acquires observation target information, the state diagnosis unit performs state diagnosis of the observation target, and the diagnosis information output unit outputs diagnosis information, so that diagnosis information and work information for performing at least one of repair and replacement of parts can be output more efficiently. In addition, the determination unit determines whether all bottom surface images inside the contour of the observation target have been captured. If it is determined that all the images cannot be captured, the partial bottom surface image is stored. If it is determined that the images can be captured, other partial bottom surface images different from the partial bottom surface image are captured, so that more efficient imaging becomes possible.

[0101] According to the image forming method of the present invention, a computer which is an image forming apparatus enables the permission / non-permission setting, identifies the operation information of the observation target, controls the movement of the moving means (mobile body) in the space, and controls the imaging of the under-image, thereby preventing imaging against the user's will. When the user's permission is obtained, the movement of the moving means (mobile body) can be controlled more safely and efficiently.

[0102] According to the image forming program of the present invention, the setting step of obtaining the user's imaging permission, the identification step of confirming the stop of the observation target (ensuring safety), the movement control step of controlling the movement of the moving means (mobile body), and the imaging control step of controlling the imaging of the imaging device can be executed. In addition, by providing an imaging control step of selecting or setting permission or rejection of imaging of the bottom surface image, it is possible to surely prevent performing imaging and diagnosis of the observation target against the user's will. In addition, by providing an imaging time specifying step of specifying the imaging possible time and a diagnosis content specifying step of specifying the diagnosis content, the diagnosis content can be specified according to the length of the imaging possible time, and more efficient diagnosis can be realized.

[0103] And the image generation apparatus of the present invention can perform subsequent photographing of the observation target more efficiently by grasping the contour of the observation target range of the observation target in advance. When the contour setting unit extracts the boundary between the observation target located within a certain distance range from the imaging device and the outside of the observation target as the characteristic points of the contour and connects the characteristic points linearly to identify the contour of the observation target range, for example, by providing a distance measuring unit in the imaging device, the contour of the observation target range can be identified more efficiently and accurately. Also, when extracting the boundary of the color shade or light and dark due to the corner or angle of the observation target as the characteristic points of the contour and connecting the characteristic points linearly to identify the contour of the observation target range, for example, by the camera mounted on the imaging device, the contour of the observation target range can be identified more efficiently and accurately.

[0104] According to the diagnostic apparatus using the image generation apparatus of the present invention, since an overlapping image of the bottom surface image and the standard information of parts acquired separately from the captured image data can be generated, each part of the observation target can be visually confirmed together with the acquired standard information of the parts. Further, the part image in the bottom surface image and the acquired standard image of the part can be directly compared by overlapping them. The image of the part specified by the part specifying unit is compared with the standard image of the part acquired by the part information acquisition processing unit, and based on the collation result of the images, image inspection data including the bottom surface image of the observation target, the comparison information of each part, and the inspection result can be acquired more efficiently. Further, these various data can be quickly displayed on a display as image data that is easy to view visually and is intuitive to understand, and can be stored in a storage medium. The necessary inspections can be performed more efficiently. The burden of the inspection work can be reduced.

[0105] According to the diagnostic information output unit of the present invention, based on the stored data of the determination processing unit, an estimate and a work ticket can be quickly created, recorded, and the estimate can be sent to the user. Further, exchanges, repair requests and reservations from the user based on the estimate, part ordering and factory schedule management generated in response to the user's request, etc. can be collectively managed, and quality control, delivery, estimation, maintenance inspection, part management, factory management, evaluation of used goods, etc. By adding further programs and device parts, the man-hours for the previous complicated administrative processes, etc. can be significantly reduced.

[0106] According to the data accumulation system of the present invention, various data generated by a plurality of image generation apparatuses or failure location specifying terminals can be more efficiently accumulated in the server. The various data accumulated in the server can be widely used, such as being used at each stage of the distribution process of the product as big data, or being used for part quality control, product development, etc.

[0107] According to the data storage system of the present invention, the imaging device is provided with a holder portion to which a mobile terminal can be detachably attached, and the mobile terminal is detachably mounted on the holder portion. Thus, the mobile terminal can replace the camera, communication device, or removable media. When application software as a shooting control and movement control unit is installed on the mobile terminal, the mobile terminal controls the shooting and movement of the imaging device. Further, the LiDAR installed on the mobile terminal can measure the distance from the imaging device to the observation target, and can identify the contour of the observation target range. For example, it is possible to reduce the labor such as lifting a vehicle for an estimate of vehicle inspection, reduce the man-hours of work, and enable an estimate with higher accuracy than visual inspection.

Brief Description of Drawings

[0108]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

[0109] Hereinafter, with reference to the drawings, an image forming apparatus, a diagnostic apparatus using the image forming apparatus, a data storage system, an image forming method, and an image forming program according to an embodiment of the present implementation will be specifically described. In particular, although the present embodiment targets an automobile or a building as an observation object, other artifacts or natural objects, etc. can be used as observation objects.

[0110] As shown in FIGS. 5 and 8(c), the data storage system 1 of the present embodiment has a failure location identification terminal 2 and a server 3. The data storage system 1 is provided with a camera 4 having a moving body 5, a camera 51, a movement control unit 52, a communication device 53, or a removable medium 54. As shown in FIG. 5, the camera 4 can enter the space between the lower surface of the floor panel 10u of the target automobile 10a and the tire contact ground surface (ground surface), the under - floor space of the building 10u as shown in FIG. 6, or the upper space (attic) above the suspended ceiling of the building, etc., and is set to an outer dimension (especially height) that can move freely in the horizontal direction.

[0111] As shown in FIGS. 5 and 8(c) for example, the imaging device 4 has a disc-shaped moving body 5. The moving body 5 is provided with a battery, a drive motor, left and right drive wheels 7a driven by the drive motor, and a steering wheel 7b steered by a servo motor. The drive motor and the servo motor can be controlled by a movement control unit 52 mounted in the moving body 5.

[0112] As shown in FIGS. 8(a) and (b), the drive wheels 7a of the imaging device 4 can be, for example, endless tracks. Three or more endless tracks are provided for the drive wheels 7a, each having an independent suspension mechanism, and each can be steered by receiving drive control. The drive wheels 7a can have, for example, 4 to 8 endless tracks, each having an independent suspension mechanism.

[0113] Also, as shown in FIG. 8(d), the imaging device 4 can have a floating moving source 7. The floating moving source 7 is a rotary wing 71 provided at the tip of each of a plurality of struts extending in the centrifugal direction on the same horizontal plane from the moving body 5, and for example, 3 to 10 rotary wings can be provided. And one or a plurality of ground wheels 7c can be provided on the moving body 5 having the rotary wings 71.

[0114] As shown in FIGS. 5, 8(b) and (c), the camera 51 can be mounted on the top surface wall of the moving body 5, for example, and its optical axis can be directed vertically upward. The camera 51 has a recording control unit 51c capable of shooting still images or moving images, and can be, for example, a CCD (Charge Coupled Devices), a CMOS (Complementary Metal Oxide Semiconductor), etc. One or a plurality of lights 50b for illuminating the observation object 10u to be the subject, and an illumination control unit for adjusting the brightness of those lights 50b can be provided on the moving body 5.

[0115] The imaging device 4 desirably includes a distance measurement unit 50c and a distance measurement control unit that controls the distance measurement unit 50c on the moving body 5. The distance measurement unit 50c can be, for example, a LiDAR, a SPAD (Single Photon Avalanche Diode), an optical sensor, a radio wave sensor, an ultrasonic sensor, or the like.

[0116] The communication device 53 can communicate wirelessly with the failure location identification terminal 2, for example, and enables wireless communication between the movement control unit 52 and the recording control unit 51c and the failure location identification terminal 2. The communication device 53 can perform communication conforming to communication standards such as wireless LAN (Wi-Fi), Bluetooth, infrared communication, or other wireless communication.

[0117] As shown in FIG. 8(c), the imaging device 4 can include a hardware interface 54h for a removable medium (flash memory) 54. The hardware interface 54h can be, for example, a card slot 54s for an SD card, a USB connection terminal 54t for a USB memory stick, or the like.

[0118] The movement control unit 52 receives a control signal output by the failure location identification terminal 2 through the communication device 53 and controls the drive wheels 7a and the steering wheels 7b. For example, the failure location identification terminal 2 has a path calculation processing unit that receives the image data photographed by the camera 51 of the imaging device 4, stored in the recording control unit 51c, and transmitted through the communication device 53, and calculates a movement path based on the received image data. The path calculation processing unit transmits the calculated movement path data to the movement control unit 52 of the imaging device 4 through the communication device 53.

[0119] Further, the movement control unit 52 can have a path calculation processing unit that calculates a traveling path that enables the most efficient imaging based on the distance data measured by the distance measurement unit 50c and stored in the distance measurement control unit. In this case, the path calculation processing unit enables autonomous traveling, acceleration, deceleration, stop, and steering by the movement control unit 52 of the imaging device 4. Furthermore, the movement control unit 52 can include a path calculation processing unit that calculates a travel path along which shooting can be performed most efficiently based on the image data captured by the camera 51 and stored in the recording control unit 51c. In this case, the path calculation processing unit enables autonomous travel, acceleration, deceleration, stop, and steering by the movement control unit 52 of the imager 4.

[0120] The failure location identification terminal 2 can be configured to include an image forming apparatus 8 and a diagnostic apparatus 9 using the same. The image forming apparatus 8 includes, for example, a contour setting unit P1, a component identification unit P2, a map assignment processing unit P3, a map setting processing unit P4, a component information acquisition processing unit P5, and an image forming processing unit P6. The diagnostic apparatus 9 includes, for example, the image forming apparatus 8, a component comparison processing unit P7, a component inspection processing unit P8, and a determination processing unit P9. Furthermore, the diagnostic apparatus 9 may include a document creation processing unit P10. As shown in FIG. 5, the failure location identification terminal 2 can use a computer 2a, and as shown in FIG. 6, the failure location identification terminal 2 can be replaced with a mobile terminal 2b.

[0121] For example, as shown in FIG. 8(a), the imager 4 is provided with a holder unit 50 on the top wall of the moving body 5 that can detachably mount a mobile terminal 6. The holder unit 50 can detachably mount a mobile terminal 6 equipped with at least one of a camera 51, a light 50b, a LiDAR (distance measuring unit) 50c, a movement control unit 52, a communication device 53, or a removable medium 54.

[0122] At this time, based on the image data captured while the mobile terminal 6 moves under the observation target by the moving body 5, the mobile terminal 6 can be used as an image forming apparatus including a contour setting unit that identifies the contour of the observation target range in the observation target (frame information on the lower side of the observation target) and sets the area within the contour as a contour area, and an image output unit that outputs, within the contour area, an image acquisition area where image data is acquired and an image non-acquisition area where image data is not acquired. Furthermore, it can be used as an image forming apparatus that forms a bottom surface image from the image data acquired within the contour area.

[0123] The mobile terminal 6 also includes a component identification unit that identifies components from feature points based on the image data within the contour area, a component information acquisition processing unit that acquires a standard image of the component identified by the component identification unit, a component comparison processing unit that compares the image of the component identified by the component identification unit with the standard image of the component acquired by the component information acquisition processing unit, and a determination processing unit that determines the component state of the component identified by the component identification unit from the comparison result in the component comparison processing unit. In this case, it can also be used as a diagnostic device.

[0124] The imaging device 4 may be, for example, a manual type as shown in FIGS. 9(a), (b), and (c). The imaging device 4 is configured by providing freely rotatable casters 7c at the left and right ends near the bottom of a rectangular moving body 5 that is elongated left and right. An operating rod 50a that slopes backward is extended from a position halfway between the left and right ends of the top wall of the moving body 5. The operating rod 50a may have a telescopic structure. Four cameras 51 that are arranged at equal intervals between the left and right ends and are directed obliquely upward and forward are provided on the front wall of the moving body 5. As shown in FIG. 9(c), two or more, for example, three, for example, three to eight or the like, a plurality of cameras 51 can be provided with respect to the moving body 5. The manual imaging device 4 is configured by including at least one of a light 50b, a distance measuring unit 50c, a communication device 53, or a removable medium on the moving body 5.

[0125] As shown in FIG. 9, the manual imaging device 4 holds and operates the operating rod 50a to move the moving body 5 below the observation object 10u shown in FIGS. 5 and 6. By grounding and rolling the freely rotatable casters 7c and operating while rolling, each camera 51 can capture images simultaneously, and a wide range can be captured more efficiently. It is desirable that the plurality of cameras 51 provided on the moving body 5 are set to the same focal length with respect to each other, and the distances from the observation object 10u that is the subject to each camera 51 are arranged to be the same with respect to each other. Also, the plurality of cameras 51 can be set to have a long focal length and a short focal length with respect to each other, and the distances from the observation object 10u that is the subject to each camera 51 can be arranged in accordance with their respective focal lengths. This enables simultaneous shooting of a wide-angle image and a magnified image. Furthermore, the plurality of cameras 51 are provided with their optical axes directed in different directions from each other, making it possible to shoot a three-dimensional image with higher speed and higher image quality.

[0126] Also, in the case of a manual imaging device, for example, a holder part for detachably mounting a mobile terminal is provided at the tip of the operating rod, and the mobile terminal can be detachably mounted to the holder part. The manual imaging device can hold the operating rod like a selfie stick and approach the mobile terminal to the lower surface part of the ceiling of a building, the lower surface part of an automobile, etc. for shooting. As described above, the mobile terminal can be used as an image forming device or a diagnostic device. Mode for Carrying Out the Invention 2

[0127] (Automobile inspection process) Hereinafter, with reference to FIGS. 1 to 5 and FIGS. 7 to 9, the image forming device 8 and the diagnostic device 9 used in the data storage system 1 of the automobile will be described according to the flow of the inspection work of the automobile 10a by the data storage system 1.

[0128] When the failure location specifying terminal 2 is activated, an input screen for information for specifying the automobile 10a as the observation target 10 is displayed on the display of the computer 2a as the failure location specifying terminal 2. On this screen, information effective for specifying the vehicle such as the name of the automobile 10a, the chassis number, the model, the type of the engine, the type, the shape of the vehicle body, etc., and vehicle inspection certificate information are input (S1).

[0129] When the imaging device 4 is installed on the ground surface (ground) of the automobile 10a below the lower surface part 10u including the floor panel, the chassis frame, the suspension, etc. of the automobile 10a which is the observation target of the automobile 10a, the imaging device 4 slowly (for example, 10 to 30 cm / s) travels on the ground surface in a random direction, and based on the imaging image data of the mounted camera 51, specifies the position of the imaging device 4 with respect to the lower surface part 10u of the automobile 10a, and stores and recognizes the imaging start position (S2).

[0130] The imaging device 4 determines whether it is located below the lower surface portion 10u of the vehicle 10a by means of a sensor (S3). The sensor can be a distance measuring unit 50c composed of a LiDAR, an optical sensor (e.g., an infrared sensor), a radio wave sensor (e.g., a transceiver antenna), an ultrasonic sensor, or the like. For example, when the distance from the imaging device 4 to the lower surface portion 10u is within a measurable distance (within 1 m or within 50 cm), it can be set to determine that it is located below the lower surface portion 10u. Also, by sensing the brightness such as the light and dark and color shade of the captured image of the camera 51 and comparing it with the brightness outside the lower surface portion 10u, it is also possible to determine that it is located below the lower surface portion 10u when it is darker or has a darker color than the brightness outside the lower surface portion 10u.

[0131] If it cannot be recognized that it is located below the lower surface portion 10u for a certain period of time (e.g., 10 seconds to 30 seconds) or more, the measurement is terminated, and it returns to the traveling start (initially placed) position or the imaging start position and stops (A) (S15). In this case, while continuing the detection, it changes the course and travels, and also performs measurement until it returns to the traveling start position or the imaging start position. If it is recognized that it is located below the lower surface portion 10u, it continues to be controlled so as to shift to the next step (S5).

[0132] If it is recognized that it is located below the lower surface portion 10u within a certain period of time (e.g., 10 seconds to 30 seconds), the imaging device 4 starts imaging a moving image (or a still image) (S5). The imaging at this time is performed in conjunction with the traveling operation. For example, the traveling can be controlled at a maximum speed of, for example, 30 to 500 cm / s, and controls such as stopping, U-turning, rotating, and decelerating can be performed.

[0133] Subsequently, it is determined whether or not the "underside recognition mode 2" is set (S6). When the "underside recognition mode 2" is set, the recognition process of the component 101 within the contour 10L of the observation target 10u is performed (S10).

[0134] (Contour setting unit P1) If it is not set to the "underside recognition mode 2" (S6), it is determined whether the edge portion of the underside is imaged (S7). If the edge portion of the underside is not imaged, the recognition process for the underside and the component 101 (S8) is performed.

[0135] (Underside recognition mode 1) The recognition process for the underside and the component 101 (S8) sets the "underside recognition mode 1" (S8a). The underside recognition mode 1 can be, for example, a recognition mode in which the outer peripheral edge of the observation target 10 is identified as the contour 10L of the observation target range, and then the component 101 (floor panel, chassis frame, suspension, bumper, side sill, fender, etc.) of the lower surface portion of the observation target is recognized. By recognizing the contour 10L of the observation target range in advance, it is possible to assume the arrangement of parts and improve the detection accuracy and inspection accuracy of components such as parts.

[0136] When set to the "underside recognition mode 1" (S8a), for example, the camera 51 (recording control unit 51c), the light 50b (lighting control unit), and the traveling drive source 7 (movement control unit 52) can be set to conditions where it is easy to recognize the outer peripheral edge of the component 101, and each can be controlled. When the camera 51 (recording control unit 51c) captures a feature point indicating the contour 10L of the lower surface portion 10u, the speed of the traveling drive source 7 can be reduced, and it can be controlled to travel slowly along the feature point and prioritize the identification of the contour 10L (S8b). Identify components such as tires and the underside component 101 and move without contacting them.

[0137] (Component identification unit P2) Furthermore, while identifying the contour 10L of the observation target range based on the (two-dimensional or three-dimensional) image data of the camera 51, the components 101 existing in the lower surface portion of the observation target 10 are recognized (S8c). The recognition of the components 101 can identify the feature points of each component 101 and store the information of the identified components 101 (S8d). The feature points of each component 101 can be determined and extracted based on, for example, different shapes, sizes, colors, arrangements, etc. for each component 101. The components 101 can be, for example, components 101 such as exhaust pipes, catalysts, mufflers, suspensions, axles, floor panels, bolts and nuts for connecting them, etc., which are visibly exposed on the lower side of the vehicle body.

[0138] (Map Assignment Processing Unit P3) When the imaging of the lower edge portion has been completed (S7), the entire contour 10L of the observation target range is recognized (S9). The contour 10L (lower edge) portion of the observation target range is stored together with the position data of those feature points (S9a). Based on the position data of the feature points of the stored contour 10L, a "lower frame map 10M" is created by assigning a two-dimensional map (or three-dimensional map) composed of a two-dimensional array (or three-dimensional array) of a plurality of planar sections 10S (or three-dimensional sections) within the contour 10L of the observation target range (S9b).

[0139] The "lower frame map 10M" can be, for example, map data in which a plurality of arranged sections 10S are assigned to image data formed to be integrated into the lower surface image 100 based on the imaging data of the observation target 10.

[0140] (Lower Recognition Mode 2) It is determined whether the creation of the "underside frame map 10M" has been completed (S9c). If it is not completed, the process returns to determine whether it is located below the lower surface portion 10u of the vehicle 10a (S3). On the screen of the failure location identification terminal 2, the uncompleted section 10N of the shooting is colored and displayed so that the completed section 10F of the shooting and the uncompleted section 10N of the shooting can be easily visually recognized. The completed section 10F of the shooting can be displayed in white or black, or by pasting an image of the corresponding section 10S that has been shot.

[0141] (Map setting processing unit P4) If the creation of the "underside frame map 10M" is completed, the inside of the "underside frame map 10M" is set as the imaging range (S9d), and the "underside recognition mode 1" is set (S9e). The "underside recognition mode 2" can be, for example, a recognition mode that identifies the lower surface portion 10u and each of its components 101 (floor panel, chassis frame, suspension, etc.) and enables inspection for each component 101 on the image data.

[0142] (Component information acquisition processing unit P5) Based on the information input (S1) on the input screen of the information for identifying the vehicle 10a which is the observation target 10, the standard information 102 of the components 101 under the vehicle 10a is acquired (S9f). The standard information 102 of the component 101 can be the part name, part number, assembly number, design drawing, service manual, parts list, (dimensions, material) color, shape, two-dimensional (or three-dimensional) image, etc. associated therewith. By acquiring the standard information 102 in advance, it becomes possible to more accurately identify the component 101 based on that information.

[0143] The standard information 102 of the component 101 can be obtained from information stored in, for example, an external storage device 3c of the server 3, or from data stored in the Internet 3a, cloud 3b, etc. The acquisition of the standard information 102 of the component 101 can be controlled, for example, by extracting the image of the component 101 from the captured image, and based on the image of the component 101, searching the stored data (2, 3, 3c) or performing an Internet 3a (3b) search to obtain the standard information 102 of the component 101.

[0144] Overlay the "standard information 102 of the lower-level component 101" on the "lower-level frame map 10M" and save (store) the formed image data (S9g). The formed image data can be displayed by overlaying an image of a translucent component 102 associated with the standard information 102 of the lower-level component 101 on the lower-level frame map 10M. When the mouse pointer is overlaid on the translucent component 102 displayed semi-transparently on the display of the computer 2a, the component 102 switches to a blinking display, and a small screen for displaying numerical data, character information, etc. of the detailed standard information 102 of the corresponding component 102 is expanded and displayed. It may be controlled to return to the original display when the mouse pointer moves away from the component 102. The same operation is possible when the computer 2a is replaced with a mobile terminal 2b.

[0145] Based on the formed image data, it is determined whether there is standard information 102 on the positions and shapes of all components 101 identified and stored in the "lower-level recognition mode 1" (S9h). If there is standard information 102 on the positions and shapes of all components 101, the data on the positions, shapes, and colors of each component 101 are compared and collated with the standard information 102 on the positions, shapes, and colors obtained (S9f) (S9j). The collation results of each component 101 (102) are temporarily stored, for example, in the imaging device 4 (51c, 54) or the failure location identification terminal 2 (S9k).

[0146] When there is no standard information 102 on the position, shape, and color of some or all of the components 101 (S9h), or after temporarily storing the collation results of all the components 101 (S9k), the intensity of the illumination of the camera 51 (recording control unit 51c), the light 50b (lighting control unit), and the traveling speed of the traveling drive source 7 (movement control unit 52) can be set and controlled respectively under conditions that can capture each component 101 of the lower surface portion 10u in detail. When the camera 51 (recording control unit 51c) captures the feature points indicating each component 101 of the lower surface portion 10u, the speed of the traveling drive source 7 can be reduced, and it can travel slowly while performing arithmetic processing on the feature points, and control can be performed to prioritize the identification of each component 101 (S9m).

[0147] (Image forming processing unit P6) After grasping the entire underbody (inside the contour 10L of the lower surface portion 10u of the vehicle 10a), recognition processing of the underbody (inside the contour 10L) and the components 101 inside the contour 10L of the observation target 10u is performed (S10). The captured images are compared with the superimposed images to collate each component 101 (S10a), and as a result of the collation, it is determined whether all the components 101 have been recognized (S10b). The positions, shapes, and colors of all the components 101 are stored permanently (S10c) in, for example, the imaging devices 4 (51c, 54) or the failure location specifying terminal 2. The position information of the permanently stored components 101 is more accurate than the temporarily stored position information.

[0148] Then, it is determined whether imaging of the entire imaging range has been completed (determination unit S10d). For example, as shown in FIG. 7(d), when the completed section 10F of the imaging is displayed on the entire underbody frame map 10M displayed on the display of the computer 2a and the uncompleted section 10N of the imaging is not displayed, it can be determined (determination unit: S10d) whether all the underbody imaging inside the contour of the underbody of the observation target vehicle (imaging range) has been imaged by the imaging device.

[0149] (Component comparison processing unit P7) It is determined (S10e) whether all temporarily stored components 101 match the position, shape, and color of the temporary storage (S9k) of the collation result of each component 101. Furthermore, it is possible to confirm whether the component 102 of the standard information obtained before creating the underbody frame map 10M is included in the component 101 recognized after creating the underbody frame map 10M, enabling the search for components 101 without omission.

[0150] If any of all the components 101 do not match the temporarily stored (S9k) position, shape, and color, the positions, shapes, and colors of all the components 101 stored in the main memory (S10c) and all the components 101 stored temporarily (S9k) are overlaid and compared and displayed on the display (S10f). The user instruction content based on the comparison result is reflected in the main memory, and duplicate temporary memories are deleted (S10g).

[0151] (Component inspection processing unit P8) An investigation of the component 101 is performed (S11). The investigation of the component 101 can be determined by comparing the component 101 on the superimposed image such as axle misalignment, underbody deformation or cracks, and oil leakage. Comparison is made with the RBG value in units of pixels for each component 101, and the component 101 with a large difference is investigated. (Judgment processing unit P9) The overall image of the underbody, the comparison information of the component 101, and the inspection result are displayed on the display (S12).

[0152] (Document creation processing unit P10) Based on the inspection result, an estimate regarding the cost of the work for updating the component state of the component and a work ticket regarding the update work are created and stored in the storage device of the computer 2a or the external storage device 3c of the server 3, etc. (S13). The recognition mode (recognition state) set to either "underbody recognition mode 1" or "underbody recognition mode 2" is initialized (S14). All processes are terminated (S15). The update work is work for updating the state of the component such as work for repairing the component, work for replacing the component, and work for fixing the component.

[0153] The computer 2a as the failure location specifying terminal 2 can store the overall image of the underrun, the comparison information (101, 102) of the component 101, and the inspection results it has memorized (S13) in, for example, the external storage device 3c of the server 3. The server 3 can receive the information transmitted from a plurality of failure location specifying terminals 2, store it together with information such as each automobile manufacturer, vehicle type, model, model year, registration year, registered prefecture, mileage, maintenance history, etc., and manage the stored data as big data that can be statistically processed. Mode 3 for carrying out the invention

[0154] (Building inspection process) Referring to FIGS. 6 and 10, regarding the image forming apparatus 8 and the diagnostic apparatus 9 used in the data storage system 1 of the building 10b, it will be shown according to the flow of the inspection work of the wall, hanging fittings, air conditioning equipment, various pipes, wiring, etc. of the lower surface portion 10u of the roof in the upper space of the suspended ceiling 10h of the building 10b by the data storage system 1.

[0155] When the failure location specifying terminal 2 and the imaging device 4 are activated, an input screen for information specifying the building 10b as the observation target 10 is displayed on the display of the mobile terminal 2b as the failure location specifying terminal 2. On this screen, information such as the type of building structure (wooden, aluminum, lightweight steel frame, heavyweight steel frame, reinforced concrete, steel-reinforced concrete, concrete-filled steel pipe structure, concrete block structure) of the building 10b, the name of the construction company or house manufacturer, model, model year, registration year, number of years since construction, registered prefecture, address, GPS (Global Positioning System) data, design drawings, maintenance history, etc. are input (S1).

[0156] The imaging device 4 is arranged above the suspended ceiling 10h below the roof or the lower surface portion 10u of the upper floor of the building 10b to be observed. The imaging device 4 can be either the imaging device 4 having any of the traveling drive sources 7 in FIGS. 8(a) to 8(c) or the imaging device 4 having the floating movement source 7 shown in FIG. 8(d). While traveling slowly (e.g., 10 to 30 cm / s) in a random direction, the imaging device 4 identifies the position of the imaging device 4 relative to the lower surface portion 10u based on the captured image data of the camera 51 mounted thereon, and stores and recognizes the imaging start position (S2).

[0157] The imaging device 4 determines whether it is located below the lower surface portion 10u by means of the distance measuring unit 50c (S3). For example, it can be set to determine that it is located below the lower surface portion 10u when the distance from the imaging device 4 to the lower surface portion 10u is within 3 m or within 1 m. Also, by sensing the brightness and darkness or color shading of the captured image of the camera 51, and discriminating the difference in brightness and color from the vertical wall outside the lower surface portion 10u, it can be determined that it is located below the lower surface portion 10u.

[0158] If it cannot be recognized that it is located below the lower surface portion 10u for a certain period of time (e.g., 30 seconds to 1 minute) or more, the measurement is terminated, and it returns to the traveling start (initial) position and stops (S15). In this case, while continuing the detection, it travels while changing the course, and if it can be recognized that it is located below the lower surface portion 10u before returning to the traveling start position, it can be controlled to shift to the next step (S5).

[0159] If it can be recognized that it is located below the lower surface portion 10u within a certain period of time (e.g., 30 seconds to 1 minute), the imaging of a video (or a still image) is started (S5). The imaging can be performed while moving at a maximum speed of, for example, 30 to 500 cm / s and including stops, U-turns, rotations, decelerations, etc.

[0160] Determine whether the "underside recognition mode 2" is set (S6). If the "underside recognition mode 2" is set, perform the recognition process of the components 101 within the contour 10L of the observation target 10u (S10).

[0161] (Contour setting unit P1) If it is not set to the "underside recognition mode 2" (S6), determine whether the underside entrance corner portion is being imaged (S7). If the underside entrance corner portion is not being imaged, perform the recognition process of the underside and the components 101 (S8).

[0162] (Underside recognition mode 1) For the recognition process of the underside and the components 101 (S8), set the "underside recognition mode 1" (S8a). The underside recognition mode 1 can be, for example, a recognition mode that identifies the outer peripheral edge of the lower surface portion 10u as the contour 10L of the observation target range.

[0163] When set to the "underside recognition mode 1", for example, the camera 51 (recording control unit 51c), the light 50b (lighting control unit), and the traveling drive source 7 or the floating movement source 71 (movement control unit 52) can be set to conditions that make it easier to recognize the outer peripheral edge of the lower surface portion 10u and controlled respectively. When the camera 51 (recording control unit 51c) captures the feature points indicating the contour 10L of the lower surface portion 10u, the movement speed of the traveling drive source 7 or the floating movement source 71 can be reduced, and it can be controlled to travel slowly along the feature points and prioritize the identification of the contour 10L (S8b). Identify surrounding pillars, walls, hanging fixtures, air conditioning equipment, pipes, wiring, etc., and move without contacting them.

[0164] (Component identification unit P2) Furthermore, while identifying the contour 10L of the observation target range based on the (two-dimensional or three-dimensional) image data of the camera 51, the components 101 of the observation target 10u are recognized (S8c). The recognition of the components 101 involves identifying the feature points of each component 101 and storing the information of the identified components 101 (S8d). The feature points of each component 101 can be determined and extracted based on, for example, different shapes, sizes, colors, arrangements, etc. for each component 101. The components 101 can be, for example, visible components 101 such as pipes, ducts, wiring, air conditioners, hanging brackets, columns, beams, walls, etc.

[0165] (Map Assignment Processing Unit P3) When the imaging of the lower edge portion has been completed (S7), the entire contour 10L of the observation target range is recognized (S9). The contour 10L (lower entrance corner) portion of the observation target range is stored together with the position data of those feature points (S9a). Based on the stored position data of the feature points of the contour 10L, a "lower frame map 10M" is created by assigning a two-dimensional map (or three-dimensional map) composed of a two-dimensional array (or three-dimensional array) of a plurality of planar sections 10S (or three-dimensional sections) within the contour 10L of the observation target range (S9b).

[0166] As shown in FIGS. 10(a) and 10(b), the imaged section 10F and the section 10N that has not yet been imaged can be color-coded and displayed on the display of the mobile terminal 2b. For example, the imager 4 can capture images along the feature points indicating the outer peripheral edge of the lower surface portion 10u and can capture images so as to go around the contour 10L of the observation target range. When going around the contour 10L of the observation target range, on the display of the mobile terminal 2b, the imaged sections 10F are displayed continuously around the displayed lower frame map 10M, and a colored filled image of the section 10N that has not yet been imaged is displayed in the center. Thus, the contour 10L of the observation target range can be identified prior to the detailed imaging of each component 101. After identifying the contour 10L of the observation target range, imaging of the section 10N that has not yet been imaged is performed, and as shown in FIG. 10(c), it is possible to finally control to display all the sections 10S within the "lower frame map 10M".

[0167] The "underside frame map 10M" can be map data in which a plurality of arranged sections 10S are assigned to image data formed and processed to be integrated into the bottom surface image 100, for example, based on the imaging data of the observation target 10u.

[0168] (Underside recognition mode 2) It is determined whether the creation of the "underside frame map 10M" is completed (S9c). If it is not completed, the process returns to determine whether it is located below the underside portion 10u (S3).

[0169] (Map setting processing unit P4) When the creation of the "underside frame map 10M" is completed, the inside of the "underside frame map 10M" is set as the imaging range (S9d), and the "underside recognition mode 1" is set (S9e). The "underside recognition mode 2" can be, for example, a recognition mode that discriminates between the underside portion 10u and each component 101 such as columns, walls, suspension fittings, air conditioning equipment, piping, and wiring, and enables inspection for each component 101 on the image data.

[0170] (Component information acquisition processing unit P5) Based on the information input (S1) on the input screen for the information identifying the building 10b which is the observation target 10, the standard information 102 of the components 101 under the observation target 10 is acquired (S9f). The standard information 102 of the components 101 can be the part name, part number, assembly number, design drawing, service manual, parts list, (dimensions, material, parts list information) color, shape, two-dimensional (or three-dimensional) image, etc. that are associated therewith. The standard information 102 of the components 101 can be acquired from the information stored in, for example, the external storage device 3c of the server 3, or from the data stored in the Internet 3a, cloud 3b, etc.

[0171] As shown in FIG. 10(c), the “standard information 102 of the lower component 101” is superimposed on the “lower frame map 10M”, and the formed image data is saved (S9g). In the formed image data, an image of the translucent component 102 with the standard information 102 of the lower component 101 associated therewith is superimposed and displayed on the lower frame map 10M. When the mouse pointer is superimposed on the translucent component 102 displayed semi-transparently on the screen, the component 102 switches to a blinking display, and a small screen for displaying numerical data, character information, etc. of the detailed standard information 102 of the corresponding component 102 is expanded and displayed. When the mouse pointer moves away from the component 102, it may be controlled to return to the original display.

[0172] Based on the formed image data, it is determined whether there is standard information 102 on the positions and shapes of all components 101 (lower surface part 10u) identified and memorized in the “lower recognition mode 1” (S9h). When there is standard information 102 on the positions and shapes of all components 101, the data on the positions, shapes, and colors of each component 101 (lower surface part 10u) is compared and collated with the acquired standard information 102 on the positions, shapes, and colors (S9j). The collation results of each component are temporarily memorized (S9k).

[0173] When there is no standard information 102 on the positions, shapes, and colors of some components 101 or all components 101 (lower surface part 10u) (S9h), or after temporarily memorizing the collation results of all components 101 (S9k), the intensity of the illumination of the camera 51 (recording control unit 51c), the light 50b (lighting control unit), and the traveling speed of the traveling drive source 7 (movement control unit 52), or the moving speed of the floating movement source 71 can be set and controlled to conditions where each component 101 of the observation target 10u can be photographed in detail. When the camera 51 (recording control unit 51c) captures the feature points indicating each component 101 of the lower surface part 10u, the speed of the traveling drive source 7 or the floating movement source 71 can be reduced, and it can be controlled to move slowly while performing arithmetic processing on the feature points to prioritize the identification of each component 101 (S9m).

[0174] (Image forming processing unit P6) After grasping the entire lower periphery (within the contour 10L of the lower surface portion 10u), recognition processing of the components 101 within the lower periphery and within the contour 10L of the lower surface portion 10u is performed (S10). The captured image is compared with the superimposed image to collate each component 101 (S10a), and based on the result of the collation, it is determined whether all components 101 have been recognized (S10b). The positions, shapes, and colors of all components 101 are stored in the memory (S10c). The position information of the components 101 stored in the memory is more accurate than the temporarily stored position information.

[0175] It is determined whether imaging of the entire imaging range has been completed (S10d). For example, as shown in FIGS. 6 and 10(c), when on the display of the mobile terminal 2b, the completed section 10F of the imaging is displayed for the entire lower frame map 10M shown, and the uncompleted section 10N of the imaging is not displayed, it can be determined whether imaging of the entire imaging range has been completed.

[0176] (Component comparison processing unit P7) It is determined whether all components 101 stored temporarily match the positions, shapes, and colors of the temporary storage (S9k) of the collation results of each component 101 (S10e). If any of all components 101 do not match the temporarily stored (S9k) positions, shapes, and colors, the positions, shapes, and colors of all components 101 stored in the memory (S10c) and all components 101 stored temporarily (S9k) are superimposed and compared and displayed on the display 2b (S10f). The user instruction content based on the comparison result is reflected in the memory, and duplicate temporary storage is deleted (S10g).

[0177] (Component inspection processing unit P8) An investigation of the component 101 is performed (S11). The investigation of the component 101 can be determined by comparing the components on the superimposed image 100 (101, 102) such as the displacement of the piping, the deformation or cracks of the wall surface or beam, and water leakage. By comparing the RBG values in units of pixels for each component, the component 101 with a large difference can be investigated.

[0178] The material can be discriminated from the image. It is possible to discriminate materials such as wood and heat insulating materials, and it becomes possible to utilize the information obtained as an index for real estate appraisal. It can also be used to detect pests such as termites. (Judgment processing unit P9) The overall image 100 of the underfloor, the comparison information (101, 102) of the component 101, and the inspection result are displayed on the display (S12).

[0179] (Document creation processing unit P10) Based on the inspection result, at least one of the inspection estimate and the work ticket is created and stored in the storage device of the mobile terminal 2b or the external storage device 3c of the server 3, etc. (S13). The recognition mode set in either "underfloor recognition mode 1" or "underfloor recognition mode 2" is reset (S14). All processes are terminated (S15).

[0180] The overall image of the underfloor, the comparison information (101, 102) of the component 101, and the inspection result stored by the mobile terminal 2b as the failure location identification terminal 2 can be stored, for example, in the external storage device 3c of the server 3. The server 3 receives the information transmitted from a plurality of failure location identification terminals 2 and stores it together with information such as the type of building structure, the name of the construction company or house manufacturer, the model, the year model, the registration year, the number of years since construction, the registered prefecture, and the maintenance history, and can manage the stored data as big data that can be statistically processed.

[0181] FIG. 11 is a hardware configuration diagram showing an example of the image forming apparatus 8, the diagnostic apparatus 9 using the image forming apparatus, and the failure location identification terminal 2 and the data storage system 20 equipped with these. These apparatuses are mutually connected to each other via a bus B with an input device 11, an output device 12, a drive device 13(18), an auxiliary storage device 14, a main storage device 15, an arithmetic processing device 16, and an interface device 17, respectively.

[0182] The input device 11 can be a keyboard, a mouse, or the like, and is used to input various signals. The output device 12 can use a display device or the like, and is used to display various windows, data, and the like. The interface device 17 is a modem, a LAN card, or the like, and is used to connect to a network.

[0183] The programs for executing the respective processes of the image forming apparatus 8, the diagnostic apparatus 9 using the image forming apparatus, and the failure location specifying terminal 2 and the data storage system 20 on which these are mounted in the present embodiment are at least a part of various programs for controlling the processes in the respective apparatuses. The various programs are provided, for example, by distribution of the recording medium 18 or downloading from a network. The recording medium 18 on which such a program is recorded can use various types of recording media such as a CD-ROM, a flexible disk, a magneto-optical disk, etc., which optically, electrically, or magnetically record information, and a semiconductor memory such as a ROM, a flash memory, etc., which electrically record information.

[0184] The programs for executing various processes are installed in the auxiliary storage device 14. The auxiliary storage device 14 stores the installed programs and also stores necessary files, data, and the like. The main storage device 15 reads and stores the program that defines the process from the auxiliary storage device 14. Then, the arithmetic processing unit 16 realizes various processes according to the program stored in the main storage device 15. Mode for Carrying Out the Invention 4

[0185] (Step of obtaining user permission) With reference to FIGS. 1 to 5, FIGS. 7, and FIG. 12, the control program (including the bottom surface image capturing program) of the bottom surface image capturing apparatus 1 having a configuration conforming to the data accumulation system 1 (data storage system 20) having the failure location specifying terminal 2 and the server 3 as described above is shown below.

[0186] The control program (including the bottom image capturing program) is incorporated as software into any of the devices in the data storage system 1 (data storage system 20). For example, it can be installed in any of the form terminal 2b for user use, the computer 2a installed in an automobile maintenance factory, or the server 3, and can be used to comprehensively control the whole. Also, it can be installed as software divided among the respective devices and operate in cooperation with each other.

[0187] When the control program (including the bottom image capturing program) of the bottom image capturing device 1 starts ST1, the control program (app including the bottom image capturing program) starts ST2. When the app starts ST2, a check box for requesting permission for monitoring is displayed on the touch panel display of the mobile terminal for user use or the in-vehicle computer (imaging control step, setting step, imaging setting section) ST3.

[0188] The timing for executing the request for permission ST3 can be when the user downloads and starts the app on the mobile terminal or the like. Also, it can be done simultaneously when the user registers for use of the app through the mobile terminal or the like and registers various information such as the user's personal information and the vehicle model. It can be determined that the registration completion is the user's permission for shooting. Furthermore, it can be set to obtain permission once or every time immediately before the app starts shooting the bottom image of the vehicle. The app can have an input field for the user to reject shooting. The app can have an input section for the user to set the rejection of shooting by themselves. When requesting permission, explanations such as the shooting location under the vehicle, the timing of shooting, the diagnosis of the vehicle based on the captured image, and the storage and output (transmission) destination of the diagnosis result can be displayed. The user can make a judgment on whether to permit while referring to the displayed contents.

[0189] If the user does not permit the shooting ST3, the setting step ST3 is executed until permission is obtained. If the user permits the shooting ST3, an input field for information identifying the vehicle to be monitored is displayed, and the input information is saved ST4. The vehicle identification information can be input by the user himself or obtained from information pre-registered in a mobile terminal, a vehicle-mounted computer, etc. Along with the vehicle identification information, the user's personal information, etc. can be obtained. Of course, the process of obtaining the user's permission is also required for obtaining personal information.

[0190] When the vehicle to be monitored stops at the monitoring position, it can be determined that it has stopped at the monitoring position based on the position information of the user's mobile terminal or the vehicle (identification step, state identification unit) ST5. Also, a sensor installed at the parking position can be made to detect the arrival of the registered vehicle ST5. When it is determined that the user's vehicle has stopped at the monitoring position, the staying time of the vehicle is estimated (imaging time specifying step, time specifying unit, time estimating unit) ST6. The estimation of the staying time ST6 can be based on the general staying time at the parking position. This general staying time at the parking position can be calculated from the average staying time set or defined for each parking position or facility using the parking position, and it is desirable to have an average staying time recording unit that defines the average staying time for each parking position or facility using the parking position. Also, regarding the average staying time recorded by the average staying time recording unit, the user's history information can be referred to, and the general staying time can be estimated by taking into account conditions such as date, time, and weather. When obtaining the user's history information, permission for using the information shall be obtained at some prior stage.

[0191] (Specify the vehicle diagnosis menu according to the service time) At the stage where the estimation ST6 of the stay time is completed, the monitoring records up to the previous time are checked ST7. The purpose of estimating the stay time ST6 is to select possible monitoring items within the estimated stay time. If monitoring has been performed previously and monitoring records up to the middle remain, it is advisable to check ST7 the existing monitoring records in order to avoid duplication of monitoring. If the monitoring records up to the previous time are within one month, by referring to the accumulated monitoring records, the appropriate selection ST8 of the current monitoring items to be performed within the estimated stay time can be made. For this reason, it is desirable to provide a monitoring status recording unit that records the history of the state for each vehicle or each user.

[0192] (Linkage between the next diagnosis and the provided service) The selection ST8 of the current monitoring items that can be performed within the estimated stay time (diagnosis content specification step) is, for example, when the monitoring location is a gas station, using the vehicle information of the vehicle and the refueling history of past users, etc., with AI (artificial intelligence) such as a program installed on the server, the end time of the service (refueling) can be estimated. Then, from the start of the next diagnosis to the estimated end time, the content of the next diagnosis that can be implemented is specified, and only that content is implemented. Therefore, it is desirable to provide a diagnosis time recording unit that records the time required for each diagnosis content.

[0193] Before the estimated end time, if the service provision (refueling) has "ended", the speed of the next diagnosis can be increased or the next diagnosis can be skipped (partially, the diagnosis is aborted), the remaining time is displayed on the app, and it can also be recorded on the server. By the server holding the diagnosis content executed so far, at the next parking position, the executed diagnosis content can be acquired, and the subsequent diagnosis can be continuously executed. Also, before the estimated end time, if the service provision (refueling) is "about to end", during the service provision (refueling), a fade-out can be performed, for example, the service provision speed is slowed down, etc., and at the same time, the speed of the next diagnosis can be increased or the next diagnosis can be skipped (partially, the diagnosis is aborted).

[0194] If the time for the provided service is determined in advance, or if the service is provided for a certain period of time or longer, set a diagnostic menu according to the time of the provided service, and perform a follow-up diagnosis using that diagnostic menu. When the service time is 5 minutes, set a diagnostic menu that will end within those 5 minutes. When the service time is 2 hours (120 minutes), set a diagnostic menu that will end within those 120 minutes (ST8). In this way, a follow-up diagnosis can be carried out according to the provided service.

[0195] (Perform the diagnosis in multiple parts) Regarding the diagnostic menu set in step ST8, execute the diagnosis in accordance with the fault location identification program (image formation program 8) shown in FIGS. 1 to 4 (setting step, identification step, movement control step, imaging control step, state diagnosis section) (ST9). Next, determine whether or not the monitoring items of the diagnostic menu selected in step ST8 have been completed (ST10). If not completed, save the diagnostic results up to that point and wait until the next stop at any monitoring position (identification step, state identification section: the bottom surface image acquisition step and the vehicle diagnosis step are associated with and interlocked with each other and executed multiple times) (ST5). When the monitoring items are completed in ST10, store the diagnostic results, determine the fault location, and if a fault location is found, form a repair estimate file and send it promptly to the user without a time lag (diagnostic information output section) (ST11). The repair estimate can be sent to the user's mobile terminal or in-vehicle computer, etc. Wait for a repair request from the user (ST12).

[0196] The moving body 5 is controlled by an imaging program installed on a failure location identification terminal 2 such as a computer 2a or a mobile terminal 2b. This imaging program executes, for example, a setting step ST3 that enables the user to set whether or not to capture an under-vehicle image of the vehicle 10a to be observed, an identification step ST5 that identifies operation information regarding the operating state of the vehicle 10a to be observed, a movement control step ST9 that controls the movement of a moving means (moving body 5) capable of moving in the space between the ground surface of the vehicle to be observed and the space, and an imaging control step ST9 that controls the imaging of the under-vehicle image by the imaging device (failure location identification terminal 2, server 3, imaging camera 4, camera 51, etc.) when controlling the movement of the moving means (moving body 5).

[0197] The user who has received the repair estimate file can send a repair request from the relevant app on a mobile terminal or an in-vehicle computer, etc. When the server 3 of the under-vehicle image imaging device 1 receives a repair request from the user at order receiving ST12, it saves the order receiving details (repair estimate file) and issues (sends) an order slip to the store designated by the user at ST13. The store that has received the order slip can place an order for parts and arrange maintenance personnel according to the date and time designated by the user. After that, if the user does not end the app at ST14, it returns to the above ST3. When the user ends the app at ST14, the execution of the app is ended at ST15.

[0198] (System for handling vehicle identification information) As shown in FIGS. 5 and 12, the bottom surface image capturing device 1 is mounted on any one of image forming devices (computer 2a, mobile terminal 2b, server 3, imaging device 4, moving body 5, camera 51, etc.) that form a bottom surface image of the observation target 10 (vehicle 10a), or has an image forming program 8 mounted on each device so that these devices cooperate with each other. Any one of the image forming devices (2 to 6) has an imaging control unit ST3 that selects or sets permission or rejection of imaging of the bottom surface image, a vehicle identification unit (acquisition unit) ST4 that identifies the vehicle that has captured the bottom surface image, and a vehicle information holding unit (acquisition unit) ST4 that stores at least one of the bottom surface image for each vehicle and the component information of the vehicle bottom surface. Further, a vehicle information extraction unit (state diagnosis unit) ST9 (diagnostic information output unit ST11) is provided that extracts at least one of the bottom surface image held by the vehicle information holding unit ST4 and the component information of the vehicle bottom surface based on the vehicle identification information of the vehicle identified by the vehicle identification unit ST4.

[0199] The vehicle identification unit ST4 obtains the information by the user inputting it himself / herself through a mobile terminal or the like, communicating with the in-vehicle computer to download it, or communicating with the IC tag of the vehicle inspection certificate. The vehicle identification information is information sufficient to identify the vehicle and each component. It can be the vehicle manufacturer name, vehicle name, model, vehicle type, body shape, model year, registration number, vehicle number, chassis number, type of automobile, use (passenger, cargo), private (commercial), seating capacity, vehicle weight, type of fuel, type of engine, displacement, rated output, length / width / height, axle weight, vehicle identification code (vehicle ID), noise, other matters described in the vehicle inspection certificate, and the user's personal information associated therewith, etc.

[0200] (System using user information) Furthermore, as shown in FIGS. 5 and 12, the bottom surface image capturing device 1 is provided with the following components in any one of the image forming devices (2 to 6) or the image forming program 8 on which each device is mounted so as to be connected. The bottom surface image capturing device 1 includes a user information holding unit STS4 that holds user information of a user who owns or uses a vehicle, and a vehicle diagnosis unit (state diagnosis unit) ST9 (image forming program 8) that acquires the captured bottom surface image and diagnoses the vehicle based on the bottom surface image, and includes a diagnosis result output unit ST11 that outputs the diagnosis result by the vehicle diagnosis unit ST9 to the user of the user information held by the user information holding unit 3 such as the server 3 (diagnosis information output unit).

[0201] The user information holding unit STS4, the vehicle diagnosis unit (movement control step, imaging control step, state diagnosis unit) ST9, the user information holding unit (diagnosis information output unit) ST11, and the diagnosis result output unit ST11 may be any one of the failure location specifying terminal 2 (computer, mobile terminal, etc.), server 3 (Internet, cloud, external storage device, etc.), imaging device 4 (mobile terminal, camera, moving body, etc.) shown in FIG. 5, or software mounted so that they cooperate with each other.

[0202] All kinds of data acquired by the bottom surface image capturing device 1 and the bottom surface image capturing program are accumulated in the big data managed by the server 3 and can be used.

Industrial Applicability

[0203] The image forming device, image forming method, image forming program, and bottom surface image capturing program of the present embodiment can be used to check the bottom surface part of an artificial object or a natural object such as a mountain or a cave. In particular, it can be used to check and inspect the bottom surface part of industrial products such as automobiles, ships, and aircraft, and artificial objects such as buildings, and to collect the obtained information.

Explanation of Reference Numerals

[0204] 1 Data storage system 2 Failure location specifying terminal 2a Same computer 2b Mobile terminal 3 Server 3a Internet 3b Cloud 3c External storage device 4 Camera 5 Mobile body 50 Holder part 50a Joystick 50b Light (lighting control unit) 50c Distance measurement unit (distance measurement control unit) 51 Camera 51c Recording control unit 52 Movement control unit (route calculation processing unit) 53 Communication device 54 Removable media 54h Hardware interface 54s Slot 54t Connection terminal 6 Mobile terminal 7 Travel drive source (floating drive source) 7a Driving wheel 7b Steering wheel 7c Universal caster (grounding wheel) 71 Rotating wing (fixed wing or hovercraft structure, etc.) 8 Image formation program P1 Contour setting unit P2 Component identification unit P3 Map assignment processing unit P4 Map setting processing unit P5 Component information acquisition processing unit P6 Image formation processing unit 9 Fault location identification program P7 Component comparison processing unit P8 Component inspection processing unit P9 Judgment processing unit P10 Document creation processing unit 10 Observation target 10a Automobile 10b Building 10u Bottom part 10h same suspended ceiling 10L same contour of the observation target range 10M lower rotation frame map 10S section 10F section where shooting has been completed 10N section where shooting has not been completed yet 100 bottom image 101 information of the photographed parts 102 information of the parts of the standard information

Claims

1. An image forming apparatus for forming an image of an undercarriage of a moving object captured by an imaging device, A space between the moving body and the ground surface of the moving body can be connected to a moving means capable of moving the space, an imaging setting unit that can set whether or not to allow the undercarriage image to be captured; a state identification unit that identifies operation information related to an operation state of the moving object; a movement control unit that controls movement of a moving means in the space when the imaging setting unit has set the imaging to be permitted and the moving object is in a state where it can be imaged based on the operation information identified by the state identification unit; an imaging control unit that controls capturing of the undercarriage image by the imaging device when the movement control unit controls the movement of the moving means; An image forming apparatus comprising:

2. a time specifying unit that specifies a service time for a service provided in a state in which the moving object can be imaged; Equipped with The movement control unit is The image forming apparatus according to claim 1 , wherein the moving means moves from when the image capturing state is reached until a service time specified by the time specifying unit has elapsed.

3. The time specifying unit is a time estimation unit that estimates a time from a start of provision of the provided service to an end of provision of the provided service based on the content of the provided service; Equipped with The image forming apparatus according to claim 2 , wherein the time estimation unit estimates the time until the end of the provision of the service by using provision history information about the provision of the service up to now.

4. An acquisition unit that acquires mobile object information related to the mobile object; a condition diagnosis unit that performs a condition diagnosis of the moving body based on the undercarriage image; a diagnostic information output unit that outputs diagnostic information related to the state diagnosis; Equipped with The diagnostic information output unit 2. The image forming apparatus according to claim 1, further comprising: outputting diagnostic information on a part constituting the movable body, the diagnostic information being included in the undercarriage image; and work information for performing at least one of repairing and replacing the part based on the diagnostic information.

5. The imaging control unit is A determination unit that determines whether or not the image of the entire underside of the inside of the contour of the moving object has been captured by the imaging device. Equipped with If the state identification unit determines that the moving object is not in a state in which it can be imaged before the imaging device captures the entire underbody image inside the contour, the imaging is stopped and the captured partial underbody image is stored; An image forming device as described in any one of claims 1 to 4, wherein when the partial underbody image is stored and the state identification unit is in a state where the moving body can be imaged, control is performed to capture another partial underbody image different from the partial underbody image within the contour.

6. An image forming method implemented in a computer that is an image forming apparatus that forms an undercarriage image of a moving object captured by an imaging device, comprising: The computer, The imaging of the undercarriage image can be set to allow or disallow. identifying operational information relating to an operational state of the moving object; When the imaging is permitted and the moving body is in a state where it can be imaged based on the identified operation information, controlling the movement of a moving means capable of moving in a space between the moving body and a ground surface of the moving body in the space; and controlling, while controlling the movement of the moving means, the imaging device to capture the undercarriage image.

7. A computer that is an image forming device that forms an undercarriage image of a moving object captured by an imaging device, A setting step for setting whether or not to allow the undercarriage image to be captured; an identification step of identifying operational information relating to an operational state of the moving object; a movement control step of controlling, in a space between the moving body and a ground surface of the moving body, movement of a moving means capable of moving in the space when the imaging is permitted and the moving body can be imaged based on the identified operation information; an imaging control step of controlling the imaging of the undercarriage image by the imaging device while controlling the movement of the moving means; and

8. A bottom image capturing program for controlling capturing an image in an image forming apparatus that forms a bottom image of an object to be observed, A bottom image capturing program comprising at least an imaging control step of selecting or setting permission or refusal of capturing a bottom image in the image forming apparatus.

9. A bottom surface image acquisition step of acquiring a captured bottom surface image; A vehicle diagnosis step of diagnosing a vehicle based on the acquired underside image, an imaging time specifying step of specifying an imaging possible time based on an imaging situation (location, service, facility, etc.) of the bottom image; 9. The underside image capturing program according to claim 7, further comprising a diagnosis content specifying step of specifying a diagnosis content to be executed in a vehicle diagnosis step based on the capturing time.

Images