Inspection support device and inspection support method

Abstract

The present invention provides an inspection support device and an inspection support method for more easily performing inspections of target components in buildings. [Solution] The inspection support device 20 of the present invention includes a processor 21, which acquires shape information regarding the shape of target members located in the target space among the constituent members of the actual house from a user terminal 12 placed in the target space within the actual house, identifies virtual target members corresponding to the target members from the portion corresponding to the target space in the three-dimensional model of the virtual house corresponding to the actual house based on the shape information, acquires inspection information associated with the virtual target members, and notifies the user of the inspection content indicated by the inspection information.

Description

Technical Field

[0001] The present invention relates to an inspection support device and an inspection support method, and more particularly to an inspection support device and an inspection support method that can be used when inspecting target members in a building.

Background Art

[0002] In a building under construction or already constructed, there is a demand for labor-saving and simplification of inspection work targeting the constituent members of the building.

[0003] On the other hand, as design data of a building, a three-dimensional model of a virtual building reproduced by BIM (Building Information Modeling) may be used in some cases (see, for example, Patent Document 1). In this case, by collating the inspection target member in the actual building (hereinafter also referred to as the actual building) with the corresponding part in the above three-dimensional model, it is possible to confirm whether the inspection target member is being used as designed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when trying to manually find the part (object) corresponding to the inspection target member in the three-dimensional model, it may be time-consuming. Also, in the inspection work, when collating the inspection target member in the actual building with the corresponding part in the three-dimensional model, it is necessary to grasp from what perspective to collate, that is, to understand the inspection content, which has been a burden for inspectors.

[0006] Therefore, the present invention has been made in view of the above problems, and its objective is to provide an inspection support device and an inspection support method for more easily performing inspections of target components in a building. [Means for solving the problem]

[0007] The above problems are solved by the inspection support device of the present invention, which includes a processor, wherein the processor acquires shape information regarding the shape of a target member located in the target space from measuring instruments placed in the target space within the building, identifies a virtual target member corresponding to the target member from the portion of a three-dimensional model of a virtual building corresponding to the building that corresponds to the target space, based on the shape information, acquires inspection information associated with the virtual target member, and notifies the user of the inspection content indicated by the inspection information. By using the inspection support device of the present invention, inspections of target components in buildings can be performed more easily.

[0008] Furthermore, in the inspection support device of the present invention, the processor may identify the placement position and orientation of the measuring instrument based on information acquired from the measuring instrument, and identify a virtual target member from the portion corresponding to the target space based on the placement position and orientation of the measuring instrument and shape information. With the above configuration, it is possible to more appropriately identify the virtual target member corresponding to the target member within the three-dimensional model of the virtual building.

[0009] Furthermore, in the inspection support device of the present invention, the processor may acquire multiple captured images of the area within the target space that includes the target member, and store at least one of the multiple captured images in a storage device. With the above configuration, the reliability of the inspection results for the target component can be ensured by storing images of the area including the target component.

[0010] Furthermore, in the inspection support device of the present invention, when the processor displays an image of the area including the target member on a display, it may also display the inspection content superimposed on the image. With the above configuration, the inspection details for the target component can be properly verified.

[0011] Furthermore, in the inspection support device of the present invention, the processor may display input objects for inputting inspection results regarding the inspection content superimposed on the captured image. With the above configuration, inputting test results can be made much easier.

[0012] Furthermore, in the inspection support device of the present invention, the processor may store the captured image and the inspection results regarding the inspection content in a storage device in association with the target member. With the above configuration, by associating the images of the inspection area and the inspection results with the target component and storing them accordingly, the inspection results can be properly managed, and the reliability of the inspection results can be ensured.

[0013] Furthermore, the aforementioned problems are solved by the inspection support method of the present invention, which includes the steps of: the processor acquiring shape information relating to the shape of a target member located in the target space from a measuring instrument placed in the target space within the building; the processor identifying a virtual target member corresponding to the target member from the portion of a three-dimensional model of a virtual building corresponding to the building that corresponds to the target space, based on the shape information; the processor acquiring inspection information associated with the virtual target member; and the processor notifying the user of the inspection content indicated by the inspection information. According to the method described above, inspections of target components in a building can be carried out more easily. [Effects of the Invention]

[0014] According to the present invention, inspection of target components in a building can be carried out more easily. [Brief explanation of the drawing]

[0015] [Figure 1] A diagram showing an information processing system including an inspection support device according to an embodiment of the present invention. [Figure 2] A diagram showing an example of a display screen of a part of a three-dimensional model of a virtual building. [Figure 3] An explanatory diagram of the functions of an inspection support device according to an embodiment of the present invention. [Figure 4] A diagram showing an example of a display screen of inspection contents. [Figure 5] A diagram showing an example of an input screen of inspection results. [Figure 6] A diagram showing an example of a display screen of inspection results. [Figure 7] A diagram of a processing flow using an inspection support device according to an embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0016] <<Regarding One Embodiment of the Present Invention>> Hereinafter, one embodiment of the present invention (hereinafter, this embodiment) will be described with reference to the accompanying drawings. This embodiment relates to a technique for supporting building inspections, and more specifically, to an information processing technique for building inspections using BIM data.

[0017] In this specification, the concept of "device" includes a single device that exhibits a specific function by itself, as well as a combination of a plurality of devices that are distributed and exist independently but cooperate (work together) to exhibit a specific function.

[0018] Also, since the basic data processing technologies (communication / transmission technology, data acquisition technology, data recording technology, data processing / analysis technology, image processing technology, and visualization technology, etc.) for realizing the content of this embodiment are well-known technologies, the description thereof will be omitted.

[0019] Furthermore, this embodiment will be described assuming that the "building" is a residence. Here, the residence may be a detached house or a single dwelling unit in an apartment building. The present invention is also applicable to buildings other than residences, such as shops, offices, commercial facilities such as movie theaters and department stores, public facilities such as hospitals and schools, factories, or other buildings with various uses.

[0020] <<Overview of the test>> Before describing the inspection support device and inspection support method according to this embodiment, we will outline the inspection performed in this embodiment. The inspection is performed on a house, and more precisely, on a house under construction. In other words, the inspection according to this embodiment is performed during the construction of a house. The inspection checks whether the components of the house, more specifically, the fixtures and building components placed in each space within the building, are installed or used correctly according to the design.

[0021] To explain in more detail, an inspector (for example, a construction worker or construction supervisor) goes to the construction site, i.e., the site of a house construction, to the space where the components to be inspected (hereinafter referred to as "target components") are located, and inspects whether the target components are located or used according to the construction instructions. The inspection content consists of inspection items, which are set for each target component. In addition, the inspection content may be set according to the work (process) performed using the target components in the construction work. The inspection content may include, for example, whether the placement position and mounting orientation of the target components are correct, whether the condition of the target components after placement is normal, whether the type and specifications of the target components are correct, and whether the target components have deteriorated. Furthermore, multiple inspections may be conducted simultaneously; that is, inspections of multiple target components may be carried out at the same time.

[0022] The inspection results are stored in a designated memory location, for example, by the inspector operating a user terminal and entering the data. On the other hand, if the inspection results for the target component are not satisfactory, the inspector issues corrective instructions to the contractor, who then corrects the placement or usage of the component. After the correction, the inspector conducts the inspection again and enters the results of the re-inspection through the user terminal.

[0023] Furthermore, in this embodiment, BIM data relating to the house under construction is used during inspection. Using the BIM data, a three-dimensional model of a virtual house (corresponding to a virtual building) that reproduces the above-mentioned house is constructed in a virtual space. Each part of this three-dimensional model corresponds to a space in the actual house (hereinafter referred to as the actual house). In addition, the part of the three-dimensional model that corresponds to a predetermined space within the actual building includes objects (hereinafter referred to as virtual members) that reproduce members (hereinafter referred to as actual members) that are actually placed in that predetermined space. In other words, the three-dimensional model includes a part that corresponds to the space in the actual house where the target members are placed (hereinafter referred to as the target space), and in that part, there are virtual members (hereinafter referred to as virtual target members) that correspond to the target members.

[0024] In other words, in this embodiment, the portion of the three-dimensional model where the virtual target member is located corresponds to the space (location) where the target member is placed in the actual house. In other words, the placement of the target member can be confirmed by the position of the virtual target member on the three-dimensional model. Furthermore, a portion or the entirety of the three-dimensional model of the virtual house can be displayed on the user terminal, as shown in Figure 2.

[0025] When inspecting a target component, the inspector confirms the specifications of the virtual target component, such as its type, shape, size, usage, and orientation when placed, and checks whether the target component is placed or used within the target space according to the confirmed specifications.

[0026] Furthermore, in this embodiment, the inspector uses a communication terminal (hereinafter referred to as the user terminal 12) that they possess, and more precisely, launches an inspection application program (hereinafter referred to as the inspection app) on the user terminal 12 and uses the inspection app to perform inspections of the target components. Specifically, the inspection app can display the inspection details regarding the target components on the display 18 (corresponding to a display unit) of the user terminal 12, and the inspector can confirm the inspection details through the display. In addition, the inspector can input inspection results through the user terminal 12 using the inspection app. The input inspection results are saved in a predetermined storage location, and construction personnel, supervisors, etc., can check the saved inspection results as needed and use the saved inspection results for report creation, etc.

[0027] Furthermore, in this embodiment, as evidence of the inspection results, photographs taken at the location where the inspection was conducted (hereinafter also referred to as the inspection location) are saved in association with the inspection results. This allows construction personnel and supervisors to check the photographs taken at the inspection location along with the inspection results.

[0028] <<Regarding the configuration of the inspection support system>> Next, the information processing system used in the aforementioned inspection (hereinafter referred to as the inspection support system 100) will be explained with reference to Figure 1. As shown in Figure 1, the inspection support system 100 consists of an application server 10 that constitutes the inspection support device 20, a user terminal 12 used by the user, and multiple databases 25 to 28. Here, "user" refers to a person who uses the functions of the inspection support device 20, and in this embodiment, for example, a person who indirectly operates the inspection support device 20 through the user terminal 12, specifically an inspector. However, it is not limited to this, and a user may also be a person who directly operates the inspection support device 20.

[0029] The user terminal 12 is a device operated by the user and consists of a smartphone, mobile phone, tablet, laptop computer, or wearable device. As mentioned above, the user terminal 12 has an inspection application installed, and when the user conducts an inspection of the target building in an actual house, the user launches the inspection application on the user terminal 12 and performs various inspection operations with the application running. The user terminal 12 can transmit information corresponding to the user's operations to the application server 10 via the network N. In the inspection support system 100, as shown in Figure 1, there are a number of user terminals 12 corresponding to the number of users using the inspection support device 20, but the number of user terminals 12 only needs to be at least one.

[0030] Furthermore, as shown in Figure 1, the user terminal 12 is equipped with a shooting device 14 such as a camera or image sensor, and the shooting device 14 captures subjects within its field of view. Images that can be captured by the shooting device 14 include still images and videos, and videos include real-time videos. The user terminal 12 can transmit images captured by the shooting device 14 to the application server 10 via the network N.

[0031] Furthermore, in this embodiment, the user terminal 12 is used as a measuring instrument when inspecting the target component. Specifically, as shown in Figure 1, the user terminal 12 is equipped with a shape sensor 16, such as a LiDAR (Laser Imaging Detection and Ranging) sensor, and measures the shape of the target component while the user terminal 12 is positioned in the space where the target component is located in an actual house (i.e., the target space) using the shape sensor 16. The user terminal 12 can transmit information indicating the shape measurement result by the shape sensor 16 (hereinafter referred to as shape information) to the application server 10 via the network N. Note that the device that performs the process of generating shape information based on the output signal of the shape sensor 16 may be the user terminal 12 or another device (for example, the application server 10).

[0032] Furthermore, in this embodiment, as shown in Figure 1, the user terminal 12 is equipped with a display 18 as a display device, and is capable of displaying information sent from the application server 10 on the display 18. The user terminal 12 is also capable of displaying images captured by the shooting device 14, more specifically, real-time video, on the display 18. Moreover, the user terminal 12 is capable of displaying information sent from the application server 10 superimposed on the captured image on the display 18. In other words, the application server 10 can use so-called AR (Augmented Reality) technology to superimpose information related to the inspection of target components and objects for receiving operations onto the captured image displayed on the display 18 of the user terminal 12.

[0033] The display 18 may be a display built into the user terminal 12 itself, or a display connected to the user terminal 12 via a wired or wireless connection. Displays connected to the user terminal 12 may include not only general stationary monitors, but also HMDs (Head Mounted Displays) such as VR goggles, and glasses-type displays that allow images to be viewed through the glasses, such as smart glasses. In this embodiment, the display 18 is assumed to be a touch panel type display built into the user terminal 12.

[0034] The application server 10, which constitutes the inspection support device 20, is an example of a computer. The application server 10 may consist of a single server, or it may consist of multiple computers arranged in parallel and distributed. Furthermore, the application server 10 may be a server computer for ASP (Application Service Provider), SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service). In this case, when the necessary information is entered on the client terminal, the server computer performs various processes and calculations based on the input information, and the calculation results are output on the client terminal side. In other words, the functions of the application server 10, which is the inspection support device 20, can be used on the client terminal (user terminal 12) side.

[0035] As shown in Figure 1, the application server 10 has a processor 21, memory 22, storage 23, and a communication interface 24. The processor 21 is composed of, for example, a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a TPU (Tensor Processing Unit), an NPU (Neural Network Processing Unit), or an ASIC (Application Specific Integrated Circuit). The memory 22 is composed of semiconductor memory such as ROM (Read Only Memory) and RAM (Random Access Memory).

[0036] The storage 23 consists of, for example, flash memory, HDD (Hard Disc Drive), SSD (Solid State Drive), FD (Flexible Disc), MO disk (Magneto-Optical disc), CD (Compact Disc), DVD (Digital Versatile Disc), SD card (Secure Digital card), or USB memory (Universal Serial Bus memory). The storage 23 may be built into the main body of the application server 10, or it may be attached to the main body of the application server 10 as an external device.

[0037] The communication interface 24 may be configured, for example, by a network interface card or a communication interface board. The application server 10 can communicate data with other devices connected to the internet or a mobile communication line, etc., via the communication interface 24.

[0038] Furthermore, the application server 10 has software installed, including programs for the operating system (OS) and application programs for inspection support. When these programs are read and executed by the processor 21, the application server 10 functions as an inspection support device 20, specifically by performing a series of information processing tasks related to inspection work. The application programs for inspection support include programs that identify parts and equipment in images through image analysis, programs that construct a three-dimensional model of a virtual house, and programs that identify virtual components present in a specified part of the three-dimensional model (specifically, the part corresponding to the target space described later). Furthermore, known technologies using BIM data can be used to construct a three-dimensional model of a virtual house and to identify virtual components within the three-dimensional model.

[0039] Multiple databases 25-28 store information related to inspections and are configured, for example, by a cloud-based database server. As shown in Figure 1, the application server 10 is connected to each database server via the network N and can freely read the information stored in each database server. In this embodiment, multiple databases 25-28 are used separately according to the content (items) of the information to be stored in each database, but various types of inspection-related information may also be consolidated and stored in a single database.

[0040] To explain the information stored in each database, the first database 25 stores information about each virtual house. As mentioned above, a virtual house is a virtual representation of a completed house under construction. Information regarding the virtual dwelling includes a three-dimensional model of the virtual dwelling, the types, shapes, colors, sizes, specifications, and placement locations of the virtual components included in the three-dimensional model, and detailed information about the actual components corresponding to the virtual components. Detailed information regarding actual components includes the type, shape, color, size, specifications, placement location, installation method, the process in which the actual components are used in the construction of the actual house, the inspection content and methods related to the actual components or the process in which the actual components are used, and the criteria for determining the quality of the construction. Thus, the first database 25 stores inspection information, including inspection content and inspection methods, in association with virtual components (i.e., virtual target components) that correspond to the actual components being inspected. Furthermore, the first database 25 stores information such as drawings (construction drawings, floor plans, etc.) of virtual houses and their corresponding actual houses, information on the structure and specifications of the actual houses, and information on the construction sites of the actual houses.

[0041] The second database 26 corresponds to the storage device of the present invention and stores information regarding the results of inspections conducted in actual houses for each actual house. The information regarding the inspection results includes identification information of the actual house where the inspection was conducted, identification information of the components (target components) that were inspected, their placement in the actual house, the inspection content, the inspection results, the location of the inspection site, and photographs of the inspection site or address information of the location where such photographs are stored. This information is stored in the second database 26 for each target component that has been inspected. In other words, in this embodiment, the inspection results regarding the inspection content and the photographs that serve as evidence thereof are stored in the second database 26 in association with the target component.

[0042] Database 327 stores summary information regarding the results of inspections conducted at actual houses, for each house. This summary information includes the identification information of the house that was inspected, the inspection details, and the inspection results. Of this information, the inspection details and inspection results are stored on a building-by-building basis, or on a construction phase (work section) basis, or on a floor-by-floor basis.

[0043] Database 428 stores information regarding the inspection status of actual houses, for each individual house. This information is used to manage the progress of inspections, and specifically indicates what types of inspections were conducted and the results of those inspections. This information is stored on a building-by-building basis, or by construction phase (work section) in the building's construction work, or by floor of the building.

[0044] <<About the functions of the inspection support device according to this embodiment>> Next, the configuration of the inspection support device 20 according to this embodiment will be described again from a functional standpoint. As shown in Figure 3, the inspection support device 20 has a location identification unit 31, a shape information acquisition unit 32, a target identification unit 33, an inspection information acquisition unit 34, a notification unit 35, a result reception unit 36, a storage unit 37, and an output unit 38. These functional units are realized through the cooperation of hardware equipment provided by the application server 10 and software including programs installed on the application server 10. Each functional unit will be described below.

[0045] (Location specifying part) The location identification unit 31 identifies the approximate location where the component to be inspected (target component) is placed in the actual house under construction, on a three-dimensional model of the virtual house corresponding to the actual house. In this embodiment, the location identification unit 31 identifies the approximate location of the target component using AR technology (specifically, self-position estimation technology) or the like, based on operations performed by the user, who is the inspector, through the user terminal 12, and information that the user terminal 12 can acquire.

[0046] To explain in more detail, when conducting an inspection, the user inputs the floor on which they are located and their initial position in the actual house via the user terminal 12. The initial position can be arbitrarily specified by the user and is, for example, the starting point from which the user moves towards the target component. The input information is transmitted from the user terminal 12 to the application server 10.

[0047] Furthermore, the user moves from their initial position towards the target component while capturing images of the interior of the actual house using the camera device 14 on the user terminal 12. The captured images at this time are real-time videos and are transmitted sequentially from the user terminal 12 to the application server 10. Subsequently, when the user reaches the space in the actual house where the target component is located, i.e., the target space, they capture a video of the area within the target space that includes the target component (the shooting range), and this video is transmitted from the user terminal 12 to the application server 10.

[0048] When the location identification unit 31 receives information transmitted from the user terminal 12, it first identifies the floor and initial position entered by the user in the three-dimensional model of the virtual house. Next, the location identification unit 31 identifies the movement path from the initial position to the target space based on video footage or the like taken while the user moves from the initial position to the target space, and then identifies the part of the three-dimensional model corresponding to the target space (hereinafter referred to as the "target space equivalent part") from the identified movement path.

[0049] Next, the position identification unit 31 identifies the placement position and orientation of the user terminal 12 in the target space based on the video of the area of ​​the target space that includes the target component. Specifically, the position identification unit 31 recognizes each device in the video by analyzing the video. The position identification unit 31 also recognizes two or more virtual components (a group of virtual components) that exist in the area corresponding to the target space in the three-dimensional model by referring to the information of virtual components stored in the first database 25. Then, the position identification unit 31 identifies the placement position and orientation of the user terminal 12 by comparing each device in the video with each of the virtual component groups. The orientation of the user terminal 12 refers to the position of the shooting device 14 equipped on the user terminal 12 in the pan direction (horizontal direction), tilt direction (vertical direction), and roll direction (rotational direction).

[0050] The position identification unit 31 then identifies the approximate position of the target member on the three-dimensional model based on the placement position and orientation of the identified user terminal 12. Here, identifying the approximate position of the target member on the three-dimensional model is equivalent to identifying the approximate position of a virtual target member corresponding to the target member.

[0051] (Shape information acquisition unit) The shape information acquisition unit 32 acquires shape information from the user terminal 12, which is placed in the target space of an actual house, when the shape sensor 16 measures the shape of the target component within the target space.

[0052] (Targeted Department) The target identification unit 33 identifies a virtual target member corresponding to the target member from the portion of the three-dimensional model of the virtual house corresponding to the target space. In this embodiment, the virtual target member is identified based on the approximate position of the target member identified by the position identification unit 31, the placement position and orientation of the user terminal 12 in the target space identified by the position identification unit 31, and the shape information acquired by the shape information acquisition unit 32.

[0053] Specifically, the target identification unit 33 identifies two or more virtual members (a group of virtual members) that are positioned in the same orientation as the user terminal 12 when viewed from a position corresponding to the placement location of the user terminal 12 in the area corresponding to the target space. The target identification unit 33 also refers to the virtual member information stored in the first database 25 and recognizes the shape of each of the virtual members in the group. Subsequently, the target identification unit 33 compares the shape of the target member indicated by the shape information acquired by the shape information acquisition unit 32 with the shape of each of the virtual members in the group recognized as described above. As a result, the target identification unit 33 determines which virtual member's shape matches or is similar to the shape of the target member. Once this determination is made, the virtual member having a shape that matches or is similar to the shape of the target member is identified as the virtual target member.

[0054] (Inspection Information Acquisition Department) The inspection information acquisition unit 34 accesses the first database 25 to acquire inspection information associated with the virtual target member identified by the target identification unit 33. Specifically, as mentioned above, the first database 25 stores inspection information associated with virtual members, and the inspection information acquisition unit 34 reads the inspection information associated with the virtual member corresponding to the virtual target member from the first database 25. In this way, the inspection information associated with the virtual target member is acquired. Furthermore, if the inspection information is also associated with the construction work process, the inspection information associated with the virtual target component and the current process should be read from the first database 25. In this case, the current process may be entered by the user through the user terminal 12, or the inspection support device 20 may automatically determine it from the progress of the construction work.

[0055] (Notification Department) The notification unit 35 notifies the user, i.e., the inspector, of the inspection content indicated by the inspection information acquired by the inspection information acquisition unit 34. Specifically, the notification unit 35 generates display data for the inspection content and transmits it to the user terminal 12, thereby displaying the inspection content (more specifically, a text object indicating the inspection content) on the display 18 of the user terminal 12. In this embodiment, as shown in Figure 4, the notification unit 35 overlays a text object indicating the inspection content (in the case shown in Figure 4, the text "high intensity") onto the video using AR technology while the captured image, more precisely, the video, is displayed on the display 18 of the user terminal 12.

[0056] To explain in more detail, the user places the user terminal 12 within the target space when performing the inspection, and uses the user terminal 12's imaging device 14 to capture a range of the target space that includes the target component (the imaging range). As a result, a video of the range including the target component is displayed on the display 18. During this time, the notification unit 35 overlays a text object indicating the inspection content onto the video. The inspection content displayed at this time is the inspection content associated with the virtual target component, that is, the inspection content for the target component. The position where the notification unit 35 displays the inspection content is not particularly limited, but as shown in Figure 4, it is preferable that it is displayed in a position corresponding to the area on the display 18 where the target component is displayed, specifically near the display area of ​​the target component.

[0057] Furthermore, if the target component is a relatively small component such as a fastening component or metal fitting, the notification unit 35 may, when displaying the inspection details, superimpose an object indicating the position where the target component should be according to the design (for example, the rectangular frame-shaped object in Figure 4) onto the captured image and display it on the display 18, as shown in Figure 4.

[0058] (Results Reception Department) The result receiving unit 36 ​​receives inspection results regarding the inspection content of the target component, specifically, it receives input of inspection results from the user. In this embodiment, the user inputs the inspection results through the user terminal 12, and the result receiving unit 36 ​​obtains information indicating the inspection results input by the user from the user terminal 12. More specifically, as shown in Figure 5, the result receiving unit 36 ​​displays an input object for inputting inspection results regarding the inspection content on the display 18 of the user terminal 12 by superimposing it on the captured image. The captured image displayed on the display 18 at this time is a captured image of the area within the target space that includes the target component, and more specifically, it is a real-time video. The input object is a touch button indicating a candidate for the inspection result (in the case shown in Figure 5, a button labeled "OK" or "NG"), and the user can input the inspection result by touching the button corresponding to the inspection result.

[0059] The method for inputting inspection results is not limited to the method described above, and other methods may be used. For example, the user may input text indicating the inspection result in an input field displayed on the screen. In this case, the user may input text indicating whether or not correction is necessary for the target part, and if so, the details of the correction. Alternatively, the user may input the inspection result by voice. In this case, the result receiving unit 36 ​​may accept the user's input of the inspection result by applying speech recognition processing and natural language processing to the user's voice. Alternatively, the user may input the inspection result by photographing the target part with the imaging device 14. In this case, the result receiving unit 36 ​​may accept the user's input of the inspection result by analyzing the captured image to identify the type, characteristics, and condition of the target part.

[0060] (Storage part) The memory unit 37 stores the inspection results received by the result receiving unit 36 ​​in the second database 26, which acts as a storage device. More specifically, the memory unit 37 stores the results of the inspections performed on the target component in the second database 26, associating them with the actual house where the inspection was performed and the target component.

[0061] Furthermore, in this embodiment, the storage unit 37 acquires an image of the area including the target member captured by the shooting device 14 of the user terminal 12 during the inspection, and stores both the captured image and the inspection results in the second database 26, associating them with the actual house and target member that were inspected. By storing the captured image, which serves as evidence of the inspection results, together with the inspection results, the reliability and credibility of the inspection results can be ensured. In this embodiment, the image captured during the inspection is a video consisting of multiple captured images (more specifically, frame images), and the storage unit 37 extracts at least one frame of captured images, preferably several frames of captured images, from the video and stores the extracted captured images together with the inspection results in the second database 26. However, it is not limited to this, and the captured video may be stored as is.

[0062] (Output section) The output unit 38 reads the inspection results of the target component stored in the second database 26 and outputs the read inspection results to the user. More specifically, the user operates the user terminal 12 to request the inspection support device 20 to provide the results of the inspection performed on the target component. Upon receiving such a request from the user terminal 12, the output unit 38 accesses the second database 26, reads the inspection results for the target component, generates display data for the inspection results, and transmits it to the user terminal 12. As a result, as shown in Figure 6, the inspection results for the target component are displayed on the display 18 of the user terminal 12 along with the target component and the inspection details. At this time, as shown in Figure 6, the floor number corresponding to the inspection location in an actual house, the process classification (work type classification) in which the target part is used, and the inspection classification may also be displayed. Furthermore, if multiple target components (inspection targets) belong to the same floor number, work type classification, and inspection classification, the inspection details and inspection results for each of the multiple target components may be displayed in a list, as shown in Figure 6.

[0063] The inspection results may be displayed on the display 18 by superimposing them onto the captured image of the area including the target component, which was captured by the user terminal 12's shooting device 14, similar to how the inspection content was displayed. Alternatively, the inspection results may be displayed on a three-dimensional model of the virtual house. For example, with the floor plan of the area corresponding to the target space of the three-dimensional model displayed on the display 18, the inspection results may be displayed near the virtual target component in the floor plan.

[0064] <<About the inspection support flow according to this embodiment>> Next, we will explain the inspection support flow, which is a data processing flow using the inspection support device 20 described above. The inspection support flow employs the inspection support method of the present invention and proceeds according to the flow shown in Figure 7. In other words, each step in the flow shown in Figure 7 corresponds to each element that constitutes the inspection support method of the present invention. Note that the processing flow shown in Figure 7 is merely an example, and unnecessary steps may be deleted, new steps added, or the order of steps rearranged, without departing from the spirit of the present invention.

[0065] The inspection support flow is triggered when the user, who is the inspector, launches the inspection application on the user terminal 12 and inputs the floor and initial position of the component to be inspected (target component) in the actual house. Once the inspection support flow is started, the processor 21 of the application server 10, which constitutes the inspection support device 20, executes a series of steps in the inspection support flow.

[0066] Specifically, the processor 21 first identifies the approximate location of the target component in the actual house using the procedure described above (S001). In step S001, the processor 21 identifies the floor on which the target component is located and its initial position based on the user's input. In addition, when performing step S001, the user moves from the initial position to the target space and captures images, or more specifically videos, of each location along the movement path using the shooting device 14 of the user terminal 12. The processor 21 acquires the video from the user terminal 12, identifies the movement path from the initial position based on the video, and identifies the part of the three-dimensional model corresponding to the target space based on the identified movement path.

[0067] Furthermore, when performing step S001, the user places the user terminal 12 in the target space where the target component exists, and captures a video of the area within the target space that includes the target component (shooting range) using the shooting device 14. The processor 21 acquires the captured video from the user terminal 12 and compares each device shown in the video with the virtual component group present in the area corresponding to the target space to determine the placement position and orientation of the user terminal 12 in the target space. Then, based on the identified placement position and orientation of the user terminal 12, the processor 21 determines the approximate location of the target component.

[0068] Subsequently, when the user terminal 12 placed in the target space measures the shape of the target component using the shape sensor 16, the user terminal 12 transmits shape information indicating the measured shape, and the processor 21 receives and acquires this shape information (S002).

[0069] Next, the processor 21 identifies a virtual target member from the area corresponding to the target space of the three-dimensional model based on the shape information acquired in step S002, the approximate position of the target member identified in step S001, and the placement position and orientation of the user terminal 12 identified in step S001 (S003). In other words, in step S003, the processor 21 identifies the shape of the target member from the shape information, refers to the information of the virtual member group derived from the placement position and orientation of the user terminal 12 among the virtual member information stored in the first database 25, and recognizes the shape of each of the virtual member group. After that, the processor 21 finds a virtual member from the virtual member group that has a shape that matches or is similar to the shape of the target member, and identifies that virtual member as the virtual target member.

[0070] Next, the processor 21 accesses the first database 25 and retrieves inspection information associated with the virtual target member identified in step S003 from the inspection information stored in the first database 25 (S004). Subsequently, the processor 21 notifies the user of the inspection content indicated by the inspection information retrieved in step S004, specifically displaying it on the display 18 of the user terminal 12 (S005). At this time, the display 18 displays a captured image, or more specifically a video, of the area within the target space that includes the target member, and the processor 21 overlays the inspection content onto the captured image and displays it on the display 18.

[0071] After step S005 is completed, the user performs an inspection of the target component according to the inspection content displayed on the display 18, and inputs the inspection results into the user terminal 12, and the processor 21 accepts the user's input of the inspection results (S006). In step S006, the processor 21 displays an input object for inputting the inspection results for the inspection content superimposed on the captured image on the display 18 of the user terminal 12. The user inputs the inspection results through the above input object, and the processor 21 obtains information indicating the inspection results entered by the user from the user terminal 12.

[0072] Next, the processor 21 stores the inspection results received in step S006 in the second database 26, associating them with the actual house and target components that were inspected (S007). In this embodiment, during the inspection, the area of ​​the target space that includes the target components is photographed by the shooting device 14 of the user terminal 12, and the processor 21 acquires the captured images, or more precisely, a video. Then, in step S007, the processor 21 extracts at least one frame from the multiple captured images (frame images) included in the video, and stores the extracted captured images together with the inspection results in the second database 26, associating them with the actual house and target components.

[0073] Subsequently, when the user operates the user terminal 12 to request inspection results for the target component, the processor 21 accesses the second database 26, reads the inspection results for the target component, and displays the read inspection results on the display 18 of the user terminal 12 (S008). This allows the user to confirm the inspection results for the target component through the display 18.

[0074] The inspection support flow ends when the series of steps described above are completed. In this embodiment, by performing the inspection support flow in the procedure described above, the inspection of target components in an actual house can be carried out more easily. Specifically, in this embodiment, when performing an inspection of a target component, the location of the target component can be identified on the three-dimensional model of the virtual house using the functions of the inspection support device 20. In addition, in this embodiment, the inspection content can be easily confirmed using the inspection support device 20, and the inspection results can be input by a relatively simple means. Furthermore, in this embodiment, captured images that serve as evidence of the inspection results can be easily obtained and saved together with the inspection results. As a result, the reliability and credibility of the inspection results can be ensured.

[0075] <<Regarding other embodiments>> Although one embodiment of the inspection support device and inspection support method of the present invention has been described above, the above embodiment is merely an example to facilitate understanding of the present invention and does not limit the present invention. In other words, the present invention can be modified and improved without departing from its spirit. Furthermore, it goes without saying that the present invention includes equivalents thereof.

[0076] Furthermore, in the above embodiment, shape information regarding the shape of the target member is obtained by measuring the shape of the target member using the shape sensor 16. However, the method for obtaining shape information is not limited to this, and other methods may be used. Specifically, for example, the target member may be photographed using the imaging device 14, and the shape of the target member may be identified by analyzing the captured image, and information regarding the identified shape may be obtained as shape information.

[0077] Furthermore, in the above embodiment, while the user moves toward the target space in an actual house, the user terminal 12 captures images (specifically, videos) using the shooting device 14. In the above embodiment, the position identification unit 31 of the inspection support device 20 identifies the position and orientation of the user terminal 12 placed in the target space based on the captured images using AR technology, and identifies the approximate position of the target component within the target space from this information. However, the embodiment is not limited to this, and the user terminal 12 may be equipped with an IMU (Inertial Measurement Unit) and a GPS (Global Positioning System) unit, and these units may measure the current position and orientation of the user terminal 12. The position identification unit 31 may then receive the measurement results of the position and orientation of the user terminal 12 from the user terminal 12 via the network N, and identify the approximate position of the target component within the target space based on this information.

[0078] Furthermore, in the above embodiment, the placement position and orientation of the user terminal 12 are determined based on information acquired from the user terminal 12 as a measuring instrument, specifically, images of the target space. In the above embodiment, virtual target members are identified from the portion corresponding to the target space in the three-dimensional model of the virtual house based on the placement position and orientation of the user terminal 12 and the shape information acquired by the shape information acquisition unit 32. However, the method is not limited to this, and other information may be added to identify the virtual target members. For example, the positional relationship between the target member and its surrounding members may be determined based on images of the range (shooting range) of the target space that includes the target member, and this positional relationship may be used together with the above information to identify the virtual target member. Such identification methods are particularly effective when the target member is a relatively small member.

[0079] Furthermore, in the above embodiment, the inspection support device 20 is configured by a server computer (specifically, an application server 10), but the present invention is not limited to the above configuration, and the inspection support device may be constructed using a personal computer (PC) or workstation for personal use. Also, some functions of the inspection support device 20 may be implemented by a user terminal 12. In this case, the application server 10 (or a PC or workstation equivalent to the application server 10) and the user terminal 12 will cooperate to configure the inspection support device 20.

[0080] Furthermore, in the above embodiment, the building on which the inspection is conducted is assumed to be a building under construction. However, the present invention is not limited to this, and the inspection support device and inspection support method can also be applied to inspections conducted on buildings that have already been completed and are in use (i.e., buildings that have been handed over to the building users). [Explanation of Symbols]

[0081] 10 Application Servers 12. User terminals (measuring instruments) 14. Shooting device 16 Shape Sensors 18 displays 20 Inspection support device 21 processors 22 memory 23 Storage 24 Communication Interfaces 25. Database 1 26. Second Database (Storage Device) 27 Third Database 28. Database No. 4 31 Location identification part 32 Measurement Information Acquisition Unit 33 Targeted Department 34. Inspection Information Acquisition Department 35 Notification Department 36 Results Reception Department 37 Memory section 38 Output section 100 Inspection Support Systems N Network

Claims

1. A test support device equipped with a processor, The aforementioned processor, From measuring instruments placed in the target space within the building, shape information regarding the shape of the target component within the target space among the building's constituent members is acquired. From the three-dimensional model of the building and the corresponding virtual building, the virtual target member corresponding to the target member is identified from the portion corresponding to the target space based on the shape information. The inspection information associated with the virtual target member is obtained, An inspection support device that notifies the user of the inspection details indicated by the aforementioned inspection information.

2. The aforementioned processor, Based on the information obtained from the measuring instrument, the placement position and orientation of the measuring instrument are determined. The inspection support device according to claim 1, which identifies the virtual target member from the portion corresponding to the target space based on the placement position and orientation of the measuring instrument and the shape information.

3. The inspection support device according to claim 1, wherein the processor acquires a plurality of captured images of the area within the target space that includes the target member, and stores at least one of the plurality of captured images in a storage device.

4. The inspection support device according to claim 3, wherein the processor displays the inspection content superimposed on the captured image when displaying the captured image of the range including the target member on a display device.

5. The inspection support device according to claim 4, wherein the processor displays an input object for inputting inspection results regarding the inspection content superimposed on the captured image.

6. The inspection support device according to claim 3, wherein the processor stores the captured image and the inspection results regarding the inspection content in the storage device in association with the target member.

7. The processor obtains shape information regarding the shape of a target component located within the target space from a measuring instrument placed in the target space within the building, The processor identifies a virtual target member corresponding to the target member from the portion of the three-dimensional model of the virtual building corresponding to the building that corresponds to the target space, based on the shape information. The processor obtains inspection information associated with the virtual target member, The processor notifies the user of the inspection content indicated by the inspection information, A testing support method having [a certain characteristic].

Images