Program, method for displaying equipment information, equipment information display system, and terminal device.

Abstract

To facilitate checking of an equipment layout at an actual construction site.SOLUTION: A program is provided, for causing a processor to implement a function for managing equipment images by group, and a function for controlling display of the equipment images by group.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to a program, a method for displaying facility information, a facility information display system, and a terminal device.

Background Art

[0002] For example, Patent Document 1 discloses a piping drawing system for creating a piping drawing by arranging icons of devices and piping lines on a grid line diagram displayed on a screen. When the starting point position of a piping line is specified at the intersection of grid lines on the grid line diagram, a piping line cursor of a certain length is displayed radially along each grid line centered on the intersection, and by setting an end point at an arbitrary position of any of the displayed piping line cursors, a piping line from the starting point to the end point is drawn. A piping drawing system is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, conventionally, when designing building facilities in a building, for example, on an installed PC terminal, drawings regarding the layout of building facilities are created. And the user imagines the layout of building facilities in the building by comparing, for example, printed paper drawings and the actual building at the construction site. In this case, the image of the layout of building facilities based on the paper drawings was left to the user. Also, even when an image of the layout of building facilities could be obtained in the actual building, when trying to change the layout of building facilities, it was necessary to recreate the drawing on the PC terminal, print out the created drawing, and compare the actual building and the drawing again, which was time-consuming.

[0005] The present invention aims to facilitate the verification of equipment placement in actual field settings. [Means for solving the problem]

[0006] The invention described in claim 1 provides a processor with a function for managing images of equipment in groups, and a function for controlling the display of the images of the equipment in groups. The system includes a function to overlay images of equipment corresponding to a selected group, such that images of the equipment located inside the boundary are displayed on the front side of the display screen, and images of the equipment located on the opposite side of the boundary are displayed on the back side of the display screen, at the corresponding positions in the captured images of the actual site. This is a program designed to achieve that. The invention described in claim 2 is that the processor has a function to accept whether or not to display for each group, and an image of the equipment corresponding to the group selected as the target of display. The aforementioned The program according to claim 1 further comprises a function to display on a display screen. The invention described in claim 3 is the program according to claim 2, further comprising the function of the processor displaying images of the equipment in different colors when a plurality of groups are selected as the object of display. The invention described in claim 4 is the program according to claim 2, further comprising the function of the processor displaying images of the equipment corresponding to a selected group at corresponding positions in a two-dimensional or three-dimensional layout diagram. Claim 5 The invention described above is such that when the processor receives a predetermined operation, The aforementioned Claims 1 to 4 The program is one of the programs described in any one of the following items. The invention described in claim 6 is the program described in claim 5, wherein the processor displays in the construction cost estimate information the cost of the equipment that penetrates the boundary and is located on the opposite side of the boundary. The invention described in claim 7 further comprises the following functions of the processor: a function to make the display of the image of the equipment follow changes in the captured image taken of the actual site, and a function to accept the setting of a plurality of references that indicate a boundary where the change in the captured image begins, as described in claims 1 to 6 The program is one of the programs described in any one of the following items. The invention described in claim 8 further comprises the function of the processor receiving the setting of the arrangement of the images of the equipment penetrating the boundary of the site, as described in claims 1 to 8. 6 The program is one of the programs described in any one of the following items. The invention described in claim 9 is a program according to any one of claims 1 to 8, wherein the equipment includes at least one of the following: air conditioning equipment, plumbing equipment, electrical equipment, building equipment, equipment relating to water or other liquids, equipment relating to gas or other gases, equipment relating to electrical wiring, and equipment relating to telecommunications. The invention described in claim 10 is a program according to any one of claims 1 to 9, wherein the group is managed for each boundary surface that divides a space. The invention described in claim 11 is a group of programs according to any one of claims 1 to 10, which are provided for each type of equipment. The invention described in claim 12 is one in which a processor manages images of equipment in groups and manages the display of images of said equipment in groups. death , The images of the equipment corresponding to the selected group are superimposed on the corresponding positions in the captured images of the actual site, with the image of the equipment located inside the boundary being displayed on the foreground of the display screen, and the image of the equipment located on the opposite side of the boundary being displayed on the background of the display screen. This is a method for displaying equipment information. The invention described in claim 13 includes a functional unit for managing images of equipment in groups, and a functional unit for controlling the display of the images of the equipment in groups. A functional unit that displays images of equipment corresponding to a selected group by superimposing them on the display screen, such that images of the equipment located inside the boundary are displayed on the front side of the display screen, and images of the equipment located on the opposite side of the boundary are displayed on the back side of the display screen, at the corresponding positions in the captured images of the actual site. This is a display system for equipment information that has [specific features / features]. The invention described in claim 14 has a processor that manages images of equipment in groups and controls the display of images of equipment in groups. death , The images of the equipment corresponding to the selected group are superimposed on the corresponding positions in the captured images of the actual site, with the image of the equipment located inside the boundary being displayed on the foreground of the display screen, and the image of the equipment located on the opposite side of the boundary being displayed on the background of the display screen. It is a terminal device. [Effects of the Invention]

[0007] According to the present invention, it is possible to easily verify the arrangement of equipment in an actual site. [Brief explanation of the drawing]

[0008] [Figure 1] This is an overall diagram of the building equipment display system of this embodiment. [Figure 2] This is an explanatory diagram illustrating the overview of the building equipment display system of this embodiment. [Figure 3] This is a functional block diagram of the terminal device of this embodiment. [Figure 4] This is a functional block diagram of the management server device of this embodiment. [Figure 5] This is an example of an equipment information management table managed by a management server device. [Figure 6] This is a functional block diagram of the layout drawing creation unit of the terminal device according to the present embodiment. [Figure 7] This is an explanatory diagram of the reception screen according to the present embodiment. [Figure 8] This is an explanatory diagram of the reception screen according to the present embodiment. [Figure 9] This is an explanatory diagram of the layer structure of the boundary image according to the present embodiment. [Figure 10] This is a diagram showing the functional blocks of the AR display unit according to the present embodiment. [Figure 11] This is an explanatory diagram of the AR display by the AR display unit according to the present embodiment. [Figure 12] This is an explanatory diagram of the tracking operation in the AR display according to the present embodiment. [Figure 13] This is an explanatory diagram of the button image displayed on the display screen where the AR display is performed. [Figure 14] This is a flowchart of the operation of the AR display unit according to the present embodiment. [Figure 15] This is the functional block of the estimate creation unit according to the present embodiment. [Figure 16] This is an example of an estimate image displayed on the display screen by the estimate creation unit according to the present embodiment. [Figure 17] This is an example of a 3D layout drawing displayed based on a pre-created layout drawing. [Figure 18] This is an example of a 2D layout drawing displayed based on a pre-created layout drawing. [Figure 19] This is an example of a 2D layout drawing displayed based on a pre-created layout drawing. [Figure 20] This is an example of a 2D layout drawing displayed based on a pre-created layout drawing.

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings.

[0010] <Building Equipment Display System 1> Figure 1 is an overall view of the building equipment display system 1 of this embodiment. As shown in Figure 1, the building equipment display system 1 of this embodiment includes a terminal device 10 that assists in the design of building equipment in a building, and a management server device 20 that manages information related to building equipment handled by the terminal device 10.

[0011] In the building equipment display system 1 of this embodiment, the terminal device 10 and the management server device 20 are able to communicate information with each other via a network. The network is not particularly limited as long as it is a communication network used for data communication between each device, and may be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet. The communication lines used for data communication may be wired or wireless, or both may be used in combination.

[0012] Furthermore, the building equipment display system 1 of this embodiment can be applied to the design of building equipment such as piping through which liquids such as tap water and sewage or refrigerants for air conditioning equipment flow, indoor units of air conditioning equipment installed indoors, and outdoor units of air conditioning equipment installed outdoors. Furthermore, the building equipment display system 1 of this embodiment is not limited to water pipes and sewer pipes, but can also be applied to other building equipment such as gas pipes through which gases such as gas flow, and electrical conduits through which electrical wiring etc. can be carried. In addition, the building equipment display system 1 of this embodiment can also be applied to telecommunications-related equipment and gas-related equipment.

[0013] Here, we will describe the outline of the building equipment display system 1 of this embodiment. Figure 2 is an explanatory diagram illustrating the overview of the building equipment display system 1 of this embodiment. As shown in Figure 2(1), the user designs the building equipment to be installed in the building on the display screen 11D of the terminal device 10. In this embodiment, the design of the building equipment is performed by the user placing equipment images 75 on the reception screen 400 displayed on the display screen 11D. In this way, the building equipment display system 1 of this embodiment allows the user to design the building equipment in a sketch-like manner by tapping on the display screen 11D. Furthermore, as shown in Figure 2(2), the terminal device 10 creates a building equipment layout diagram 71 based on the user's design of the building equipment. In this description of the embodiment, the creation of the building equipment layout diagram 71 is referred to as "layout diagram creation".

[0014] Then, as shown in Figure 2(3), the terminal device 10 displays a superimposed image 800 on the display screen 11D, in which the equipment image 75 is superimposed on the captured image 120 of the building to be designed. Here, the equipment image 75 and the captured image 120 of the actual building are in a corresponding positional relationship. When the captured image 120 changes by moving the display screen 11D of the terminal device 10, the equipment image 75 follows the changed captured image 120, and the display direction of the equipment image 75 changes. In this description of the embodiment, the superimposed display of the equipment image 75 and the captured image 120 is called "augmented reality (AR) display". Here, augmented reality (AR) is the process of expanding the world seen by the user by adding information such as shapes and text to an image of a real landscape.

[0015] Subsequently, as shown in Figure 2(4), the terminal device 10 creates an estimate image 850 (an example of estimate information) showing the estimated cost for the construction of the building equipment, based on the designed building equipment layout diagram 71. The terminal device 10 then displays the created estimate image 850 on the display screen 11D. In this description of the embodiment, the process of creating the estimate image 850 and displaying it on the display screen 11D is referred to as "estimate creation".

[0016] In this embodiment of the building equipment display system 1, "layout plan creation," "AR display," and "estimate creation" are interconnected. Therefore, the user can repeat simulations any number of times on the terminal device 10, for example, by rearranging the layout of building equipment based on the results of the AR display or based on the estimated cost. In particular, in this embodiment of the building equipment display system 1, the user can perform these series of tasks—layout plan creation, AR display, and estimate creation—all at once on the display screen 11D. The following describes in detail each component of the building equipment display system 1 mentioned above.

[0017] [Terminal device 10] Figure 3 is a functional block diagram of the terminal device 10 of this embodiment.

[0018] As shown in Figure 1, the terminal device 10 (an example of a display means) can be a portable terminal device such as a tablet or a mobile phone, such as a smartphone, that can be carried around by the user. The terminal device 10 in this embodiment has a touch panel unit 11 that displays images and accepts user input, and a camera 12 that takes pictures. The terminal device 10 also has a communication device (not shown) that receives information from other devices such as a management server device 20 and transmits information to other devices. The touch panel unit 11 receives user input and instructions regarding the images displayed on the display screen 11D through touch operations performed by the user on the display screen 11D.

[0019] Camera 12 (an example of a shooting means) is provided on the terminal device 10 on the side opposite to the touch panel unit 11. The camera 12 then displays the captured image 120 on the touch panel unit 11. In this embodiment, the terminal device 10 is held and used by the user with the user (user's face) facing directly towards the display screen 11D of the touch panel unit 11. In this case, the shooting direction of the camera 12 of the terminal device 10 is the same as the direction the user's face is facing (the direction of the user's gaze). The captured image 120 displayed on the display screen 11D is the same as the scenery the user sees.

[0020] Furthermore, as shown in Figure 3, the terminal device 10 includes a layout diagram creation unit 30 (an example of a reception means) that creates a layout diagram 71, an augmented reality (AR) display unit 40 (an example of a control means) that overlays images of building equipment based on the created layout diagram 71 onto the captured image 120 for the user to see, and an estimate creation unit 50 that creates estimate information based on the created layout diagram 71.

[0021] [Management Server Device 20] Figure 4 is a functional block diagram of the management server device 20 in this embodiment. Figure 5 shows an example of the equipment information management table 210 managed by the management server device 20.

[0022] As shown in Figure 4, the management server device 20 includes a building equipment database (DB) unit 21 that stores information about building equipment used in the layout drawing creation unit 30, a layout drawing storage unit 22 that stores information about the layout drawing 71 created in the terminal device 10, and a construction drawing creation unit 23 that creates construction drawings to be used as construction drawings based on the layout drawing 71.

[0023] (Architectural equipment DB part 21) The building equipment database unit 21 stores an equipment information management table 210 that manages information on multiple types of building equipment. As shown in Figure 5, the equipment information management table 210 includes the following items: an "equipment ID" that can identify building equipment, a "name" which is a specific name assigned to each piece of building equipment, an "equipment image ID" which identifies the equipment image 75 used to create the layout diagram 71, a "unit price" which is the price set for each piece of building equipment, and "product information" which shows detailed information about the building equipment.

[0024] For example, in the case of an indoor unit of an air conditioning system as an example of building equipment, the following are associated with "Equipment ID": AC001, "Name": Air Conditioning Indoor Unit, "Equipment Image ID": ac001.jpg, "Unit Price": ¥200,000, and "Product Information": URL001. The "Product Information" also stores address information for accessing the database where the product information is stored. Furthermore, the product information includes detailed product information such as product images and design drawings of the building equipment, the model number of the building equipment, the manufacturer information of the building equipment, and the specifications of the building equipment (e.g., dimensions and service life).

[0025] The building equipment DB 21 transmits the contents of the equipment information management table 210 to the terminal device 10 based on a request from the terminal device 10. The building equipment DB 21 also manages the equipment information management table 210 based on settings received from the management user, such as setting unit prices or adding or deleting equipment images 75.

[0026] (Layout diagram storage unit 22) The layout diagram storage unit 22 acquires information about the layout diagram 71 created by the terminal device 10. The layout diagram storage unit 22 then stores the acquired information about the layout diagram 71. Furthermore, the layout diagram storage unit 22 transmits the stored information about the layout diagram 71 to the terminal device 10 in response to a request from the terminal device 10.

[0027] (Construction drawing creation section 23) The construction drawing creation unit 23 acquires information about the layout drawing 71 from the layout drawing storage unit 22. Then, the construction drawing creation unit 23 creates construction drawings of the building based on the acquired layout drawing 71. The construction drawings can be in the format of, for example, CAD (Computer-aided Design) drawings and are more detailed than the layout drawing 71 in this embodiment. The construction drawing creation unit 23 then transmits the information of the created construction drawings to the requested destination. For example, the construction drawing creation unit 23 can transmit the information of the created construction drawings to the terminal device 10, the printing device, or other devices.

[0028] Next, the functional configurations of the terminal device 10 in this embodiment will be described in detail. (Layout drawing creation unit 30) Figure 6 is a functional block diagram of the layout diagram creation unit 30 of the terminal device 10 in this embodiment. Figures 7 and 8 are explanatory diagrams of the reception screen 400 of this embodiment.

[0029] As shown in Figure 6, the layout plan creation unit 30 includes an equipment information acquisition unit 31 that acquires information on building equipment, a reception screen display unit 32 that receives user requests for operations related to the creation of the layout plan 71, and a layout plan management unit 33 that manages the created layout plan 71.

[0030] The equipment information acquisition unit 31 acquires the equipment information management table 210 (see Figure 5) from the management server device 20. The equipment information acquisition unit 31 then acquires the building equipment information stored in the equipment information management table 210. Furthermore, the equipment information acquisition unit 31 sends the acquired building equipment information to the reception screen display unit 32.

[0031] The reception screen display unit 32 draws the layout diagram 71 created by the user's operation on the display screen 11D. In this embodiment, the reception screen display unit 32 draws a layout diagram 71 displayed in three dimensions (hereinafter referred to as the three-dimensional layout diagram 713) or a layout diagram 71 displayed in two dimensions (the two-dimensional layout diagram 712).

[0032] The layout plan management unit 33 manages the information of the layout plan 71 created by the user on the reception screen 400 described above. Specifically, for each created layout plan 71, the layout plan management unit 33 stores the layout of the boundary image 73 and the equipment image 75 in the boundary image 73. Then, in response to a request from the reception screen display unit 32, the layout plan management unit 33 passes the stored layout plan information 71 to the reception screen display unit 32. The layout plan management unit 33 also sends the stored layout plan information 71 to the AR display unit 40 and the estimate creation unit 50. Furthermore, the layout plan management unit 33 transmits the layout plan information 71 to the management server device 20.

[0033] Next, we will explain the reception screen display unit 32 in detail. As shown in Figure 7, the reception screen display unit 32 of this embodiment displays boundary images 73 that show the boundary surfaces that divide space in a building, and equipment images 75 that show the building equipment set up in the building, on the display screen 11D. The reception screen display unit 32 of this embodiment displays the 3D layout diagram 713 as a bird's-eye view. The 3D layout diagram 713 includes a 3D boundary image 733, which is a boundary image 73 displayed in 3D, and a 3D equipment image 753, which is an equipment image 75 displayed in 3D.

[0034] The 3D boundary image 733 (an example of a building image) is an image that shows the design space corresponding to the actual building in which the user will arrange building equipment. The 3D boundary image 733 is created based on numerical information regarding the size of the actual building received from the user. The 3D boundary image 733 also includes an inner surface 73N that shows the inner surface of the room space and an outer surface 73G that shows the outer surface of the room space. This inner surface 73N and outer surface 73G are formed for the floor on the lower vertical side of the room space, the ceiling on the upper vertical side of the room space, and the four walls in the horizontal direction, respectively. The space between the inner surface 73N and the outer surface 73G is the thickness portion (interior) of the ceiling, floor, and walls, respectively.

[0035] In this description of the embodiment, the ceiling, floor, and wall sections that constitute the boundary between adjacent room spaces and the outside in a building will be referred to as "boundary sections" unless otherwise specified.

[0036] As will be described later, the 3D equipment image 753 is displayed based on the placement of the 2D equipment image 752, which is placed by the user in the 2D layout diagram 712. In this embodiment, the 3D equipment image 753 cannot be edited in the 3D layout diagram 713. However, it is possible to enable editing of the 3D equipment image 753 in the 3D layout diagram 713.

[0037] Furthermore, when the reception screen display unit 32 receives a user slide operation on the display screen 11D for the 3D layout diagram 713, it rotates the 3D layout diagram 713 in any direction (up, down, left, or right). Additionally, when the reception screen display unit 32 receives a user pinch-out operation on the display screen 11D for the 3D layout diagram 713, it enlarges the 3D layout diagram 713, and when it receives a pinch-in operation, it shrinks the 3D layout diagram 713.

[0038] Furthermore, as shown in Figures 8(A) and 8(B), the reception screen display unit 32 of this embodiment displays the two-dimensional layout diagram 712 in plan view. The two-dimensional layout diagram 712 includes a two-dimensional boundary image 732, which is a boundary image 73 displayed in two dimensions, and a two-dimensional equipment image 752, which is an equipment image 75 displayed in two dimensions.

[0039] The two-dimensional layout diagram 712 is displayed on the display screen 11D in response to the user's touch operation on the three-dimensional layout diagram 713. That is, when the reception screen display unit 32 receives a user tap operation on one boundary area (wall, ceiling, or floor) in the three-dimensional layout diagram 713 (see Figure 7), it displays the two-dimensional layout diagram 712 of the boundary area that was tapped on the display screen 11D. In this case, the reception screen display unit 32 does not display the two-dimensional layout diagrams 712 of other boundary areas that were not tapped on the display screen 11D.

[0040] Furthermore, when a user performs a tap operation on the 3D layout diagram 713, the reception screen display unit 32 displays a 2D layout diagram 712 on the reception screen 400, as shown in Figure 8(A), which shows the inner surface 73N side of the boundary where the tap operation was performed. The 2D layout diagram 712 showing the inner surface 73N side displays a 2D boundary image 732 as viewed from the inner surface 73N side of the 2D layout diagram 712, and a 2D equipment image 752 placed on the inner surface 73N of the 2D boundary image 732.

[0041] In this building, building equipment is mainly installed inside the room space. Furthermore, the interior of the room space greatly influences the interior finish of the room space. Therefore, in this embodiment, the reception screen display unit 32 displays the 2D layout diagram 712 showing the inner surface 73N of the boundary on the reception screen 400 in response to a tap operation on the 3D layout diagram 713, prioritizing it over the outer surface 73G, so that the user can perform the layout work starting with the equipment image 75 for the inner surface 73N of the boundary.

[0042] Furthermore, the reception screen display unit 32 displays a grid 91 on the two-dimensional boundary image 732. The grid 91 is displayed by arranging multiple lines in a grid pattern. The grid 91 allows two-dimensional equipment images 752, which are linear building equipment such as pipes, to be placed only along the lines. The grid 91 also allows two-dimensional equipment images 752, which are rectangular building equipment such as equipment machines, to be placed so that their centers lie on the intersections of the lines. In this embodiment, the grid 91 specifies the placement of the two-dimensional equipment images 752 to some extent. This makes it easy for the user to place two-dimensional equipment images 752 or connect equipment to each other by tapping on the display screen 11D.

[0043] The reception screen display unit 32 then receives from the user the placement of the 2D equipment image 752 relative to the 2D boundary image 732. Based on the placement of the 2D equipment image 752 received from the user, the reception screen display unit 32 draws a 2D layout diagram 712. The reception screen display unit 32 also reflects the placement of the 2D equipment image 752 in the 2D layout diagram 712 into the 3D layout diagram 713. Furthermore, the reception screen display unit 32 sends the updated information of the 3D layout diagram 713 to the layout diagram management unit 33.

[0044] Furthermore, when the reception screen display unit 32 receives a user's slide operation on the two-dimensional layout diagram 712 displayed on the display screen 11D, it displays the two-dimensional layout diagram 712 in any direction (up, down, left, or right). In addition, when the reception screen display unit 32 receives a user's pinch-out operation on the two-dimensional layout diagram 712 displayed on the display screen 11D, it enlarges the two-dimensional layout diagram 712, and when it receives a pinch-in operation, it shrinks the two-dimensional layout diagram 712.

[0045] Next, we will specifically explain the operation button images displayed on the reception screen 400 when the 3D layout diagram 713 is displayed. As shown in Figure 7, the reception screen display unit 32 displays a new creation button 611, a read button 612, and a save button 613 on the reception screen 400 where the three-dimensional layout diagram 713 is displayed.

[0046] The "Create New" button 611 is a button image that accepts user selection when creating a new layout drawing 71 of building equipment in a building. When the "Create New" button 611 is selected, the reception screen display unit 32 requests the user to set the room space of the building, which will be displayed as a 3D boundary image 733. Specifically, when the "Create New" button 611 is selected, the reception screen display unit 32 accepts input from the user for the size of the room space, that is, the distance and thickness of each boundary. Then, based on the input distance values, the reception screen display unit 32 creates the boundary image 73 (3D boundary image 733, 2D boundary image 732) and displays it on the reception screen 400.

[0047] The read button 612 is a button image that accepts a user's request to read a layout diagram 71 previously created and saved by the user and display it on the display screen 11D. The read button 612 is also a button image that accepts a user's request to read a layout diagram 71 created based on measurement information measured by a measuring device (described later) and display it on the display screen 11D. The save button 613 is a button image that accepts an operation from the user to store and save the layout diagram 71 created by the user in the terminal device 10. In this embodiment of the building equipment display system 1, the layout diagram 71 can be stored in the layout diagram management unit 33 of the terminal device 10, as well as in the management server device 20.

[0048] Furthermore, as shown in Figure 7, the reception screen display unit 32 displays the layer management button 62 on the reception screen 400 where the 3D layout diagram 713 is displayed. The layer management button 62 is a button image that accepts user requests to manage the layer configuration of the boundary image 73. The layer management button 62 includes a layer display switching button 621 and a layer addition button 622.

[0049] Figure 9 is an explanatory diagram of the layer structure of the boundary image 73 in this embodiment. In this embodiment, the boundary image 73 has multiple layers at each boundary, and each layer can accept the placement of equipment images 75. For example, as shown in Figure 9(A), the boundary image 73 at each boundary can be composed of a three-layer structure consisting of a first layer 73a, a second layer 73b, and a third layer 73c. ​​Then, as shown in Figure 9(B), for example, the first layer 73a can display equipment images 75 related to air conditioning equipment, the second layer 73b can display equipment images 75 related to plumbing equipment, and the third layer 73c can display equipment images 75 related to electrical equipment. The layer configuration of the boundary image 73 is reflected in both the two-dimensional layout diagram 712 and the three-dimensional layout diagram 713. In addition, in this embodiment, the display color of the equipment images 75 can be displayed differently for each layer.

[0050] The layer display switching button 621 accepts the switching of the display / hide status of the equipment image 75 for each layer in the multiple layers. The layer addition button 622 is used when adding a new layer. As described above, in this embodiment, the layer structure allows for the management of building equipment according to different types of building equipment, such as air conditioning equipment, plumbing equipment, and electrical equipment.

[0051] Furthermore, as shown in Figure 7, the reception screen display unit 32 displays an AR display button 63 and an estimate creation button 64 on the reception screen 400 where the 3D layout diagram 713 is displayed. The AR display button 63 (an example of an instruction image) is a button image that receives a user instruction to execute an AR display, which superimposes the created layout diagram 71 onto the captured image 120. When the AR display button 63 is selected, the reception screen display unit 32 causes the AR display unit 40 to execute the AR display on the display screen 11D. The AR display will be explained in detail later.

[0052] The estimate creation button 64 (an example of a third instruction image) is a button image that the user adds an instruction to create an estimate based on the created layout diagram 71. When the estimate creation button 64 is selected, the reception screen display unit 32 causes the estimate creation unit 50 to display the estimate image 850 on the display screen 11D. The estimate display will be explained in detail later.

[0053] Next, we will specifically explain the operation button images displayed on the reception screen 400 when the 2D layout diagram 712 is displayed. As shown in Figure 8(A), the reception screen display unit 32 of this embodiment displays equipment image drawing buttons 65 related to the display of a two-dimensional equipment image 752 on the reception screen 400 which displays the two-dimensional arrangement diagram 712 on the inner surface 73N side. In this embodiment, the equipment image drawing buttons 65 include a drawing button 651, a delete button 652, a connection button 653, a drilling button 654, and an equipment selection button 655.

[0054] The drawing button 651 is a button image that accepts a setting to make the 2D equipment image 752 of the selected building equipment available for addition to the 2D boundary image 732. When the drawing button 651 is selected, the reception screen display unit 32 adds and displays the 2D equipment image 752 based on the tapped position each time the user performs a tap operation on the 2D boundary image 732.

[0055] As shown in Figure 8(A), when, for example, an indoor unit of an air conditioning system is selected as a building facility, and the drawing button 651 is selected, the reception screen display unit 32 then displays a 2D equipment image 752 corresponding to the indoor unit at the position where the user tapped in the 2D boundary image 732. Furthermore, as shown in Figure 8(A), when, for example, piping is selected as building equipment and the drawing button 651 is selected, the reception screen display unit 32 then displays a two-dimensional equipment image 752 corresponding to the piping, connecting the positions where the user has performed consecutive tap operations on the two-dimensional boundary image 732.

[0056] The delete button 652 is a button image that accepts a setting to make the 2D equipment image 752 available for deletion. When the delete button 652 is selected, the reception screen display unit 32 deletes the 2D equipment image 752 that the user subsequently tapped.

[0057] The connection button 653 is a button image that accepts the display of a joint to connect the intersecting parts when two-dimensional equipment images 752, which are displayed in a straight line such as a pipe, are displayed intersecting.

[0058] The perforation button 654 is a button image that accepts an operation to display the two-dimensional equipment image 752 in a direction that penetrates the boundary. When the perforation button 654 is selected, the reception screen display unit 32 displays the two-dimensional equipment image 752 that the user subsequently tapped, showing it penetrating the two-dimensional boundary image 732. The reception screen display unit 32 will execute the display that penetrates the boundary regardless of whether the operation based on the perforation button 654 is performed on the two-dimensional equipment image 752 on the inner surface 73N side or the outer surface 73G side.

[0059] The equipment selection button 655 is a button image that accepts the selection of building equipment to be added to the 2D equipment image 752 based on the operation of the drawing button 651. When the equipment selection button 655 is selected, the reception screen display unit 32 displays on the reception screen 400 a list of multiple building equipment options that can be selected as the 2D equipment image 752. The reception screen display unit 32 displays the multiple building equipment options based on the equipment information management table 210 acquired by the equipment information acquisition unit 31. The reception screen display unit 32 then displays the building equipment selected based on the operation of the equipment selection button 655 on the 2D boundary image 732 in response to the user's tap operation after the operation of the drawing button 651, as described above.

[0060] Furthermore, as shown in Figure 8(A), the reception screen display unit 32 of this embodiment displays a reference setting button 661, a grid setting button 662, a cancel button 671, a redo button 672, an outside display button 681, and a bird's-eye view button 69 on the reception screen 400 which displays the two-dimensional arrangement diagram 712 of the inner surface 73N side.

[0061] The reference setting button 661 is a button image for accepting the setting of a reference image 95, which serves as the reference for starting the movement of the display screen 11D of the terminal device 10 when performing AR display. In this embodiment, the reference image 95 is arrow-shaped. After the reference setting button 661 is selected, the user can tap on one boundary to display the reference image 95 at the position of the tap on the two-dimensional boundary image 732 of the boundary, such that the tip of the arrow points towards the back of the display screen 11D. Furthermore, the display of the reference image 95 can be set for either the inner surface 73N or the outer surface 73G of the two-dimensional boundary image 732.

[0062] Furthermore, the setting of the reference image 95 on the two-dimensional arrangement diagram 712 is also reflected in the three-dimensional arrangement diagram 713. As shown in Figure 7, when the reference image 95 is set on the inner surface 73N, the arrow is positioned so that it points towards the outer surface 73G. Also, when the reference image 95 is set on the outer surface 73G, the arrow is positioned so that it points towards the inner surface 73N. Reference image 95 will be explained in detail later.

[0063] The grid setting button 662 accepts the user's setting for the grid spacing 91. When the grid setting button 662 is selected, it displays multiple spacing options on the reception screen 400. Multiple spacing options are available for the grid 91, and the user can select one of them as desired.

[0064] The undo button 671 is a button image that accepts a command to cancel the user's previous action. The redo button 672 accepts instructions to perform an operation again that was canceled by the cancel button 671.

[0065] The external display button 681 is a button image that accepts an operation to switch the display from the state showing the inner surface 73N side of the 2D layout diagram 712 to the outer surface 73G side of the 2D layout diagram 712, which is the opposite side. As shown in Figure 8(A), when the inner surface 73N side of the 2D layout diagram 712 is displayed, the 2D equipment image 752 located on the inner surface 73N side of the 2D boundary image 732 is displayed. In contrast, when the external display button 681 is selected, as shown in Figure 8(B), the display switches to the state showing the outer surface 73G side of the 2D layout diagram 712, and the 2D equipment image 752 located on the outer surface 73G side of the 2D boundary image 732 is displayed.

[0066] In this embodiment, when the inner surface 73N side of the 2D layout diagram 712 is displayed, the 2D equipment image 752 located on the inner surface 73N is displayed, but the 2D equipment image 752 located on the outer surface 73G is not displayed. On the other hand, when the outer surface 73G side of the 2D layout diagram 712 is displayed, the 2D equipment image 752 located on the outer surface 73G is displayed, but the 2D equipment image 752 located on the inner surface 73N is not displayed. This prevents the display of the 2D layout diagram 712 from becoming cluttered in this embodiment.

[0067] The bird's-eye view button 69 (an example of the second instruction image) is a button image that accepts an operation to switch the display from the 2D layout diagram 712 to the 3D layout diagram 713. As shown in Figure 8(A), when the bird's-eye view button 69 is selected while the 2D layout diagram 712 is displayed on the reception screen 400, the reception screen 400 displaying the 3D layout diagram 713 is displayed, as shown in Figure 7.

[0068] Next, we will explain the reception screen 400, which displays the two-dimensional layout diagram 712 of the outer surface 73G. As shown in Figure 8(B), the reception screen display unit 32 of this embodiment displays a two-dimensional layout diagram 712 showing the outer surface 73G side on the reception screen 400. The two-dimensional layout diagram 712 showing the outer surface 73G side displays a two-dimensional boundary image 732 as seen from the outer surface 73G side of the two-dimensional layout diagram 712, and a two-dimensional equipment image 752 placed on the outer surface 73G of the two-dimensional boundary image 732.

[0069] Furthermore, as shown in Figure 8(B), the reception screen display unit 32 of this embodiment displays basically the same button images on the reception screen 400 that displays the two-dimensional arrangement diagram 712 on the outer surface 73G side as on the reception screen 400 that displays the two-dimensional arrangement diagram 712 on the inner surface 73N side as shown in Figure 8(A). However, on the reception screen 400 that displays the two-dimensional arrangement diagram 712 on the outer surface 73G side, the inner display button 682 is displayed instead of the outer display button 681.

[0070] The inner display button 682 is a button image that accepts an operation to switch the display from the outer surface 73G side of the two-dimensional layout diagram 712 to the inner surface 73N side of the two-dimensional layout diagram 712, which is the opposite side. When the inner display button 682 is selected, the display on the reception screen 400 switches from the state showing the outer surface 73G side of the two-dimensional layout diagram 712 as shown in Figure 8(B) to the state showing the inner surface 73N side of the two-dimensional layout diagram 712 as shown in Figure 8(A).

[0071] (AR display section 40) Figure 10 shows the functional blocks of the AR display unit 40 in this embodiment. Figure 11 is an explanatory diagram of the AR display by the AR display unit 40 of this embodiment. Figure 12 is an explanatory diagram of the tracking operation in the AR display of this embodiment.

[0072] As shown in Figure 10, the AR display unit 40 includes an image acquisition unit 41 that acquires the captured image 120, an operation identification unit 42 that identifies the movement of the position and orientation of the display screen 11D, an overlay display unit 43 that overlays the arrangement diagram 71 onto the captured image 120, and an adjustment unit 44 that adjusts the display position of the arrangement diagram 71.

[0073] The image acquisition unit 41 acquires the captured image 120 taken by the camera 12 (see Figure 1). The image acquisition unit 41 then sends the information of the acquired image 120 to the operation identification unit 42 and the superimposed display unit 43. The image acquisition unit 41 always acquires the captured image 120 from the camera 12 when AR display is being performed.

[0074] The motion identification unit 42 identifies the movement of the captured image 120 acquired from the captured image acquisition unit 41. For example, when a user moves the terminal device 10, the orientation of the camera 12 installed on the terminal device 10 changes, and the content of the captured image 120 captured by the camera 12 changes. In other words, the captured image 120 moves along with the movement of the terminal device 10. The motion identification unit 42 then identifies the amount of movement of the image in the vertical, horizontal, and forward / backward directions based on the feature points of the image displayed in the captured image 120. The operation identification unit 42 then sends information about the amount of movement of the identified captured image 120 to the superimposed display unit 43.

[0075] The operation identification unit 42 may also determine the position and orientation of the display screen 11D using, for example, a motion sensor mounted on the terminal device 10. In this case, the motion sensor can be a 3-axis gyro sensor that detects the angular velocity (rotational speed) of the terminal device 10, a 3-axis accelerometer that detects the acceleration of the terminal device 10, and a 3-axis geomagnetic sensor that detects the Earth's magnetic field and determines the absolute direction of the terminal device 10.

[0076] The superimposed display unit 43 acquires the captured image 120 from the captured image acquisition unit 41. Furthermore, the superimposed display unit 43 acquires information from the layout diagram creation unit 30. Then, as shown in Figure 11, the superimposed display unit 43 displays the superimposed image 800, which is created by superimposing the 3D equipment image 753 of the layout diagram 71 onto the captured image 120, on the display screen 11D of the terminal device 10. Furthermore, in this embodiment, when displaying the 3D equipment image 753, the superimposed display unit 43 displays both the 3D equipment image 753 located on the inner surface 73N and the 3D equipment image 753 located on the outer surface 73G.

[0077] Furthermore, the superimposed display unit 43 determines the display size of the 3D equipment image 753 based on the characteristics of the room space obtained from the captured image 120. The superimposed display unit 43 extracts characteristics of the room space from the captured image 120, such as the corners of the room, the width of the walls in the left-right direction, and the width of the walls in the up-down direction (height of the ceiling from the floor). The superimposed display unit 43 then aligns the corners of the boundaries in the 3D boundary image 733 with the corners of the room in the captured image 120, and aligns the size of the boundaries in the 3D boundary image 733 with the size of the boundaries such as the walls in the captured image 120. In this way, the superimposed display unit 43 of this embodiment determines the display position and display size (scale) of the 3D arrangement diagram 713 on the display screen 11D.

[0078] In this embodiment, the building equipment display system 1 requests the user to start the AR display when the user holds the terminal device 10 (display screen 11D) at a predetermined position in the room space of the actual building and orients the display screen 11D in a predetermined direction. The position and orientation of the display screen 11D of the terminal device 10 requested by the user are determined by the position and orientation of the reference image 95 that was set in advance by the user in the boundary image 73.

[0079] In the building equipment display system 1, the initial state is one in which the display screen 11D is oriented in a predetermined direction at a predetermined position, and the user moves the position and orientation of the display screen 11D. Furthermore, the building equipment display system 1 changes the display of the 3D equipment image 753 according to the amount of movement of the display screen 11D from the initial state. As a result, even if the user moves the display screen 11D, a tracking display is achieved in which the 3D equipment image 753 is displayed on the 3D boundary diagram 733 at the position placed by the user relative to the captured image 120 that shows the actual building. The AR display will be explained in more detail below.

[0080] When the superimposed display unit 43 starts displaying the 3D equipment image 753 of the layout diagram 71, it displays the 3D equipment image 753 in a position and direction based on the reference image 95 specified by the user when creating the layout diagram. For example, as shown in Figure 8(A), suppose that when creating the layout diagram, the user specified a reference image 95 in the direction from the inner surface 73N side to the outer surface 73G side in the center of the 2D layout diagram 712 that displays the inner surface 73N of one wall.

[0081] In this case, when the superimposed display unit 43 starts displaying the 3D equipment image 753 of the layout diagram 71, it displays the 3D equipment image 753 as seen from the inner surface 73N side of one of the walls on the display screen 11D. Furthermore, the superimposed display unit 43 displays the 3D equipment image 753 located on the inner surface 73N of one of the walls on the front side of the display screen 11D, and the 3D equipment image 753 located on the outer surface 73G of one of the walls on the back side.

[0082] The superimposed display unit 43 then changes the display content of the 3D equipment image 753 displayed on the display screen 11D based on the movement amount information of the captured image 120 acquired from the operation identification unit 42. For example, as shown in Figure 12(A), when the user moves the terminal device 10 in the left-right direction, the display direction of the 3D equipment image 753 is changed in the left-right direction to follow the left-right changes in the captured image 120 displayed on the display screen 11D. This tracking operation also applies to the vertical movement of the terminal device 10.

[0083] Furthermore, as shown in Figure 12(B), when the terminal device 10 moves in the front-to-back direction as the user moves, the display direction of the 3D equipment image 753 is changed in the front-to-back direction to follow the front-to-back changes in the captured image 120 displayed on the display screen 11D.

[0084] Furthermore, if a user who was inside a room in an actual building moves to the outside of the room (wall), the display screen 11D will change from showing the 3D equipment image 753 viewed from the inner surface 73N of the boundary to showing the 3D equipment image 753 viewed from the outer surface 73G of the boundary to showing the 3D equipment image 753 viewed from the outer surface 73G of the boundary to the front of the display screen 11D.

[0085] As described above, when the user changes the orientation or position of the terminal device 10, the superimposed display unit 43 displays the 3D equipment image 753 in accordance with the changes in the captured image 120 displayed on the display screen 11D. In this way, in the building equipment display system 1 of this embodiment, when a user views a captured image 120 of actual building equipment through the display screen 11D, the 3D equipment image 753 is displayed on the captured image 120, making it feel as if the building equipment is actually there.

[0086] Next, we will explain the button images displayed on the AR display screen 11D. Figure 13 is an explanatory diagram of the button images displayed on the display screen 11D where AR display is performed.

[0087] As shown in Figure 11, the superimposed display unit 43 displays the bird's-eye view button 69 and the button ON image 81 on the display screen 11D where the superimposed image 800 is displayed. When the superimposed image 800 is displayed and the bird's-eye view button 69 is selected, the superimposed image display unit 43 displays a reception screen 400 (see Figure 7) on the display screen 11D that displays the 3D layout diagram 713. For example, it is assumed that a user may want to change the layout diagram 71 by grasping an image of the layout of the building equipment of the target building from the superimposed image 800 in the AR display. In such a case, in this embodiment, the bird's-eye view button 69 is displayed on the display screen 11D where the superimposed image 800 is displayed, so that the user can immediately start changing the layout of the equipment images 75 in the layout diagram 71.

[0088] Furthermore, in this embodiment, the bird's-eye view button 69 is provided at the lower end of the display screen 11D in the left-right direction, thereby improving operability when the user holds the terminal device 10 with both hands and switches to the display screen 11D where the 3D layout diagram 713 is displayed.

[0089] The button ON image 81 is a button image that receives instructions to display a group of button images related to correcting the display of the 3D equipment image 753 on the display screen 11D where the superimposed image 800 is displayed. When the button ON image 81 is selected, the superimposed display unit 43 displays the group of button images related to correcting the display of the 3D equipment image 753 on the display screen 11D. The button images related to correcting the display of the 3D equipment image 753 will be described later. In this embodiment, when displaying the superimposed image 800, in order to secure as wide an area as possible for displaying the captured image 120, the other button images are displayed only when the button ON image 81 is operated, and the other button images are not displayed unless the button ON image 81 is operated.

[0090] As shown in Figure 13(A), when the button ON image 81 (see Figure 11) is selected, the adjustment unit 44 displays the button OFF image 82, reference selection button 83, rotation button 84, left / right movement button 85, up / down movement button 86, and zoom button 87 on the display screen 11D where the superimposed image 800 is displayed. The adjustment unit 44 then accepts requests from the user to adjust the display position of the 3D equipment image 753 relative to the captured image 120 in the superimposed image 800 in response to the operation of these buttons. In this embodiment, the button OFF image 82, the reference designation button 83, the rotation button 84, the left / right movement button 85, the up / down movement button 86, and the zoom in / out button 87 are displayed at the lower edge of the display screen 11D in the left / right direction.

[0091] The button OFF image 82 is a button image that allows the user to request that the button image displayed on the display screen 11D be hidden in response to the operation of the button ON image 81 (see Figure 11).

[0092] In this embodiment, the reference designation buttons 83 include reference button 1 831, reference button 2 832, and reference button 3 833. As shown in Figure 13(B), reference button 1 831, reference button 2 832, and reference button 3 833 correspond to the first reference image 951, second reference image 952, and third reference image 953, respectively, which are reference images 95 set at predetermined positions in the three-dimensional arrangement diagram 713.

[0093] For example, the user selects the Reference 1 button 831 at a location in the actual building corresponding to where the first reference image 951 is displayed in the 3D layout drawing 713. When the Reference 1 button 831 is operated, the adjustment unit 44 redisplays the 3D equipment image 753 on the display screen 11D, with the first reference image 951 as the center of the display. The superimposed display unit 43 then uses the position and orientation of the terminal device 10 when the Reference 1 button 831 was operated as the origin, and changes the display direction of the 3D equipment image 753 that has been redisplayed, following the subsequent changes in the captured image 120.

[0094] Similarly, the adjustment unit 44 accepts the selection of the reference 2 button 832 at a position corresponding to the location where the second reference image 952 is displayed in the 3D layout diagram 713, thereby redisplaying the 3D equipment image 753 with the second reference image 952 as the center on the display screen 11D. Furthermore, the adjustment unit 44 accepts the selection of the reference 3 button 833 at a position corresponding to the location where the third reference image 953 is displayed in the 3D layout diagram 713, thereby redisplaying the 3D equipment image 753 with the third reference image 953 as the center on the display screen 11D. Thus, in this embodiment, the reference position of the display of the 3D equipment image 753 relative to the captured image 120 in the superimposed image 800 is adjusted based on multiple different criteria set in the 3D arrangement diagram 713.

[0095] As shown in Figure 13(A), the rotation button 84 is a button image that accepts an operation to rotate the 3D equipment image 753 in the superimposed image 800 around a predetermined virtual rotation axis as the center of rotation. The left / right movement button 85 is a button image that accepts an operation to move the entire 3D equipment image 753 in the superimposed image 800 in the left / right direction on the display screen 11D. The up / down movement button 86 is a button image that accepts an operation to move the entire 3D equipment image 753 in the superimposed image 800 in the vertical direction on the display screen 11D. The zoom button 87 is a button image that accepts operations to enlarge or reduce the overall size of the 3D equipment image 753 in the superimposed image 800.

[0096] In this embodiment, the display position of the 3D equipment image 753 is adjusted using the reference designation button 83, but the embodiment is not limited to this configuration. For example, in the terminal device 10 of this embodiment, instead of using the reference designation button 83, the display position of the 3D equipment image 753 in the superimposed image 800 may be automatically adjusted based on characteristic locations of the room space, such as corners in the building or predetermined reference markers, obtained from the captured image 120.

[0097] Figure 14 is a flowchart illustrating the operation of the AR display unit 40 in this embodiment. As shown in Figure 14, the AR display unit 40 determines whether the AR display button 63 (see Figure 7) has been selected (step 101). If the AR display button 63 is not selected (NO in step 101), the process returns to step 101. On the other hand, if the AR display button 63 is selected (YES in step 101), the AR display unit 40 acquires the captured image 120 from the camera 12 (step 102). Furthermore, the AR display unit 40 superimposes a 3D equipment image 753 based on the reference image 95 onto the acquired captured image 120 and begins displaying the superimposed image 800 on the display screen 11D (step 103).

[0098] Next, it is determined whether or not there is a change in the movement of the captured image 120 (step 104). If there is a change in the movement of the captured image 120 (YES in step 104), a superimposed image 800 in which the 3D equipment image 753 follows the captured image 120 is displayed on the display screen 11D (step 105).

[0099] Then, if there is no change in movement of the captured image 120 (NO in step 104), or if the tracking display is shown in step 105, it is determined whether or not an instruction to adjust the position of the 3D equipment image has been received from the user (step 106). An instruction to adjust the position of the 3D equipment image is determined based on the operation of one of the button images: the rotation button 84, the left / right movement button 85, the up / down movement button 86, and the zoom in / out button 87 (see Figure 13). Then, if an instruction to adjust the position of the 3D equipment image has been received from the user (YES in step 106), the position adjustment of the 3D equipment image is performed (step 107).

[0100] Subsequently, if the user does not provide instructions for adjusting the position of the 3D equipment image (NO in step 106), or if the position of the 3D equipment image is adjusted in step 107, it is determined whether the user has pressed the reference designation button 83 (see Figure 13) (step 108). If the user has pressed the reference designation button 83 (YES in step 108), the 3D equipment image 753 is redisplayed based on the reference designation button 83 (step 109).

[0101] Furthermore, if the reference designation button 83 is not operated (NO in step 108), or if the 3D equipment image 753 is redisplayed based on the reference designation button 83 in step 109, it is determined whether or not the bird's-eye view button 69 (see Figure 13) has been operated (step 110). If the bird's-eye view button 69 has been operated (YES in step 110), the display screen 11D is moved from the superimposed image 800 to the reception screen 400 where the 3D layout diagram 713 is displayed, and the series of AR display processes is terminated. On the other hand, if the bird's-eye view button 69 has not been operated (NO in step 110), the process returns to step 104 to determine again whether or not there has been a change in movement of the captured image 120. As described above, in the building equipment display system 1 of this embodiment, AR display is performed on the terminal device 10.

[0102] (Estimate Creation Department 50) Next, we will explain how to prepare an estimate using this embodiment. Figure 15 shows the functional blocks of the estimate creation unit 50 in this embodiment. As shown in Figure 15, the estimate creation unit 50 (an example of an estimate creation means) includes a layout plan information acquisition unit 51 that acquires information from the layout plan 71, a cost information acquisition unit 52 that acquires cost information for building equipment, and an estimate image creation unit 53 that creates an estimate image.

[0103] The layout plan information acquisition unit 51 acquires information about the layout plan 71 created by the user from the layout plan creation unit 30. The layout plan information acquisition unit 51 then sends the acquired layout plan 71 information to the estimate image creation unit 53.

[0104] The price information acquisition unit 52 acquires price information regarding the unit price of building equipment from the equipment information management table 210 held by the layout drawing creation unit 30. The price information acquisition unit 52 then sends the acquired price information for each building equipment to the estimate image creation unit 53.

[0105] The estimate image creation unit 53 identifies the types and quantities of building equipment used in the layout drawing 71 from the layout drawing information acquisition unit 51. Then, the estimate image creation unit 53 calculates the cost of constructing the building equipment created as the layout drawing 71 based on the building equipment cost information received from the cost information acquisition unit 52.

[0106] Furthermore, the estimation image creation unit 53 of this embodiment identifies the type and quantity of building equipment installed in a direction that penetrates the boundary. For example, in an actual building, if building equipment is installed in a direction that penetrates a boundary such as a wall, it will be necessary to perform work such as drilling into the wall. Therefore, when calculating the estimated cost, even if the same building equipment is used, the cost of construction will differ between building equipment that requires drilling and building equipment that does not. Accordingly, the estimation image creation unit 53 of this embodiment sets a different cost for building equipment installed in a direction that penetrates the boundary, such as setting a higher cost for building equipment that does not require drilling.

[0107] Then, the estimate image creation unit 53 creates an estimate image 850 based on the estimated cost for the construction of building equipment calculated based on the layout drawing 71 and displays it on the display screen 11D.

[0108] Figure 16 is an example of an estimate image displayed on the display screen 11D by the estimate creation unit 50 of this embodiment. As shown in Figure 16, the display screen where the estimate image 850 is displayed shows a 3D layout diagram 713, a new creation button 611, a read button 612, a save button 613, an estimate print button 88, a close button 89, and an AR display button 63.

[0109] The print estimate button 88 is a button image that receives a command from the user to print the estimate image 850. The close button 89 is a button image for receiving instructions from the user to hide the estimate image 850. When the close button 89 is operated, the display of the estimate image 850 disappears, and the 3D layout diagram 713 that was displayed behind the estimate image 850 is displayed. The display screen 11D then transitions to the reception screen 400 (see Figure 7) where the 3D layout diagram 713 is displayed. In the terminal device 10 of this embodiment, if the user wants to review the layout of the building equipment again after confirming the estimated amount, they can do so immediately by operating the close button 89.

[0110] When the AR display button 63 is operated, the system switches to a display screen 11D where the superimposed image 800 is displayed. In the terminal device 10 of this embodiment, if the user wants to see the image of the building equipment layout again using the superimposed image 800 after confirming the estimated price, they can do so immediately by operating the AR display button 63.

[0111] Furthermore, in this embodiment, the close button 89 and the AR display button 63 are provided at the lower, left-right edges of the display screen 11D, respectively, thereby improving operability when a user holds the terminal device 10 with both hands and transitions to the display screen 11D that displays the reception screen 400 or the superimposed image 800.

[0112] Next, we will explain a specific example of using a pre-prepared layout diagram 71. In the example described above, the layout diagram 71 was created in response to user operations on the reception screen 400. Here, the layout diagram 71 is created based on measurement information that measures the arrangement of building equipment already installed in the actual building (hereinafter referred to as existing building equipment), and can be displayed on the reception screen 400.

[0113] For example, if existing building facilities are pipes such as water supply pipes, a movable measuring device is used inside the pipes to determine the layout of the pipes. The measuring device comprises a camera unit capable of capturing images of the surroundings, a distance measuring unit for measuring the distance the device moves, and a posture measuring unit for measuring the orientation of the device (i.e., the direction the device is facing). By actually moving this measuring device, for example, inside a pipe, the arrangement (i.e., structure) of existing building equipment is identified based on the information of the distance moved and the posture. Furthermore, by moving the measuring device while capturing images with the camera unit, equipment images of the existing building equipment are obtained. In addition, information about the shooting position within the existing building equipment is associated with the equipment images. Note that the equipment images may be either video or still images.

[0114] The measurement information of existing building facilities measured by the measuring device is converted into a format usable as a layout diagram 71 and stored in the layout diagram storage unit 22 of the management server device 20. In addition, the photographic information of existing building facilities measured by the measuring device is stored in the layout diagram storage unit 22 along with the information of the shooting location.

[0115] Figure 17 is an example of a three-dimensional layout diagram 713 displayed based on a pre-created layout diagram 71. As shown in Figure 17, in response to the user's operation on the read button 612, a layout diagram 71, which was created in advance using the measuring device described above, is read out. The read layout diagram 71 is displayed on the reception screen 400 of the display screen 11D. Figure 17 shows an example of a 3D layout diagram 713. The user can then refer to the 3D layout diagram 713 to confirm the arrangement of existing building equipment in the building.

[0116] Figures 18, 19, and 20 are examples of two-dimensional layout diagrams 712 displayed based on a pre-created layout diagram 71. As shown in Figure 18(A), the reception screen display unit 32 displays a two-dimensional layout diagram 712 on the display screen 11D in response to a user tap on the three-dimensional layout diagram 713 (see Figure 17). As described above, the two-dimensional layout diagram 712 shown in Figure 18(A) is not displayed by a user drawing on the reception screen 400, but is displayed based on a pre-created layout diagram 71 obtained from the layout diagram storage unit 22.

[0117] Furthermore, the reception screen display unit 32 displays a detailed information button 90 on the reception screen 400. The detailed information button 90 is a button image that accepts the display of detailed information about the equipment image 75.

[0118] As shown in Figure 18(B), when the detailed information button 90 is selected, the reception screen display unit 32 displays the shooting information button 901 and the product information button 902. The shooting information button 901 is a button for displaying equipment images, which are images of existing building equipment, on the display screen 11D. In this example, the reception screen display unit 32 plays a video of the equipment images in response to the user's operation on the shooting information button 901. Furthermore, the product information button 902 is a button used to display information related to the building equipment corresponding to the equipment image 75 displayed on the reception screen 400 on the display screen 11D. The reception screen display unit 32 displays detailed information about the building equipment product in response to the user's operation on the product information button 902.

[0119] Then, as shown in Figure 19(A), after the shooting information button 901 (see Figure 18(B)) of the detailed information button 90 is selected, when the user taps on a 2D equipment image 752 in the equipment image 75, the equipment shooting image 92 corresponding to that 2D equipment image 752 is played. When the equipment shooting image 92 is displayed, a mark 92M is superimposed on the 2D equipment image 752 that the equipment shooting image 92 is displayed on. This mark 92M corresponds to the shooting position of the video being played as the equipment shooting image 92. Then, as shown in Figure 19(B), the mark 92M moves on the 2D equipment image 752 in accordance with the progress of the video of the equipment shooting image 92. Furthermore, the mark 92M can be moved on the 2D equipment image 752 by the user's slide operation. When the user moves the mark 92M to a predetermined position, the reception screen display unit 32 displays the equipment image 92 taken at that predetermined position.

[0120] Furthermore, as shown in Figure 20(A), the reception screen display unit 32 can receive changes to the pre-created layout diagram 71 from the user. The reception screen display unit 32 accepts changes to the layout diagram 71 via the equipment image drawing button 65. For example, as explained with reference to Figures 19(A) and 19(B), after the user checks the equipment photographed image 92, the user may determine that there are areas in existing building equipment that require repair. In such a case, as shown in Figure 20(A), the user performs operations such as deleting or drawing the 2D equipment image 752 of the target area on the reception screen 400. Note that the 2D equipment image 752 shown by the dashed line in Figure 20(A) is the equipment image 75 of the building equipment that has been modified by the user. The layout diagram 71, after the modification operation of the equipment image 75 has been performed, is stored in the layout diagram storage unit 22 of the management server device 20.

[0121] Furthermore, as shown in Figure 20(B), after the product information button 902 (see Figure 18(B)) of the detailed information button 90 is selected, if the user taps on the modified 2D equipment image 752, the product information 93 for the building equipment corresponding to that 2D equipment image 752 is displayed. The product information 93 is displayed based on the information for each building equipment stored in the equipment information management table 210. Specifically, the product information 93 displays information about the building equipment such as name and model number, product image, manufacturer name, price, and service life, which would be included in a product catalog.

[0122] Furthermore, in the example described above, the equipment image 92 is displayed on the display screen 11D in response to an operation on the 2D equipment image 752, but it may also be displayed in response to an operation on the 3D equipment image 753. In addition, the product information 93 may be displayed, for example, by the user tapping the building equipment section in the estimate image 850. Furthermore, product information 93 can also be displayed in response to tap operations on the user-created layout diagram 71, as explained with reference to Figures 8(A) and 8(B).

[0123] Here, the hardware configuration of the terminal device 10 and the management server device 20 in this embodiment will be described. Each terminal device 10 and management server device 20 are equipped with a CPU (Central Processing Unit) as a computing means, a working memory as a main memory means, a magnetic disk drive (HDD: Hard Disk Drive) as an auxiliary storage means, a network interface, a display mechanism including a display device, an audio mechanism, and input devices such as a keyboard and mouse. The auxiliary storage means stores the OS program and application programs. These programs are then loaded into working memory and executed by the CPU, thereby realizing the functions of each component in the terminal device 10 and the management server device 20.

[0124] Furthermore, the programs that implement the series of functions in the building equipment display system 1 of this embodiment on the terminal device 10 and the management server device 20, respectively, may be provided not only by means of communication, but also by being stored on various recording media.

[0125] In this embodiment of the building equipment display system 1, the layout plan 71 created by the user in the layout plan creation unit 30 is superimposed on the captured image 120. However, the system is not limited to creating the layout plan 71 in the layout plan creation unit 30. For example, the terminal device 10 may download a layout plan or other drawings that can be converted to a layout plan, such as CAD drawings, from the management server device 20. The terminal device 10 may then superimpose the equipment image onto the captured image 120 based on the downloaded drawing.

[0126] Furthermore, in the building equipment display system 1 of this embodiment, the superimposed image 800 displayed on the terminal device 10 is not temporarily stored in working memory and is not designed to be saved and read from the auxiliary storage means. However, in the building equipment display system 1, the video and still images of the superimposed image 800 may be saved in the auxiliary storage means of the terminal device 10 or the management server device 20.

[0127] <Other> (((1))) A building equipment display system characterized by comprising: a means for photographing buildings; a display means for displaying images on a display screen; a reception means for receiving requests from a user via a reception image displayed on the display screen for display of a layout diagram in which equipment images indicating building equipment are arranged on a building image corresponding to a building; and a control means for controlling the display screen to display a superimposed image on the photographed image of the building taken by the photographing means, in response to a user operation on an instruction image displayed on the display screen on which the layout diagram is displayed, by superimposing the equipment images on the photographed image of the building. (((2))) The building equipment display system according to (((1))), characterized in that the control means causes the reception image to be displayed on the display screen in which the superimposed image is displayed in response to a user operation on the second instruction image displayed on the display screen. (((3))) The building equipment display system according to (((2))), characterized in that the control means displays the instruction image and the second instruction image on the lower side of the display screen when the display screen is in an upright position. (((4))) The building equipment display system according to (((1))) or (((2))), characterized in that the control means displays on the display screen, in response to a user's operation on a third instruction image displayed on the display screen on which the layout diagram is displayed, estimate information relating to an estimate for the construction of building equipment created based on the layout diagram. (((5))) The building equipment display system according to (((1))), characterized in that the receiving means receives from the user the setting of the arrangement of equipment images in a direction that penetrates the boundary portion constituting the boundary of the building. (((6))) The building equipment display system according to (((5))), comprising an estimation preparation means for preparing an estimate for the construction of building equipment based on the layout drawing received by the reception means, wherein the estimation preparation means sets a different estimated amount from that for equipment images that do not penetrate the boundary when an equipment image penetrating the boundary is selected. (((7))) The building equipment display system according to (((1))), characterized in that the control means displays the superimposed image by making the equipment image follow the captured image when the captured image displayed on the display screen changes as the display screen moves, and the receiving means receives the setting of the reference for the start of movement of the display screen from the user on the receiving image on which the layout diagram is displayed. (((8))) The building equipment display system according to (((7))), characterized in that the control means displays the superimposed image based on the setting of a plurality of different criteria. (((9))) The building equipment display system according to (((1))), characterized in that the receiving means receives the arrangement of the equipment image from the user in the arrangement diagram which is displayed in two dimensions, and the control means displays the equipment image which is displayed in three dimensions on the display screen based on the arrangement diagram which is displayed in two dimensions, as received by the receiving means. (((10))) The building equipment display system according to (((1))), characterized in that the receiving means receives from the user the display of the layout diagram created based on measurement information obtained from a measuring means for measuring the layout of building equipment already installed in a building. (((11))) The building equipment display system according to (((1))), characterized in that it displays detailed information about the building equipment in response to user operations on the layout diagram. (((12))) The building equipment display system according to (((11))) is characterized in that the aforementioned detailed information is an image of the building equipment already installed in the building. (((13))) A terminal device comprising: a means for photographing a building; a means for displaying an image on a display screen; a means for receiving a request from a user via a reception image displayed on the display screen for displaying a layout diagram in which equipment images indicating building equipment are arranged on a building image corresponding to the building; and a control means for controlling the display screen to display a superimposed image on the display screen in which the equipment images are superimposed on a photographed image of the building taken by the photographing means, in response to a user operation on an instruction image displayed on the display screen on which the layout diagram is displayed. (((14))) A program that enables a computer, which functions as a portable terminal device, to implement the following functions: a function to photograph buildings; a function to display images on a display screen; a function to receive requests from the user via a reception image displayed on the display screen for the display of a layout diagram in which equipment images indicating building equipment are placed on a building image corresponding to a building; and a function to control the display screen to display a superimposed image on the photographed image of the building, which was taken by the photographing means, by superimposing the equipment images on the photographed image, in response to the user's operation on the instruction image displayed on the display screen on which the layout diagram is displayed. Furthermore, according to the above-mentioned (((1))) to (((14))), it is possible to create drawings regarding the arrangement of building equipment in a building and to easily compare the created drawings with the actual building. [Explanation of symbols]

[0128] 1…Building equipment display system, 10…Terminal device, 11D…Display screen, 12…Camera, 30…Layout plan creation unit, 40…AR display unit, 50…Estimate creation unit, 63…AR display button, 64…Estimate creation button, 69…Bird's-eye view button, 71…Layout plan, 73…Boundary image, 75…Equipment image, 120…Captured image, 400…Reception screen, 800…Overlay image, 850…Estimate image

Claims

1. In the processor, A function to manage images of equipment by group, A function to control the display of images of the aforementioned equipment by group, The system includes a function to overlay images of equipment corresponding to a selected group, such that images of the equipment located inside the boundary are displayed on the front side of the display screen, and images of the equipment located on the opposite side of the boundary are displayed on the back side of the display screen, at the corresponding positions in the captured images of the actual site. A program to achieve this.

2. The aforementioned processor, A function that allows users to choose whether or not to display each group, A function to display images of the equipment corresponding to the group selected as the target of display on the display screen, The program according to claim 1, further comprising:

3. The aforementioned processor, When multiple groups are selected as targets for display, a function is provided to display the images of the equipment in different colors. The program according to claim 2, further comprising:

4. The aforementioned processor, A function to display images of the equipment corresponding to the selected group at corresponding locations in a two-dimensional or three-dimensional layout diagram. The program according to claim 2, further comprising:

5. The aforementioned processor, When a predetermined operation is received, the estimated construction information for the image of the equipment on the display screen is displayed. The program according to any one of claims 1 to 4.

6. The processor is The construction cost estimate information includes the cost of the equipment that penetrates the boundary and is located on the opposite side of the boundary. The program according to claim 5.

7. The aforementioned processor, A function that allows the display of the equipment's image to track changes in the captured image of the actual site, A function that accepts the setting of multiple reference points indicating a boundary where the change in the captured image begins, The program according to any one of claims 1 to 6, further comprising:

8. The aforementioned processor, A function that accepts settings for the placement of images of the equipment that penetrate the boundary of the site. The program according to any one of claims 1 to 6, further comprising:

9. The aforementioned facilities include at least one of the following: air conditioning equipment, plumbing equipment, electrical equipment, building equipment, equipment related to water or other liquids, equipment related to gas or other gases, equipment related to electrical wiring, and equipment related to telecommunications. The program according to any one of claims 1 to 8.

10. The aforementioned group is managed for each boundary surface that divides the space. The program according to any one of claims 1 to 9.

11. The aforementioned groups are provided according to the type of equipment. The program according to any one of claims 1 to 10.

12. The processor, The images of the equipment are managed by group, The display of images of the aforementioned equipment is managed by group, The images of the equipment corresponding to the selected group are superimposed on the corresponding positions in the captured images of the actual site, with the image of the equipment located inside the boundary being displayed on the foreground of the display screen and the image of the equipment located on the opposite side of the boundary being displayed on the background of the display screen. How to display equipment information.

13. A functional unit that manages images of equipment by group, A functional unit that controls the display of images of the aforementioned equipment by group, A functional unit that displays images of equipment corresponding to a selected group by superimposing them on the display screen, such that images of the equipment located inside the boundary are displayed on the front side of the display screen, and images of the equipment located on the opposite side of the boundary are displayed on the back side of the display screen, at the corresponding positions in the captured images of the actual site. A display system for equipment information.

14. It has a processor, The aforementioned processor, The images of the equipment are managed by group, The display of images of the aforementioned equipment is controlled by group, The images of the equipment corresponding to the selected group are superimposed on the corresponding positions in the captured images of the actual site, with the image of the equipment located inside the boundary being displayed on the foreground of the display screen and the image of the equipment located on the opposite side of the boundary being displayed on the background of the display screen. Terminal device.

Images