Building Management Systems and Programs

Abstract

To provide a building management system which facilitates building management work, such as making an estimate for repair or renovation quickly, through simple operation, using a low-cost device, without using high-cost computer resources.SOLUTION: A building management system includes: a communication unit which receives building information from a portable information terminal having a three-dimensional data generation function of imaging the interior of a room of a building and generating three-dimensional data of the interior of the room, an object generation function of generating an object in the room from the three-dimensional data, and a communication function; a white model generation unit which combines wall objects included in the building information received by the communication unit to generate a white model of the building including at least one room; and a BIM data allocation unit which allocates BIM data to at least one target object selected from among an object of the white model, fitting objects and home equipment objects.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a building management system and a program. [Background technology]

[0002] For example, Patent Document 1 discloses an invention of a work management support device aimed at efficiently performing work related to structures such as social infrastructure (particularly repair work and inspection work). The work management support device includes an imaging means capable of acquiring real-world images of a target area including the structure, and a support information generation means for acquiring 3D shape information corresponding to the target area and generating virtual support information indicating that virtual work has been performed over time on the 3D shape information. The real-world images and virtual support information are synchronized in real time and can be displayed on a display means. This invention is said to be able to visualize the progress of work, facilitating consensus building among multiple parties, for example, on the target area, work content, and estimates of work-related quantities for work such as repairing, renovating, and replenishing structures.

[0003] For example, Patent Document 2 discloses an invention for an interior construction management system for interior construction work, such as restoration work. The interior construction management system includes: a drawing registration means for registering drawing information for each property provided by the client; a photo registration means for registering photographic information based on actual photographs taken within the property; an estimate creation means for determining restoration work and setting the dimensions of the area to be restored based on the drawings and / or photographs registered by the drawing registration means and the photo registration means; an estimate creation means for creating an estimate showing at least a list, quantity, and price of products selected as necessary for each restoration work based on the set dimensional information; and a schedule creation means for creating a restoration work schedule based on the selected products and dimensional information. This invention is said to enable construction personnel to place orders, receive orders, and perform work in a focused and efficient manner, while simultaneously clarifying and reducing costs. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-155553 [Patent Document 2] Japanese Patent Publication No. 2020-155124 Summary of the Invention [Problem to be solved by the invention]

[0005] The work management support device described in Patent Document 1 synchronizes virtual support information and real-world video in real time, making it effective for tracking work progress. However, while the invention described in this document includes the idea of ​​generating objects from 3D shape information as virtual support information, it does not include the idea of ​​associating the objects with BIM (Building Information Modeling) data. In other words, the 3D shapes captured by the work management support device described in this document are diverse and not standardized, and the objects generated are also diverse, making it difficult to organize data when calculating construction estimates and reducing the accuracy of estimates. Furthermore, managing a wide variety of objects places a heavy processing load on the computer system, and providing a good system environment requires expensive hardware resources.

[0006] Furthermore, the interior construction management system described in Patent Document 2 allows construction personnel to place orders, receive orders, and perform work in a centralized and efficient manner, while simultaneously clarifying and reducing costs. However, in the invention described in this document, the dimensions of the work area are set based on registered drawings and photographs, which makes creating the drawings time-consuming and the dimensions read from the photographs may be inaccurate. Furthermore, there is no concept of treating products, etc., as objects and associating BIM data with those objects. Therefore, as with the business management support device described in Patent Document 1, the organization of calculations for construction estimates, etc., becomes cumbersome, which can hinder the accuracy of the estimates.

[0007] The object of the present invention is to provide a building management system that can measure and obtain three-dimensional data of the interior of a building with simple operations, obtain the walls, windows, doors, home appliances, etc. within the interior as objects to which individual attribute information can be assigned, and then assign BIM data, which serves as attribute information, to the objects, thereby enabling building management tasks such as creating estimates to be performed quickly with simple operations using inexpensive devices, without using expensive computer resources.

[0008] Note that the BIM data in this invention does not refer to the broader BIM data used in all architectural information modeling methods, but refers to simplified 3D data and attribute information limited to data related to building maintenance management, renovation, etc. used in the building management system of this invention, and these are collectively referred to as BIM data in this specification. Examples of attribute information for BIM data in this invention include the common code of the corresponding object, company code, large / medium / subcategory, manufacturer name, product name, product code, size, material unit price, construction unit price, unit, remarks, various codes, and associated repair work. [Means for solving the problem]

[0009] In order to solve the above problem, in a first aspect of the present invention, a building management system is provided that includes: a mobile information terminal that includes: a 3D data generation function that captures an image of the inside of a room of a building and generates 3D data of the inside of the room by measuring the distance from the imaging point to the imaged object; an object generation function that generates, from the 3D data, at least one object selected from wall objects corresponding to each of the walls of the room, fixture objects corresponding to each of single or multiple fixtures installed in the room, and household equipment objects corresponding to each of single or multiple household equipment installed in the room; and a communication function that transmits building information including multiple wall objects for at least a single room. The building management system includes: a communication unit that receives the building information; a white model generation unit that combines multiple wall objects included in the building information received by the communication unit to generate a white model of the building including at least a single room; and a BIM data allocation unit that allocates BIM data to at least one target object selected from the objects of the white model, the fixture objects, and the household equipment objects.

[0010] The image processing device may further include a white model shaping unit that shapes the white model by applying correction to at least one of the wall objects that constitute the white model.

[0011] The system may further include an object image generation unit that generates a two-dimensional or three-dimensional object image representing at least one object selected from the objects of the white model, the fixture objects, and the housing equipment objects, and the communication unit may transmit data of the object image and list data of all or part of the BIM data to the mobile information terminal or other information processing device. In this case, the communication unit may receive a request from the mobile information terminal or other information processing device that includes designation of at least one object represented in the object image and BIM data corresponding to the designated object, and in response to the request, the BIM data allocation unit may change or newly allocate the BIM data of the designated object to the BIM data included in the request.

[0012] The system may further include an estimate generation unit that references all or part of the BIM data and generates an estimate of the repair and renovation costs of the building, and the communication unit may transmit the estimate data to the mobile information terminal or other information processing device.

[0013] The system may further include an inference unit that receives at least one input object selected from the white model object, the fixture object, and the home appliance object, and outputs one or more BIM data candidates corresponding to the input object as a result of inference. The inference unit may perform inference using a supervised trained neural network trained on the input object and correct answer information corresponding to the input object, or an unsupervised trained neural network trained to appropriately classify and output BIM data corresponding to the input object. In this case, the BIM data assignment unit may automatically assign the most probable BIM data candidate output by the inference unit as a result of inference using the assignment target object as input to the assignment target object. Alternatively, the system may further include a communication unit that transmits the BIM data candidate output by the inference unit as a result of inference using the assignment target object to the mobile information terminal or other information processing device used by the user, and the BIM data assignment unit may assign the BIM data candidate selected by the user from the BIM data candidate transmitted by the communication unit to the assignment target object.

[0014] The system may further include a grade registration unit that registers a grade for at least one registered object selected from the white model object, the building fixture object, and the home appliance object, and the BIM data allocation unit may allocate BIM data corresponding to the grade registered in the grade registration unit to the registered object.

[0015] The system may further include a comment information storage unit that stores comment information related to at least one selected object selected from the objects of the white model, the fixture objects, and the household equipment objects, linked to the selected object. In this case, the comment information storage unit may further store position information indicating a position of a focus point in the selected object, and may further include an object image generation unit that generates a two-dimensional or three-dimensional object image representing one or more objects selected from the objects of the white model, the fixture objects, and the household equipment objects, the objects including at least the selected object, and a focus point image generation unit that generates a focus point image indicating the focus point of the selected object so as to be superimposed on the object image, and the communication unit may transmit data of the object image with the focus point image superimposed and the comment information to the mobile information terminal or another information processing device.

[0016] The comment information holding unit may further hold image data or electronic data related to the comment information, in which case the communication unit transmits the image data or electronic data, in addition to the image data and comment information of the object, to the mobile information terminal or other information processing device.

[0017] In addition, the building information may include captured image data of the interior of the room at a position corresponding to the point of interest, the comment information storage unit may further store the captured image data linked to the point of interest, and the communication unit may transmit the captured image data in addition to data of the object image on which the point of interest image is superimposed and the comment information.

[0018] The comment information storage unit may store imaging information indicating the shooting position and shooting direction when the captured image data was acquired, and the focus point image generation unit may generate the focus point image and generate the shooting position and shooting direction so as to be superimposed on the object image.

[0019] The system may further include a drawing / perspective generation unit that uses the BIM data of the target objects assigned by the BIM data assignment unit to generate a floor plan, a three-dimensional drawing, or a perspective drawing of the building that is composed of the target objects.

[0020] In a second aspect of the present invention, a program is provided for causing a computer to realize the following functions from a mobile information terminal: a 3D data generation function for capturing an image of the interior of a room of a building and generating 3D data of the interior of the room by measuring the distance from the imaging point to the object being imaged; an object generation function for generating, from the 3D data, at least one object selected from wall objects corresponding to each of the walls of the room, fixture objects corresponding to each of single or multiple fixtures installed in the room, and household equipment objects corresponding to each of single or multiple household equipment installed in the room; and a communication function for transmitting building information including multiple wall objects for at least a single room.The program provides a program for causing a computer to realize the following functions from a mobile information terminal: a communication function for receiving the building information; a white model generation function for combining multiple wall objects included in the building information received by the communication unit to generate a white model of the building including at least a single room; and a BIM data assignment function for assigning BIM data to at least one target object selected from the objects of the white model, the fixture objects, and the household equipment objects.

[0021] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of a network system 10 that uses a building management system 100. FIG. [Figure 2] 1 is a functional block diagram showing an example of a building management system 100, a mobile information terminal 200, and an information processing device 300. FIG. [Figure 3] FIG. 2 is a functional block diagram showing an example of a white model shaping unit 106. [Figure 4] 2 is a flow diagram showing an example of the operation of the building management system 100. FIG. [Figure 5] FIG. 10 is a flow diagram showing an example of a white model shaping process 512. [Figure 6] FIG. 10 is a flow diagram showing an example of BIM data allocation processing 518. [Figure 7] FIG. 10 is a flow diagram showing another example of the BIM data allocation process 518. [Figure 8] 10 is a flowchart showing another example of the operation of the building management system 100. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0023] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0024] 1 is a conceptual diagram showing an example of a network system 10 that uses a building management system 100. The network system 10 includes the building management system 100, a mobile information terminal 200, an information processing device 300, and a network 400, and the building management system 100 and the mobile information terminal 200 are connected to each other via the network 400, and the building management system 100 and the information processing device 300 are connected to each other via the network 400.

[0025] The building management system 100 is an information processing device 300 that functions as a server, and the mobile information terminal 200 is a portable or mobile information processing device 300 such as a smartphone or a pad computer. The information processing device 300 is a general information processing device 300 such as a desktop personal computer or a notebook personal computer. The mobile information terminal 200 and the information processing device 300 function as clients that issue requests to the building management system 100, which is the server, and process responses to the requests from the server. The network 400 is a data communication means that connects the mobile information terminal 200 and the information processing device 300, which are clients, to the building management system 100, which is the server. Examples of the network 400 include a wireless LAN (Local Area Network) or a wired LAN that conforms to the Internet Protocol, a mobile phone network, Ethernet, a serial communication line, Bluetooth (registered trademark) wireless communication, etc.

[0026] The hardware configurations of the building management system 100, mobile information terminal 200, and information processing device 300 are all common, and may include, for example, a motherboard, a CPU (Central Processing Unit) connected to a system bus on the motherboard, cache memory, main memory, a GPU (Graphics Processing Unit), graphics memory, mass storage devices such as HDDs (Hard Disk Drives) and SSDs (Solid State Drives), input devices such as keyboards and pointing devices, output control devices such as displays and printers, etc. By cooperating with this hardware through software, the building management system 100, mobile information terminal 200, and information processing device 300 can realize the functions described below.

[0027] FIG. 2 is a functional block diagram showing an example of the building management system 100, the mobile information terminal 200, and the information processing device 300.

[0028] The building management system 100 includes a communication unit 102, a white model generation unit 104, a white model shaping unit 106, a BIM data allocation unit 108, an inference unit 110, a grade registration unit 112, an object image generation unit 114, an estimate generation unit 116, a comment information storage unit 118, a focus point image generation unit 120, a memory unit 122, a control unit 124, and a drawing / perspective generation unit 126. The mobile information terminal 200 includes a camera module 202, a LiDAR (Light Detection and Ranging) module 204, a 3D data generation unit 206, an object generation unit 208, a communication unit 210, a display input unit 212, a memory unit 214, and a control unit 216. The information processing device 300 includes an input unit 302, an output unit 304, a display unit 306, a memory unit 308, a communication unit 310, and a control unit 312.

[0029] The camera module 202 captures images of the interior of a room in a building, and the LiDAR module 204 measures the distance from the imaging point to the imaged object. Based on the measurements by the LiDAR module 204, the 3D data generation unit 206 generates 3D data of the interior of the room. The object generation unit 208 generates objects within the room from the 3D data generated by the 3D data generation unit 206. Examples of the generated objects include wall objects corresponding to each wall of the room, fixture objects corresponding to one or more fixtures installed in the room, and home appliance objects corresponding to one or more home appliances installed in the room. The object generation unit 208 generates at least one of these example objects. The communication unit 210 transmits and receives data. The communication unit 210 transmits building information including multiple wall objects for at least one room generated by the object generation unit 208.

[0030] The display input unit 212 is, for example, a liquid crystal display equipped with a touch panel, and the memory unit 214 stores programs and data. The control unit 216 controls the camera module 202, the LiDAR module 204, the 3D data generation unit 206, the object generation unit 208, the communication unit 210, the display input unit 212, and the memory unit 214.

[0031] The input unit 302 is, for example, a pointing device such as a keyboard or a mouse, and the output unit 304 is, for example, a printer. The display unit 306 is, for example, a liquid crystal display, and the storage unit 308 is, for example, a mass storage device such as a HDD. Programs and data are recorded in the storage unit 308. The communication unit 310 transmits and receives data. The control unit 312 controls the input unit 302, the output unit 304, the display unit 306, the storage unit 308, and the communication unit 310.

[0032] The communication unit 102 transmits and receives data to and from the mobile information terminal 200 or the information processing device 300. In particular, the communication unit 102 receives building information transmitted from the mobile information terminal 200. As described above, the building information includes a plurality of wall objects for at least a single room generated by the object generation unit 208. The communication unit 102 also transmits object image data and list data of all or part of the BIM data to the mobile information terminal 200 or the information processing device 300.

[0033] The white model generating unit 104 combines a plurality of wall objects included in the building information received by the communication unit 102 to generate a white model of a building including at least one room.

[0034] The white model shaping unit 106 shapes the white model by applying correction to at least one of the wall objects that make up the white model. Fig. 3 is a functional block diagram showing an example of the white model shaping unit 106. For example, the white model shaping unit 106 has the following correction functions: tilt correction 150, distortion correction 152, module pitch correction 154, unnecessary wall elimination correction 156, small room elimination correction 158, straightening correction 160, perimeter correction 162, and wall division correction 164.

[0035] Tilt correction 150 rotates the white model to correct tilt. Distortion correction 152 corrects two wall objects so that they are in a straight line parallel to the X-axis or Y-axis. Module pitch correction 154 moves each object so that it aligns with the grid lines. Unnecessary wall removal correction 156 removes unnecessary wall objects. Small room removal correction 158 removes wall objects that make up small rooms. Straightening correction 160 combines two wall objects into one. Periphery correction 162 complements missing peripheral walls. Wall division correction 164 divides wall objects.

[0036] The BIM data allocation unit 108 allocates BIM data to at least one target object selected from the white model object, the fixture object, and the housing equipment object.

[0037] The inference unit 110 can be used to assign BIM data. At least one input object selected from white model objects, fixture objects, and housing equipment objects is input to the inference unit 110, and one or more BIM data candidates corresponding to the input object are output as the inference result. The inference unit 110 performs inference using a supervised trained neural network that has been trained using the input object and correct answer information corresponding to the input object. Alternatively, the inference unit 110 performs inference using an unsupervised trained neural network that has been trained to appropriately classify and output BIM data corresponding to the input object.

[0038] As an example of using the inference unit 110, the BIM data allocation unit 108 can automatically assign to the object to be allocated the BIM data with the highest probability among the BIM data candidates output by the inference unit 110 as a result of inference using the object to be allocated as input.

[0039] Alternatively, as another example using the inference unit 110, the BIM data allocation unit 108 can allocate BIM data corresponding to a user's selection from among the BIM data candidates presented to the user as a result of the inference to the allocation target object. That is, the communication unit 102 transmits the inference result of the inference unit 110 to the mobile information terminal 200 or information processing device 300 used by the user, and the BIM data allocation unit 108 allocates BIM data corresponding to the user's selection operation from among the BIM data candidates transmitted by the communication unit 102 to the allocation target object.

[0040] The BIM data allocation unit 108 can also allocate BIM data to objects (registered objects) according to the grade registered in the grade registration unit 112. The grade registration unit 112 registers a grade for at least one registered object selected from white model objects, fixture objects, and housing equipment objects.

[0041] It is also possible to change BIM data that has already been assigned, or to assign new BIM data to an unassigned object. In this case, the communication unit 102 receives a request including the designation of at least one object and the BIM data corresponding to the designated object from the mobile information terminal 200 or the information processing device 300, and in response to the request, the BIM data assignment unit 108 changes or newly assigns the BIM data of the designated object to the BIM data included in the request.

[0042] The object image generation unit 114 generates a two-dimensional or three-dimensional object image representing at least one object selected from a white model object, a fixture object, and a home appliance object. Data of the generated object image is transmitted by the communication unit 102 to the mobile information terminal 200 or the information processing device 300.

[0043] The estimate generation unit 116 references all or part of the BIM data to generate an estimate of the cost of repairing and renovating the building. The communication unit 102 transmits the generated estimate data to the mobile information terminal 200 or the information processing device 300.

[0044] The comment information storage unit 118 stores comment information related to at least one selected object selected from the white model objects, the fixture objects, and the household equipment objects, in association with the selected object.

[0045] The comment information storage unit 118 can further store position information indicating the position of the point of interest in the selected object, and in this case, the point of interest image generation unit 120 generates a point of interest image indicating the point of interest in the selected object so as to be displayed superimposed on the object image generated by the object image generation unit 114. Data of the object image on which the point of interest image is superimposed and the comment information are transmitted by the communication unit 102 to the mobile information terminal 200 or the information processing device 300.

[0046] The comment information storage unit 118 can further store image data or electronic data related to the comment information. In this case, the communication unit 102 transmits the image data or electronic data, in addition to the image data and comment information of the object, to the mobile information terminal 200 or the information processing device 300.

[0047] Furthermore, if the building information received by the communication unit 102 includes captured image data of the interior of a room at a position corresponding to the point of interest, the comment information storage unit 118 stores the captured image data linked to the point of interest, and the communication unit 102 transmits the captured image data to the mobile information terminal 200 or the information processing device 300 in addition to the data of the object image on which the image of the point of interest is superimposed and the comment information.

[0048] The comment information storage unit 118 can store imaging information indicating the shooting position and shooting direction when the captured image data linked to the focus point was acquired. In this case, the focus point image generation unit 120 generates the focus point image and also generates the shooting position and shooting direction so that they are superimposed on the object image and displayed.

[0049] The drawing / perspective generating unit 126 uses the BIM data of the target objects assigned by the BIM data assigning unit 108 to generate a plan view, a three-dimensional view, or a perspective view of a building made up of the target objects.

[0050] The memory unit 122 is a mass storage device such as a HDD, and stores programs and data. In particular, the memory unit 122 stores BIM data and object information. The control unit 124 controls the communication unit 102, the white model generation unit 104, the white model shaping unit 106, the BIM data allocation unit 108, the inference unit 110, the grade registration unit 112, the object image generation unit 114, the estimate generation unit 116, the comment information storage unit 118, the focus point image generation unit 120, the memory unit 122, and the drawing / perspective generation unit 126.

[0051] 4 is a flow diagram showing an example of the operation of the building management system 100. First, the camera module 202 of the mobile information terminal 200 captures an image of the interior of the building, and the LiDAR module 204 measures the distance to objects present in the room (step 500). From the distance measurement results obtained by the LiDAR module 204, the three-dimensional data generation unit 206 generates three-dimensional data (3D data) of the interior of the room (step 502). Furthermore, from the generated 3D data, the object generation unit 208 generates objects for the interior of the room (step 504).

[0052] The generated object has, as its attributes, category information including a category indicating the category to which the object belongs, as well as identification information (UUID: Universally Unique Identifier), location information, size information such as height and width, and identification information of related objects if the object is related to other objects. Examples of categories include walls, doors, openings, windows, bathtubs, beds, chairs, dishwashers, fireplaces, ovens, refrigerators, sinks, sofas, stairs, storage, stoves, desks, televisions, toilets, and washing and drying appliances. In this specification, objects belonging to the building structure category, such as walls and wall openings, are referred to as wall objects, objects belonging to the fixture category, such as windows and doors, are referred to as fixture objects, and objects belonging to the home appliance category, such as bathtubs, beds, chairs, dishwashers, and fireplaces, are referred to as home appliance objects.

[0053] The generated object (including category information) is transmitted to the building management system 100 as building information (step 506), and the communication unit 102 of the building management system 100 receives the building information (step 508). The building information may include image data captured in step 500 or image data captured separately. The generated object may be transmitted to the building management system 100 in real time in synchronization with the generation of the object, or the generated object may be temporarily recorded in the storage unit 214 and then transferred from the storage unit 214 to the building management system 100.

[0054] The objects received by the communication unit 102 in step 508 are input to the white model generation unit 104, which generates a white model (step 510). By generating a white model, it is possible to place fittings objects corresponding to fittings and household equipment objects corresponding to household equipment in the generated white model, thereby making it possible to simulate the indoor structure that makes up a room in a real building.

[0055] The white model is generated, for example, according to the following algorithm: That is, by referencing wall objects included in the received object, and using the object identification information (UUID), position information, and size information included in the category information of the wall object, vertex information for the wall object is constructed from the position information of the wall object and the width information included in the size information, and a determination is made as to whether the vertices of each wall object are connected. Connected wall objects are sequentially connected, and if the start and end points of the vertices of a connected series of wall objects match, the space enclosed by the series of wall objects is recognized as a room. This determination and recognition process is performed for all wall objects, recognizing one or more rooms, and wall objects that are not determined to constitute a room are recognized as walls. In this way, a white model consisting of one or more rooms and walls is constructed.

[0056] However, the white model generated in step 510 may contain distorted walls that should be straight, unnecessary walls, rooms that do not exist in reality, or walls that should be missing. These defects are noise caused by improperly generated wall objects and result in a deviation from the actual room structure. If this deviation remains, the image simulating the room may display distorted or missing walls, or non-existent walls, and the fixtures and fixtures and fixtures placed in the room may appear unnatural, which is inconvenient. Furthermore, if wall objects that should be present are missing or unnecessary wall objects remain, the BIM data that will be assigned later will not be assigned properly. Therefore, to eliminate the deviation between the actual room structure and the white model, the white model is shaped (step 512).

[0057] 5 is a flow diagram showing an example of the white model shaping process. The shaping of the white model is processed by the white model shaping unit 106 in the following order: tilt correction (step 540), which rotates the white model to correct the tilt; distortion correction (step 542), which corrects two wall objects so that they are straight lines parallel to the X-axis or Y-axis; module pitch correction (step 544), which moves each wall object along a grid line; unnecessary wall deletion correction (step 546), which deletes unnecessary wall objects; small room deletion correction (step 548), which deletes wall objects that constitute small rooms; straightening correction (step 550), which combines two wall objects into one; perimeter correction (step 552), which complements missing perimeter walls; and wall division correction (step 554), which divides wall objects. Note that it is not necessary to perform all of the correction processes from step 540 to step 554; any of the correction processes may be performed as needed.

[0058] By shaping the white model, the discrepancy between the actual room structure and the white model is eliminated, and BIM data can be appropriately assigned to the wall objects that make up the white model.

[0059] Next, the grade registration unit 112 of the building management system 100 generates a grade registration request and sends it to the client side (step 514). Note that the processing on the client side after the mobile information terminal 200 sends the building information to the building management system 100 in step 506 may continue to be performed by the mobile information terminal 200, or after appropriate login processing, an information processing device 300 other than the mobile information terminal 200 may take over the processing as a client. Therefore, the grade registration request in step 514 is sent to the mobile information terminal 200 or information processing device 300 operating as a client.

[0060] The mobile information terminal 200 or the information processing device 300 that received the grade registration request performs the grade registration operation (step 516), and the BIM data allocation unit 108 of the building management system 100 that received the grade registration allocates the BIM data corresponding to the registered grade to the wall objects, fittings objects, and housing equipment objects that make up the white model (step 518). Note that if no grade is registered, a preset default grade may be applied.

[0061] In a broad sense, BIM data refers to data used in modeling methods for information about buildings, but the BIM data in this invention does not refer to the broad BIM data used in all modeling methods for building information, but refers to simplified 3D data and attribute information limited to data related to building maintenance management, renovation, etc. used in the building management system 100, and these are collectively referred to as BIM data. Examples of attribute information for BIM data in this invention include the common code of the corresponding object, company code, large / medium / small classification, manufacturer name, product name, product code, size, material unit price, construction unit price, unit, remarks, various codes, and associated repair work.

[0062] The assignment of BIM data at step 518 does not need to be performed for all wall objects, fixture objects, and home appliance objects, and BIM data may be assigned only to some of the wall objects, fixture objects, or home appliance objects. Also, step 518 may be skipped without assigning BIM data to all objects.

[0063] The BIM data allocation in step 518 may be performed using the inference results from the inference unit 110. FIG. 6 is a flow diagram showing an example of the BIM data allocation process (step 518). First, the BIM data allocation unit 108 identifies a target object that is the target of the BIM data allocation process (step 560) and inputs the target object into the trained model of the inference unit 110 (step 562). The inference unit 110 infers BIM data candidates for the input target object (step 564) and passes the inferred BIM data candidates to the BIM data allocation unit 108. The BIM data allocation unit 108 assigns the BIM data that shows the highest probability among the BIM data candidates to the target object (step 566). The BIM data allocation unit 108 determines whether there are any remaining target objects (step 568). If it determines that there are remaining target objects, it repeats the process from step 560; if it determines that there are no remaining target objects, it ends the process (step 570). By performing this type of processing, the BIM data that is inferred to be the most appropriate for the target object can be automatically assigned, eliminating the need for user operation for assignment and improving user convenience.

[0064] Furthermore, the BIM data allocation in step 518 may be based on a user's selection with reference to the inference results by the inference unit 110. FIG. 7 is a flow diagram showing another example of the BIM data allocation process (step 518). Similar to the process in FIG. 6, the target object is identified (step 560), input to the trained model (step 562), and BIM data candidates are inferred (step 564). The inferred BIM data candidates are then transmitted to the mobile information terminal 200 or the information processing device 300 (step 572). The mobile information terminal 200 or the information processing device 300 that has received the BIM data candidates displays the BIM data candidates (step 574), and the user performs a selection operation to select BIM data to be allocated from the displayed BIM data candidates (step 576). The BIM data allocation unit 108, which has received information on the selection result, allocates the selected BIM data selected by the user to the target object (step 578). The BIM data allocation unit 108 determines whether there are any remaining target objects (step 568), and if it is determined that there are, repeats the processing from step 560, and if it is determined that there are no remaining objects, ends the processing (step 570). By performing such processing, BIM data according to the user's selection can be allocated from among BIM data candidates inferred to be appropriate for the target object, and the user's BIM data allocation operation can be assisted, thereby improving user convenience.

[0065] Next, the object image generation unit of the building management system 100 generates object images for all or some of the wall objects, fittings objects, and home appliance objects (step 520). The object images are three-dimensional or two-dimensional images, and if BIM data is assigned to an object, an image such as a product image associated with the BIM data is generated as the image of the object.

[0066] The generated object image is transmitted to the mobile information terminal 200 or the information processing device 300 together with the list of BIM data (step 522). The mobile information terminal 200 or the information processing device 300 receives the object image and displays it on the display input unit 212 or the display unit 306 together with the list of BIM data (step 524). The user of the mobile information terminal 200 or the information processing device 300 references the displayed object image and performs an operation to add or modify the BIM data (step 526). The results of the addition or modification operation in step 526 are transmitted to the building management system 100, and the BIM data allocation unit 108 newly adds or reassigns the BIM data of the corresponding object to reflect the transmitted operation results (step 528). The process then returns to step 522 and repeats.

[0067] When the user completes the addition and modification operations in step 526 and wishes to request an estimate, the mobile information terminal 200 or the information processing device 300 requests the building management system 100 to create an estimate in response to the user's estimate request operation (step 530). The estimate generation unit 116 of the building management system 100, which has received the estimate request, generates an estimate by referencing the BIM data and transmits the generated estimate data to the requesting mobile information terminal 200 or the information processing device 300 (step 532). The mobile information terminal 200 or the information processing device 300, which has received the estimate data, displays it on the display input unit 212 or the display unit 306, or outputs it to the output unit 304 (step 534).

[0068] In addition to generating the estimate in step 532, the building management system 100 can generate a floor plan, a three-dimensional drawing, or a perspective drawing (hereinafter referred to as "drawings, etc.") of the building using the drawing / perspective generation unit 126, and can transmit the generated drawings, etc., together with or separately from the estimate, to the mobile information terminal 200 or the information processing device 300. In this case, the mobile information terminal 200 or the information processing device 300 that has received the drawings, etc. outputs the drawings, etc. (step 536) after outputting the estimate in step 534.

[0069] According to the above-mentioned building management system 100, a white model of a building's rooms is generated using a mobile information terminal 200 equipped with an object generation function, which simplifies tasks such as on-site dimension measurements and drawing drawings of existing buildings during building renovation work, etc., and significantly reduces work time.

[0070] Furthermore, according to the building management system 100 described above, the generated white model is shaped and simplified BIM data is assigned to each object, such as a wall, fixture, or appliance, so that an estimate can be easily created by referring to the product unit price, etc., of the BIM data.

[0071] Furthermore, with the above-mentioned building management system 100, BIM data can be added and changed interactively, making it easy to add and revise estimates, significantly improving work efficiency. In addition, the interactive addition and change function to BIM data makes it easy to input, manage history, and share repair information in building maintenance management, improving work efficiency.

[0072] Furthermore, in the above-described building management system 100, the BIM data of the present invention is limited to simplified BIM data, which reduces hardware resources and keeps the system usage costs low.

[0073] The building management system 100 described above can be used not only for creating estimates but also for property management as shown in Fig. 8. That is, when the mobile information terminal 200 or the information processing device 300 issues a property management request (step 580) and displays (step 584) the object image received from the building management system 100 in response to the request (step 582), the user of the mobile information terminal 200 or the information processing device 300 specifies the position of a point of interest on the displayed object, enters a comment, and performs operations to select a captured image (step 586), and transmits the position of the point of interest, the comment, and the captured image to the building management system 100 (step 588). The building management system 100 receives the position of the point of interest, the comment, and the captured image (step 590), and the comment information storage unit 118 stores them. The comment information holding unit 118 generates an object image by superimposing a point of interest image (for example, a pin image) on the point of interest position of the object related to the received point of interest position, comment, and captured image, and transmits the object image to the mobile information terminal 200 or the information processing device 300 (step 592). The mobile information terminal 200 or the information processing device 300 displays the received object image with the point of interest image together with the comment (step 594). This makes it possible to manage the point of interest, for example, when a stain or scratch is discovered on a wall of a property, by recording the stain or scratch as a point of interest and recording and updating comments and images.

[0074] The comment information storage unit 118 can store information on the shooting position and shooting direction (imaging information) in addition to the position of the point of interest, comments, and captured images, and can superimpose the shooting position and shooting direction on an object image with an image of the point of interest. The comment information storage unit 118 can also store image data as attribute information of the corresponding selected object (BIM data), and can further store electronic data that can be viewed in the same format as when printed on paper. The image data and electronic data can be displayed and printed out in the same way as comments.

[0075] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0076] The above-described embodiment can also be understood as a program for running a computer. That is, the program can be understood as a program for causing a computer to implement a 3D data generation function for capturing an image of the interior of a room in a building and measuring the distance from the image capture point to generate 3D data of the interior of the room, an object generation function for generating, from the 3D data, at least one object selected from wall objects corresponding to each wall of the room, fixture objects corresponding to one or more fixtures installed in the room, and household equipment objects corresponding to one or more household equipment installed in the room, and a communication function for transmitting building information including the wall objects for at least a single room, a communication function for receiving building information from a mobile information terminal 200 having the following functions: a white model generation function for combining the wall objects included in the building information received by the communication unit 102 to generate a white model of a building including at least a single room; and a BIM data assignment function for assigning BIM data to at least one target object selected from the white model objects, fixture objects, and household equipment objects.

[0077] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings, is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0078] 10...Network system, 100...Building management system, 102...Communication unit, 104...White model generation unit, 106...White model shaping unit, 108...BIM data allocation unit, 110...Inference unit, 112...Grade registration unit, 114...Object image generation unit, 116...Estimate generation unit, 118...Comment information storage unit, 120...Focus point image generation unit, 122...Memory unit, 124...Control unit, 126...Drawing / perspective generation unit, 150...Tilt correction, 152...Distortion correction, 154...Module pitch correction, 156...Incorrect Main wall removal correction, 158...small room removal correction, 160...straightening correction, 162...perimeter correction, 164...wall division correction, 200...mobile information terminal, 202...camera module, 204...LiDAR module, 206...3D data generation unit, 208...object generation unit, 210...communication unit, 212...display input unit, 214...memory unit, 216...control unit, 300...information processing device, 302...input unit, 304...output unit, 306...display unit, 308...memory unit, 310...communication unit, 312...control unit, 400...network.

Claims

1. a communication unit that receives the building information from a mobile information terminal that includes a three-dimensional data generation function that captures an image of the interior of a room of a building and generates three-dimensional data of the interior of the room by measuring the distance from the imaging point to the object being imaged, an object generation function that generates, from the three-dimensional data, at least one object selected from a wall object corresponding to each wall of the room, a fixture object corresponding to each of a single or multiple fixtures installed in the room, and a household equipment object corresponding to each of a single or multiple household equipment devices arranged in the room, and a communication function that transmits building information including a plurality of the wall objects for at least one of the rooms; a white model generation unit that generates a white model of the building including at least one of the rooms by combining a plurality of the wall objects included in the building information received by the communication unit; a BIM data allocation unit that allocates BIM data to at least one target object selected from the white model object, the fixture object, and the housing equipment object; Building management system.

2. a white model shaping unit that shapes the white model by applying correction to at least one of the wall objects that constitute the white model. The building management system according to claim 1 .

3. an object image generation unit that generates a two-dimensional or three-dimensional object image representing at least one object selected from the white model object, the fixture object, and the home equipment object; The communication unit transmits the object image data and a list data of all or part of the BIM data to the mobile information terminal or other information processing device. The building management system according to claim 1 .

4. The communication unit receives a request from the mobile information terminal or other information processing device, the request including designation of at least one object represented in the object image and BIM data corresponding to the designated object; In response to the request, the BIM data allocation unit changes or newly allocates the BIM data of the specified object to the BIM data included in the request. The building management system according to claim 3.

5. The system further includes an estimate generating unit that generates an estimate of repair and renovation costs for the building by referring to all or part of the BIM data, The communication unit transmits the estimate data to the mobile information terminal or other information processing device. The building management system according to claim 1 .

6. An inference unit that receives at least one input object selected from the white model object, the fixture object, and the housing equipment object, and outputs, as a result of inference, one or more BIM data candidates according to the input object, The inference unit performs inference using a supervised trained neural network trained using the input object and correct answer information corresponding to the input object, or an unsupervised trained neural network trained to appropriately classify and output BIM data corresponding to the input object. The building management system according to claim 1 .

7. The BIM data allocation unit automatically allocates the most probable BIM data to the target object among the BIM data candidates output by the inference unit as a result of inference using the target object as an input. The building management system according to claim 6.

8. Further, a communication unit is provided that transmits BIM data candidates output by the inference unit as a result of inference using the target object as input to the mobile information terminal or other information processing device used by the user, The BIM data allocation unit allocates BIM data corresponding to a selection operation of the user from among the BIM data candidates transmitted by the communication unit to the target object. The building management system according to claim 6.

9. a grade registration unit that registers a grade for at least one registered object selected from the white model object, the fixture object, and the home equipment object; The BIM data allocation unit allocates BIM data corresponding to the grade registered in the grade registration unit to the registered object. The building management system according to claim 1 .

10. a comment information storage unit that stores comment information related to at least one selected object selected from the white model object, the fixture object, and the home equipment object, by linking the comment information to the selected object. The building management system according to claim 1 .

11. the comment information storage unit further stores position information indicating a position of a point of interest in the selected object; an object image generation unit that generates a two-dimensional or three-dimensional object image representing one or more objects selected from the objects of the white model, the fixture objects, and the home equipment objects, the selected objects including at least the selected objects; a focus point image generating unit that generates a focus point image indicating the focus point of the selected object so as to be displayed superimposed on the object image, The communication unit transmits data of the object image on which the point-of-interest image is superimposed and the comment information to the mobile information terminal or other information processing device. The building management system according to claim 10.

12. the comment information storage unit further stores image data or electronic data related to the comment information; The communication unit transmits the image data or the electronic data of the object in addition to the image data and the comment information to the mobile information terminal or other information processing device. The building management system of claim 11.

13. the building information includes captured image data of the interior of the room at a position corresponding to the point of interest, the comment information storage unit further stores the captured image data in association with the point of interest; The communication unit transmits the captured image data in addition to data of the object image on which the point-of-interest image is superimposed and the comment information. The building management system of claim 11.

14. the comment information storage unit stores imaging information indicating a shooting position and a shooting direction when the captured image data was acquired; The point-of-interest image generating unit generates the point-of-interest image and generates the photographing position and the photographing direction so as to be superimposed on the object image. The building management system of claim 13.

15. The BIM data allocation unit further includes a drawing / perspective generation unit that generates a floor plan, a three-dimensional drawing, or a perspective drawing of the building composed of the target object by using the BIM data of the target object allocated by the BIM data allocation unit. The building management system according to claim 1 .

16. a receiving function for receiving the building information from a mobile information terminal having a three-dimensional data generating function for capturing an image of the interior of a room of a building and generating three-dimensional data of the interior of the room by measuring the distance from the imaging point to the object being imaged; an object generating function for generating, from the three-dimensional data, at least one object selected from a wall object corresponding to each wall of the room, a fixture object corresponding to each of a single or multiple fixtures installed in the room, and a household equipment object corresponding to each of a single or multiple household equipment devices arranged in the room; and a transmitting function for transmitting building information including a plurality of the wall objects for at least one of the rooms; a white model generating function that generates a white model of the building including at least one of the rooms by combining a plurality of the wall objects included in the building information received by the receiving function; A BIM data allocation function that allocates BIM data to at least one target object selected from the white model object, the fixture object, and the housing equipment object; A program to make the above happen on a computer.

Images