Meeting Support System

Abstract

To provide a conference support system capable of making it unnecessary for a client and a person in charge of design concerning design of a building to go to a conference and capable of recalling a concrete image concerning each object used for the conference.SOLUTION: A conference support system includes: a first server 10 for providing a virtual conference space VS that can be displayed through display units 406, 412 with respect to a real space in front of a plurality of user terminals 40; a second server 20 for transmitting BIM model data of a building to the first server 10 to provide a BIM model to the virtual conference space VS; and a third server 30 for transmitting material information data on each object constituting the BIM model to the first server 10 to provide a material model to the virtual conference space VS. A designer terminal and a client terminal are included in the user terminal 40. The designer terminal 40 integrates a material model with respect to the BIM model displayed in the virtual conference space VS to create a BIM model VM for a conference.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a conference support system. [Background technology]

[0002] When explaining the design of a house or other building to a client, or when discussing the created design with the client and revising the design by changing various building materials such as walls and flooring according to the client's requests, it has traditionally been the case that the client would visit a design office or home builder, or that a designer from the design office (including designers and sales staff, hereafter referred to as "designers") would visit the client's home and then hold discussions using a computer.

[0003] Architectural design begins with the architectural design phase, which involves determining the layout of a building (architecture) while taking into account the site conditions and surrounding environment, and then designing the interior and exterior, floor plan, and decoration. This architectural design phase uses commonly available computer-aided design (CAD) software to create not only two-dimensional (2D) drawings but also three-dimensional (3D) drawings, including perspective drawings. Recently, building information modeling (BIM) models have become commonplace in the design of buildings, allowing multiple design professionals, including designers, to share information about the building. A BIM model is a three-dimensional model that contains information about the shape, materials, dimensions, and location of building components (also known as objects, including beams, columns, and walls) in a virtual three-dimensional space. Using this BIM model, it is possible to create three-dimensional drawings of the three-dimensional model from various angles, as well as cut the three-dimensional drawings in various ways and create two-dimensional drawings, such as floor plans (including floor plans) and elevations (including frame drawings) from various angles. That is, each component that makes up a three-dimensional model contains various pieces of specification information, and this specification information can be changed, modified, or added to, and a history of changes, additions, etc. can also be kept.

[0004] Therefore, by using the BIM model downloaded to a computer to explain things, designers can provide clients with a design that is clear and easy to evoke a realistic image.

[0005] Patent Document 1 proposes a design user terminal that communicates with a Web catalog server that stores a Web catalog and designs a BIM model of a building. The design user terminal has a first communication unit that connects to the Web catalog server via a network and transmits to the Web catalog server a first intermediate file for requesting an object data list that compiles object data from one or more manufacturers for objects that constitute the BIM model, a first storage unit that receives a second intermediate file via the first communication unit and stores the received second intermediate file, formed by inputting the object data list into the first intermediate file, and a design unit that designs a BIM model of the building, reads the object data list in the second intermediate file stored in the first storage unit, and imports object data selected from the object data list into the BIM model. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-56551 Summary of the Invention [Problem to be solved by the invention]

[0007] According to the design user terminal described in Patent Document 1, the design user can design a BIM model using the object data list in the second intermediate file. That is, since the products related to each object that makes up the BIM model are stored in a data list, when a product to be applied to the actual BIM model is selected from the data list, the data is automatically entered into the design document database. This allows product data from manufacturers to be efficiently imported into the database in the BIM model, and the selected product data can be automatically entered as specification information for the target object.

[0008] However, because only text data (character information) related to the selected product data is imported as the specification information for the target object, although the character information is added, it does not contribute to evoking an image of the target object. Therefore, to make it easier to evoke an image, it is possible to manually set product-related textures for each object in the BIM model, but setting textures for each object requires a great deal of effort and time.

[0009] Furthermore, when the above-mentioned consultations between the client and the designer take place, it takes time for one party to travel to the other party or for both parties to travel to a certain location, and considering that the final design is usually decided after multiple design changes, the issue of time required for travel at each consultation (meeting) becomes even more pronounced.

[0010] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a meeting support system that eliminates the need for one or both parties to travel to a meeting between a client and a designer regarding building design, and generates a BIM model for a meeting that allows users to evoke concrete images of each object that makes up the BIM model. [Means for solving the problem]

[0011] In order to achieve the above object, one aspect of the conference support system according to the present invention is to a first server to which a plurality of user terminals are connected via a network, the first server providing a virtual conference space that can be displayed by each user terminal through a display unit in relation to the real space in front of each user terminal; a second server that transmits BIM model data of a building to the first server and provides a BIM model based on the BIM model data to the virtual conference space; a third server that transmits material information data regarding each object constituting the BIM model to the first server and provides a material model based on the material information data to the virtual conference space; The plurality of user terminals include a designer terminal that designs the BIM model and a client terminal, The designer terminal connected to the first server integrates the material model with the BIM model displayed in the virtual conference space to create a conference BIM model.

[0012] According to this aspect, the first server provides a virtual conference space that can be displayed by both the user terminal (designer terminal) and the client terminal on their respective displays in the real space in front of the user terminal. A conference is held in which the designer gives an explanation using the conference BIM model displayed in the virtual conference space, the client receives the explanation, and the client proposes various requests and demands to the designer. This eliminates the need for one or both of the client and the designer to travel to the conference location. Furthermore, the designer terminal connected to the first server integrates a material model based on material information data into the BIM model displayed in the virtual conference space to create a conference BIM model. In other words, by making each object that makes up the BIM model a specific material model, the conference BIM model, in which various objects such as walls, flooring, and ceilings are composed of material models, allows the client to visualize each object in a specific image.

[0013] When the imaging unit of each user terminal captures, for example, a video of the real space ahead, the first server provides a virtual conference space in the real space ahead that can be displayed via the display unit of the user terminal.

[0014] A designer, for example, creates a BIM model on his or her own designer terminal, and then sends the BIM model data of the created BIM model to a second server and stores it. The designer terminal then sends the BIM model data from the second server to a first server, and the BIM model is generated in a virtual conference space provided by the first server so that it can be displayed on the display unit of the user terminal. The designer terminal then sends material information data from a third server to the first server, and a material model based on the material information data is generated in the virtual conference space so that it can be displayed on the display unit of the user terminal. In this way, the first server provides a virtual conference space that can be viewed via the display unit of each user terminal, and also provides the BIM model and material model so that they can be viewed in the virtual conference space. Based on the operation of the designer terminal, the BIM model and material model are integrated in the virtual conference space to generate a conference BIM model.

[0015] When generating a virtual conference space, which is a meeting place between the designer and the client, the first server grants access rights to the designer terminal and the client terminal, and only user terminals that have been granted access rights can display the virtual conference space (participate in the virtual conference) via the display unit of that user terminal.

[0016] In another aspect of the conference support system according to the present invention, the third server stores a plurality of pieces of material information data for one of the objects; In response to a request from the client terminal, the designer terminal changes the material model and updates the conference BIM model.

[0017] According to this embodiment, multiple material information data for one object are stored in the third server, and in response to a request from the client terminal, the designer terminal changes the material model and updates the conference BIM model.This makes it possible to generate multiple types of conference BIM models in which various material models are applied to each object in response to a request from the client in a single meeting, allowing for intensive design meetings to be held within a set time frame, which ultimately leads to a reduction in the number of meetings.

[0018] Furthermore, if the material model desired by the client does not exist in the material information data provided by the third server, the designer can request the third server to add the requested material information data, and the third server can add the requested material information data to the database and send a message to the designer terminal indicating that the material information data has been added.

[0019] Another aspect of the conference support system according to the present invention is After the material model is changed and the meeting BIM model is updated, change information data of the material model is sent to the third server; When the change to the material model is approved by the third server, the update of the BIM model for the meeting is officially decided.

[0020] According to this embodiment, the update of the conference BIM model due to changes in the material model is officially decided with the approval of the third server that provides material information data to the virtual conference space, so that the conference BIM model based on the material model, which can change in various ways depending on requests from the client, etc., can be confirmed in a chronological order according to the changes that occur.If the conference is in progress, the conference BIM model that is currently decided can be confirmed, and at the end of the last conference, the final decided design can be confirmed.

[0021] Here, it is possible for the designer terminal or second server, etc., to store a BIM model for a meeting as data each time it is changed depending on the meeting, but in this case, for example, the material model for an object decided in a single meeting is determined after approval by a third server that provides material information data, thereby finalizing the material model for the object decided for each meeting, allowing the meeting to proceed in a manner where there is no misunderstanding between the client and the designer, leading to the finalization of the design.

[0022] In another aspect of the conference support system according to the present invention, the first server provides an avatar creation function for creating an avatar of a user who uses the user terminal; The display units of both the designer terminal and the client terminal display a situation in which a designer avatar explains a model using the conference BIM model in the virtual conference space.

[0023] According to this aspect, the display unit displays a designer avatar created by the avatar creation function of the first server explaining the BIM model for the conference in the virtual conference space, giving both the designer and the client a sense of realism as if they were participating in the same conference. Here, the designer avatar includes images of various people, animals, animated characters, etc., as well as the designer's own image. The designer's own image can be captured by a capturing device (capturing unit) such as an external camera installed in or connected to the designer's terminal.

[0024] In another aspect of the conference support system according to the present invention, The display units of both the designer terminal and the client terminal further display client avatars participating in the conference in the virtual conference space.

[0025] According to this aspect, by displaying a client avatar in addition to the designer avatar, the video of the conference participants gathered in the virtual conference space for the conference creates an even more realistic feeling that both the designer and the client are participating in the same conference. Here, like the designer avatar, the client avatar also includes images of various people, animals, animated characters, etc., as well as images of the client avatar itself captured by a capturing means (capturing unit) such as a camera equipped on or connected to the client terminal.

[0026] Another aspect of the conference support system according to the present invention is the user terminal includes a head-mounted display; The avatar of the user wearing the head-mounted display is displayed on the display unit in a state where the user is not wearing the head-mounted display.

[0027] According to this aspect, by having either or both of the designer and the client wear head-mounted displays while participating in a conference, for example, the designer's facial movements and eye movements can be more faithfully reproduced in the avatars that simulate the designer, further enhancing the sense of realism of participating in the conference. Furthermore, since the first server that provides the virtual conference space has a hide function that deletes the display of the head-mounted display that is actually being worn from the avatar, the avatars of the designer and the client displayed on the display unit of the user terminal are not wearing head-mounted displays, allowing the designer and the client to feel as if they are actually participating in a conference.

[0028] Another aspect of the conference support system according to the present invention is the first server provides a 3D pen that performs three-dimensional drawing on the virtual conference space; At least one of the designer avatar and the client avatar in the virtual conference space uses the 3D pen to draw in the virtual conference space in three dimensions.

[0029] According to this aspect, at least one of the designer avatar and the client avatar in the virtual conference space can use a 3D pen to draw in 3D in the virtual conference space, thereby specifically and clearly indicating with the 3D pen which object in the conference BIM model they would like to change the material model for, etc. In addition, both the designer avatar and the client avatar can use the 3D pen to draw various comments, hand-drawn drawings, etc. in 3D in the virtual conference space.

[0030] Another aspect of the conference support system according to the present invention is The first server provides an automatic transcription function for transcribing speech in a conference for the virtual conference space; The method is characterized in that automatically transcribed text is drawn in the virtual conference space.

[0031] According to this method, the audio from the meeting is automatically transcribed and displayed in the virtual meeting space, preventing participants from forgetting to hear what the other party is saying. Furthermore, participants can reconsider what the other party and themselves have said through the text, allowing the designer and client to proceed with the meeting with a common understanding.

[0032] In another aspect of the conference support system according to the present invention, The automatic transcription function includes a translation function; The method is characterized in that the translated and automatically transcribed text is drawn in the virtual conference space.

[0033] According to this aspect, the automatically transcribed text after translation is displayed in the virtual conference space, so that even in a conference with a foreign client who does not understand Japanese, the conference can proceed smoothly with mutual understanding, and a conference can be held without time lag with a client who is in a foreign country, for example.

[0034] Another aspect of the conference support system according to the present invention is The first server provides a history storage function for storing data on the contents of the conference held in the virtual conference space, and a history playback function for playing back the stored data.

[0035] According to this aspect, by saving the data of the meeting contents and playing back the saved data, the meeting contents can be reviewed after the meeting, and both the designer and the client can share decisions regarding changes to the material model, etc. Also, for example, by referring to the history of multiple meetings, a client can request a meeting BIM model to which a material model that has not yet been reviewed has been applied.

[0036] Another aspect of the conference support system according to the present invention is The first server provides a sticky note pasting function for pasting sticky notes in the virtual conference space.

[0037] According to this aspect, matters decided in a meeting, or matters that the designer or client wants to confirm or communicate to the other party in the meeting, can be written on sticky notes and pasted in the virtual meeting space, which effectively serves as a memo of the decisions made, matters to be confirmed or communicated, etc.

[0038] Another aspect of the conference support system according to the present invention is The first server provides the virtual conference space with a capture function for acquiring still images of the conference.

[0039] According to this aspect, the capture function for capturing still images of the conference makes it possible to capture important scenes in the conference or scenes that the user wishes to keep as still images. [Effects of the Invention]

[0040] As can be understood from the above explanation, the conference support system of the present invention makes it possible for a client and a designer to hold a conference regarding the design of a building without one or both parties having to travel to the conference site, and allows the client and designer to conjure up concrete images of each object that makes up the BIM model. [Brief explanation of the drawings]

[0041] [Figure 1] 1 is a diagram illustrating an overall configuration of a conference support system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer that forms the conference support system according to the embodiment. [Figure 3A] FIG. 2 is a diagram illustrating an example of a functional configuration of a first server. [Figure 3B] FIG. 2 is a diagram illustrating an example of a functional configuration of a second server. [Figure 3C] FIG. 10 is a diagram illustrating an example of a functional configuration of a third server. [Figure 3D] FIG. 2 illustrates an example of a functional configuration of a designer terminal. [Figure 3E] FIG. 2 illustrates an example of a functional configuration of a client terminal. [Figure 4] FIG. 10 is a diagram showing the status of a conference in a virtual conference space displayed on a display unit of a user terminal. [Figure 5] FIG. 10 is a diagram showing an example of a flooring model list as an example of a material model based on material information data. [Figure 6] FIG. 10 is a diagram showing a state in which a transcription of an audio message and attached sticky notes are displayed in a virtual conference space displayed on a display unit of a user terminal. DETAILED DESCRIPTION OF THE INVENTION

[0042] Hereinafter, an example of a conference support system according to an embodiment will be described with reference to the accompanying drawings. In this specification and drawings, substantially identical components may be designated by the same reference numerals to avoid redundant description.

[0043] [Meeting support system according to an embodiment] An example of a conference support system according to an embodiment will be described with reference to FIGS. 1 to 6. Here, FIG. 1 is a diagram showing the overall configuration of the conference support system according to an embodiment, and FIG. 2 is a diagram showing an example of the hardware configuration of a computer forming the conference support system according to an embodiment. Also, FIGS. 3A to 3E are diagrams showing examples of functional configurations of a first server, a second server, a third server, a designer terminal, and a client terminal, respectively. FIG. 4 is a diagram showing the status of a conference in a virtual conference space displayed on a display unit of a user terminal. Furthermore, FIG. 5 is a diagram showing an example of a flooring model list as an example of a material model based on material information data, and FIG. 6 is a diagram showing a state in which a voice transcript and attached sticky notes are displayed in a virtual conference space displayed on a display unit of a user terminal.

[0044] The conference support system 100 is formed by connecting a first server 10, a second server 20, a third server 30, and a plurality of user terminals 40 via a network 50. The first server 10 provides a virtual conference space VS that can be displayed on the display unit of a user terminal of a user who has been granted access rights, for example.

[0045] The first server 10 grants the second server 20, the third server 30, and each user terminal 40 access rights to the system, and each user terminal 40 that has been granted access rights is able to enjoy the various services provided by the first server 10.

[0046] 4, when the user terminals 40 capture video of the real space in front of them via their image capturing units 402, 411, the first server 10 provides a virtual conference space VS that can be displayed by each user terminal 40 on its display units 406, 412. The first server 10 also provides various other options to the virtual conference space VS, which will be described in detail below.

[0047] The first server 10 is a cloud server owned by a software company or the like that provides a conference using the virtual conference space VS.

[0048] The second server 20 stores BIM model data related to BIM models (such as a BIM design model and a BIM structural model) created by a designer (an example of a user) on a designer terminal 40. By operating the designer terminal, the BIM model data is transmitted from the second server 20 to the first server 10, and a BIM model based on the BIM model data is generated for the virtual conference space VS provided by the first server 10 so as to be displayable via the display units 406, 412 of the user terminal 40.

[0049] The second server 20 is a cloud server owned by a house builder or the like that creates a BIM model of a building, provides the BIM model to a client, constructs a building based on the BIM model, and sells the building to the client.

[0050] The third server 30 stores material information data for each object that constitutes a BIM model based on the BIM model data. By operating the designer terminal, the material information data is transmitted from the third server 30 to the first server 10, and a material model based on the material information data is generated for the virtual conference space VS provided by the first server 10 so as to be displayable via the display unit of the user terminal 40. Here, the objects that constitute the BIM model of a building include columns, beams, walls, floors, ceilings, etc.

[0051] The third server 30 is a cloud server owned by a company that creates a database of products from various building material manufacturers for various building materials (objects in a BIM model) that make up a building, and provides the database to house manufacturers and the like.

[0052] The user terminals 40 include a designer terminal owned by a designer who explains the building design using the conference BIM model VM in the virtual conference space VS, and a client terminal owned by the building owner or a client considering purchasing the building.

[0053] Although not shown in the figure, the designer terminal 40 may access a cloud server of Autodesk, which provides the BIM software Autodesk Revit (registered trademark), and create a BIM model on the designer terminal 40 or make various modifications and changes.Therefore, the cloud server of Autodesk, not shown, may also be connected to the first server 10, the designer terminal 40, etc. via the network 50.

[0054] The user terminal 40 includes a tablet 40A, a smartphone 40B, a personal computer 40C, and a head mounted display 40D (HMD).

[0055] The network 50 includes a public network such as the Internet, a wireless network such as a mobile phone network, a dedicated network such as a VPN (Virtual Private Network), a LAN (Local Area Network), and the like.

[0056] 2, the first server 10, the second server 20, the third server 30, and the user terminal 40 are all comprised of information processing devices such as personal computers (PCs), workstations (WSs), tablets, etc. The following description focuses on the first server 10 as a computer, but the other servers such as the second server 20 basically have the same hardware configuration.

[0057] 2, the computer constituting the first server 10 includes a CPU (Central Processing Unit) 11, a main storage device 12, an auxiliary storage device 13, a communication IF (interface) 14, and an input / output IF 15, which are interconnected by a connection bus 16. The main storage device 12 and the auxiliary storage device 13 are computer-readable recording media. Note that the above components may be provided separately, or some of the components may not be provided.

[0058] The CPU 11 is also called an MPU (Microprocessor) or a processor, and may be a single processor or a multiprocessor. The CPU 11 is a central processing unit that controls the entire first server 10, which is a computer. The CPU 11, for example, deploys a program stored in the auxiliary storage device 13 in an executable manner in the working area of ​​the main storage device 12, and controls peripheral devices through the execution of the program, thereby providing functions that meet a predetermined purpose.

[0059] The main memory device 12 stores computer programs executed by the CPU 11, data processed by the CPU 11, etc. The main memory device 12 includes, for example, a flash memory, a random access memory (RAM), and a read-only memory (ROM). The auxiliary memory device 13 stores various programs and various data on a readable and writable recording medium, and is also called an external memory device. The auxiliary memory device 13 stores, for example, an operating system (OS), various programs, various tables, etc. The OS includes, for example, a communication interface program that exchanges data with external devices connected via the communication IF 14. The external devices connected to the first server 10 include, for example, information processing devices and external memory devices such as personal computers (PCs), workstations (WSs), servers, and mobile terminals, such as the second server 20, the third server 30, and the user terminal 40 connected to the network.

[0060] The auxiliary storage device 13 is used, for example, as a storage area that supplements the main storage device 12, and stores computer programs executed by the CPU 11, data processed by the CPU 11, etc. The auxiliary storage device 13 is a silicon disk including nonvolatile semiconductor memory (flash memory, EPROM (Erasable Programmable ROM)), a hard disk drive (HDD: Hard Disk Drive), a solid state drive, etc. Examples of the auxiliary storage device 13 include drives for removable recording media such as CD drives, DVD drives, and BD drives. Examples of removable recording media include CDs, DVDs, BDs, USB (Universal Serial Bus) memories, and SD (Secure Digital) memory cards.

[0061] The communication IF 14 is an interface with the network to which the first server 10 is connected. The communication IF 14 is connected to the second server 20, the third server 30, and each user terminal 40 via the network, and provides a virtual conference space VS that can be displayed by each user terminal 40 through the display units 406, 412 in the real space in front of the user terminal 40.

[0062] The input / output IF 15 is an interface for inputting and outputting data between devices connected to the first server 10. Input devices such as a keyboard, a touch panel, a mouse, or other pointing device, and a microphone are connected to the input / output IF 15. The first server 10 receives operation instructions and the like from an operator who operates the input device via the input / output IF 15.

[0063] Furthermore, output devices such as display devices such as a liquid crystal panel (LCD: Liquid Crystal Display) or an organic EL panel (EL: Electroluminescence) and a printer, a speaker, etc. are connected to the input / output IF 15. The first server 10 outputs data and information processed by the CPU 11 and data and information stored in the main storage device 12 and the auxiliary storage device 13 via the input / output IF 15.

[0064] Next, an example of the functional configuration of various servers and user terminals will be described with reference to FIGS. 3A to 3E and 4 to 6. FIG.

[0065] 3A, the first server 10 provides various functions, such as at least a first communication unit 101, a virtual conference space generation unit 102, a BIM model setting unit 103, an avatar generation unit 104, a sticky note attachment unit 105, a 3D drawing unit 106, a voice transcription unit 107, a translation unit 108, a still image capture unit 109, a history saving and playback unit 110, a first display unit 111, and a first storage unit 112, by executing a program using a CPU 11. Note that at least some of the above processing functions may be provided by a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), or the like, and similarly, at least some of the above processing functions may be provided by dedicated LSIs (Large Scale Integration) such as an FPGA (Field-Programmable Gate Array), a numerical calculation processor, an image processing processor, or other digital circuits.

[0066] The first communication unit 101 provides a virtual conference space VS that can be displayed on the display units 406, 412 when capturing an image of the real space ahead via the image capturing units 402, 411 of the user terminal 40. Furthermore, the first communication unit 101 provides various options for the virtual conference space VS, including three-dimensional drawing using a 3D pen.

[0067] The virtual conference space generation unit 102 provides a virtual conference room selected by the designer terminal 40 or the like from among various virtual conference rooms in the real space in front of the user terminal 40. This virtual conference room includes a large conference room, a small conference room, a seminar room, an outdoor room, etc., and the user can also freely set a virtual conference room according to their preferences.

[0068] In the example shown in FIG. 4, a small conference room is set as the virtual conference space VS and is displayed on the display units 406 and 412.

[0069] The BIM model setting unit 103 sets the placement position of the BIM model that will be the subject of the meeting in the virtual conference space VS, and receives the BIM model data stored in the second server 20 through operation from the designer terminal 40, and places the BIM model based on the BIM model data at the set position in the virtual conference space VS.

[0070] 4 shows a state in which a BIM model based on BIM model data transmitted from the second server 20 is placed at a set position in the virtual conference space VS, and a conference BIM model VM is generated by applying material models based on material information data transmitted from the third server 30 to various objects that make up this BIM model. Note that the following explanation will focus on a case in which a client requests a change to the flooring among the objects, and in response to this request, the designer changes the material model related to the flooring to another material model.

[0071] The avatar generation unit 104 generates an avatar of the designer who owns the designer terminal 40, an avatar of the client who owns the client terminal 40, etc. Here, images of various people, animals, animation characters, etc. are applied to the avatar, and further, images of the designer or client themselves are applied.

[0072] The example shown in Figure 4 shows a designer avatar A1 based on an image of the designer himself and a client avatar A2 based on an image of the client himself participating in a conference with a conference BIM model VM in between.

[0073] When a designer or the like generates an avatar using his or her own image, an image of the designer or the like taken by the imaging unit 402, 411 equipped on the designer terminal 40 or the client terminal 40, or by an external imaging means such as a camera connected to the designer terminal 40, is sent to the first server 10, and the image is processed by the avatar generation unit 104 of the first server 10 to generate a designer avatar or the like.

[0074] Furthermore, if a designer or other person is wearing a head-mounted display, their image is captured by the camera in the head-mounted display, and video data relating to facial and eye movements is imported into the avatar generation unit 104, which displays the detailed movements of the designers and clients participating in the meeting, giving the user an even more realistic feeling as if they were actually participating in the same meeting.

[0075] Furthermore, when a designer or the like is wearing a head-mounted display, the avatar generation unit 104 has a non-display function that causes the designer avatar A1 and the client avatar A2 to be displayed on the display units 406, 412 of the user terminal 40 when they are not wearing a head-mounted display (when the head-mounted display is removed from the avatar).

[0076] In this way, by displaying the designer avatar A1 and the client avatar A2 on the display units 406 and 412 without wearing a head-mounted display, the participants can feel as if they are actually participating in a real conference.

[0077] In the illustrated example, the first server 10 has an avatar generation unit 104, but the user terminal 40 may also have an avatar generation unit, and each user terminal 40 may independently generate an avatar and provide the avatar to the virtual conference space VS.

[0078] As shown in Fig. 6, the sticky note pasting unit 105 writes down on sticky notes N matters decided in a meeting, matters that the designer or client wants to confirm or communicate to the other party in the meeting, etc., and pastes the sticky notes N in the virtual meeting space VS. The pasted sticky notes N function as a memorandum of decisions, matters to be confirmed or communicated, etc. In the example shown in Fig. 6, it is written on the sticky note N that at the meeting on XX month △ day, the flooring was changed to D4 in the material information data.

[0079] 5, material information data relating to flooring, which is an example of an object, stored in the third server 30 is transmitted to the first server 10 by operating the designer terminal 40, and a flooring model list (a list of multiple flooring models F (examples of material models)) based on the material information data is provided to the virtual conference space VS so that it can be displayed via the display units 406, 412. In the example shown in FIG. 5, a total of 16 flooring models F are illustrated, arranged in four vertical rows, A to D, and four horizontal columns, 1 to 4.

[0080] In the example shown in Figure 6, a flooring model F with material information data: A1 is integrated into a conference BIM model VM, and client avatar A2, upon viewing this, requests that flooring model F be changed from material information data: A1 to other material information data: D4.

[0081] The three-dimensional drawing unit 106 provides a 3D pen (not shown) that performs three-dimensional drawing on the virtual conference space VS, and the designer avatar A1 and the client avatar A2 in the virtual conference space VS use the 3D pen to perform three-dimensional drawing on the virtual conference space VS as needed.

[0082] The 3D drawing unit 106 allows the client to use a 3D pen to specifically and clearly indicate which object in the BIM model VM for the meeting the client wants to change in the material model. Additionally, various other comments, hand-drawn drawings, etc. can be drawn in 3D in the virtual meeting space VS.

[0083] The speech transcription unit 107 renders automatically transcribed text in the virtual conference space VS, as shown in Fig. 6. In the example shown in Fig. 6, a client avatar A2 requests that a flooring model F based on material information data: A1, which is currently set in the conference BIM model VM, be changed to a flooring model F based on other material information data: D4, and this voice is displayed as transcription L1. In response, the designer avatar A1 replies that the flooring model F will be changed from material information data: A1 to material information data: D4 and the conference BIM model VM will be updated, and this voice is displayed as transcription L2.

[0084] The speech transcription unit 107 automatically transcribes speech during a meeting and displays the text in the virtual meeting space VS, preventing participants from forgetting what the other party said. Furthermore, participants can review what the other party and themselves said through the text, allowing the designer and client to proceed with the meeting with a common understanding.

[0085] The translation unit 108 displays the automatically transcribed text after translation in the virtual conference space VS (an example of the translation is not shown). For example, if the client is a foreigner who cannot understand Japanese, the conference can proceed smoothly with mutual understanding even in a conference with the client. Furthermore, even if the client is in a foreign country, the conference can be held without time lag.

[0086] The still image capture unit 109 acquires still images of various scenes from a meeting in the virtual meeting space VS. For example, it can acquire important scenes from a meeting or scenes that the client wants to keep as still images. For example, by acquiring still images of a meeting BIM model VM each time the flooring is changed to various material models, the client can compare each still image consecutively or side by side.

[0087] The history saving and playback unit 110 saves data of the conference contents in the virtual conference space VS and also plays back the saved data. In this way, by saving data of the conference contents and playing back the saved data, the conference contents can be reviewed after the conference and decisions regarding changes to the flooring material model can be shared between the designer and the client. In addition, by referring to the history of multiple conferences, the client can request the creation of a new conference BIM model VM to which a flooring material model that has not yet been confirmed (for example, material models such as material information data: B1 and material information data: C3 in Figure 5) has been applied.

[0088] The first display unit 111 displays the virtual conference space VS provided in the real space in front of each user terminal 40, the virtual conference room that forms the virtual conference space VS, the BIM model and conference BIM model VM that are placed at predetermined positions in the virtual conference space VS, the placement of the designer avatar A1 and client avatar A2, the line of sight of both, etc., and checks whether a realistic conference is being displayed on the display units 406, 412 of each user terminal 40.

[0089] The first storage unit 112 stores video data of the conference together with audio data, stores still image data acquired by the still image capture unit 109, and also stores data recorded when various options are applied. Furthermore, the first storage unit 112 stores the addresses of the user terminals 40 with access authority, the first server 10, the second server 20, and the third server 30.

[0090] On the other hand, as shown in FIG. 3B, the second server 20 provides at least various functions of a second communication unit 201 and a second storage unit 202 by the CPU 11 executing a program.

[0091] The second communication unit 201 transmits the BIM model data stored in the second storage unit 202 to the first communication unit 101. A BIM model based on the BIM model data received by the first server 10 is provided to the virtual conference space VS displayed via the display units 406, 412 in the real space in front of each user terminal 40. Here, for the second and subsequent conferences, a conference BIM model VM has already been created in which specific material models have been integrated for several objects, and therefore the created conference BIM model VM is also stored in the second storage unit 202.

[0092] At the next meeting, the created BIM model VM for the meeting will be provided to the virtual meeting space VS, and the material models of already set objects will be changed and material models for objects that have not yet been set will be integrated, allowing the design meeting to proceed further.

[0093] The second storage unit 202 further stores a conference BIM model VM determined in one conference, and further stores a conference BIM model VM of a building that is finally determined.

[0094] On the other hand, as shown in FIG. 3C, the third server 30 provides various functions of at least a third communication unit 301, a material model change approval unit 302, and a third storage unit 303 by the CPU 11 executing a program.

[0095] The third communication unit 301 provides the virtual conference space VS with material information data for each object stored in the third storage unit 303. For each object, a data list of each company's products is created and a material model that can evoke a specific image is set. Therefore, by generating a conference BIM model by integrating material models according to the client's requests for each object based on the material information data for each object provided to the virtual conference space VS, conference participants can evoke a specific image of each object, and therefore a specific image of each room or the entire building.

[0096] Furthermore, product texture settings can be applied automatically and quickly to each object in the BIM model, making it possible to confirm changes to various material models with a concrete image each time during a set meeting.

[0097] After the material model is changed, the conference BIM model VM is updated, and the third communication unit 301 receives the material model change information data, the material model change approval unit 302 approves the material model change, and with this approval, officially decides to update the conference BIM model VM.

[0098] In this way, by officially deciding to update the conference BIM model VM due to changes in the material model with the approval of the material model change approval unit 302, the conference BIM model VM based on the material model, which can change in various ways depending on requests from the client, etc., can be confirmed each time in a chronological order according to the changes in the material model. If the conference is in progress, the conference BIM model VM that has been decided at the time can be confirmed, and at the end of the last conference, the final design can be confirmed.

[0099] The third memory unit 303 stores the material models of each object that constitutes the conference BIM model VM determined in one meeting, and further stores the material models of each object that constitutes the conference BIM model VM of the building that is finally determined.

[0100] On the other hand, as shown in Figure 3D, the designer terminal 40 provides various functions, such as at least a fourth communication unit 401, a fourth photographing unit 402, a BIM model operation unit 403, a material model integration and modification unit 404, various option selection and operation unit 405, a fourth display unit 406, and a fourth memory unit 407, by executing a program by the CPU 11.

[0101] The fourth communication unit 401 is connected to the first communication unit 101 of the first server 10, and when a video of the real space ahead is captured through the fourth imaging unit 402, the virtual conference space VS provided by the first server 10 can be displayed through the fourth display unit 406.

[0102] In addition, the fourth communication unit 401 connects to the second communication unit 201 of the second server 20 and transmits the BIM model data stored in the second memory unit 202 to the first server 10, and also connects to the third communication unit 301 of the third server 30 and transmits the material information data stored in the third memory unit 303 to the first server 10.

[0103] The fourth image capturing unit 402 captures video of the real space in front of the designer terminal 40, and also captures video of the designer. In the captured video of the real space in front, a virtual conference space VS provided by the first server 10 can be displayed via the fourth display unit 406, and the captured video of the designer himself is transmitted to the first server 10 and used by the avatar generation unit 104 to create his own designer avatar A1.

[0104] The BIM model operation unit 403 transmits BIM model data from the second server 20 to the first server 10 and imports a BIM model based on the BIM model data into the virtual conference space VS. During the conference, when the designer avatar A1 explains to the client avatar A2 using the BIM model and the conference BIM model VM, the size of the conference BIM model VM, the viewing angle from the client avatar A2, etc. are changed in various ways, and the position of the conference BIM model VM is changed.

[0105] The material model integration / change unit 404 allows the designer to execute an operation in the virtual conference space VS to integrate a material model based on material information data for each object in the BIM model, and executes an operation to change the integrated material model to another material model in response to a request from the client.

[0106] The option selection / operation unit 405 selects one or more of the various options provided by the first server 10 to the virtual conference space VS, specifically, a sticky note function, a 3D drawing function, a voice transcription function, a translation function, a still image capture function, a history saving function, and a history playback function, and performs the operation to be applied to the virtual conference space VS.

[0107] The fourth storage unit 407 stores BIM model data designed by a designer, and in addition to transmitting the BIM model data from the second server 20 to the first server 10 and providing it to the virtual conference space VS, the BIM model data in the fourth storage unit 407 can also be transmitted to the first server 10. Furthermore, like the first storage unit 112, the fourth storage unit 407 stores video data from conferences together with audio data, stores still image data acquired by the still image capture unit 109, and also stores records as data when various options are applied.

[0108] On the other hand, as shown in FIG. 3E, the client terminal 40 provides various functions of at least a fifth communication unit 410, a fifth image capturing unit 411, and a fifth display unit 412 by the CPU 11 executing a program.

[0109] The fifth communication unit 410 is connected to the first communication unit 101 of the first server 10, and when a video of the real space ahead is captured through the fifth imaging unit 411, the virtual conference space VS provided by the first server 10 can be displayed through the fifth display unit 412.

[0110] The fifth image capturing unit 411 captures video of the real space in front of the client terminal 40, and also captures video of the client. In the captured video of the real space in front, a virtual conference space VS provided by the first server 10 can be displayed via the fifth display unit 412, and the captured video of the client himself is transmitted to the first server 10 and used by the avatar generation unit 104 to create his own client avatar A2.

[0111] According to the conference support system 100 shown in the figure, a virtual conference space VS is provided by the first server 10 in the real space in front of the user terminal 40, which can be displayed by both the designer terminal 40 and the client terminal 40 included in the user terminal 40 through their respective display units 406, 412, and a conference is held in which the designer gives an explanation using the conference BIM model VM provided in the virtual conference space VS, the client receives the explanation, and the client proposes various requests and demands to the designer, thereby eliminating the need for one or both of the client and the designer to travel to the conference location for the conference.

[0112] Furthermore, the designer terminal 40 connected to the first server 10 integrates a material model based on material information data into the BIM model displayed in the virtual conference space VS to create a conference BIM model VM.In other words, by making each object that makes up the BIM model a specific material model, the conference BIM model, in which various objects such as walls, flooring, and ceilings are composed of material models, enables the client to imagine a specific image of the design.

[0113] It should be noted that the present invention is not limited to the configurations shown here, and other embodiments may be possible in which other components are combined with the configurations described in the above embodiments. In this regard, the present invention can be modified within the scope of the present invention, and can be appropriately determined depending on the application form. [Explanation of symbols]

[0114] 10: First server 20: Second server 30: Third server 40, 40A, 40B, 40C, 40D: User terminal (designer terminal, client terminal) 50: Network 100: Meeting support system 402: 4th Filming Division (Filming Division) 406: 4th display section (display section) 411: 5th Filming Division (Filming Division) 412: 5th display section (display section) VS: Virtual Meeting Space VM: BIM model for the meeting (BIM model) F: Flooring model (material model) A1: Designer Avatar (Avatar) A2: Client avatar (avatar) L1, L2: Audio transcription (transcription) N: Sticky notes

Claims

1. a first server to which a plurality of user terminals are connected via a network, the first server providing a virtual conference space that can be displayed by each user terminal through a display unit in relation to the real space in front of each user terminal; a second server that transmits BIM model data of a building to the first server and provides a BIM model based on the BIM model data to the virtual conference space; a third server that transmits material information data regarding each object constituting the BIM model to the first server and provides a material model based on the material information data to the virtual conference space; The plurality of user terminals include a designer terminal that designs the BIM model and a client terminal, The designer terminal connected to the first server integrates the material model with the BIM model displayed in the virtual conference space to create a conference BIM model, the third server stores a plurality of pieces of material information data for one of the objects, In response to a request from the client terminal, the designer terminal changes the material model and updates the conference BIM model; After the material model is changed and the meeting BIM model is updated, change information data of the material model is sent to the third server; A conference support system characterized in that, when the change to the material model is approved by the third server, an update to the conference BIM model is officially decided.

2. The second server stores the BIM model for the meeting determined in one meeting and the BIM model for the meeting determined finally, The third server stores the material models of each object that constitutes the conference BIM model determined in one meeting and the material models of each object that constitutes the conference BIM model that is finally determined. The conference support system described in claim 1.

3. the first server provides an avatar creation function for creating an avatar of a user who uses the user terminal; The conference support system described in claim 1 or 2, characterized in that the display units of both the designer terminal and the client terminal display a situation in which a designer avatar explains a model using the conference BIM model in the virtual conference space.

4. 4. The conference support system according to claim 3, wherein the display units of both the designer terminal and the client terminal further display client avatars participating in the conference in the virtual conference space.

5. the user terminal includes a head-mounted display; 5. The conference support system according to claim 4, wherein an avatar of the user wearing the head-mounted display is displayed on the display unit in a state where the user is not wearing the head-mounted display.

6. the first server provides a 3D pen that performs three-dimensional drawing on the virtual meeting space; 5. The conference support system according to claim 4, wherein at least one of the designer avatar and the client avatar in the virtual conference space uses the 3D pen to perform three-dimensional drawing in the virtual conference space.

7. The first server provides an automatic transcription function for transcribing speech in a conference for the virtual conference space; The conference support system according to claim 1 or 2, wherein automatically transcribed characters are drawn in the virtual conference space.

8. The automatic transcription function includes a translation function; The conference support system according to claim 7, wherein translated and automatically transcribed characters are displayed in the virtual conference space.

9. 3. The conference support system according to claim 1, wherein the first server provides a history storage function for storing data of conference contents in the virtual conference space and a history playback function for playing back the stored data.

10. 3. The conference support system according to claim 1, wherein the first server provides a sticky note pasting function for pasting sticky notes in the virtual conference space.

11. 3. The conference support system according to claim 1, wherein the first server provides the virtual conference space with a capture function for capturing still images of the conference.

Images