Information processing device and information processing method

Abstract

This information processing device: holds knowledge information indicating data on a site including a 3D model of the site and the position corresponding to the data; displays, on a display device, a first virtual space constructed by a 3D model of a project management room; displays, in the first virtual space, a miniature 3D model obtained by reducing the 3D model of the site and display content which is data selected from the knowledge information; identifies, on the basis of the 3D model of the site and the position of the selected data, coordinates which are in the miniature 3D model and which correspond to the position of the data corresponding to the display content; and highlights the identified coordinates in the miniature 3D model.

Description

Information processing device and information processing method Incorporation by Reference

[0001] This application claims priority from Japanese Patent Application No. 2023-204445, filed December 4, 2023, the contents of which are incorporated herein by reference.

[0002] The present invention relates to an information processing device and an information processing method.

[0003] Background of the present invention is JP 2022-505469 A (Patent Document 1), which states that "multiple visualization objects may be provided to represent one or more data sets in a virtual 3D space. The visualization objects may include funnels, containers, name cards, etc. The visualization objects may be arranged in a circular carousel that can rotate around the virtual camera or the user's position in the VR / AR environment. Individual data points within the visualization objects may be rotated, sized, positioned, colored, or otherwise characterized based on attributes of the corresponding data points. Individual data points may also be animated as transitions between visualization objects in a unified view. Voice commands may be interpreted as part of an interactive environment that can provide views of visualization objects to multiple devices simultaneously" (see Abstract).

[0004] Special Publication No. 2022-505469

[0005] The technology described in Patent Literature 1 places individual data points in a visualization object in a virtual 3D space according to the attributes of the data points, but does not link the data placed in the virtual 3D space with the location in the site where the data was obtained. Therefore, one aspect of the present invention displays the correspondence between the data placed in the virtual space and the location in the site in a manner that is easy for participants to recognize, thereby making discussions about the site among participants more efficient.

[0006] In order to solve the above problem, one aspect of the present invention employs the following configuration: An information processing device includes a processor and a memory, and is connected to a display device, the memory stores knowledge information indicating data related to the site, including a 3D model of the site, and locations corresponding to the data, and virtual space management information indicating a first virtual space constructed using the 3D model of a project management room, the processor displays the first virtual space on the display device, displays a miniature 3D model that is a reduced version of the 3D model of the site, accepts selection of data included in the knowledge information, displays the selected data in the first virtual space as display content, and, based on the 3D model of the site and the locations indicated by the knowledge information of the selected data, identifies coordinates within the miniature 3D model that correspond to the location of data corresponding to the display content, and highlights the identified coordinates within the miniature 3D model.

[0007] According to one aspect of the present invention, the correspondence between data placed in a virtual space and the location at the site is displayed in a manner that is easy for participants to recognize in the virtual space, thereby making discussions about the site among participants more efficient.

[0008] Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments.

[0009] 1 is a block diagram showing an example of the configuration of a discussion space providing system in Example 1. FIG. 2 is a diagram showing an example of the data configuration of a knowledge DB (DataBase) in Example 1. FIG. 3 is a diagram showing an example of the data configuration of virtual space management information in Example 1. FIG. 4 is a diagram showing an example of the data configuration of miniature 3D model management information in Example 1. FIG. 5 is a diagram showing an example of the data configuration of display content management information in Example 1. FIG. 6 is a diagram showing an example of the data configuration of corresponding coordinate management information in Example 1. FIG. 7 is a diagram showing an example of the data configuration of user management information in Example 1. A flowchart showing an example of initial setting processing in Example 1. A flowchart showing an example of routine processing in Example 1. A flowchart showing an example of participant instruction response processing in Example 1. A flowchart showing an example of room layout update processing in Example 1. A floor plan view showing an example of a project management room in Example 1. A diagram showing an example of the screen configuration of a query input screen in Example 1. A diagram showing an example of the screen configuration of a search result display screen in Example 1.

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment, the same components are generally designated by the same reference numerals, and repeated explanations will be omitted. It should be noted that this embodiment is merely an example for realizing the present invention, and does not limit the technical scope of the present invention.

[0011] 1 is a block diagram showing an example of the configuration of a discussion space providing system. The discussion space providing system includes a discussion space providing device 100, participant terminals 200 used by participants, and worker terminals 300 used by on-site workers, and each participant terminal 200 and each worker terminal 300 is connected to the discussion space providing device 100 via a network 400 such as the Internet.

[0012] The discussion space providing device 100 provides a site 3D space and a project management room as a 3D (Three-Dimensional) virtual space (hereinafter simply referred to as a virtual space or space). Examples of sites include large factories and plants, and areas within large factories and plants (such as rooms within large factories and plants) where work is carried out. The site 3D space is a 3D space constructed as a copy of the site space.

[0013] The project management room is a virtual space that is different from the site 3D space. The project management room displays, for example, 3D models (3D models including three-dimensional shape information) that construct the site 3D space, miniature 3D models of equipment installed at the site, and data related to the site. Hereinafter, these miniature 3D models (i.e., reduced 3D models) will also be referred to as miniature 3D models, and the data related to the site displayed in the project management room will also be referred to as display content.

[0014] The participant terminal 200 may be, for example, a PC (Personal Computer), smartphone, or tablet terminal on which an application is installed that allows participants to enter a virtual space such as the Metaverse and interact with other participants, the administrator of the discussion space providing system, and on-site workers, or it may be a headset (for example, a VR (Virtual Reality) headset) that includes a goggle-type display device, headphones, a microphone, and a gyro sensor.

[0015] The worker terminal 300 is, for example, a PC, smartphone, or tablet terminal carried by a worker on-site and having an application installed that allows the worker on-site to communicate with participants using the participant terminal 200.

[0016] Furthermore, the worker terminal 300 periodically measures the position of the worker on-site using, for example, a GPS (Global Positioning System) function or PDR (Pedestrian Dead Reckoning) using angular velocity obtained by a gyro sensor or the like, and transmits the measured position to the discussion space providing device 100, whereby the worker's position information is accumulated in the knowledge DB 121 described below. Note that the worker's position may be captured by, for example, a surveillance camera installed on-site and periodically transmitted to the discussion space providing device 100, whereby the worker's position information is accumulated in the knowledge DB 121.

[0017] A participant using the participant terminal 200, for example, causes an icon representing the participant to enter the virtual space via an input device (e.g., a mouse, keyboard, touch panel, or microphone) provided on the participant terminal 200. An image corresponding to the position, orientation, and viewing angle of the icon is displayed on the output device of the participant terminal 200. For example, via the input device provided on the participant terminal 200, the participant can move the icon in the virtual space within the virtual space or to a different virtual space, or change the field of view displayed on the display device provided on the participant terminal 200 (e.g., change the viewing angle or line of sight, zoom in or out, etc.).

[0018] After an avatar representing the participant enters the virtual space, the participant can input voice or text via an input device provided in the participant terminal 200, and the input voice or text can be output from the corresponding icon on an output device provided in the participant terminal 200 of the participant who has entered the virtual space. This allows the participant using the participant terminal 200 to hold a discussion with other participants in the virtual space.

[0019] Furthermore, after an avatar representing the participant enters the virtual space, the participant can input voice or text via an input device provided in the participant terminal 200, and the input voice or text can be output from an output device provided in the worker terminal 300. This allows the participant using the participant terminal 200 to hold a discussion with the worker on-site.

[0020] In this way, a participant is a person who can enter the virtual space, and a worker is a person who does not enter the virtual space but can interact with participants. Hereinafter, the administrator, participants, and workers will be collectively referred to as users.

[0021] The discussion space providing device 100 is configured by a computer having, for example, a CPU (Central Processing Unit) 101, a memory 102, an auxiliary storage device 103, a communication device 104, an input device 105, and an output device 106.

[0022] The CPU 101 is an example of a processor and executes programs stored in the memory 102. The memory 102 includes a non-volatile storage element, Read Only Memory (ROM), and a volatile storage element, Random Access Memory (RAM). The ROM stores unchanging programs (e.g., a Basic Input / Output System (BIOS)). The RAM is a high-speed, volatile storage element such as Dynamic Random Access Memory (DRAM), and temporarily stores programs executed by the CPU 101 and data used when the programs are executed.

[0023] The auxiliary storage device 103 is a large-capacity, non-volatile storage device such as a magnetic storage device (HDD (Hard Disk Drive)) or a flash memory (SSD (Solid State Drive)), and stores programs to be executed by the CPU 101 and data to be used when the programs are executed. In other words, the programs are read from the auxiliary storage device 103, loaded into the memory 102, and executed by the CPU 101.

[0024] The input device 105 is a device that receives input from an operator, such as a keyboard, mouse, or microphone. The output device 106 is a device that outputs the results of program execution in a format that can be recognized by the operator, such as a display device, printer, or speaker.

[0025] The communication device 104 is a network interface device that controls communication with other devices in accordance with a predetermined protocol, and includes a serial interface such as a USB (Universal Serial Bus).

[0026] A part or all of the programs executed by the CPU 101 may be provided to the discussion space providing device 100 via a network from a removable medium (CD-ROM, flash memory, etc.) which is a non-transitory storage medium or an external computer equipped with a non-transitory storage medium, and stored in the non-volatile auxiliary storage device 103 which is a non-transitory storage medium. For this reason, the discussion space providing device 100 may have an interface for reading data from removable media. This also applies to the participant terminal 200 and the worker terminal 300.

[0027] The discussion space providing device 100 is a computer system configured on one physical computer or on multiple logically or physically configured computers, and may operate in separate threads on the same computer, or may operate on a virtual computer constructed on multiple physical computer resources. The same applies to the participant terminals 200 and the worker terminals 300.

[0028] The CPU 101 includes, for example, a knowledge setting unit 111, a virtual space management unit 112, a room management unit 113, a user management unit 114, a miniature 3D model management unit 115, and a display content management unit 116.

[0029] The knowledge setting unit 111 sets the knowledge DB 121, which will be described later. The virtual space management unit 112 manages the virtual space. The room management unit 113 manages the project management room, which is an example of a virtual space. The user management unit 114 manages users of the discussion space providing system. The miniature 3D model management unit 115 manages the miniature 3D models placed in the project management room. The display content management unit 116 manages the display content placed in the project management room.

[0030] For example, the CPU 101 functions as a knowledge setting unit 111 by operating in accordance with a knowledge setting program loaded into the memory 102, and functions as a virtual space management unit 112 by operating in accordance with a virtual space management program loaded into the memory 102. The same relationship between programs and functional units applies to the other functional units included in the CPU 101.

[0031] Note that some or all of the functions of the functional units included in the CPU 101 may be realized by hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

[0032] The auxiliary storage device 103 holds, for example, a knowledge database (DB) 121 , virtual space management information 122 , miniature 3D model management information 123 , display content management information 124 , corresponding coordinate management information 125 , and user management information 126 .

[0033] The knowledge DB 121 includes data related to the site (for example, data indicating the environment, equipment status, worker status or behavior, etc., at the site or a portion of the site). The virtual space management information 122 includes information for managing multiple virtual spaces including the site 3D space and the project management room.

[0034] The miniature 3D model management information 123 includes information for managing miniature 3D models placed in the project management room. The display content management information 124 includes information for managing display content placed in the project management room. The corresponding coordinate management information 125 includes information indicating which coordinates in the miniature 3D models placed in the project management room correspond to the (real) positions of display content placed in the project management room. The user management information 126 includes information for managing users including managers, participants, and workers.

[0035] In addition, some or all of the information stored in the auxiliary storage device 103 may be stored in the memory 102 or in an external database connected to the discussion space providing device 100.

[0036] In this embodiment, the information used by the discussion space providing system does not depend on the data structure and may be expressed in any data structure. In this embodiment, the information is expressed in a table format, but the information can be stored in a data structure appropriately selected from, for example, a list, a database, or a queue.

[0037] 2 is a diagram showing an example of the data configuration of the knowledge DB 121. The knowledge DB 121 includes, for example, a data ID column 1211, a data type column 1212, a data column 1213, a tag column 1214, a feature column 1215, a position column 1216, a related object column 1217, and a time column 1218. The data ID column 1211 holds a data ID that identifies data stored in the knowledge DB 121 (that identifies a record in the knowledge DB 121).

[0038] The data type column 1212 holds information indicating the data type, such as image, video, voice, audio, motion, acceleration, gyro, geomagnetism, heart rate, pulse, electrical skin resistance, sweat rate, body temperature, brain waves, point of gaze, air temperature, humidity, pressure (voltage, oil pressure, air pressure, etc.), and current.

[0039] The data type may include, for example, document data such as work procedures, equipment manuals, work reports, and memos for the site or equipment installed at the site, audio data of meetings related to the site or equipment at the site, and a 3D model showing the three-dimensional shape of the site, a partial area of ​​the site, or equipment installed at the site.

[0040] The data column 1213 holds data corresponding to the data ID and the data type. The tag column 1214 holds tags assigned to the data, which are used, for example, for data searches using search queries. Note that tags do not need to be highly readable and may be, for example, feature quantities of high-dimensional vectors obtained by feature extraction processing.

[0041] The feature column 1215 holds feature values ​​related to the data. The location column 1216 holds information indicating the (real) location where the data was obtained or the (real) location of an associated object (described later) of the data. The associated object column 1217 holds information indicating an associated object of the data. An associated object includes an object or event related to the data, and the object or event that is the measurement target of the data, as well as the object or event described in the data, are both examples of an associated object of the data. The time column 1218 holds information indicating the time corresponding to the data in the data column 1213 (for example, the acquisition time of the data).

[0042] For example, the data with the data ID D001 in FIG. 2 is 3D model data showing the shape of plant A at the time, and the position (x １ , y １ , z １ ) indicates the center of gravity of plant A. For example, data with a data ID of D002 is audio data of a meeting at the relevant time regarding factory B, and the position (x ２ , y ２ , z ２ ) indicates the center of gravity of factory B. For example, data with a data ID of D003 indicates the position (x ３ , y ３ , z ３ ) and a tag indicating that no abnormality has occurred at that position is attached.

[0043] Furthermore, for example, data with a data ID of D004 is image data of cooling device A at the time, and a tag indicating that an accident has occurred in cooling device A of the image data at the time is attached, and the position (x ４ , y ４ , z ４ ) indicates the center of gravity of cooling device A. For example, data with a data ID of D005 is voice data of cooling device A at the time, and a tag indicating that an accident has occurred in cooling device A of the voice data at the time is attached. ４ , y ４ , z ４) indicates the center of gravity of the cooling device A.

[0044] For example, data with a data ID of D006 is document data (e.g., a manual) at the time of the relevant time regarding transformer A, and a tag indicating that work A is described in the document is attached. ５ , y ５ , z ５ ) indicates the center of gravity of transformer A. For example, data with a data ID of D007 is the position data of a worker with a user ID of U003, and the position at the time is (x ７ , y ７ , z ７ ) has been shown to be

[0045] 3 is a diagram showing an example of the data configuration of the virtual space management information 122. The virtual space management information 122 includes, for example, a space ID column 1221, a space type column 1222, a data ID column 1223, a room ID column 1224, a data column 1225, a display size column 1226, a description column 1227, and a destination space ID column 1228.

[0046] The space ID column 1221 holds a space ID that identifies a virtual space. A different space ID is defined for each different virtual space. The space type column 1222 holds a space type that indicates the type of virtual space. The space type includes, for example, "site," which indicates that the virtual space is a site 3D space, and "room," which indicates that the virtual space is a project management room.

[0047] The data ID column 1223 holds the data ID of the 3D model data that constructs the site 3D space. In a record whose space type is "site", a data ID is stored in the data ID column 1223, but in a record whose space type is "room", no data ID is stored in the data ID column 1223 (it may be a null value).

[0048] The room ID column 1224 holds a room ID that identifies the 3D model data that constructs the project management room. In a record whose space type is "room," a room ID is stored in the room ID column 1224, but in a record whose space type is "site," no room ID is stored in the room ID column 1224 (a null value may be stored).

[0049] The data column 1225 holds 3D model data for constructing a virtual space of the corresponding space type. The display size column 1226 holds information indicating the display size of the 3D model indicated by the data column 1225 (for example, the ratio to the size of the original 3D model).

[0050] The description column 1227 holds a description of the 3D model indicated by the data column 1225. The example of FIG. 3 shows that the shape of room1.glb is a rectangular parallelepiped with a side ratio of x:y:z. The connected space ID column 1228 holds information indicating virtual spaces to which participants (icons) in the virtual space can move. The example of FIG. 3 shows that a participant in the virtual space with space ID S001 can move to a virtual space with space ID S002 and a virtual space with space ID S003, and that a participant in the virtual space with space ID S002 can move to a virtual space with space ID S001.

[0051] 4 is a diagram showing an example of the data configuration of the miniature 3D model management information 123. The miniature 3D model management information 123 includes, for example, a miniature ID column 1231, a data ID column 1232, a data column 1233, a room ID column 1234, a display size column 1235, a display position column 1236, and a display angle column 1237.

[0052] The miniature ID column 1231 holds a miniature ID for identifying a miniature 3D model. The data ID column 1232 holds data IDs of 3D model data that are miniaturized to construct a miniature 3D model. The data column 1233 holds 3D model data that are miniaturized to construct a miniature 3D model.

[0053] The room ID field 1234 holds the room ID of the project management room in which the miniature 3D model is located. The display size field 1235 indicates the display size of the miniature 3D model (e.g., a ratio to the size of the original 3D model). The display position field 1236 indicates the display position (e.g., center of gravity position) of the miniature 3D model in the project management room indicated by the room ID field 1234. The display angle field 1237 indicates the display angle of the miniature 3D model in the project management room indicated by the room ID field 1234.

[0054] 5 is a diagram showing an example of the data configuration of the display content management information 124. The display content management information 124 includes, for example, a content ID column 1241, a data ID column 1242, a data type column 1243, a data column 1244, a room ID column 1245, a display size column 1246, a display position column 1247, and a display angle column 1248.

[0055] The content ID column 1241 holds a content ID for identifying the display content. The data ID column 1242 holds a data ID for the display content. The data type column 1243 holds a data type for the display content. The data column 1244 holds information indicating the data that constitutes the display content.

[0056] The room ID field 1245 holds the room ID of the project management room in which the display content is located. The display size field 1246 indicates the display size of the display content (e.g., the size ratio relative to a display frame having a predetermined shape and size). The display position field 1247 indicates the display position (e.g., the center of gravity position) of the display content in the project management room indicated by the room ID field 1245. The display angle field 1237 indicates the display angle of the display content in the project management room indicated by the room ID field 1245.

[0057] 4, the display content with the content ID C001 indicates an administration screen. The administration screen indicates an operation screen by the administrator (for example, a screen displayed on the display of the discussion space providing device 100 or another terminal used by the administrator). Therefore, no data ID is stored in the data ID column 1242 of the display content with the content ID C001, and a URL (Uniform Resource Locator) for viewing the administration screen, for example, is stored in the data column 1244.

[0058] The management screen can also display a query input screen and a search display screen, which will be described later. Participants whose icons are placed in the project management room can view the management screen, making it easier for participants to communicate with the administrator, and ultimately making it easier for the administrator to obtain consent from participants when adding display content or miniature 3D models to the project management room using the query input screen and search display screen, which will be described later. Furthermore, by displaying a web browser or the like on the management screen, participants can link with external applications through communication with the administrator while remaining in the project management room.

[0059] For example, when data of an audio type is displayed as display content, a player for playing the audio may be displayed as display content, or text transcribed from the audio may be displayed as display content.

[0060] 6 is a diagram showing an example of the data configuration of the corresponding coordinate management information 125. The corresponding coordinate management information 125 includes, for example, a content ID column 1251, a miniature ID column 1252, a coordinates within a miniature column 1253, and an emphasis flag column 1254. The content ID column 1251 holds a content ID. The miniature ID column 1252 holds a coordinates within a miniature column.

[0061] The miniature coordinates column 1253 indicates coordinates in the miniature 3D model indicated by the miniature ID column 1252 that correspond to the (real) position of the displayed content of the content ID indicated by the content ID column 1251. The highlight flag column 1254 holds a highlight flag that indicates whether the coordinates in the miniature corresponding to the displayed content are highlighted in the miniature 3D model (for example, if the highlight flag is "1", it is highlighted, and if it is "0", it is not highlighted).

[0062] 7 is a diagram showing an example of the data configuration of the user management information 126. The user management information 126 includes, for example, a user ID field 1261, icon data 1262, a space ID field 1263, a coordinate field 1264, an orientation field 1265, a participant flag field 1266, an administrator flag field 1267, and a worker flag field 1268.

[0063] The user ID column 1261 holds a user ID for identifying a user. The icon data 1262 holds data of the user's icon displayed in the virtual space. The space ID column 1263 holds the space ID of the virtual space in which the user (icon) is currently present. The coordinate column 1264 holds coordinates indicating the current location of the user (icon) in the virtual space.

[0064] The orientation column 1265 holds information indicating the current orientation of the user (icon) in the virtual space (if the user is a participant or administrator, the viewing angle displayed on the terminal used by the user).

[0065] The participant flag column 1266 holds a participant flag that is a flag indicating whether the user is a participant (for example, if the participant flag is "1", the user is a participant, and if "0", the user is not a participant). The administrator flag column 1267 holds an administrator flag that is a flag indicating whether the user is an administrator (for example, if the administrator flag is "1", the user is an administrator, and if "0", the user is not an administrator). The worker flag column 1268 holds a worker flag that is a flag indicating whether the user is a worker (for example, if the worker flag is "1", the user is a worker, and if "0", the user is not a worker).

[0066] 8 is a flowchart showing an example of an initial setting process by the discussion space providing device 100. The knowledge setting unit 111 registers information in the knowledge DB 121 (S801). Specifically, for example, the knowledge setting unit 111 may register information in the knowledge DB 121 according to input from an administrator to the input device 105, or may register information received from an external system in the knowledge DB 121.

[0067] The virtual space management unit 112 registers information about the site 3D space and the project management room in the virtual space management information 122 (S802). Specifically, for example, the virtual space management unit 112 selects data to be registered as the site 3D space from data whose data type is a 3D model stored in the knowledge DB 121 in accordance with input from the administrator via the input device 105. Furthermore, the virtual space management unit 112 acquires the display size, destination space ID, and description of the selected data in accordance with input from the administrator via the input device 105. The virtual space management unit 112 issues a space ID for the selected data and registers the issued space ID, the space type "site," the data ID of the selected data, the selected data, the acquired display size, destination space ID, and description in the virtual space management information 122. This registers information about the site 3D space.

[0068] Furthermore, for example, the virtual space management unit 112 acquires data (3D model data) as a project management room, as well as the display size, destination space ID, and description of the data, in accordance with input from the administrator via the input device 105. The virtual space management unit 112 issues a space ID and a room ID for the acquired data, and registers the issued space ID and room ID, the space type "room," the acquired data, and the acquired display size, destination space ID, and description in the virtual space management information 122. This registers the information of the project management room.

[0069] The virtual space management unit 112 may register multiple site 3D spaces in the virtual space management information 122, or may register multiple project management rooms in the virtual space management information 122. Furthermore, the virtual space management unit 112 does not need to acquire descriptions of the site 3D spaces and the project management rooms (does not need to register them in the virtual space management information 122).

[0070] The miniature 3D model management unit 115 places the miniature 3D model in the project management room (S803). Specifically, for example, the miniature 3D model management unit 115 displays data of a data type of 3D model stored in the knowledge DB 121 on the output device 106 (it may display a list of the data, or it may display 3D model data corresponding to a search query generated by an input from the administrator to the input device 105), and selects data to be placed in the project management room as a miniature 3D model from the data of a data type of 3D model stored in the knowledge DB 121 in accordance with the input from the administrator to the input device 105.

[0071] Furthermore, for example, the miniature 3D model management unit 115 determines the room ID of the project management room in which the miniature 3D model is to be placed, as well as the display size, display position, and display angle of the miniature model in the project management room, in accordance with input from the administrator to the input device 105. The miniature 3D model management unit 115 issues a miniature ID, and registers the issued miniature ID, the data ID of the selected data, the selected data, the determined room ID, display size, display position, and display angle in the miniature 3D model management information 123. As a result, the miniature 3D model is placed in the project management room.

[0072] Note that predetermined limitations may be imposed on the display size, display position, and / or display angle of the miniature 3D model. For example, the display size of the miniature 3D model may be determined so that the volume of the miniature 3D model is equal to or less than a predetermined multiple (e.g., 1 / 10 or less) of the volume of the project management room and the entire area of ​​the miniature 3D model fits within the project management room. Furthermore, for example, the display position of the miniature 3D model may be limited to the center of the floor of the project management room (e.g., a position within a predetermined distance from the center of gravity of the floor).

[0073] Note that one miniature 3D model may be placed in one project management room, or multiple miniature 3D models may be placed therein, or no miniature 3D model may be placed therein.

[0074] The display content management unit 116 arranges the display content in the project management room (S804). Specifically, for example, the display content management unit 116 displays the data stored in the knowledge DB 121 on the output device 106 (it may display a list of the data, or it may display data corresponding to a search query generated by an input from the administrator to the input device 105), and selects data to be arranged in the project management room as display content from the data stored in the knowledge DB 121 in accordance with the input from the administrator to the input device 105. However, because the management screen is not stored in the knowledge DB 121, the display content management unit 116 accepts input, for example, the URL of the management screen.

[0075] Furthermore, for example, the display content management unit 116 determines the room ID of the project management room in which the display content is to be placed, as well as the display size, display position, and display angle of the display content within the project management room, in accordance with input from the administrator via the input device 105. The display content management unit 116 issues a content ID, and registers the issued content ID, the data ID of the selected data, the selected data, the determined room ID, display size, display position, and display angle in the display content management information 124. As a result, the display content is placed in the project management room.

[0076] Note that a predetermined restriction may be imposed on the display size, display position, and / or display angle of the display content. For example, the display size of the display content may be determined so that the entire area of ​​the display content fits within the project management room. Furthermore, for example, the display position of a management screen, which is an example of display content, may be restricted to a first wall surface of the project management room, and the display positions of display content other than the management screen may be restricted to a wall surface different from the first wall surface of the project management room.

[0077] Note that one display content may be arranged in one project management room, or multiple display contents may be arranged in one project management room, or no display content may be arranged in one project management room.

[0078] The display content management unit 116 associates the placed miniature 3D model with the display content for each project management room (S805). Specifically, for each project management room, the display content management unit 116 identifies coordinates (intra-miniature coordinates) corresponding to the position of the display content in the miniature 3D model based on the position of the display content placed in the project management room in step S804 indicated by the knowledge DB 121, the position of the miniature 3D model placed in the project management room in step S803 indicated by the knowledge DB 121, and the 3D model data of the miniature 3D model.

[0079] Furthermore, the display content management unit 116 acquires, for example, information indicating whether to highlight the specified coordinates within the miniature 3D model (i.e., a highlight flag) in accordance with an input from the administrator to the input device 105. The display content management unit 116 registers the content ID of the display content for which the coordinate correspondence has been specified, the miniature ID of the miniature 3D model, the specified coordinates, and the highlight flag in the corresponding coordinate management information 125.

[0080] In the project management room, for example, coordinates within a miniature whose emphasis flag is set to "1" may be highlighted by lighting or flashing, or by placing a pin at the coordinates within the miniature, or by connecting the coordinates within the miniature whose emphasis flag is set to "1" with the displayed content corresponding to the coordinates within the miniature with a ray of light.

[0081] The display content management unit 116 may also highlight coordinates in the site 3D space that correspond to coordinates in the miniature whose highlight flag is set to "1" (for example, the coordinates are identified from the coordinates in the miniature and the position indicated in the knowledge DB 121 of the site 3D model that corresponds to the miniature 3D model). This makes it easier for participants viewing the site 3D space to check in more detail the location corresponding to the highlighted display content in the site 3D space.

[0082] The user management unit 114 executes the initial registration of the user (S806) and ends the initial setting process. Specifically, for example, the user management unit 114 acquires icon data, a participant flag, an administrator flag, and a worker flag in accordance with input from the administrator via the input device 105. Furthermore, the user management unit 114 issues a user ID and registers the issued user ID and the acquired icon data, participant flag, administrator flag, and worker flag in the user management information 126.

[0083] 9 is a flowchart showing an example of a routine process executed after the initial setting process by the discussion space providing device 100. In the routine state, the discussion space providing device 100 repeatedly executes routine processes in parallel, including, for example, a process of updating the knowledge DB 121 (S901), a process of managing entry and exit of participants and managers into the virtual space (S902), a process of managing user coordinates and orientation (S903), a process of responding to instructions from participants (S904), and a process of updating the layout of the project management room (S905).

[0084] In step S901, the knowledge setting unit 111 updates the knowledge DB 121 in accordance with a knowledge DB update instruction input by an administrator to the input device 105 or received from an external device or system. The knowledge DB update instruction includes, for example, information on a record to be newly added to the knowledge DB 121, update information on the contents of a record registered in the knowledge DB 121, or information on a registered record to be deleted.

[0085] In step S902, the user management unit 114 receives an entry instruction from, for example, the participant terminal 200 of a participant who has not yet entered the virtual space. The entry instruction includes, for example, the user ID of the participant, the space ID of the virtual space to which the participant wishes to enter, and the coordinates and orientation within the virtual space.

[0086] The user management unit 114 stores the space ID, coordinates, and orientation indicated by the entry instruction in a record corresponding to the user ID indicated by the entry instruction in the user management information 126. The user management unit 114 also places icon data corresponding to the user ID indicated by the entry instruction at the coordinates indicated by the entry instruction in the virtual space of the space ID indicated by the entry instruction, in the orientation indicated by the entry instruction.

[0087] Furthermore, if there is a miniature 3D model with the same data ID as the virtual space (site 3D space) in which the icon is placed, the user management unit 114 may also place the icon in the same orientation at the corresponding coordinates in the miniature 3D model. In this way, participants can enter the virtual space. Entry of the administrator into the virtual space can also be achieved in a similar manner.

[0088] The entry instruction does not have to include the space ID, coordinates, and / or direction, in which case the space ID, coordinates, and / or predetermined initial values ​​at the time of entry of the participant are set.

[0089] In step S902, the user management unit 114 receives an exit instruction from the participant terminal 200 of a participant currently in the virtual space. The exit instruction includes, for example, the user ID of the participant.

[0090] The user management unit 114 changes the space ID, coordinates, and orientation of the record in the user management information 126 that corresponds to the user ID indicated by the exit instruction to null values. The user management unit 114 also deletes the icon data that corresponds to the user ID indicated by the exit instruction from the virtual space. In this way, the participant can exit the virtual space. The administrator can also exit the virtual space in a similar manner.

[0091] A participant entering the virtual space inputs instructions to move an icon and change its orientation to the input device (e.g., a gyro sensor, keyboard, and / or mouse) of the participant terminal 200, and in step S903, the user management unit 114 receives the movement instruction and orientation change instruction, including the participant's user ID, from the participant terminal 200, and reflects the post-movement coordinates and post-orientation direction indicated by the instruction for the user ID in the user management information 126.

[0092] Furthermore, the user management unit 114 changes the coordinates and orientation of the icon of the user ID in the virtual space to the coordinates after the movement and the orientation after the change. If there is a miniature 3D model with the same data ID as the virtual space (site 3D space) in which the icon is placed, the user management unit 114 may also place the icon in the changed orientation at the corresponding coordinates in the miniature 3D model. The administrator's management of the coordinates and orientation of the icon is also achieved in a similar manner.

[0093] As mentioned above, the worker has not entered the virtual space. Therefore, in step S903, the user management unit 114 periodically acquires, for example, the latest location of a user (i.e., a worker) whose worker flag is set to 1 in the user management information 126 from the knowledge DB 121 (for example, data whose data ID is 007 in FIG. 1 ), updates the coordinate values ​​of the icon corresponding to that worker in the user management information 126 according to the acquired location, and places the icon of that worker at that location in the construction site 3D space.

[0094] Furthermore, if there is a miniature 3D model with the same data ID as the virtual space (site 3D space) in which the icon is placed, the user management unit 114 may also place the icon at the corresponding coordinates in the miniature 3D model. This allows a participant whose icon is placed in the project management room to easily understand which site 3D space other participants are looking at or which site the worker is at, and ultimately allows the participant to easily move to the site 3D space that the other participant is looking at or the site 3D space corresponding to the site where the worker is located by the space movement process in step S1005 described below.

[0095] The participant instruction response process in step S904 will be described in detail later with reference to Fig. 10. The room layout update process in step S905 will be described in detail later with reference to Fig. 11.

[0096] 10 is a flowchart showing an example of the participant instruction response process in step S904. The room management unit 113 receives from the participant terminal 200 various instructions input by the participants to the input device of the participant terminal 200 (S1001). The room management unit 113 determines the type of instruction received in step S1001 (S1002). The types of instructions received from the participant terminal 200 in step S1001 include, for example, an instruction to select display content in the project management room, an instruction to select another user's icon, an instruction to move in space, and an instruction to change the visual status of a miniature 3D model.

[0097] When the room management unit 113 determines that the type of instruction is an instruction to select display content in the project management room (S1002: display content selection instruction), it refers to the corresponding coordinate management information 125 and changes the display of the coordinates within the miniature corresponding to the selected display content in the miniature 3D model arranged in the project management room (S1003). Specifically, for example, the room management unit 113 changes the emphasis flag corresponding to the coordinates within the miniature (from "1" to "0" and from "0" to "1").

[0098] The processing of step S1003 allows participants to recognize coordinates within the miniature that correspond to the displayed content on the miniature 3D model, thereby deepening their understanding of the displayed content and facilitating deeper discussions among participants. Furthermore, if coordinates within the on-site 3D space that correspond to coordinates within the miniature whose highlight flag is set to "1" are also highlighted, participants viewing the on-site 3D space can also recognize the coordinates that correspond to the displayed content, thereby deepening their understanding of the displayed content and facilitating deeper discussions among participants.

[0099] In addition, in step S1003, the room management unit 113 may, for example, search the knowledge DB 121 for 3D model data that shows the same related matter as the display content selected in step S1002, and place the found 3D model data in the project management room as a miniature 3D model.

[0100] If the room management unit 113 determines that the type of instruction is an instruction to select the icon of another user within the project management room or within the miniature 3D model (S1002: Select other user), it mediates interaction with the selected other user for the participant terminal 200 that sent the instruction in step S1001 (S1004).

[0101] Specifically, for example, if the icon of another user selected in step S1002 is the icon of a participant or administrator different from the participant of the participant terminal 200 that sent the instruction in step S1001, the room management unit 113 causes, for example, a chat or an audio / video call to be performed using the icon in the virtual space between the participant terminal 200 of the participant and the participant terminal 200 of the different participant or the terminal of the administrator (e.g., the discussion space providing device 100). Also, for example, if the icon of another user selected in step S1002 is the icon of a worker, the room management unit 113 causes an audio / video call to be performed between the participant terminal 200 of the participant and the worker terminal 300 of the worker.

[0102] Also, for example, if the icon of another user selected in step S1002 is the icon of a participant or administrator other than the participant of the participant terminal 200 that sent the instruction in step S1001, the room management unit 113 may display the field of view corresponding to the icon of the other user on the output device of the participant terminal 200 that sent the instruction in step S1001, or may display the field of view corresponding to the icon of the participant of the participant terminal 200 that sent the instruction in step S1001 on the output device of the participant terminal 200 of the participant corresponding to the icon of the other user.

[0103] Also, for example, if the icon of another user selected in step S1002 is the icon of a participant or administrator different from the participant of the participant terminal 200 that sent the instruction in step S1001 and is an icon placed in the on-site 3D space, the room management unit 113 may display the data in the project management room on the output device of the terminal of the other user.

[0104] The process of step S1003 allows participants to easily interact with other participants, managers, or on-site workers. In particular, participants who recognize that icons are located in the same on-site 3D space can easily interact with each other, making it easier to deepen discussions among participants who share issues or goals.

[0105] If the room management unit 113 determines that the type of instruction is a spatial movement instruction indicating the user ID of the participant, the space ID of the destination virtual space, and the coordinates and orientation in the destination virtual space (S1002: spatial movement instruction), it moves the icon of the participant to the coordinates in the destination virtual space in the orientation (S1005). Also, in step S1005, the room management unit 113 changes the space ID, coordinates, and orientation of the user ID in the user management information 126 to the values ​​indicated by the spatial movement instruction.

[0106] The spatial movement instruction does not have to include coordinates and / or direction, in which case it is set to the coordinates and / or predetermined initial values ​​of the participant immediately after movement in the destination virtual space.

[0107] Also, for example, when a participant's icon is placed in a project management room, if a miniature 3D model placed in the project management room is selected via the input device of the participant's participant terminal 200, a spatial movement instruction to move the icon to the virtual space (on-site 3D space) corresponding to the miniature 3D model is sent to the discussion space providing device 100.

[0108] Also, for example, when a participant's icon is placed in the on-site 3D space, by accepting input of a room ID via the input device of the participant's participant terminal 200, a spatial movement instruction to move the icon to the project management room of that room ID is sent to the discussion space providing device 100.

[0109] The processing of step S1005 allows participants using participant terminals 200 to easily move between different virtual spaces. In particular, when the site 3D space is a large space such as a plant, a participant who wants to check a first location and a second location that are far from each other in the site 3D space can, for example, select a first location in a miniature 3D model corresponding to the site 3D space from the project management room to move to the first location in the site 3D space, confirm the first location, return to the project management room, and then select a second location in a miniature 3D model corresponding to the site 3D space from the project management room to move to the second location in the site 3D space. This allows the participant to move between the first and second locations in a short time without having to directly move an icon from the first location to the second location, which are far from each other in the site 3D space.

[0110] If the room management unit 113 determines that the type of instruction is an instruction to change the viewing state of a miniature 3D model (S1002: instruction to change the viewing state of a miniature 3D model), it changes the viewing state of the miniature 3D model indicated by the instruction, which is displayed on the output device 106 of the participant terminal 200 that sent the instruction, to the viewing state indicated by the instruction (S1006).

[0111] For example, when the participant terminal 200 of a participant whose icon is placed in the project management room receives the selection of a partial area of ​​a miniature 3D model placed in the project management room, for example, via input to an input device, it performs a predetermined motion such as zooming in or out on the area and sends a miniature 3D model visibility state change instruction to the discussion space providing device 100 to change the visibility state of the miniature 3D model.

[0112] The processing of step S1006 makes it easier for participants to visually identify areas of concern in the miniature 3D model, which in turn makes it easier for participants to recognize and solve problems they have.

[0113] In the above example, the processes of steps S1003 to S1006 are executed in accordance with various instructions from the participant terminal 200. However, the processes of steps S1003 to S1006 may also be executed in accordance with instructions from an administrator (for example, in accordance with input to the input device 105 of the discussion space providing device 100 or the terminal owned by the administrator). Furthermore, some of the processes of steps S1003 to S1006 may be executed only by instructions from some users, such as the administrator or participants who have been given authorization in advance. In particular, if the process of step S1003 were to be executed by instructions from all participants, many parts of the miniature 3D model would be highlighted, making it difficult to recognize important parts. Therefore, it is desirable that the process be executed only by instructions from some users, such as the administrator.

[0114] 11 is a flowchart showing an example of a room layout update process. The room management unit 113 determines whether a query for adding display content has been input (S1201). In step S1201, the room management unit 113 determines whether it has acquired, for example, a query input to the input device of the participant terminal 200 by a participant whose icon has been placed in the project management room, or a query input to the input device 105 of the discussion space providing device 100 (or a terminal owned by the administrator) by an administrator whose icon has been placed in the project management room. Specific examples of queries will be described later with reference to FIG. 13. Note that the query in step S1201 may be input only by the administrator.

[0115] If the room management unit 113 determines that a query has been entered (S1201: YES), it searches the knowledge DB 121 for data corresponding to the query, and displays the found data as a miniature 3D model if the data type of the found data is a 3D model, or as display content if the data type of the found data is other than a 3D model in the project management room where the icon of the participant or administrator who entered the query is located (S1202).

[0116] The display size, display position, and display angle of the hit data may be specified by input by the participant or administrator who entered the query, or the hit data may be automatically displayed at a predetermined display size and a predetermined display angle in an available area of ​​the project management room (so as not to overlap with the display area of ​​a miniature 3D model that has already been displayed or other display content). Also, in step S1202, predetermined restrictions may be imposed on the display size, display position, and / or display angle of the miniature 3D model and display content.

[0117] In step S1202, the room management unit 113 also updates the miniature 3D model management information 123 or the display content management information 124 in accordance with the newly displayed 3D model or display content.

[0118] If the room management unit 113 determines that a query has not been input (S1201: NO), the process proceeds to step S1203. The room management unit 113 determines whether a predetermined trigger has been fired, for example, by constantly monitoring the knowledge DB 121 (S1203).

[0119] If the room management unit 113 determines that the trigger has been fired (S1203: YES), it displays data that meets the predetermined conditions corresponding to the trigger in the project management room previously associated with the trigger (S1204). Note that in step S1204, if the data type of the data that meets the predetermined conditions corresponding to the trigger is a 3D model, it displays the data as a miniature 3D model, and if the data type of the data that meets the predetermined conditions corresponding to the trigger is other than a 3D model, it displays the data as display content (S1204). If the room management unit 113 determines that the trigger has not been fired (S1203: NO), it transitions to step S1205.

[0120] For example, the trigger may be "position (x ３ , y ３ , z ３The data type of the trigger is "temperature" and the value of the data is 20°C or more. ３ , y ３ , z ３ 1 is data whose data type is temperature," and a project management room whose room ID is R001 is associated with the trigger in advance. In this case, when a record whose data ID is D003 in the example of FIG. 1 (i.e., data in which the temperature at the location is 20° C. or higher) is added to the knowledge DB 121, the room management unit 113 displays the data corresponding to the record as display content in the project management room whose room ID is R001.

[0121] Also, for example, assume that "an accident occurs in cooling device A" is predetermined as the trigger, that data that satisfies a predetermined condition corresponding to the trigger is "data whose related matter is cooling device A," and that a project management room whose room ID is R001 is pre-associated with the trigger. In this case, when records whose data IDs are D004 and D005 in the example of Fig. 1 (i.e., data for which it is determined that an accident has occurred in cooling device A based on the related matter and tag) are added to the knowledge DB 121, the room management unit 113 displays the data corresponding to the records as display content in the project management room whose room ID is R001.

[0122] The display size, display position, and display angle of the data corresponding to the trigger may be specified by input by the participant or administrator who entered the query, or the data corresponding to the trigger may be automatically displayed at a predetermined display size and a predetermined display angle in an available area of ​​the project management room (so as not to overlap with the display area of ​​a miniature 3D model that has already been displayed or other display content). Also in step S1205, predetermined restrictions may be imposed on the display size, display position, and / or display angle of the miniature 3D model and display content.

[0123] In step S1204, the room management unit 113 updates the miniature 3D model management information 123 or the display content management information 124 in accordance with the newly displayed 3D model or display content.

[0124] The room management unit 113 determines whether it has received an instruction to change the display position, shape, or size of a miniature 3D model or display content already placed in the project management room, or an instruction to delete the miniature 3D model or display content (S1205). In step S1203, the room management unit 113 determines whether it has acquired, for example, a query input into the input device of the participant terminal 200 by a participant whose icon has been placed in the project management room, or a query input into the input device 105 of the discussion space providing device 100 (or a terminal owned by the administrator) by whom an icon has been placed in the project management room. Note that the instruction in step S1205 may be executable only by the administrator.

[0125] If the room management unit 113 determines that it has received the instruction in step S1205 (S1205: YES), it changes the display position, shape, or display size of the existing miniature 3D model or display content, or deletes it in accordance with the instruction (S1206).If the room management unit 113 determines that it has not received the instruction in step S1205 (S1205: NO), it ends the room layout update process.

[0126] In step S1206, the room management unit 113 updates the miniature 3D model management information 123 or the display content management information 124 in accordance with the change in the display position, shape, or display size, or the deletion, executed in step S1205.

[0127] Fig. 12 is a plan view showing an example of a project management room. In the example of Fig. 12, a project management room 1270 is a rectangular parallelepiped and has a first wall 1280, a second wall 1290, a third wall 1292, and a fourth wall 1294. A management screen 1281 is displayed as display content on the first wall 1280. In addition, display content 1291, display content 1293, and display content 1295 are displayed on the second wall 1290, the third wall 1292, and the fourth wall 1294, which are walls different from the first wall 1280, respectively.

[0128] A miniature 3D model 1271 is placed in the center of the floor of the project management room 1270 (or at a position a predetermined height above the center of the floor). A ray 1273 connects coordinates 1272 within the miniature of the miniature 3D model 1271 to the display content 1291, thereby highlighting the coordinates 1272 within the miniature and indicating that the (real) position of the display content 1291 corresponds to the coordinates 1272 within the miniature.

[0129] Additionally, an icon 1274 of a participant whose participant ID is U010 and an icon 1275 of a participant whose participant ID is U015 are placed in the project management room 1270.

[0130] In this way, since the project management room 1270 can arrange multiple display contents for the site, participants can view the site objects collectively, even at large sites such as plants. In particular, when the participant terminal 200 is an immersive device such as a VR headset, it is difficult to view all of the display contents collectively if the participant moves only within the 3D site space, so the above-mentioned effect of providing the project management room 1270 is particularly pronounced. Furthermore, in the project management room 1270, participants can easily recognize which position in the miniature 3D model (i.e., the site) the display content corresponds to.

[0131] Furthermore, by displaying the management screen 1281 on the first wall surface 1280, which is a wall surface on which other display contents are not displayed, participants can easily distinguish between the management screen 1281 and other display contents.

[0132] 13 is a diagram showing an example of the screen configuration of a query input screen for inputting a query in step S1201. The query input screen 1300 is displayed on, for example, the output device 106 of the discussion space providing device 100, the output device of the terminal owned by the administrator, or the output device of the participant terminal 200.

[0133] The query input screen 1300 includes, for example, a search keyword input area 1310 and a search condition setting area 1320. The room management unit 113 searches the knowledge DB 121 according to the search keyword and / or the search condition.

[0134] The search condition setting area 1320 includes, for example, a search type setting area 1321 , a time setting area 1322 , a data type setting area 1323 , a location setting area 1324 , a related matter setting area 1325 , and a free input area 1326 .

[0135] The search type setting area 1321 is an area for setting a search range based on, for example, a search keyword. When "content" is selected in the search type setting area 1321, data containing the search keyword in the content (e.g., sentences in a document file, file names of each file, etc.) is searched for in the knowledge DB 121.

[0136] When "tag" is selected in the search type setting area 1321, for example, data whose tags contain the search keyword is searched for in the knowledge DB 121. When "full text" is selected in the search type setting area 1321, for example, data whose contents and / or tags contain the search keyword is searched for in the knowledge DB 121.

[0137] The time setting area 1322 is an area for setting search conditions relating to the time of data, and the data type setting area 1323 is an area for setting search conditions relating to the data type.

[0138] The position setting area 1324 is an area for setting search conditions related to the position of data. In the example of Fig. 13, a center and a radius can be set in the position setting area 1324, and data indicating a position within the sphere defined by the set center and radius can be searched from the knowledge DB 121. Note that the position setting area 1324 may be set in an area of ​​a shape other than a sphere, and the name of a room or section within a site may also be set.

[0139] The related thing setting area 1325 is an area for setting search conditions for related things of data. In the example of Fig. 13, the related thing setting area 1325 displays boxes that allow the user to select related things contained in the knowledge DB 121. The free input area 1326 is an area for the user to freely input text indicating search conditions.

[0140] Examples of queries that can be specified using the query input screen 1300 are described below. For example, displaying data within a radius of 1 meter from a certain location whose data type is document, displaying data whose data type is image that matches the search keyword "danger" and whose time is this week, constantly displaying data whose data type is temperature (thermometer measurements) located at a certain location in real time, displaying all data whose time is the current day, etc. are all examples of queries.

[0141] 14 is a diagram showing an example of the screen configuration of a search result display screen. The search result display screen 1400 is a screen that displays data hit by a search using a query entered on the query input screen 1300, and is displayed, for example, on the output device that was displaying the query input screen 1300.

[0142] The search result display screen 1400 includes, for example, a 3D model display area 1410, a search result display area 1420, a button 1430, and a button 1440. The 3D model display area 1410 displays a 3D model (extracted from the 3D model data in the knowledge DB 121) of an area including positions and related matters corresponding to the data included in the search results. The positions and related matters corresponding to the data included in the search results are also shown on the 3D model displayed in the 3D model display area 1410 (the positions and related matters are indicated by the numbers "1" and "2" in the 3D model display area 1410 in FIG. 14 ).

[0143] The search result display area 1420 displays the search results based on the search keywords and search conditions specified on the query input screen 1300. In the example of Fig. 14, the search result display area 1420 displays data, data type, time, tag, location, and related matter, but any item included in the knowledge DB 121 may be displayed.

[0144] In addition, items extracted from the knowledge DB 121 such as data, data type, time, tag, location, and related things may also be displayed superimposed on the 3D model in the 3D model display area 1410.

[0145] In addition, in the search result display area 1420, for example, the data may be arranged in chronological order, or the data may be sorted by data type and then arranged in chronological order. In addition, in the search result display area 1420, for example, the data may be sorted by related matter and then arranged in chronological order.

[0146] When data displayed in search result display area 1420 is selected and then button 1430 is selected, the selected data is displayed in step S1202. When button 1440 is selected, the screen returns to query input screen 1300, or search result display screen 1400 simply ends.

[0147] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0148] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the program, table, and file that implements each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0149] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected.

Claims

1. An information processing device comprising a processor and a memory, and connected to a display device, wherein the memory holds knowledge information indicating data relating to a site including a 3D model of the site and a location corresponding to the data, and virtual space management information indicating a first virtual space constructed by a 3D model of a project management room, wherein the processor displays the first virtual space on the display device, displays a miniature 3D model that is a reduced version of the 3D model of the site in the first virtual space, accepts a selection of data included in the knowledge information, displays the selected data in the first virtual space as display content, identifies coordinates within the miniature 3D model that correspond to the location of data corresponding to the display content based on the 3D model of the site and the locations indicated by the knowledge information of the selected data, and highlights the identified coordinates within the miniature 3D model.

2. An information processing device as described in claim 1, wherein the processor, after accepting entry of a participant into the first virtual space, accepts input of a query to search the knowledge information, searches the knowledge information for data corresponding to the query, and displays the searched data in the first virtual space as the display content.

3. An information processing device as described in claim 1, wherein the knowledge information indicates a type of data related to the site, and the processor receives an instruction to update the knowledge information, updates the knowledge information based on the update instruction, and when it is determined that a specified trigger has been activated when the knowledge information is updated, retrieves data from the knowledge information that satisfies a specified condition corresponding to the specified trigger, and displays the retrieved data in the first virtual space as the display content.

4. An information processing device according to claim 1, wherein the processor displays, in the first virtual space, a management screen, which is a screen of a terminal used by an administrator who manages the first virtual space, as the display content.

5. An information processing device according to claim 4, wherein the project management room includes a floor and a plurality of wall surfaces, and the processor places the miniature 3D model on the floor surface, places the management screen on a wall surface included in the plurality of wall surfaces, and places the display content other than the management screen on a wall surface included in the plurality of wall surfaces but other than the wall surface on which the management screen is placed.

6. An information processing device as described in claim 1, wherein the processor receives an instruction to change the viewing state of the miniature 3D model by a predetermined motion, and changes the viewing state of the miniature 3D model being displayed in accordance with the instruction.

7. An information processing device according to claim 1, wherein the processor is configured such that the virtual space management information indicates a second virtual space that is constructed using a 3D model of the site and is different from the first virtual space.

8. An information processing device as described in claim 7, wherein the processor: accepts entry of a participant into the first virtual space or the second virtual space; places a participant icon representing the participant in the virtual space in which the participant is entering; displays a field of view based on the placement of the participant icon on the display device; and moves the participant icon placed in either the first virtual space or the second virtual space to the other.

9. An information processing device as described in claim 8, wherein the knowledge information indicates a position of a worker at the site, and the processor: identifies coordinates corresponding to the position of the worker in the miniature 3D model based on the 3D model of the site and the position indicated by the knowledge information of the worker; places a worker icon indicating the worker at the identified coordinates in the miniature 3D model; identifies a position corresponding to the position of the worker in the 3D model of the site based on the position indicated by the 3D model of the site and the knowledge information of the worker; and places the worker icon at the identified position in the second virtual space.

10. An information processing device according to claim 9, wherein the processor accepts a selection of an icon representing the participant or the worker, and executes a predetermined interaction process with the participant or the worker corresponding to the selected icon.

11. An information processing device according to claim 10, wherein the processor displays, as the predetermined interaction process, the field of view of the participant or worker corresponding to the selected icon on the display device.

12. An information processing device according to claim 7, wherein the processor calculates coordinates in the 3D model of the site that correspond to the identified coordinates based on the position indicated by the knowledge information of the 3D model of the site and the identified coordinates, and highlights the calculated coordinates in the second virtual space.

13. An information processing method by an information processing device, the information processing device having a processor and a memory, and connected to a display device, the memory holding knowledge information indicating data regarding the site including a 3D model of the site and a position corresponding to the data, and virtual space management information indicating a first virtual space constructed by a 3D model of a project management room, the information processing method comprising the steps of: the processor displays the first virtual space on the display device; the processor displays a miniature 3D model, which is a reduced version of the 3D model of the site, in the first virtual space; the processor accepts selection of data included in the knowledge information; the processor displays the selected data in the first virtual space as display content; the processor identifies coordinates in the miniature 3D model that correspond to the position of data corresponding to the display content based on the 3D model of the site and the positions indicated by the knowledge information of the selected data; and the processor highlights the identified coordinates in the miniature 3D model.

Images