In-hoistway apparatus arrangement assistance system

Abstract

Provided is a system making it easy for a worker to revise three-dimensional model data of an in-hoistway apparatus or the like in accordance with actual circumstances, and making it possible to assist with the work of arranging an apparatus in a hoistway. This in-hoistway apparatus arrangement assistance system comprises: an image acquisition unit for acquiring an image of the inside of a hoistway of an elevator in real space; a data acquisition unit for acquiring coordinate data of the inside of the hoistway and three-dimensional model data of an object to be installed in the hoistway from a data management unit that stores the coordinate data and the three-dimensional model data; a display unit for displaying an augmented reality image generated by superimposing the three-dimensional model data onto the image of the inside of the hoistway in real space; an operation detection unit for detecting a selection operation for individually selecting an object and a movement operation on the selected object in the augmented reality image; an update unit for updating the three-dimensional model data so as to reflect a change in orientation and location of the object due to the movement operation; and a transmission unit for transmitting the updated three-dimensional model data to the data management unit.

Description

Elevator shaft equipment layout support system

[0001] The present disclosure relates to a system for assisting in-hoistway equipment layout.

[0002] An elevator exterior display system is known that includes a display unit that displays a three-dimensional model of the elevator, and the display unit detects changes in the user's field of view and displays an image of the three-dimensional model projected into the field of view (see, for example, Patent Document 1).

[0003] Japanese Patent Publication No. 2021-116159

[0004] As described above, the technology disclosed in Patent Document 1 allows on-site workers to easily check the layout of equipment and the like to be installed in the elevator hoistway that has been designed in advance. However, the situation inside the hoistway may differ from that assumed at the time of design. In such cases, the technology disclosed in Patent Document 1 makes it difficult to modify the 3D model data in accordance with the on-site situation.

[0005] The present disclosure has been made to solve such problems, and its purpose is to provide an in-hoistway equipment layout support system that enables on-site workers to easily check the layout of equipment, etc. to be installed in an elevator hoistway that has been designed in advance, and that enables easy correction of 3D model data of equipment, etc. in accordance with the actual situation in the hoistway, thereby supporting the work of arranging equipment in the hoistway.

[0006] The elevator shaft equipment placement assistance system according to the present disclosure includes an image acquisition unit that acquires an image of the interior of an elevator elevator shaft in real space; a data acquisition unit that acquires coordinate data of the interior of the elevator shaft and three-dimensional model data of objects to be installed in the elevator shaft from a data management unit that stores the coordinate data and the three-dimensional model data; a display unit that displays an augmented reality image generated by superimposing the three-dimensional model data of the objects acquired from the data management unit on the image of the interior of the elevator shaft in the real space; an operation detection unit that detects a selection operation for individually selecting the objects and a movement operation for the selected objects in the augmented reality image; an update unit that updates the three-dimensional model data of the objects to reflect changes in orientation and position of the objects due to the movement operation; and a transmission unit that transmits the updated three-dimensional model data of the objects to the data management unit.

[0007] The elevator shaft equipment placement support system according to the present disclosure allows on-site workers to easily check the placement of equipment, etc. to be installed in the elevator shaft that has been designed in advance, and also allows them to easily modify the three-dimensional model data of the equipment, etc. according to the actual situation in the elevator shaft, thereby supporting the work of placing equipment in the elevator shaft.

[0008] 1 is a block diagram showing the overall configuration of the in-shaft equipment layout support system according to Embodiment 1. FIG. 2 is a block diagram showing the configuration of a processing unit of AR glasses of the in-shaft equipment layout support system according to Embodiment 1. FIG. 3 is a block diagram showing the configuration of a terminal processing unit in a modified example of the in-shaft equipment layout support system according to Embodiment 1. FIG. 4 is a flow diagram showing an example of a beacon terminal position specifying process for the in-shaft equipment layout support system according to Embodiment 1. FIG. 5 is a diagram showing the start of communication of the AR glasses of the in-shaft equipment layout support system according to Embodiment 1. FIG. 6 is a diagram showing an example of a main operation screen displayed on the AR glasses of the in-shaft equipment layout support system according to Embodiment 1. FIG. 7 is a diagram showing an example of a beacon terminal setting screen displayed on the AR glasses of the in-shaft equipment layout support system according to Embodiment 1. FIG. 8 is a diagram showing an example of a beacon terminal selection screen displayed on the AR glasses of the in-shaft equipment layout support system according to Embodiment 1. FIG. 9 is a perspective view showing an example of installation of a beacon terminal in the in-shaft equipment layout support system according to Embodiment 1. FIG. 10 is a diagram showing an example of a 3D coordinate data display screen displayed on the AR glasses of the in-shaft equipment layout support system according to Embodiment 1. 1 is a diagram illustrating designation of the position of a beacon terminal in three-dimensional coordinate data within a hoistway in the hoistway in the hoistway in-hoistway equipment layout support system according to Embodiment 1. FIG. 2 is a flow diagram illustrating an example of coordinate alignment processing in the hoistway in-hoistway equipment layout support system according to Embodiment 1. FIG. 3 is a diagram illustrating coordinate alignment in the hoistway in-hoistway equipment layout support system according to Embodiment 1. FIG. 4 is a diagram illustrating coordinate alignment in the hoistway in-hoistway equipment layout support system according to Embodiment 1. FIG. 5 is a flow diagram illustrating an example of augmented reality image display processing in the hoistway in-hoistway equipment layout support system according to Embodiment 1. FIG. 6 is a diagram illustrating an example of a three-dimensional data display selection screen displayed on AR glasses in the hoistway in-hoistway equipment layout support system according to Embodiment 1. FIG. 7 is a diagram illustrating an example of an augmented reality image when looking at the front wall within a hoistway in the hoistway in the hoistway in-hoistway equipment layout support system according to Embodiment 1. FIG. 8 is a diagram illustrating an example of an augmented reality image when looking at the landing side from within a hoistway in the hoistway in the hoistway equipment layout support system according to Embodiment 1.1 is a flow diagram showing an example of processing for moving an object within an augmented reality image of the in-hoistway equipment layout support system according to Embodiment 1. FIG. 2 is a perspective view illustrating an example of equipment interference in the in-hoistway equipment layout support system according to Embodiment 1. FIG. 3 is a diagram illustrating an example of a screen displayed on the AR glasses of the in-hoistway equipment layout support system according to Embodiment 1 when selecting a three-dimensional data operation. FIG. 4 is a diagram illustrating an example of a screen display during an object selection operation in the in-hoistway equipment layout support system according to Embodiment 1. FIG. 5 is a diagram illustrating an example of a screen display after an object has been moved in the in-hoistway equipment layout support system according to Embodiment 1. FIG. 6 is a flow diagram showing an example of processing for adding an object such as an obstacle that has caused a layout change in the in-hoistway equipment layout support system according to Embodiment 1. FIG. 7 is a diagram illustrating an example of a screen displayed on the AR glasses of the in-hoistway equipment layout support system according to Embodiment 1. FIG. 8 is a diagram illustrating an example of a screen display during an object selection operation in the in-hoistway equipment layout support system according to Embodiment 1. FIG. 9 is a diagram illustrating an example of a screen display during an object addition in the in-hoistway equipment layout support system according to Embodiment 1. FIG. 1 is a diagram illustrating a confirmation operation of data change contents in the intra-hoistway equipment layout support system according to Embodiment 1. FIG. 2 is a flow chart illustrating an example of equipment layout processing at another work location in the intra-hoistway equipment layout support system according to Embodiment 1. FIG. 3 is a diagram illustrating an example of a case where a beacon terminal of the intra-hoistway equipment layout support system according to Embodiment 1 is moved to another work location. FIG. 4 is a diagram illustrating an example of a case where a beacon terminal of the intra-hoistway equipment layout support system according to Embodiment 1 is additionally installed at another work location. FIG. 5 is a flow chart illustrating an example of a revision process of 3D model data in the intra-hoistway equipment layout support system according to Embodiment 1. FIG. 6 is a diagram illustrating an example of a case where 3D model data is revised using a PC in the intra-hoistway equipment layout support system according to Embodiment 1. FIG. 7 is a diagram illustrating an example of a case where 3D model data is revised using AR glasses in the intra-hoistway equipment layout support system according to Embodiment 1.Fig. 1 is a block diagram showing the overall configuration of a first modified example of the hoistway equipment layout support system according to embodiment 1. Fig. 2 is a block diagram showing the overall configuration of a second modified example of the hoistway equipment layout support system according to embodiment 1. Fig. 3 is a block diagram showing the configuration of a processing unit of VR glasses in the second modified example of the hoistway equipment layout support system according to embodiment 1. Fig. 4 is a flow diagram showing an example of equipment layout processing in the second modified example of the hoistway equipment layout support system according to embodiment 1. Fig. 5 is a diagram showing an example of a case where three-dimensional model data is revised using VR glasses in the second modified example of the hoistway equipment layout support system according to embodiment 1. Fig. 6 is a diagram showing an example of a configuration for realizing the functions of the terminal processing unit, glasses processing unit, positioning device processing unit, and VR glasses processing unit of the hoistway equipment layout support system according to embodiment 1.

[0009] An embodiment of a hoistway equipment layout support system according to the present disclosure will be described with reference to the accompanying drawings. In each drawing, the same or corresponding parts are designated by the same reference numerals, and redundant explanations are appropriately simplified or omitted. For convenience, the following description may express the positional relationship of each structure based on the illustrated state. Note that the present disclosure is not limited to the following embodiments, and any combination of the embodiments, any modification of any component of each embodiment, or any omission of any component of each embodiment are possible within the scope of the present disclosure.

[0010] Embodiment 1. A first embodiment of the present disclosure will be described with reference to Fig. 1 to Fig. 39. Fig. 1 is a block diagram showing the overall configuration of an in-hoistway equipment layout support system. Fig. 2 is a block diagram showing the configuration of a processing unit of AR glasses in the in-hoistway equipment layout support system. Fig. 3 is a block diagram showing the configuration of a terminal processing unit in a modified example of the in-hoistway equipment layout support system.

[0011]

[0033] Figure 4 is a flow chart showing an example of a process for identifying the position of a beacon terminal in the intra-shaft equipment layout support system. Figure 5 is a diagram showing the start of communication by the AR glasses of the intra-shaft equipment layout support system. Figure 6 is a diagram showing an example of a main operation screen displayed on the AR glasses of the intra-shaft equipment layout support system. Figure 7 is a diagram showing an example of a beacon terminal setting screen displayed on the AR glasses of the intra-shaft equipment layout support system. Figure 8 is a diagram showing an example of a beacon terminal selection screen displayed on the AR glasses of the intra-shaft equipment layout support system. Figure 9 is a perspective view showing an example of installation of a beacon terminal in the intra-shaft equipment layout support system. Figure 10 is a diagram showing an example of a three-dimensional coordinate data display screen displayed on the AR glasses of the intra-shaft equipment layout support system. Figure 11 is a diagram explaining the designation of the position of a beacon terminal in three-dimensional coordinate data within a hoistway in the intra-shaft equipment layout support system.

[0012] Fig. 12 is a flow chart showing an example of a coordinate alignment process in the hoistway in-equipment layout support system. Fig. 13 to Fig. 15 are diagrams for explaining coordinate alignment in the hoistway in-equipment layout support system.

[0013] Fig. 16 is a flow chart showing an example of an augmented reality image display process of the hoistway in-equipment layout support system. Fig. 17 is a diagram showing an example of a 3D data display selection screen displayed on AR glasses in the hoistway in-equipment layout support system. Fig. 18 is a diagram showing an example of an augmented reality image when looking at the front wall inside the hoistway in the hoistway in the hoistway in-equipment layout support system. Fig. 19 is a diagram showing an example of an augmented reality image when looking at the landing side from inside the hoistway in the hoistway in the hoistway in the hoistway in-equipment layout support system.

[0014] Fig. 20 is a flow diagram showing an example of processing for moving an object within an augmented reality image of the hoistway equipment layout support system. Fig. 21 is a perspective view illustrating an example of equipment interference in the hoistway equipment layout support system. Fig. 22 is a diagram showing an example of a screen displayed on the AR glasses of the hoistway equipment layout support system when selecting a 3D data operation. Fig. 23 is a diagram showing an example of a screen display when an object is selected in the hoistway equipment layout support system. Fig. 24 is a diagram showing an example of a screen display after an object has been moved in the hoistway equipment layout support system.

[0015] Fig. 25 is a flow diagram showing an example of an object addition process for an obstacle or the like that has caused a layout change in the hoistway equipment layout support system. Fig. 26 is a diagram showing an example of a screen when object attribute information is input in the hoistway equipment layout support system. Fig. 27 is a diagram showing an example of a screen display when an object is added in the hoistway equipment layout support system. Fig. 28 is a diagram showing an example of a display of the distance between objects in the hoistway equipment layout support system. Fig. 29 is a diagram explaining an example of object generation in the hoistway equipment layout support system.

[0016] Fig. 30 is a flow chart showing an example of a process for confirming changes in the hoistway equipment layout support system. Fig. 31 is a diagram for explaining the process for confirming changes in data in the hoistway equipment layout support system.

[0017] Fig. 32 is a flow chart showing an example of equipment layout processing at another work location of the hoistway equipment layout support system. Fig. 33 is a diagram showing an example of a case where a beacon terminal of the hoistway equipment layout support system is moved to another work location. Fig. 34 is a diagram showing an example of a case where a beacon terminal of the hoistway equipment layout support system is additionally installed at another work location.

[0018] Fig. 35 is a flow chart showing an example of a process for revising three-dimensional model data in the hoistway equipment layout support system. Fig. 36 is a diagram showing an example of revising three-dimensional model data using a PC in the hoistway equipment layout support system. Fig. 37 is a diagram showing an example of revising three-dimensional model data using AR glasses in the hoistway equipment layout support system.

[0019] Fig. 38 is a block diagram showing the overall configuration of a first modified example of an intra-hoistway equipment layout support system. Fig. 39 is a block diagram showing the overall configuration of a second modified example of an intra-hoistway equipment layout support system. Fig. 40 is a block diagram showing the configuration of a processing unit of VR glasses in the second modified example of an intra-hoistway equipment layout support system. Fig. 41 is a flow diagram showing an example of equipment layout processing in the second modified example of an intra-hoistway equipment layout support system. Fig. 42 is a diagram showing an example of revising three-dimensional model data using VR glasses in the second modified example of an intra-hoistway equipment layout support system. Fig. 43 is a diagram showing an example of a configuration that realizes the functions of the terminal processing unit, glass processing unit, positioning device processing unit, and VR glass processing unit of the intra-hoistway equipment layout support system.

[0020] As shown in Fig. 1 , one configuration example of the elevator shaft equipment layout support system according to this embodiment includes a terminal device 100, a cloud server 200, and AR glasses 300. The terminal device 100 is an information processing device such as a PC, a smartphone, or a tablet terminal. The terminal device 100 includes a terminal communication unit 101, a terminal processing unit 110, and a human interface unit 103.

[0021] The device processing unit 110 has a function of performing calculations on data and the like to execute predetermined processing. The device communication unit 101 is a communication interface that enables the terminal device 100 to communicate with the cloud server 200. The device communication unit 101 may be configured as a physically structured circuit, in which case the device communication unit 101 may also be referred to as a communication circuit. The device communication unit 101 can transmit data received from the device processing unit 110 to the cloud server 200. The device communication unit 101 can also receive data transmitted from the cloud server 200 and pass the received data to the device processing unit 110.

[0022] The human interface unit 103 is an interface between the terminal device 100 and the user. The human interface unit 103 includes, for example, a display unit and an operation unit. The display unit is a liquid crystal display, an LED display, an organic EL display, or the like that can display various information to the user. The operation unit is a keyboard, a mouse, or the like that allows the user to input various information. The human interface unit 103 may also include a touch panel, a non-contact panel, or the like that serves as both the display unit and the operation unit.

[0023] Cloud server 200 is a server device realized by a collection of multiple information processing devices. Cloud server 200 includes a server communication unit 201 and a data management unit 202. Note that server communication unit 201 and data management unit 202 may be provided in a server device consisting of a single information processing device, rather than a collection of multiple information processing devices.

[0024] The server communication unit 201 is a communication interface through which the cloud server 200 communicates with each of the terminal device 100 and the AR glasses 300. The server communication unit 201 may be configured as a physically structured circuit, in which case the server communication unit 201 may also be referred to as a communication circuit. The server communication unit 201 is capable of transmitting data received from the data management unit 202 to one or both of the terminal device 100 and the AR glasses 300. The server communication unit 201 is also capable of receiving data transmitted from the terminal device 100 or the AR glasses 300 and passing the received data to the data management unit 202.

[0025] The data management unit 202 manages various data handled by the hoistway equipment layout support system. The data managed by the data management unit 202 includes coordinate data within the elevator hoistway 1 and three-dimensional model data of objects installed within the hoistway 1. The data management unit 202 stores the coordinate data within the hoistway 1 and the three-dimensional model data of objects. The coordinate data within the hoistway 1 is data for representing positions within the hoistway 1 on a three-dimensional coordinate system having, for example, an X-axis, a Y-axis, and a Z-axis. The objects installed within the hoistway 1 include, for example, various devices installed within the hoistway 1, such as control panels, junction boxes, weighing devices, governors, and hoists, as well as wiring connecting these devices. The three-dimensional model data of the objects installed within the hoistway 1 is data representing the shapes and sizes of the three-dimensional models of these devices, wiring, etc., as well as their positions and orientations within the hoistway 1 on the three-dimensional coordinate system. The data managed by the data management unit 202 may include images captured by a field worker using a camera.

[0026] The AR glasses 300 are glasses-type or helmet-type devices worn on the face or head of a user. AR is an abbreviation for "Augmented Reality." The AR glasses 300 include a glass communication unit 301, a display unit 302, an operation unit 303, a camera 304, a sensor 305, a glass storage unit 306, and a glass processing unit 310.

[0027] The glass communication unit 301 is a communication interface for the AR glasses 300 to communicate with the cloud server 200. The glass communication unit 301 may be configured as a physically structured circuit, in which case the glass communication unit 301 may also be referred to as a communication circuit. The glass communication unit 301 is capable of transmitting data received from the glass processing unit 310 to the cloud server 200. The glass communication unit 301 is also capable of receiving data transmitted from the cloud server 200 and passing the received data to the glass processing unit 310.

[0028] The camera 304 captures images of real space. The images include not only photographs but also images or videos in which part or the entire elevator shaft is mapped. For example, the camera 304 is provided facing forward of the AR glasses 300 and can capture images in the same direction as the orientation of the face of a user wearing the AR glasses 300. This allows the camera 304 to capture images of real space in the same range as the field of view of the user wearing the AR glasses 300, depending on the orientation of the face of the user.

[0029] The display unit 302 and the operation unit 303 are interfaces between the AR glasses 300 and a user wearing the AR glasses 300. The display unit 302 is a liquid crystal device, an LED device, an organic EL device, or the like that can display various information to the user. The display unit 302 is visualized in front of the eyes of the user wearing the AR glasses 300. The operation unit 303 is, for example, a remote controller, a button, a switch, a stick, a touchpad, or the like that allows the user to operate the AR glasses 300 and input various information. Note that one or both of an operation detection unit 313 and a gesture detection unit 316, which will be described later, may be used together with or instead of the operation unit 303.

[0030] The sensor 305 is for detecting the behavior of the AR glasses 300 in accordance with the behavior of a user wearing the AR glasses 300. The sensor 305 is, for example, a 6DoF (Six-Degrees of Freedom) sensor. The sensor 305, which is a 6DoF sensor, can detect acceleration in the directions of the X-axis, Y-axis, and Z-axis, as well as rotational angular acceleration around the X-axis, Y-axis, and Z-axis. Such a sensor 305 can detect changes in the position and orientation of the AR glasses 300 in three-dimensional space.

[0031] The glass storage unit 306 stores various data related to the operation control of the AR glasses 300. For example, the glass storage unit 306 stores coordinate data within the elevator shaft 1 and three-dimensional model data of objects transmitted from the cloud server 200. The glass storage unit 306 also stores images captured by the camera 304. The glass processing unit 310 has a function of performing calculations and the like mainly using the data stored in the glass storage unit 306, and executing predetermined processing required for the operation control of the AR glasses 300.

[0032] Communication between the terminal device 100 and the cloud server 200 may be wired or wireless. On the other hand, communication between the AR glasses 300 and the cloud server 200 is preferably wireless. One or both of the communication between the terminal device 100 and the cloud server 200 and the communication between the AR glasses 300 and the cloud server 200 may be performed via a communication network such as the Internet. In communication via a communication network, there are usually multiple communication nodes on the communication path. In this case, communication between the AR glasses 300 and the communication node closest to the AR glasses 300 is preferably wireless.

[0033] The hoistway equipment layout assistance system according to this embodiment may or may not include the terminal device 100. When the hoistway equipment layout assistance system includes the terminal device 100, some or all of the functions described below that are provided in the glass processing unit 310 of the AR glasses 300 may be provided in the terminal processing unit 110 of the terminal device 100. In this way, the data processing load can be distributed, and the processing load on the glass processing unit 310 of the AR glasses 300 can be reduced. This can contribute to making the AR glasses 300 smaller and lighter.

[0034] 2, the glass processing unit 310 has, as its functional configuration, an image acquisition unit 311, a data acquisition unit 312, a behavior detection unit 315, a display control unit 320, an operation detection unit 313, and an update unit 314. The image acquisition unit 311 acquires an image of the inside of the elevator hoistway 1 in real space. When a user wearing the AR glasses 300 turns their face toward the elevator hoistway 1 in real space, the camera 304 captures an image of the inside of the hoistway 1 in real space. The image acquisition unit 311 acquires the image captured by the camera 304, i.e., the image of the inside of the elevator hoistway 1 in real space.

[0035] The data acquisition unit 312 acquires coordinate data within the hoistway 1 and three-dimensional model data of objects installed within the hoistway 1 from the data management unit 202 of the cloud server 200. The server communication unit 201 of the cloud server 200 transmits the coordinate data within the hoistway 1 and the three-dimensional model data of objects installed within the hoistway 1, which are stored in the data management unit 202, to the AR glasses 300. The glass communication unit 301 of the AR glasses 300 receives the coordinate data and three-dimensional model data transmitted from the cloud server 200. Then, the data acquisition unit 312 acquires the coordinate data and three-dimensional model data received by the glass communication unit 301.

[0036] The behavior detection unit 315 detects the behavior of the AR glasses 300. Behavior of the AR glasses 300 also occurs in the AR glasses 300 in accordance with the behavior of the user wearing the AR glasses 300. The behavior detection unit 315 uses the detection results of the sensor 305 of the AR glasses 300 to detect relative changes in the position of the AR glasses 300, i.e., movement, and changes in the orientation of the AR glasses 300, as the behavior of the AR glasses 300.

[0037] 1, the elevator shaft equipment layout support system further includes a first beacon terminal 410 and a second beacon terminal 420. Each of the first beacon terminal 410 and the second beacon terminal 420 indicates a reference position in real space. The first beacon terminal 410 includes a first beacon transmitter 411. The first beacon transmitter 411 transmits a preset first beacon signal. The second beacon terminal 420 includes a second beacon transmitter 421. The second beacon transmitter 421 transmits a preset second beacon signal.

[0038] The glass processing unit 310 further includes a coordinate alignment unit 321 that performs coordinate alignment. The coordinate alignment is performed by using information about the reference positions indicated by the first beacon terminal 410 and the second beacon terminal 420 to match the position in the elevator shaft 1 in real space with the position on the coordinate axis of the coordinate data acquired from the data management unit. Specifically, for example, the first beacon terminal 410 and the second beacon terminal 420 are arranged vertically in the elevator shaft 1 in real space. That is, the first beacon terminal 410 and the second beacon terminal 420 have different vertical positions and the same horizontal position in real space. These first beacon terminal 410 and second beacon terminal 420 determine the Z-axis direction. Also, for example, the first beacon terminal 410 determines the coordinate origin.

[0039] A user wearing the AR glasses 300 first moves to a position where the first beacon terminal 410 is within the field of view of the camera 304. When the glass communication unit 301 of the AR glasses 300 receives the first beacon signal transmitted from the first beacon terminal 410, the coordinate alignment unit 321 identifies the position of the AR glasses 300 relative to the first beacon terminal 410 in real space using the reception strength of the first beacon signal, etc. Also, when the user operates, for example, the operation unit 303 to specify the position of the first beacon terminal 410 in an image captured by the camera 304, the position of the AR glasses 300 relative to the first beacon terminal 410 in the image is identified. Then, the coordinate alignment unit 321 aligns the reference position indicated by the first beacon terminal 410 in the real space and the image. As a result, the coordinate origin is aligned.

[0040] Next, the user wearing the AR glasses 300 moves to a position where the second beacon terminal 420 is within the field of view of the camera 304. Then, by a similar procedure, the coordinate alignment unit 321 aligns the reference position indicated by the second beacon terminal 420 in the real space and in the image. As described above, the first beacon terminal 410 and the second beacon terminal 420 are aligned in the vertical direction in the elevator shaft 1 in the real space. Therefore, the coordinate alignment unit 321 can identify the Z-axis direction from the reference positions indicated by the first beacon terminal 410 and the second beacon terminal 420, respectively.

[0041] The first beacon terminal 410 and the second beacon terminal 420 are arranged vertically in the elevator shaft 1 in real space. On this premise, even if only one of the first beacon terminal 410 and the second beacon terminal 420 is within the field of view of the camera 304 and the other is outside the field of view of the camera 304, the Z-axis direction can be identified as long as the AR glasses 300 can receive a beacon signal from the other beacon terminal. For example, when the position of the user wearing the AR glasses 300 is a position where the first beacon terminal 410 is within the field of view of the camera 304 and the glass communication unit 301 of the AR glasses 300 can receive the second beacon signal transmitted from the second beacon terminal 420, the coordinate alignment unit 321 can identify the Z-axis direction using the reception strength of the first beacon signal and the second beacon signal, etc.

[0042] The X-axis direction is specified, for example, using a direction arranged horizontally in real space. For example, the coordinate alignment unit 321 aligns the X-axis in the image with the lower edge of the opening of the elevator shaft 1, which serves as the entrance / exit on the landing side. The remaining Y-axis direction can be specified as a direction perpendicular to both the X-axis and Z-axis directions. In this way, the coordinate alignment unit 321 performs coordinate alignment to match the position in the elevator shaft 1 in real space with the position on the coordinate axes of the coordinate data acquired from the data management unit.

[0043] The display control unit 320 causes the display unit 302 to display an augmented reality image generated by superimposing the 3D model data of the object on an image of the interior of the elevator shaft 1 in real space. The display control unit 320 generates an augmented reality image by superimposing the 3D model data of the object acquired by the data acquisition unit 312 on the image of the interior of the elevator shaft 1 in real space acquired by the image acquisition unit 311. The display control unit 320 then causes the display unit 302 to display the generated augmented reality image. In this way, the display unit 302 displays an augmented reality image in which the 3D model data of the object acquired from the data management unit 202 is superimposed on the image of the interior of the elevator shaft 1 in real space. At this time, the display control unit 320 can correctly superimpose the 3D model data of the object on the image of the interior of the elevator shaft 1 in real space by using the coordinate axes aligned by the coordinate alignment unit 321.

[0044] Furthermore, the display control unit 320 changes the generated augmented reality image in accordance with the behavior of the AR glasses 300 detected by the behavior detection unit 315, i.e., changes in the position and orientation of the display unit 302 provided on the AR glasses 300. In other words, the display unit 302 changes the augmented reality image to be displayed in accordance with changes in the position and orientation of the display unit 302. This makes it possible to display on the display unit 302 an augmented reality image that matches the field of view that would be visible to the user if the user were not wearing the AR glasses 300, in accordance with the movement, orientation, posture, etc. of the user wearing the AR glasses 300.

[0045] The operation detection unit 313 detects a selection operation for individually selecting an object within the augmented reality image and a movement operation for the selected object. The user can perform a selection operation and a movement operation, for example, by operating the operation unit 303 of the AR glasses 300. A selection operation is an operation for individually selecting an object within the augmented reality image displayed on the display unit 302. A movement operation is an operation for moving an object selected by the selection operation within the augmented reality image displayed on the display unit 302. The operation detection unit 313 detects a selection operation for individually selecting an object within the augmented reality image and a movement operation for the selected object.

[0046] The update unit 314 updates the 3D model data of the object to reflect changes in the orientation and position of the object due to the move operation. Updating refers to saving new information at the time of execution without overwriting previous information. After the revision is complete, unnecessary information may be deleted. The update unit 314 updates the 3D model data of the object that has been moved to data on the orientation and position of the object after the move. Note that if the object that has been moved is deformable, such as a flexible wiring, the update unit 314 updates not only the orientation and position of the 3D model data of the object but also the shape to that after the move operation.

[0047] The glass communication unit 301 acquires the 3D model data of the object updated by the update unit 314. Then, the glass communication unit 301 transmits the updated 3D model data of the object to the cloud server 200. The server communication unit 201 of the cloud server 200 receives the 3D model data of the object transmitted from the AR glasses 300. The glass communication unit 301 also transmits images stored in the glass storage unit 306 to the cloud server 200. The server communication unit 201 of the cloud server 200 receives the images transmitted from the AR glasses 300. The data management unit 202 of the cloud server 200 updates the stored 3D model data to the 3D model data received by the server communication unit 201. In this sense, the glass communication unit 301 is a transmission unit that transmits the updated 3D model data of the object to the data management unit 202.

[0048] With the hoistway equipment layout support system configured as described above, it is possible to confirm before installation whether each piece of equipment specified in the 3D model data created during design can be placed as specified on site. If the equipment cannot be placed as specified, the 3D model data can be manipulated on site to change it to an optimal layout that takes into account the on-site conditions, and the changes can be reflected in the 3D model data. This allows on-site workers to easily check the layout of equipment, etc. to be installed in the elevator hoistway as designed in advance, and also makes it easy to modify the 3D model data of equipment, etc. according to the actual conditions in the hoistway, thereby supporting the equipment layout work in the hoistway.

[0049] In one modified example of the AR glasses 300 according to this embodiment, the glasses processing unit 310 may further include a gesture detection unit 316, as shown in FIG. 2 . A user wearing the AR glasses 300 can hold out their hand in front of them, allowing the camera 304 of the AR glasses 300 to capture an image of the user's hand. Note that the user's hand may be a bare hand, or may be a hand wearing, for example, work gloves or gloves. The gesture detection unit 316 detects a gesture made by the user's hand captured by the camera 304. For example, the gesture detection unit 316 extracts the user's hand from the image captured by the camera 304. The gesture detection unit 316 then detects a gesture made by the extracted user's hand.

[0050] In this case, the operation detection unit 313 can detect a selection operation and a movement operation by a gesture. The operation detection unit 313 determines whether the gesture detected by the gesture detection unit 316 is a specific gesture representing a selection operation or a movement operation. If the gesture detected by the gesture detection unit 316 is a specific gesture representing a selection operation, the operation detection unit 313 detects the selection operation. If the gesture detected by the gesture detection unit 316 is a specific gesture representing a movement operation, the operation detection unit 313 detects the movement operation.

[0051] Specifically, for example, a gesture representing a selection operation is a gesture of pointing at an object in an augmented reality image with a finger. Furthermore, a gesture representing a movement operation is a gesture of dragging a selected object within the augmented reality image. This allows the user to intuitively perform selection and movement operations while viewing the augmented reality image.

[0052] In one variation of the AR glasses 300 according to this embodiment, when a move operation is performed on a first object selected by a selection operation within the augmented reality image, the update unit 314 updates the three-dimensional model data of the object to reflect changes in the orientation and position of the first object due to the move operation, and also updates the three-dimensional model data of the second object to reflect changes in the orientation and position of a second object associated with the first object due to the move operation on the first object. For example, if the first object is a control panel, an example of the second object associated with it would be wiring connected to the control panel.

[0053] In this way, the device and the wiring connected to the device can be moved simultaneously. This improves the efficiency of the moving operation. Furthermore, if the device and the wiring connected to the device are moved separately, an unrealizable arrangement that ignores the associated relationships may occur. According to this modified example, it is possible to prevent such an unrealizable arrangement that ignores the associated relationships.

[0054] In one variation of the AR glasses 300 according to this embodiment, the glass processing unit 310 may further include a model data generation unit 317, as shown in FIG. 2 . The model data generation unit 317 detects an object from an image of the interior of the elevator shaft 1 in real space and generates three-dimensional model data of the object. In this case, the display control unit 320 generates an augmented reality image by superimposing the three-dimensional model data of the object acquired from the data management unit 202 and the three-dimensional model data of the object generated by the model data generation unit 317 on the image of the interior of the elevator shaft 1 in real space. In other words, the display unit 302 displays an augmented reality image generated by superimposing the three-dimensional model data of the object acquired from the data management unit 202 and the three-dimensional model data of the object generated by the model data generation unit 317 on the image of the interior of the elevator shaft 1 in real space. In this way, if an object not included in the three-dimensional model data managed by the data management unit 202 is present in the elevator shaft 1 in real space, the object can be easily captured as three-dimensional model data. Such data reflecting the actual conditions inside the elevator shaft 1 can be utilized not only when installing an elevator, but also when remodeling or modernizing the elevator.

[0055] In one variation of the AR glasses 300 according to this embodiment, the glasses processing unit 310 may further include a distance measurement unit 318, as shown in FIG. 2 . The distance measurement unit 318 measures the distance between objects in the augmented reality image. Specifically, for example, when one object is selected by a selection operation and another object is further selected by a selection operation, the distance measurement unit 318 measures the distance between the two selected objects. The distance measurement unit 318 can calculate the distance between the objects based on, for example, three-dimensional model data of the objects. In this way, a user wearing the AR glasses 300 can easily check the distance between equipment, wiring, and the like to be installed at the work site. This allows clearance between equipment to be easily secured, improving work efficiency, and the like.

[0056] In one variation of the AR glasses 300 according to this embodiment, the display unit 302 may be capable of displaying attribute information of an object superimposed on an augmented reality image. The attribute information of the object includes the type, name, material, color, etc. of the object. The display unit 302 may also be capable of displaying numerical information of three-dimensional model data of the object superimposed on the augmented reality image. In this way, a user wearing the AR glasses 300 can easily check information about the equipment, wiring, etc. to be installed at the work site. This can improve work efficiency, etc.

[0057] Furthermore, in this case, the glass processing unit 310 may further include an attribute information acquisition unit 319, as shown in FIG. 2 . The attribute information acquisition unit 319 inputs attribute information of an object. For example, a user can input attribute information of an object by selecting an object in an augmented reality image and operating the operation unit 303 or using a gesture. The attribute information acquisition unit 319 acquires the attribute information of the object input in this manner. In this case, the display unit 302 may display the attribute information of the object acquired by the attribute information acquisition unit 319 superimposed on the augmented reality image. Furthermore, the glass communication unit 301 may transmit the attribute information of the object acquired by the attribute information acquisition unit 319 to the cloud server 200. Then, the data management unit 202 may store the attribute information of the object received by the server communication unit 201 in association with three-dimensional model data of the object.

[0058] In one modified example of the hoistway equipment layout support system according to this embodiment, as shown in Fig. 3, the system may further include one of a BIM data generation unit 111 and a two-dimensional drawing data generation unit 112. In the example shown in Fig. 3, the terminal processing unit 110 of the terminal device 100 includes the BIM data generation unit 111 and the two-dimensional drawing data generation unit 112. Note that the BIM data generation unit 111 and the two-dimensional drawing data generation unit 112 are not limited to being provided in the terminal device 100. Alternatively, for example, the BIM data generation unit 111 and the two-dimensional drawing data generation unit 112 may be provided in the cloud server 200, the AR glasses 300, or the like.

[0059] The BIM data generation unit 111 generates BIM (Building Information Modeling) data from the 3D model data of the object. In this case, for example, the server communication unit 201 of the cloud server 200 transmits to the terminal device 100 coordinate data within the hoistway 1 and 3D model data of objects to be installed in the hoistway 1, which are stored in the data management unit 202. The terminal communication unit 101 of the terminal device 100 receives the coordinate data and 3D model data transmitted from the cloud server 200. Then, the BIM data generation unit 111 generates BIM data from the coordinate data and 3D model data received by the terminal communication unit 101. By including such a BIM data generation unit 111, BIM data that reflects the on-site situation can be generated from the 3D data as needed, and the BIM data can be provided to contractors involved in the construction of buildings, including elevator installation.

[0060] The 2D drawing data generator 112 also generates 2D drawing data from the 3D model data of the object. In this case, as with BIM data, the 2D drawing data generator 112 acquires coordinate data within the hoistway 1 and 3D model data of objects to be installed within the hoistway 1, which are stored in the data management unit 202 of the cloud server 200, and generates 2D drawing data from the acquired coordinate data and 3D model data. By providing this 2D drawing data generator 112, 2D drawing data reflecting the on-site situation can be generated from the 3D data as needed, and drawings reflecting the on-site situation can be provided to contractors, clients, architectural firms, etc. involved in building construction, including elevator installation. Drawings to be attached to documents submitted in legally required procedures can also be easily created.

[0061] Next, an example of operation in each processing step of the elevator shaft equipment layout support system configured as above will be described. First, the beacon terminal position identification process will be described with reference to the flow chart of Figure 4 together with Figures 5 to 11. First, in step S100, when the AR glasses 300, the first beacon terminal 410, and the second beacon terminal 420 are turned on, in the following step S101, as shown in Figure 5, the glass communication unit 301 of the AR glasses 300 starts communication and connects to the Internet.

[0062] In the following step S102, the display control unit 320 of the AR glasses 300 displays a main operation screen (menu screen) on the display unit 302. An example of the main operation screen displayed on the display unit 302 is shown in FIG. 6. On this main operation screen, by pressing the "menu" button, it is possible to, for example, select a beacon terminal, confirm the position of the beacon terminal, cancel a beacon terminal position that has been confirmed once, display a help screen, etc. After step S102, the hoistway equipment layout assistance system then performs the process of step S103.

[0063] In step S103, the glass processing unit 310 determines whether the property in which the elevator shaft 1 of the target elevator is to be installed and the elevator car number have been specified by operation of the operation unit 303 or gesture. If the property and car number have not been specified, the glass processing unit 310 returns to step S102 and continues processing. On the other hand, if the property and car number have been specified, the glass processing unit 310 then performs processing of step S104.

[0064] In step S104, the display control unit 320 of the AR glasses 300 displays a beacon terminal (indoor positioning device) setting screen on the display unit 302. An example of the beacon terminal setting screen displayed on the display unit 302 is shown in FIG. 7. In the following step S105, the glass processing unit 310 determines whether or not a beacon terminal has been detected, that is, whether or not the glass communication unit 301 has received a beacon signal transmitted from the beacon terminal, based on the received reception strength. If a beacon terminal is not detected, the glass processing unit 310 returns to step S104 and continues processing. On the other hand, if a beacon terminal is detected, the glass processing unit 310 then performs processing in step S106.

[0065] In step S106, the display control unit 320 of the AR glasses 300 displays a beacon terminal (indoor positioning device) selection screen on the display unit 302. An example of the beacon terminal selection screen displayed on the display unit 302 is shown in FIG. 8. In the example described here, as shown in FIG. 9, it is assumed that a first beacon terminal 410 and a second beacon terminal 420 are installed in real space, i.e., in the actual elevator shaft 1. Then, when the beacon terminal to be used this time is selected by the user's operation, in the subsequent step S107, the display control unit 320 of the AR glasses 300 displays three-dimensional coordinate data within the elevator shaft 1 on the display unit 302.

[0066] At this time, the glass communication unit 301 accesses the cloud server 200 and requests coordinate data within the hoistway 1. In response to this request, the server communication unit 201 of the cloud server 200 acquires the coordinate data within the hoistway 1 stored in the data management unit 202 and transmits it to the AR glasses 300. Then, the display control unit 320 causes the display unit 302 to display the three-dimensional coordinate data within the hoistway 1 received by the glass communication unit 301. An example of a three-dimensional coordinate data display screen displayed on the display unit 302 is shown in FIG. 10. After step S107, the glass processing unit 310 then performs processing of step S108.

[0067] In step S108, for example, as shown in FIG. 11, the position of the beacon terminal in the three-dimensional coordinate data in the elevator shaft 1 displayed on the display unit 302 is specified. Then, in the following step S109, the glass processing unit 310 determines whether or not the positions in real space and the positions in the three-dimensional coordinate data match for all beacon terminals selected in step S106. If the positions in real space and the positions in the three-dimensional coordinate data do not match for all beacon terminals, the glass processing unit 310 returns to step S106 and continues processing. On the other hand, if the positions in real space and the positions in the three-dimensional coordinate data match for all beacon terminals, the series of beacon terminal position identification processes is completed.

[0068] Next, the coordinate alignment process will be described with reference to the flow chart of FIG. 12 together with FIGS. 13 to 15. Here, an example will be described in which the beacon terminals are installed in a vertical line. First, in step S200, as shown in FIG. 13, the field worker (user) moves to the floor where the second beacon terminal 420 is installed. In the following step S201, the user reconfirms the position of the second beacon terminal 420 in the real space and the position in the three-dimensional coordinate data near the second beacon terminal 420, and if necessary, corrects the position designation. Then, when the correction is completed, the glass processing unit 310 next performs the process of step S202.

[0069] In step S202, the image acquisition unit 311 acquires an image of the landing opening of the hoistway 1 captured by the camera 304. At the same time, the coordinate alignment unit 321 acquires detection axis information of the sensor 305 ( FIG. 14 ). Then, in the following step S203, the coordinate alignment unit 321 adjusts the coordinate origin and the Z-axis direction of the sensor 305 based on the reference position indicated by the beacon terminal. The coordinate alignment unit 321 also aligns the X-axis direction of the sensor 305 with the lower edge of the image of the landing opening of the hoistway 1 ( FIG. 15 ). In this way, the coordinate alignment unit 321 corrects the detection axes (X-axis, Y-axis, and Z-axis) of the sensor 305.

[0070] Next, the display process of an augmented reality image including an object will be described with reference to the flow chart of Fig. 16 together with Fig. 17 to Fig. 19. If the property, the car number, and the beacon terminal have been designated in step S300, the glass processing unit 310 determines in the following step S301 whether or not three-dimensional data display has been selected by operation of the operation unit 303 or gesture (Fig. 17). If three-dimensional data display has been selected, the glass processing unit 310 then performs the process of step S302.

[0071] In step S302, the glass communication unit 301 accesses the cloud server 200 and requests 3D model data of the object. In response to this request, the server communication unit 201 of the cloud server 200 acquires the 3D model data of the object stored in the data management unit 202 and transmits it to the AR glasses 300. The display control unit 320 then aligns the position and orientation of the display of the 3D model data of the object received by the glass communication unit 301 with the aligned coordinates.

[0072] In the next step S303, the display control unit 320 causes the display unit 302 to display an augmented reality image generated by superimposing the 3D model data of the object on an image of the interior of the elevator shaft 1 in real space. This makes it possible to display an augmented reality image in which the 3D model data of the object is superimposed on an image of the interior of the elevator shaft 1 in real space in accordance with the behavior of the user wearing the AR glasses 300.

[0073] For example, as shown in Fig. 18, when a user wearing AR glasses 300 faces the front wall of the hoistway 1 from the work floor, an augmented reality image is displayed that virtually recreates the situation in which various devices, wiring, etc. that are scheduled to be installed on the front wall of the hoistway 1 are actually installed, according to the user's field of view at that time. Also, as shown in Fig. 19, when a user wearing AR glasses 300 faces the landing side from the work floor in the hoistway 1, an augmented reality image is displayed that virtually recreates the situation in which various devices, wiring, etc. that are scheduled to be installed on the landing side of the hoistway 1 are actually installed, according to the user's field of view at that time.

[0074] Next, the process of moving an object within an augmented reality image will be described with reference to the flow chart of FIG. 20 together with FIGS. 21 to 24. In step S400, an augmented reality image in which 3D model data of the object is superimposed on an image of the elevator shaft 1 in real space is displayed on the display unit 302 of the AR glasses 300. In step S401, the user checks through the AR glasses 300 whether objects such as equipment and wiring interfere with each other. Then, in step S402, the user determines whether no particularly interfering equipment, wiring, etc. are found and whether the current design layout is possible. If the current design layout is possible, the process ends. On the other hand, for example, as shown in FIG. 21, if interfering equipment, wiring, etc. are found, the glasses processing unit 310 next performs the process of step S403.

[0075] In step S403, the user selects a 3D data operation from the operation menu displayed on the display unit 302 by the display control unit 320 ( FIG. 22 ). The user then selects an object within the augmented reality image to move it to eliminate interference ( FIG. 23 ). To enable selection of the overlapping object at the back, the user may select the object from a list of object names, for example. The display control unit 320 also displays the movable range of the selected object within the augmented reality image. The movable range of the selected object can be determined, for example, by the excess length of the wiring connected to the object. After step S403, the glass processing unit 310 then performs the process of step S404.

[0076] In step S404, the user moves an object within the augmented reality image. In the following step S405, the display control unit 320 determines whether the moved object is within the object's movable range (the range of the extra length of the connected wiring). If the moved object is within the object's movable range, the glass processing unit 310 then performs processing in step S406. In step S406, the display control unit 320 generates an augmented reality image after the object has been moved and displays it on the display unit 302 ( FIG. 24 ). The update unit 314 then updates the 3D model data of the moved object to reflect the state of the object after the move operation.

[0077] Next, the process of adding an object, such as an obstacle, that caused a change in object placement will be described with reference to the flowchart of FIG. 25 together with FIGS. 26 to 29 . In step S500, if the placement of an object (such as equipment or wiring) is changed, the glass processing unit 310 then performs the process of step S501. In step S501, the user checks and identifies the placement, size, and material of the obstacle that caused the change in object placement using the augmented reality image or the like displayed on the display unit 302. In the subsequent step S502, the user operates the operation menu displayed on the display unit 302 by the display control unit 320 to select "Add / Delete 3D Data." In step S503, the user inputs information such as the size and material of the obstacle in the augmented reality image ( FIG. 26 ). After step S503, the glass processing unit 310 then performs the process of step S504.

[0078] In step S504, the display control unit 320 generates an augmented reality image in which an obstacle object is additionally arranged based on the information input in step S503, and displays the image on the display unit 302 ( FIG. 27 ). When two objects, including the added obstacle, are selected in the augmented reality image, the distance measurement unit 318 measures the distance between the two objects, including the obstacle. The display control unit 320 displays the distance between the obstacle and the object on the display unit 302 ( FIG. 28 ). This allows the user to confirm whether the distance between the obstacle and the object satisfies the required clearance conditions. Then, in the following step S505, when the user performs a confirmation operation, the object addition process ends. Note that the object addition in step S503 may be performed by the model data generation unit 317 detecting the obstacle from an image of the elevator shaft 1 in real space and generating three-dimensional model data of the obstacle, as shown in FIG. 29 .

[0079] Next, the process of confirming the changes will be described with reference to the flow chart of FIG. 30 together with FIG. 31 . First, in step S600, as shown in FIG. 31 , a user wearing AR glasses 300 looks around inside the elevator shaft 1 and confirms the contents of the changed augmented reality image displayed on the display unit 302. Then, in the following step S601, if the object can be placed in the position displayed in the changed augmented reality image, the series of confirmation processes ends. On the other hand, if the object cannot be placed in the position displayed in the changed augmented reality image (step S602), the process of moving the object in the augmented reality image ( FIG. 20 ) and the process of adding an object such as an obstacle that caused the position change ( FIG. 25 ) are performed again.

[0080] Next, the equipment arrangement process at another work site will be described with reference to the flow chart of Fig. 32 together with Fig. 33 and Fig. 34. First, in step S700, if there is another work site, in step S701, a beacon terminal is installed at the other work site as necessary. Note that it is desirable to install a beacon terminal at another work site, since it is expected that the position accuracy will be improved by installing the beacon terminal as close to the work site as possible. In this case, for example, as shown in Fig. 33, the second beacon terminal 420 installed at the previous work site may be moved, or as shown in Fig. 34, a new third beacon terminal 430 may be added and installed.

[0081] Then, in steps S702 to S707, the object placement is confirmed and corrected using the augmented reality image, in the same manner as at the previous work site. That is, in step S702, a beacon terminal position identification process (FIG. 4) is performed, and in the subsequent step S703, a coordinate alignment process (FIG. 12) is performed. In addition, in the subsequent step S704, an augmented reality image including the object is displayed (FIG. 16), and in the subsequent step S705, an object movement process (FIG. 20) is performed within the augmented reality image. Then, in the subsequent step S706, an object addition process (FIG. 25) is performed to add an obstacle or other object that caused the placement change, and in the subsequent step S707, a process to confirm the changes (FIG. 30) is performed.

[0082] On the other hand, if there are no other work sites and work at all sites has been completed in step S700, the hoistway equipment layout assistance system then performs the process of step S708. If there is a revision by the user in step S708, the glasses communication unit 301 transmits the 3D model data of the object updated by the update unit 314 and the user's revision instruction to the cloud server 200. Then, the server communication unit 201 of the cloud server 200 receives the 3D model data of the object transmitted from the AR glasses 300 (step S709).

[0083] In the following step S710, the data management unit 202 of the cloud server 200 updates the stored 3D model data to the 3D model data received by the server communication unit 201. On the other hand, if the revision process has not been performed on the 3D model data in step S708, in step S711 the data management unit 202 temporarily stores the 3D model data until the revision process is performed.

[0084] Next, the revision process for 3D model data will be described with reference to the flow chart in Fig. 35 together with Figs. 36 and 37. First, in step S800, the user displays and views the 3D model data. Then, in the following step S801, if the revised 3D model data does not contain any revision details or revision history, the user adds the revision details and revision history to the revised 3D model data in step S802.

[0085] This revision process may be performed using a PC or the like at the work site or at a location separate from the work site, as shown in Fig. 36. Alternatively, as shown in Fig. 37, the revision process may be performed by a user wearing AR glasses 300 at the work site by checking an augmented reality image on which the changed three-dimensional model data is superimposed.

[0086] Next, a first modified example of the hoistway equipment layout assistance system according to this embodiment will be described with reference to Fig. 38. In this first modified example, the hoistway equipment layout assistance system further includes an indoor positioning device 400. The indoor positioning device 400 is a parent device of the beacon terminal. In the illustrated example, a first beacon terminal 410 and a second beacon terminal 420 are provided as child devices. The indoor positioning device 400 includes a positioning device communication unit 401, a positioning device processing unit 402, and a positioning device storage unit 403. The first beacon terminal 410 includes a first beacon communication unit 412. The second beacon terminal 420 includes a second beacon communication unit 422.

[0087] The indoor positioning device 400 and the first beacon terminal 410 can communicate with each other via a positioning device communication unit 401 and a first beacon communication unit 412. The indoor positioning device 400 and the second beacon terminal 420 can communicate with each other via a positioning device communication unit 401 and a second beacon communication unit 422. The indoor positioning device 400 and the AR glasses 300 can communicate with each other via a positioning device communication unit 401 and a glass communication unit 301. The indoor positioning device 400 and the cloud server 200 can communicate with each other via a positioning device communication unit 401 and a server communication unit 201.

[0088] Although not shown in the figure, the first beacon terminal 410 and the second beacon terminal 420 can each transmit a beacon signal. The first beacon terminal 410 and the second beacon terminal 420 transmit and stop the beacon signal according to a control signal transmitted from the indoor positioning device 400. In this way, the indoor positioning device 400, which is the parent device, controls the operation of the first beacon terminal 410 and the second beacon terminal 420, which are child devices.

[0089] In this first modified example, the positioning device processing unit 402 of the indoor positioning device 400 realizes some or all of the functions of the image acquisition unit 311, data acquisition unit 312, operation detection unit 313, update unit 314, behavior detection unit 315, gesture detection unit 316, model data generation unit 317, distance measurement unit 318, attribute information acquisition unit 319, display control unit 320, and coordinate alignment unit 321 described above. Furthermore, the positioning device communication unit 401 of the indoor positioning device 400 realizes the functions of the transmission unit described above. In the illustrated example, the AR glasses 300 are not provided with a glass storage unit 306 or a glass processing unit 310. Therefore, the positioning device processing unit 402 of the indoor positioning device 400 realizes the functions of all of the units described above. However, the AR glasses 300 may have the glass storage unit 306 and the glass processing unit 310 provided therein, and only some of the functions of the units described above may be provided in the indoor positioning device 400.

[0090] As described above, in this first modified example, the indoor positioning device 400 includes some or all of the image acquisition unit 311, data acquisition unit 312, operation detection unit 313, and update unit 314, as well as the aforementioned transmission unit. This makes it possible to distribute the load associated with relatively high-load three-dimensional data processing, etc., and reduce the processing load on the glasses processing unit 310 of the AR glasses 300. This can contribute to making the AR glasses 300 smaller and lighter.

[0091] The hoistway equipment layout assistance system according to this embodiment may or may not include the terminal device 100. When the hoistway equipment layout assistance system includes the terminal device 100, some or all of the functions described below that are provided in the glass processing unit 310 of the AR glasses 300 may be provided in the terminal processing unit 110 of the terminal device 100. In this way, the data processing load can be distributed, and the processing load on the glass processing unit 310 of the AR glasses 300 can be reduced. This can contribute to making the AR glasses 300 smaller and lighter.

[0092] Next, a second modified example of the hoistway equipment layout support system according to this embodiment will be described with reference to Figures 39 to 42. In this second modified example, the hoistway equipment layout support system further includes VR glasses 500. Figure 39 shows an example configuration of the second modified example of the hoistway equipment layout support system. In the illustrated example, the hoistway equipment layout support system includes a cloud server 200, AR glasses 300, and VR glasses 500.

[0093] The VR glasses 500 are glasses-type, helmet-type, or other devices worn on the user's face or head. VR is an abbreviation for "Virtual Reality." The VR glasses 500 basically have a configuration in which the camera 304 is removed from the AR glasses 300. Although not shown here, the VR glasses 500 may be equipped with a camera that captures the movement of the user's eyeballs as a function other than the camera 304. This function fulfills the role of the VR glasses operation unit 520, which will be described later. That is, the VR glasses 500 include a glass communication unit 301, a display unit 302, a sensor 305, a glass memory unit 306, and a VR glasses processing unit 510. The glass communication unit 301, display unit 302, sensor 305, and glass memory unit 306 of the VR glasses 500 each have the same functions as the glass communication unit 301, display unit 302, sensor 305, and glass memory unit 306 of the AR glasses 300, respectively. In the illustrated example, the VR glasses operation unit 520 is provided as an accessory device to the VR glasses 500. The VR glasses operation unit 520 corresponds to the operation unit 303 of the AR glasses 300. The VR glasses operation unit 520 is, for example, a remote controller, a button, a switch, a stick, a touchpad, etc., which the user uses to operate the VR glasses 500 and input various types of information.

[0094] The user of the VR glasses 500 does not need to be in the actual elevator shaft 1, and is usually in a location different from the actual elevator shaft 1. For this reason, the user of the VR glasses 500 does not need to move around much, and therefore communication between the VR glasses 500 and the cloud server 200 may be wired or wireless. Note that communication between the VR glasses 500 and the cloud server 200 may be performed via a communication network such as the Internet.

[0095] The glass storage unit 306 of the VR glasses 500 stores various data related to the operation control of the VR glasses 500. For example, the glass storage unit 306 of the VR glasses 500 stores coordinate data and three-dimensional model data of objects inside the hoistway 1 transmitted from the cloud server 200. Furthermore, as will be described later, an image of the inside of the hoistway 1 captured by a camera may be transmitted from the cloud server 200 to the VR glasses 500. In this case, the glass storage unit 306 of the VR glasses 500 further stores the image of the inside of the hoistway 1 transmitted from the cloud server 200.

[0096] The VR glasses processing unit 510 has a function of performing calculations and the like using data stored mainly in the glasses storage unit 306 of the VR glasses 500, and executing predetermined processing necessary for controlling the operation of the VR glasses 500. As shown in Fig. 40 , the VR glasses processing unit 510 has, as its functional configuration, a data acquisition unit 312, a behavior detection unit 315, a display control unit 320, an operation detection unit 313, and an update unit 314.

[0097] The functions of the data acquisition unit 312 and the behavior detection unit 315 of the VR glasses processing unit 510 are equivalent to the functions of the data acquisition unit 312 and the behavior detection unit 315 of the AR glasses 300.

[0098] The display control unit 320 of the VR glasses processing unit 510 causes the display unit 302 of the VR glasses 500 to display a virtual reality image generated based on the three-dimensional model data of the object. The display control unit 320 of the VR glasses processing unit 510 generates a virtual reality image based on the three-dimensional model data of the object acquired by the data acquisition unit 312 of the VR glasses processing unit 510. The display control unit 320 of the VR glasses processing unit 510 then causes the generated virtual reality image to be displayed on the display unit 302 of the VR glasses 500. In this way, the display unit 302 of the VR glasses 500 displays the virtual reality image generated based on the three-dimensional model data of the object acquired from the data management unit 202.

[0099] Furthermore, the display control unit 320 of the VR glasses processing unit 510 changes the virtual reality image that it generates in response to the behavior of the VR glasses 500 detected by the behavior detection unit 315 of the VR glasses processing unit 510, i.e., changes in the position and orientation of the display unit 302 provided in the VR glasses 500. In other words, the display unit 302 of the VR glasses 500 changes the virtual reality image that it displays in response to changes in the position and orientation of the display unit 302. This makes it possible to display on the display unit 302 of the VR glasses 500 a virtual reality image that is tailored to the field of view of the user wearing the VR glasses 500 in response to the movement, orientation, posture, etc. of the user.

[0100] The operation detection unit 313 of the VR glasses processing unit 510 detects a selection operation for individually selecting an object within a virtual reality image and a movement operation for the selected object. The user can perform a selection operation and a movement operation, for example, by operating the VR glasses operation unit 520. A selection operation is an operation for individually selecting an object within a virtual reality image displayed on the display unit 302 of the VR glasses 500. A movement operation is an operation for moving an object selected by the selection operation within the virtual reality image displayed on the display unit 302 of the VR glasses 500. The operation detection unit 313 of the VR glasses processing unit 510 detects a selection operation for individually selecting an object within a virtual reality image and a movement operation for the selected object.

[0101] The update unit 314 of the VR glasses processing unit 510 updates the 3D model data of the object to reflect changes in the orientation and position of the object due to the movement operation. Updating refers to saving new information at the time of execution without overwriting previous information. After the revision is complete, unnecessary information may be deleted. The update unit 314 updates the 3D model data of the object that has been moved to data on the orientation and position of the object after the movement. Note that if the object that has been moved is deformable, such as a flexible wiring, the update unit 314 updates not only the orientation and position of the 3D model data of the object but also the shape to that after the movement operation.

[0102] The glass communication unit 301 of the VR glasses processing unit 510 acquires the 3D model data of the object updated by the update unit 314 of the VR glasses processing unit 510. Then, the glass communication unit 301 of the VR glasses processing unit 510 transmits the updated 3D model data of the object to the cloud server 200. The server communication unit 201 of the cloud server 200 receives the 3D model data of the object transmitted from the VR glasses 500. The data management unit 202 of the cloud server 200 updates the stored 3D model data to the 3D model data received by the server communication unit 201.

[0103] As described above, the data managed by the data management unit 202 may include images captured by a site worker using a camera. In this case, the data acquisition unit 312 of the VR glasses processing unit 510 may further acquire images of the interior of the hoistway 1 from the data management unit 202 of the cloud server 200. The server communication unit 201 of the cloud server 200 transmits the images of the interior of the hoistway 1 stored in the data management unit 202 to the VR glasses 500. The glass communication unit 301 of the VR glasses 500 receives the images of the interior of the hoistway 1 transmitted from the cloud server 200. Then, the data acquisition unit 312 of the VR glasses processing unit 510 acquires the images of the interior of the hoistway 1 received by the glass communication unit 301 of the VR glasses processing unit 510.

[0104] In this case, the display control unit 320 of the VR glasses processing unit 510 may cause the display unit 302 of the VR glasses 500 to display a virtual reality image generated by superimposing 3D model data of the object on an image of the interior of the elevator shaft 1 in virtual space. The display control unit 320 of the VR glasses processing unit 510 generates a virtual reality image by superimposing the 3D model data of the object on an image of the interior of the elevator shaft 1 in real space acquired by the data acquisition unit 312 of the VR glasses processing unit 510. The display control unit 320 of the VR glasses processing unit 510 then displays the generated virtual reality image on the display unit 302 of the VR glasses 500. In this way, the display unit 302 of the VR glasses 500 displays a virtual reality image in which the 3D model data of the object acquired from the data management unit 202 is superimposed on an image of the interior of the elevator shaft 1 in real space.

[0105] When the virtual reality image includes an image of the inside of the elevator shaft 1 in real space, the VR glasses processing unit 510 may have a model data generation unit 317 as shown in Fig. 40. The model data generation unit 317 of the VR glasses processing unit 510 has the same function as the model data generation unit 317 of the AR glasses 300.

[0106] The main difference between a virtual reality image including an image of the inside of the elevator shaft 1 in real space and an augmented reality image is as follows: The augmented reality image is displayed with coordinate alignment on the display unit 302 of the glasses worn by a user who is in the elevator shaft 1 in real space. In contrast, the user of the glasses on the display unit 302 that displays the virtual reality image does not need to be in the elevator shaft 1 in real space, and therefore coordinate alignment is not required.

[0107] 40, the VR glasses processing unit 510 in this second modified example may have a distance measurement unit 318. The distance measurement unit 318 of the VR glasses processing unit 510 has the same function as the distance measurement unit 318 of the AR glasses 300. In other words, the distance measurement unit 318 of the VR glasses processing unit 510 measures the distance between objects in a virtual reality image.

[0108] Furthermore, the display unit 302 of the VR glasses 500 in this second modified example may be capable of displaying attribute information of an object superimposed on a virtual reality image. The display unit 302 of the VR glasses 500 may be capable of displaying numerical information of three-dimensional model data of the object superimposed on a virtual reality image. In this case, the glasses processing unit 310 has an attribute information acquisition unit 319, as shown in FIG. 40 . The attribute information acquisition unit 319 has the same function as the attribute information acquisition unit 319 of the AR glasses 300.

[0109] Next, the equipment placement process in the second modified example will be described with reference to the flow chart of FIG. 41 together with FIG. 42. First, in step S900, a beacon terminal is installed at the work site. In the following step S901, a beacon terminal position identification process ( FIG. 4 ) is performed, and in step S902, a coordinate alignment process ( FIG. 12 ) is performed. In the following step S903, if the designer's support is not required for the equipment placement change process, in step S904, an augmented reality image including an object is displayed on the display unit 302 of the AR glasses 300 worn by the site worker ( FIG. 16 ), and in the following step S905, an object is moved within the augmented reality image ( FIG. 20 ). Then, in the following step S906, an object addition process ( FIG. 25 ) is performed to add an obstacle or other object that caused the placement change, and in the following step S907, a confirmation process ( FIG. 30 ) of the change content is performed.

[0110] On the other hand, if the designer's support is required for the device layout change process in step S903, then in step S908, a virtual reality image including the object is displayed on the display unit 302 of the VR glasses 500 worn by the designer, and then in step S909, the object is moved within the virtual reality image, as shown in Fig. 42. Then, in step S910, an object such as an obstacle that caused the layout change is added, and then in step S907, the change content is confirmed.

[0111] According to the second modified example described above, support for equipment placement work can be provided at the work site or a location different from the work site by another user (such as a designer or an expert) using the VR glasses 500. Note that, for example, multiple AR glasses 300 may be used, and virtual reality images may be displayed on the AR glasses 300 instead of the VR glasses 500.

[0112] 43 is a diagram showing an example of a configuration for realizing the functions of the device processing unit 110, the glasses processing unit 310, the positioning device processing unit 402, and the VR glasses processing unit 510 in this embodiment. Hereinafter, the device processing unit 110, the glasses processing unit 310, the positioning device processing unit 402, and the VR glasses processing unit 510 will not be distinguished from one another and will simply be referred to as processing units.

[0113] The functions of the processing unit are realized by, for example, a processing circuit. The processing circuit may include a processor 601 and a memory 602. The processing circuit may be dedicated hardware 603. A part of the processing circuit may be formed as dedicated hardware 603, and the processing circuit may further include a processor 601 and a memory 602. The processing circuit may also include a device that combines the processor 601 and the memory 602. In the example shown in the figure, a part of the processing circuit is formed as dedicated hardware 603. In the example shown in the figure, the processing circuit further includes a processor 601 and a memory 602.

[0114] The processing circuitry, part of which is at least one dedicated hardware 603, may be, for example, a single circuit, a multiple circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. When the processing circuitry comprises at least one processor 601 and at least one memory 602, the functionality of the processing unit is realized by software, firmware, or a combination of software and firmware.

[0115] The software and firmware are written as programs and stored in memory 602. The processor 601 realizes the functions of each unit by reading and executing the programs stored in memory 602. The processor 601 is also called a CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose Computing on Graphics Processing Units), TPU (Tensor Processing Unit), central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, or DSP. The memory 602 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, an EPROM, or an EEPROM, or a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD.

[0116] In this way, the processing circuit of the processing unit can realize each function of the processing unit by hardware, software, firmware, or a combination of these. When the processing circuit of the processing unit includes at least a processor 601 and a memory 602, the processor 601 executes a program stored in the memory 602 in the processing unit, and the hardware and software of the processing unit work together to realize the function of each unit included in the processing unit.

[0117] The present disclosure can be used for an elevator shaft equipment layout support system that supports the layout design of various devices, wiring, etc. in an elevator shaft.

[0118] 1 Elevator shaft 100 Terminal device 101 Terminal communication unit 110 Terminal processing unit 111 BIM data generation unit 112 2D drawing data generation unit 103 Human interface unit 200 Cloud server 201 Server communication unit 202 Data management unit 300 AR glasses 301 Glass communication unit 302 Display unit 303 Operation unit 304 Camera 305 Sensor 306 Glass memory unit 310 Glass processing unit 311 Image acquisition unit 312 Data acquisition unit 313 Operation detection unit 314 Update unit 315 Behavior detection unit 316 Gesture detection unit 317 Model data generation unit 318 Distance measurement unit 319 Attribute information acquisition unit 320 Display control unit 321 Coordinate alignment unit 400 Indoor positioning device 401 Positioning device communication unit 402 Positioning device processing unit 403 Positioning device storage unit 410 First beacon terminal 411 First beacon transmission unit 412 First beacon communication unit 420 Second beacon terminal 421 Second beacon transmission unit 422 Second beacon communication unit 430 Third beacon terminal 500 VR glasses 510 VR glasses processing unit 520 VR glasses operation unit 601 Processor 602 Memory 603 Dedicated hardware

Claims

1. An image acquisition unit that acquires images of the inside of an elevator shaft in real space, A data acquisition unit that acquires coordinate data and 3D model data from a data management unit that stores coordinate data within the elevator shaft and 3D model data of objects installed within the elevator shaft, A display unit that displays an augmented reality image generated by superimposing 3D model data of the object acquired from the data management unit onto an image of the elevator shaft in the real space, An operation detection unit that detects selection operations for individually selecting objects within the augmented reality image and movement operations for the selected objects, An update unit updates the 3D model data of the object to reflect the changes in the orientation and position of the object caused by the aforementioned movement operation, An elevator shaft equipment placement support system comprising: a transmission unit that transmits the updated 3D model data of the object to the data management unit; and

2. The system further includes a behavior detection unit that detects changes in the position and orientation of the display unit, The elevator shaft equipment placement support system according to claim 1, wherein the display unit changes the augmented reality image displayed in accordance with changes in the position and orientation of the display unit.

3. A camera that photographs the user's hands, The system further includes a gesture detection unit that detects the user's hand gestures captured by the aforementioned camera, The elevator shaft equipment placement support system according to claim 1 or 2, wherein the operation detection unit is capable of detecting the selection operation and the movement operation performed by the gesture.

4. The elevator shaft equipment placement support system according to claim 1 or 2, wherein the update unit updates the 3D model data of the object to reflect the change in orientation and position of the first object as a result of the movement operation when the movement operation is performed on the first object selected by the selection operation in the augmented reality image, and updates the 3D model data of the second object to reflect the change in orientation and position of the second object associated with the first object as a result of the movement operation to the first object.

5. The system further includes a model data generation unit that detects an object from an image of the elevator shaft in the real space and generates three-dimensional model data of the object. The elevator shaft equipment placement support system according to claim 1 or 2, wherein the display unit displays the augmented reality image generated by superimposing the three-dimensional model data of the object acquired from the data management unit and the three-dimensional model data of the object generated by the model data generation unit onto an image of the elevator shaft in the real space.

6. The elevator shaft equipment placement support system according to claim 1 or 2, further comprising a distance measuring unit for measuring the distance between objects in the augmented reality image.

7. A beacon terminal is provided within the elevator shaft in the real space to indicate a reference position in the real space, The elevator shaft equipment placement support system according to claim 1 or claim 2, further comprising: a coordinate alignment unit that uses information regarding the reference position indicated by the beacon terminal to perform coordinate alignment to match the position in the elevator shaft in the real space with the position on the coordinate axis of the coordinate data acquired from the data management unit.

8. The elevator shaft equipment placement support system according to claim 7, further comprising a positioning device provided in a manner that can communicate with the beacon terminal, and which includes part or all of the image acquisition unit, the data acquisition unit, the operation detection unit, and the update unit, and the transmission unit.

9. The elevator shaft equipment placement support system according to claim 1 or claim 2, wherein the display unit is capable of displaying the attribute information of the object superimposed on the augmented reality image.

10. The elevator shaft equipment placement support system according to claim 1 or claim 2, further comprising an attribute information acquisition unit that acquires attribute information of the input object.

11. The display unit is capable of displaying a virtual reality image generated based on the three-dimensional model data of the object acquired from the data management unit. The elevator shaft equipment placement support system according to claim 1, wherein the operation detection unit is capable of detecting selection operations for individually selecting the objects in the virtual reality image and movement operations for the selected objects.

12. The elevator shaft equipment placement support system according to claim 11, wherein the update unit updates the 3D model data of the object to reflect the change in orientation and position of the first object as a result of the movement operation when the movement operation is performed on the first object selected by the selection operation in the virtual reality image, and updates the 3D model data of the second object to reflect the change in orientation and position of the second object associated with the first object as a result of the movement operation to the first object.

13. The elevator shaft equipment placement support system according to claim 11 or claim 12, further comprising a distance measuring unit for measuring the distance between objects in the virtual reality image.

14. The elevator shaft equipment placement support system according to claim 11 or claim 12, wherein the display unit is capable of displaying the attribute information of the object superimposed on the virtual reality image.

15. The elevator shaft equipment placement support system according to any one of claims 1, 2, 11, and 12, further comprising a BIM data generation unit that generates BIM data from the three-dimensional model data of the aforementioned object.

16. The elevator shaft equipment placement support system according to any one of claims 1, 2, 11, and 12, further comprising a two-dimensional drawing data generation unit that generates two-dimensional drawing data from three-dimensional model data of the aforementioned object.