Display device, display control method, and display control program

Abstract

To provide a display device that allows a worker to more accurately recognize a construction position even when a position of a construction surface in an overlay display is misaligned with the position of the construction surface in an actual real space when a virtual object is overlaid and displayed in accordance with a position in the real space.SOLUTION: A display device includes: a display control unit (21) that displays, as a virtual object, a construction position image showing a construction position whose position is determined based on a virtual construction surface, which is a construction surface recognized in data for an overlay display; and a spatial distance measurement unit (25) that acquires a position of an actual construction surface in a three-dimensional space, which is the actual construction surface in a real space. The display control unit (21) changes a display position of the construction position image when there is a positional deviation between a position of an actual construction surface in the three-dimensional space and a position of a virtual construction surface in the three-dimensional space.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a display device that overlays virtual objects in an arrangement according to positions in the real space onto the field of view of a worker viewing the real space where a building is to be constructed, or onto an image representing the real space. [Background technology]

[0002] Conventionally, in construction work, etc., a method has been proposed in which information necessary for the work is displayed superimposed on the real space on a display device such as smart glasses worn by a worker, and the work is carried out while the work is being carried out. For example, Patent Document 1 discloses a projection device that scans the real space, recognizes the spatial shape, scales the drawing data to fit the real space, and maps and displays it on the real space. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-163466 Summary of the Invention [Problem to be solved by the invention]

[0004] Using the above technology, it is conceivable that an image showing the installation position of a construction target can be overlaid onto the real space where the work is to be carried out. For example, in the construction of a building, this overlay display technology can be used when constructing elevator shafts and landings.

[0005] In many cases, a construction object is constructed on a predetermined construction surface in real space. In this case, the virtual object displayed by the overlay display is displayed on the predetermined construction surface. However, due to errors caused by the actual construction, it is possible that the position of the construction surface recognized in the overlay display may be misaligned with the position of the construction surface (actual construction surface) in real space. When such a misalignment occurs, it is possible that the worker will not be able to accurately grasp the construction position of the construction object.

[0006] One aspect of the present invention aims to provide a display device that allows a worker to more accurately recognize the construction position when a virtual object is overlaid with an arrangement corresponding to the position in real space, even if the position of the construction surface in the overlay display is misaligned with the position of the construction surface in the actual real space. [Means for solving the problem]

[0007] In order to solve the above problems, the display device of the present invention is a display device that overlays a virtual object on the field of view of a worker viewing a real space as a construction work target, or on an image representing the real space, in an arrangement corresponding to the position of the real space, and is equipped with a display control unit that displays, as the virtual object, a construction position image indicating a construction position whose position is determined based on a virtual construction surface, which is the construction surface recognized in the data for the overlay display, and a position acquisition unit that acquires the position in three-dimensional space of the actual construction surface, which is the actual construction surface of the real space, and the display control unit is configured to change the display position of the construction position image when there is a positional discrepancy between the position in three-dimensional space of the actual construction surface and the position in three-dimensional space of the virtual construction surface.

[0008] In order to solve the above problems, the display control method of the present invention is a display control method that overlays a virtual object on the field of view of a worker viewing a real space as a construction work target, or on an image representing the real space, in an arrangement corresponding to the position of the real space, and includes a display control step that displays, as the virtual object, a construction position image indicating a construction position whose position is determined based on a virtual construction surface, which is the construction surface recognized in the data for the overlay display, and a position acquisition step that acquires the position in three-dimensional space of the actual construction surface, which is the actual construction surface of the real space, and in the display control step, if there is a positional discrepancy between the position in three-dimensional space of the actual construction surface and the position in three-dimensional space of the virtual construction surface, the method changes the display position of the construction position image.

[0009] The display device according to each aspect of the present invention may be realized by a computer. In this case, a display control program that causes the computer to operate as each part (software element) of the display device to realize the display device on a computer, and a computer-readable recording medium on which the program is recorded, also fall within the scope of the present invention. [Effects of the Invention]

[0010] According to the present invention, the display position of the construction position image can be changed so as to reduce the deviation between the position of the virtual construction surface and the position of the actual construction surface, thereby enabling the worker to more accurately recognize the construction position. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 10 is a perspective view showing an example of a state in which a construction position image OB as a virtual object is overlaid and displayed in real space. [Figure 2] 1 is a block diagram showing an outline of the configuration of smart glasses 1 according to an embodiment of the present invention. [Figure 3] FIG. 1 is a diagram showing a state in which smart glasses (display device) 1 according to the present embodiment are attached to a safety helmet 111. [Figure 4] FIG. 2 is a perspective view showing an outline of an elevator hall 200 and a hoistway 102 to which the elevator is to be constructed. [Figure 5] 1 is a perspective view showing an outline of an opening 101 as viewed from inside a hoistway 102. FIG. [Figure 6] FIG. 10 is a perspective view for explaining a method for recognizing a positional deviation between a virtual construction surface and an actual construction surface. [Figure 7] 10 is a perspective view showing the state of overlay display in a display mode in which the display position of each displayed construction position image OB can be manually moved. FIG. [Figure 8] FIG. 10 is a perspective view showing the state of a wall surface when the wall surface has a complex shape. [Figure 9] 1 is a diagram schematically showing six virtual construction surfaces 400X1, 400X2, 400Y1, 400Y2, 400Z1, and 400Z2 surrounding a predetermined construction work environment. [Figure 10] 3 is a flowchart showing a processing flow of an elevator installation method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, one embodiment of the present invention will be described in detail.

[0013] (Outline of smart glass application examples) 3 is a diagram showing a state in which smart glasses (display device) 1 according to this embodiment are attached to a safety helmet 111. In this embodiment, a situation is assumed in which a worker who is installing an elevator during the construction of a building wears the smart glasses 1 together with the safety helmet 111.

[0014] The smart glasses 1 are optically transparent head-mounted displays. A wearer of the smart glasses 1 can view the external real space and can also view the projected image within the real space. That is, the smart glasses 1 overlay a virtual object as construction reference information related to elevator construction in a position corresponding to the position in the real space in the field of view of a worker viewing the real space including the elevator landing and hoistway. Note that while this embodiment is intended for elevator construction work, it is not limited thereto and can be applied to any construction work as long as the construction work involves installing an object to be installed at a predetermined position in real space.

[0015] FIG. 1 shows an example of a state in which a construction position image OB as a virtual object is overlaid on real space. As shown in the figure, the construction position image OB as a virtual object is displayed on a wall surface 300 in real space. This allows the worker to confirm the position in three-dimensional space where construction work should be performed in real space. This eliminates the need for tasks such as measuring and confirming the position where construction work should be performed in real space, thereby improving workability. Details of FIG. 1 will be described later.

[0016] In this embodiment, smart glasses 1, which are optically transparent head-mounted displays, are used as the display device, but the present invention is not limited to this. For example, a video-transparent head-mounted display that displays both an image of the external real space captured by a camera and an image of a virtual object may be used. Furthermore, instead of a head-mounted display, the display device according to the present invention may be a portable information display terminal such as a tablet PC or a notebook PC, or a portable display and camera connected to an information processing device wirelessly or via a cable.

[0017] The virtual object as construction reference information is displayed based on 3D CAD data or 2D CAD data generated based on the design specifications of the construction target. This CAD data may be generated based on information of a 3D model generated as BIM (Building Information Modeling), for example. In BIM, various information such as the quantity, product number, dimensions, material, performance, and price of each part is included in the 3D model as object information, so at least one of this information may be displayed as construction reference information.

[0018] (Details of smart glasses configuration) 2 is a block diagram showing an outline of the configuration of the smart glasses 1. As shown in the figure, the smart glasses 1 include a control unit 2, a projection unit 3, an imaging unit 4, a storage unit 5, a communication unit 6, and an audio input / output unit 7. The control unit 2 is a block that performs various information processing in the smart glasses 1, and includes a display control unit 21, a position recognition unit 22, an input control unit 23, a warning control unit 24, and a spatial distance measurement unit (position acquisition unit) 25.

[0019] The display control unit 21 controls the projection unit 3 to overlay and display a virtual object as construction reference information related to elevator construction. The projection unit 3 projects an image onto the half mirror, allowing the worker wearing the smart glasses 1 to view the projected image while viewing the external real space. The display control unit 21 makes the projected image for the right eye and the projected image for the left eye different, thereby displaying the virtual object as if it were three-dimensionally positioned at a predetermined position in real space. This allows the worker to recognize the construction position as a three-dimensional position in real space.

[0020] The display control unit 21 reads out construction reference information including three-dimensional CAD data or two-dimensional CAD data stored in the storage unit 5 and controls the display of the virtual object. Here, the three-dimensional CAD data or two-dimensional CAD data also includes information indicating a positional relationship with an object existing in real space. In addition, the display control unit 21 controls the display position of the virtual object based on the position in real space recognized by the position recognition unit 22.

[0021] The display control unit 21 may acquire construction reference information from outside via the communication unit 6. The communication unit 6 may communicate with a PC or server on a local network via a wireless LAN, or with an external server via the Internet, for example. This makes it possible to update the data stored in the storage unit 5 and acquire new data.

[0022] The position recognition unit 22 recognizes the three-dimensional position of the smart glasses 1 in real space and the three-dimensional direction in which the smart glasses 1 are facing by recognizing the captured images of markers M1 and M2 (details will be described later) placed at predetermined positions in real space captured by the imaging unit 4. The position recognition unit 22 is also capable of position recognition with 6DoF (Degree of Freedom). This allows the worker to once have their position recognized by the position recognition unit 22 using the markers M1 and M2, and then view the virtual object while maintaining its relative positional relationship with real space even if they subsequently change their viewing direction or move.

[0023] The input control unit 23 is a block that receives and processes various instruction inputs from the worker. For example, an instruction input is an input to an input interface as a virtual object. That is, the display control unit 21 displays an image as the input interface as a virtual object in real space, and the instruction input from the worker is received by image recognition of the worker's virtual touch with a finger or the like.

[0024] Furthermore, for example, an instruction input from a worker may be accepted by voice input from the voice input / output unit 7. That is, the input control unit 23 recognizes the voice received by the voice input / output unit 7 and recognizes the content of the instruction from the worker, thereby accepting the instruction input from the worker.

[0025] Furthermore, an instruction input from a worker may be accepted by an external input device via the communication unit 6. For example, the communication unit 6 communicates with various input devices such as a Bluetooth (registered trademark) compatible input controller or a wireless keyboard, and an instruction input is accepted as an input by a worker to the input device.

[0026] The warning control unit 24 controls some kind of warning to be issued to the worker wearing the smart glasses 1. Examples of warning methods include projecting and displaying the warning content using an overlay display by the display control unit 21, outputting a warning sound or a voice indicating the warning content from the audio input / output unit 7, and so on.

[0027] The warning condition is that the position recognition unit 22 recognizes that the worker is located within a predetermined area near the elevator shaft. This allows a warning to be issued when there is an increased risk of the worker falling into the elevator shaft, thereby further improving the safety of the worker.

[0028] At least one function of the display control unit 21, the position recognition unit 22, the input control unit 23, and the warning control unit 24 included in the control unit 2 may be realized in an external computer via communication.

[0029] The spatial distance measuring unit 25 recognizes the distance to an object in real space using a distance sensor such as LiDAR (Light Detection and Ranging) and recognizes the actual shape of the real space. This enables processing such as changing the display position of a virtual object according to the relationship between the display position of the virtual object as construction reference information and the position of the actual object in real space. For example, as will be described in detail later, the spatial distance measuring unit 25 measures the actual construction surface, which is the actual construction surface in real space, and the display control unit 21 changes the display position of the virtual object based on this measurement result. Note that the spatial distance measuring unit 25 is not essential, and a configuration may be adopted in which the distance to an object in real space is recognized by an external spatial distance measuring device and the information is acquired via the communication unit 6.

[0030] (Example of a real space where an elevator is installed) Fig. 4 is a perspective view showing an outline of an elevator hall 200 and a hoistway 102, which are the targets of elevator construction. The figure shows a state in which the elevator doors have not yet been installed, during the construction stage of a building. In this state, an opening 101 serving as an elevator entrance exists between the hall 200 and the hoistway 102. The hall 200 has a floor surface 201 and a wall surface 202.

[0031] Two markers M1 and M2 are provided on the floor surface 201. As described above, the markers M1 and M2 are used by the position recognition unit 22 to recognize the positions in real space by performing image recognition of the markers M1 and M2 in the captured image. Each of the markers M1 and M2 is made up of a rectangular planar member, and a predetermined pattern is formed on its surface. By arranging such markers M1 and M2 at predetermined positions near the opening 101 on the floor surface 201 of the hall 200, the position recognition unit 22 can accurately recognize the positions of the hall 200 and the elevator shaft 102 in three-dimensional space.

[0032] In this embodiment, two markers are provided for each landing, but this is not limited to this, and one marker, or three or more markers may be provided. The more markers provided, the higher the accuracy of position recognition, but if two markers are provided, position recognition can be performed with sufficient accuracy.

[0033] Furthermore, in this embodiment, the markers M1 and M2 are provided at the hall 200, but the markers may also be provided within the hoistway 102.

[0034] Markers M1 and M2 are placed at predetermined positions on floor surface 201 that are specified as marker installation positions in construction reference information including 3D CAD data or 2D CAD data. The installation positions of these markers M1 and M2 require high precision in order to accurately match the positions in real space with the displayed positions of the virtual objects. Therefore, it is preferable that the markers be placed based on a predetermined position reference object with high placement position accuracy in real space. An example of the position reference object is piano wire 103 placed in elevator shaft 102.

[0035] FIG. 5 is a perspective view showing the opening 101 as seen from inside the elevator shaft 102. As shown in the figure, two piano wires 103, 103 are suspended from predetermined positions at the top of the elevator shaft 102 during the construction stage of a building. In other words, the piano wires 103, 103 are placed in fixed positions without being affected by the construction status of each floor. Therefore, by using this piano wire as a position reference (reference line), it is possible to maintain a high level of accuracy in the installation positions of the markers M1, M2. Note that this is not limited to piano wire, and any strong wire member with a sufficiently thin diameter may be used.

[0036] Alternatively, a laser beam may be used as a position reference, for example. That is, by irradiating a laser beam vertically downward from a predetermined position in the upper part of the elevator shaft 102, or by irradiating a laser beam vertically upward from a predetermined position in the lower part of the elevator shaft 102, the laser beam can be positioned at a fixed position, similar to piano wire. However, with laser beams, the thickness of the beam tends to increase as the distance from the light source increases, so there is a possibility that errors due to the thickness of the beam may be affected depending on the height of the building. Alternatively, a reference point (architectural markings) marked on the landing of each floor may be used as a position reference.

[0037] Furthermore, multiple landings 200 will be provided for each floor for one elevator, and it is preferable to install the markers at the same location in relation to the elevator entrance / exit, i.e., at the same relative position in relation to the entrance, at each landing 200.

[0038] In many cases, all halls 200 for the same elevator have the same structural arrangement. In this case, since markers M1 and M2 can be installed in the same locations relative to the elevator entrances at each hall 200, the marker installation work can be performed using the same standards and work. This allows the marker installation work to be performed efficiently. In addition, the data for overlay display can also have a common data structure among multiple halls 200.

[0039] Each of the markers M1 and M2 may be provided with a two-dimensional code. In Fig. 4, each of the markers M1 and M2 is provided with a two-dimensional code QR1 or QR2. The two-dimensional codes QR1 and QR2 indicate information about the floor on which the marker M1 or M2 is located. In other words, when the position recognition unit 22 recognizes the markers M1 and M2 in the captured image, it also recognizes the two-dimensional codes QR1 and QR2, thereby enabling it to recognize information about the floor on which the read marker M1 or M2 is located.

[0040] The floor information included in the marker is not limited to a two-dimensional code. For example, the marker may display text such as "3F," which may be acquired as floor information through character recognition. Furthermore, the marker may be colored specific to each floor or display an image with a specific shape, allowing the floor to be recognized through image recognition.

[0041] (Example of adjusting the overlay display position) Next, an example of adjusting the positional deviation of the construction position image OB in the overlay display will be described with reference to Fig. 1. The construction position image OB is an image showing the position where a predetermined construction object is to be constructed. The display control unit 21 determines the display position of the construction position image OB based on a virtual construction surface, which is the construction surface recognized in the data for the overlay display.

[0042] As described above, construction objects are often constructed on a predetermined construction surface in real space. In this case, the virtual object displayed by the overlay display is displayed on the predetermined construction surface. However, due to errors caused by actual construction, it is possible that the position of the construction surface recognized in the overlay display may differ from the position of the construction surface (actual construction surface) in real space.

[0043] 101 in FIG. 1 shows a state in which the construction position image OB is displayed at a position shifted toward the front with respect to the wall surface 300, which is the actual construction surface. In other words, a positional shift occurs between the virtual construction surface and the wall surface 300, which is the actual construction surface. By recognizing this positional shift, the display control unit 21 adjusts the display position of the construction position image OB so that it is on the wall surface 300, which is the actual construction surface, as shown in 102 in FIG. 1. This type of display control allows the worker to more accurately recognize the construction position.

[0044] The construction position image OB is not limited to being displayed on the virtual construction surface, but may be displayed at a position a predetermined distance away from the virtual construction surface. That is, the display control unit 21 recognizes the positional relationship between the construction position image OB and the virtual construction surface based on the data for overlay display, and changes the display position of the construction position image OB based on this positional relationship. Therefore, the display position of the construction position image OB can be appropriately changed whether or not the position of the virtual construction surface and the position of the construction position image OB match.

[0045] (Method for recognizing the positional deviation between the virtual construction surface and the actual construction surface) 6 is a perspective view for explaining a method for recognizing the positional deviation between the virtual construction surface and the actual construction surface. In the example shown in the figure, there is a first wall surface 300A and a second wall surface 300B arranged perpendicular to it, and there are a first virtual construction surface 400A and a second virtual construction surface 400B as virtual construction surfaces corresponding to these, respectively. There is a deviation between the position of the first virtual construction surface 400A and the position of the first wall surface 300A, and there is also a deviation between the position of the second virtual construction surface 400B and the position of the second wall surface 300B.

[0046] As described above, the spatial distance measuring unit 25 can recognize the distance to an object existing in real space. Therefore, the spatial distance measuring unit 25 can measure the three-dimensional position of the marker M by recognizing the distance and direction of the marker M placed on the floor surface, and can measure the distance LA1 between the marker M and the first wall surface 300A and the distance LB1 between the marker M and the second wall surface 300B by measuring the three-dimensional positions of the first wall surface 300A and the second wall surface 300B. Furthermore, the spatial distance measuring unit 25 can measure the distance LA2 between the marker M and the first virtual construction surface 400A and the distance LB2 between the marker M and the second virtual construction surface 400B based on the measured position of the marker M and the display positions of the first virtual construction surface 400A and the second virtual construction surface 400B displayed by the display control unit 21. Therefore, the distance LA3, which is the positional deviation between the first wall surface 300A and the first virtual construction surface 400A, is calculated by LA3 = LA1 - LA2, and the distance LB3, which is the positional deviation between the second wall surface 300B and the second virtual construction surface 400B, is calculated by LB3 = LB1 - LB2.

[0047] (Example of how to change the display position) When the positional deviation between the virtual construction surface and the actual construction surface is measured as described above, the display position of the construction position image OB is changed based on this positional deviation. There are three methods for changing the display position of the construction position image OB based on the positional deviation:

[0048] A first method is a method in which the display position of the construction position image OB is automatically changed based on the positional deviation. When the positional deviation from the actual construction surface is measured for one or more virtual construction surfaces recognized in the overlay display, the display control unit 21 identifies the construction position image OB to be displayed based on each virtual construction surface. Then, the display control unit 21 changes the display position of each construction position image OB according to the amount of positional deviation of the corresponding virtual construction surface.

[0049] According to this method, the position of the virtual construction surface and the position of the corresponding actual construction surface are automatically recognized, and the display position of the construction position image OB is changed according to the amount of positional deviation. Therefore, the worker can view the construction position image OB at a more accurate position without performing processing to correct the positional deviation.

[0050] A second method is one in which the display position of the construction position image OB is changed by a worker's input of a numerical value based on the positional deviation. When the positional deviation from each of one or more virtual construction surfaces recognized in the overlay display is measured, the display control unit 21 displays the amount of positional deviation between each virtual construction surface and the corresponding actual construction surface. For example, when the worker selects a specific virtual construction surface in the overlay display, this display may display the amount of positional deviation for the selected virtual construction surface. Alternatively, the construction position image OB, whose display position is set based on the selected virtual construction surface, may be highlighted, and when the worker selects a construction position image OB from among them, a display position editing screen may be displayed that allows the display position of the selected construction position image OB to be changed.

[0051] A third method is a method in which the worker manually moves and changes the display position of the construction position image OB in the overlay display. When the positional deviations between one or more virtual construction surfaces recognized in the overlay display and the actual construction surface are measured, the display control unit 21 transitions to a display mode in which the display position of each displayed construction position image OB can be manually moved in response to an instruction from the worker. That is, the display control unit 21 changes the display position of the construction position image OB identified by the worker in the three-dimensional space according to the movement direction instructed by the worker. In this display mode, for example, the input control unit 23 may be capable of recognizing the position and movement of the worker's fingers, allowing the worker to pinch and move the construction position image OB with his or her fingers. Furthermore, the input control unit 23 may be capable of displaying a finger image in the overlay display space based on the worker's operation input to the input controller, thereby allowing the worker to virtually pinch the construction position image OB.

[0052] 7 shows the state of the overlay display in this display mode. 701 shows a state in which a worker is holding the construction position image OB, which is displayed offset from the wall surface 300, with his fingers F. 702 shows a state in which the worker has pushed the construction position image OB up to the wall surface 300.

[0053] In measuring the positional deviation between the virtual construction surface and the actual construction surface, for example, if an error occurs in measuring the position of the actual construction surface, simply changing the display position of the construction position image OB based on the measurement result of the positional deviation between the virtual construction surface and the actual construction surface is likely to result in the positional deviation due to the measurement error remaining. In contrast, with the above method, the worker can virtually move the display position of the construction position image OB by hand, so that the display position of the construction position image OB can be accurately adjusted to match the placement position of the actual construction surface.

[0054] When changing the display position of the construction position image OB, the construction position image OB at a display position determined by measuring the positional deviation between the virtual construction surface and the actual construction surface may also be displayed at the same time. This allows the worker to recognize the difference between the construction position image OB at the display position determined by measurement, thereby preventing the construction position image OB from being moved to an incorrect position due to the worker's misrecognition.

[0055] Furthermore, positional deviations between the virtual construction surface and the actual construction surface often occur only in the direction perpendicular to the virtual construction surface. Therefore, the direction in which the worker can change the display position of the construction position image OB may be limited to the direction perpendicular to the virtual construction surface. This prevents the display position of the construction position image OB from being changed unnecessarily in the vertical or horizontal directions.

[0056] Furthermore, when the worker virtually pinches the construction position image OB and moves it toward the wall surface 300, the display control unit 21 may automatically change the display position of the construction position image OB in accordance with the amount of positional deviation of the measured virtual construction surface. This allows the worker to automatically correct the display position of the construction position image OB by virtually pinching and pressing the construction position image OB, as in the first method described above.

[0057] (Recognition of the correspondence between the virtual construction surface and the actual construction surface) Next, a method for accurately recognizing the correspondence between the virtual construction surface and the actual construction surface will be described. When the shape of the wall surface that serves as the reference for the construction position image is complex, it may be difficult to recognize the correspondence between the virtual construction surface and the actual construction surface. Figure 8 shows the state of a wall surface when the wall surface has a complex shape. In the example shown in the figure, the actual construction surface has a 1-1 wall surface 300A1 and a 1-2 wall surface 300A2, and their normal directions are the same. Furthermore, the virtual construction surface has a 1-1 virtual construction surface 400A1 and a 1-2 virtual construction surface 400A2, and their normal directions are the same.

[0058] In such a case, since there are multiple wall surfaces with the same normal direction, there are multiple combinations of correspondences between each wall surface and each virtual construction surface. For example, it is not possible to determine whether the virtual construction surface corresponding to the 1-1 wall surface 300A1 is the 1-1 virtual construction surface 400A1 or the 1-2 virtual construction surface 400A2 based solely on the normal direction. In this case, it is possible to perform correspondence by taking into account, for example, the positional relationship with other wall surfaces and the amount of positional deviation. However, there are problems such as the algorithm becoming complicated and the possibility of correspondence failure when the shape becomes more complex.

[0059] Alternatively, the display control unit 21 may recognize the correspondence between the virtual construction surface and the actual construction surface through an instruction input by the worker, and recognize the amount of positional deviation based on the correspondence. For example, as shown in Fig. 8, the 1-1 wall surface 300A1 is first selected by touching it with the worker's finger, and then the 1-1 virtual construction surface 400A1 is selected by touching it with the worker's finger, thereby allowing the display control unit 21 to recognize that the two are in a correspondence relationship. This allows the display position of the construction position image OB to be appropriately changed even in a situation where the correspondence between the virtual construction surface and the actual construction surface is difficult to understand.

[0060] (Another example of how to change the display position of the construction location image) Next, another example of a method for changing the display position of a construction position image will be described. For example, as shown in FIG. 9, six virtual construction planes 400X1, 400X2, 400Y1, 400Y2, 400Z1, and 400Z2 are provided surrounding a specified construction work environment, and the display position of each construction position image is determined based on one of the virtual construction planes 400X1, 400X2, 400Y1, 400Y2, 400Z1, and 400Z2. In this case, based on the measurement results of the positional deviation between each virtual construction plane and the corresponding actual construction plane, the display control unit 21 changes the display position of each virtual construction plane and also changes the display position of the construction position image corresponding to each virtual construction plane. This makes it possible to collectively change the display positions of the corresponding construction position images for each of the six virtual construction planes surrounding a specified construction work environment.

[0061] In the above example, the display positions of all construction position images corresponding to the virtual construction surface whose display positions are changed are changed, but it is also possible to change the display position of only a specific construction position image. This specific construction position image may be selected by the worker or may be set in advance in the construction reference information data stored in the storage unit 5.

[0062] In addition, among the six virtual construction surfaces, virtual construction surfaces on which the display position is not permitted to be changed may be preset, which prevents the display position of a virtual construction surface on which the display position does not need to be changed, such as a virtual construction surface corresponding to a floor, from being changed due to an operation error or a misunderstanding by the worker.

[0063] (Update of construction reference information data) As described above, when the display position of the construction position image is changed, the display control unit 21 may be configured to add information regarding the change in display position to the construction reference information data stored in the storage unit 5. Furthermore, the construction reference information data to which information regarding the change in display position has been added may be further uploaded to an external server. This makes it possible to share the latest construction reference information data to which information regarding the change in display position has been added on the server. In other words, even when another worker works in the same construction work environment, the construction reference information data for which the display position has been adjusted can be used.

[0064] In addition, there may be cases where the construction reference information is created as data that can be commonly used at multiple locations in a single building. For example, it may be possible to use common construction reference information on each floor, such as on intermediate floors of a building. In this case, the storage unit 5 stores the construction reference information data as common data. Here, if the display position of the construction position image is changed while a worker is working on a specific floor where the common construction reference information can be used, the display control unit 21 simply adds information regarding the change in display position to the common construction reference information data stored in the storage unit 5. This allows the construction reference information data with the adjusted display position to be used when a worker works on another floor where the common construction reference information can be used.

[0065] (Elevator installation method processing flow) Next, the process flow of the elevator installation method according to this embodiment will be described with reference to Fig. 10. When elevator installation begins, first, in step 1 (hereinafter referred to as S1), a marker installation jig MJ having markers M1 and M2 attached thereto is installed at a predetermined position in the hall 200.

[0066] Next, in S2, the worker wears the smart glasses 1 on the floor where he or she is to work, and the marker corresponding to the floor where the worker is located is captured by the imaging unit 4, and position recognition processing is performed by the position recognition unit 22. Here, the position recognition unit 22 identifies the floor where the worker is located based on the information included in the marker.

[0067] The position recognition unit 22 may determine whether a marker is installed on the floor where the worker is located based on the distance to the marker. This allows the marker to be properly recognized as the marker installed on the floor where the worker is located, even if markers on other floors are included in the image capture range of the image capture unit 4. The distance to the marker may be determined based on the size of the marker in the captured image, or a distance sensor such as LiDAR (Light Detection and Ranging) may be used.

[0068] Once the position recognition is performed, in S3, the display control unit 21 reads out data of the construction reference information according to the floor recognized by the position recognition unit 22, based on at least one of the selected process and part. Then, the display control unit 21 performs an overlay display of the virtual object based on the read-out data. Once the worker has used the markers M1 and M2 to have the position recognition unit 22 recognize their position, they can view the virtual object with its relative positional relationship with real space maintained, even if they subsequently change their viewing direction or move.

[0069] Next, in S4, the positional deviation between the virtual construction surface and the actual construction surface is measured. Then, in S5, it is determined whether or not it is necessary to change the display position of the construction position image. This determination is made, for example, by presenting a dialog to the worker asking whether or not to change the display position of the construction position image, and based on the worker's response to the dialog. Here, the worker may be asked which of the first to third methods described above will be used as a method for changing the display position of the construction position image, or the change method may be set in advance, or the change method may be fixed. Furthermore, if there is a positional deviation between the virtual construction surface and the actual construction surface, or if the amount of positional deviation is equal to or greater than a predetermined range, the display control unit 21 may decide to change the display position of the construction position image without confirming with the worker.

[0070] If the result of S5 is No, that is, if the display position of the construction position image does not need to be changed, the process proceeds to S7. On the other hand, if the result of S5 is Yes, that is, if the display position of the construction position image needs to be changed, in S6 the display control unit 21 changes the display position of the construction position image by one of the first to third methods described above.

[0071] Thereafter, in S7, the worker performs the construction work while checking the construction reference information overlaid and displayed by the display control unit 21. Note that the construction work includes not only the actual construction work but also the check work after the construction work is completed.

[0072] Next, in S8, the completion of the construction work is confirmed. If the answer is No in S8, that is, if the construction work is continuing, the processing from S5 is repeated; if the answer is Yes in S8, that is, if the construction work has been completed, the processing is terminated.

[0073] The above construction method and smart glasses 1 enable elevator construction to be carried out safely and efficiently. Such effects will also contribute to the achievement of Goal 11.c of the Sustainable Development Goals (SDGs) advocated by the United Nations, which states, "Support the development of sustainable and resilient buildings using local materials in least developed countries, including through financial and technical assistance."

[0074] [Software implementation example] The functions of the smart glasses 1 (hereinafter referred to as the "device") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (particularly each part included in the control unit 2).

[0075] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0076] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0077] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0078] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0079] (summary) The display device according to aspect 1 of the present invention is a display device that overlays a virtual object onto the field of view of a worker viewing a real space as the target of construction work, or onto an image representing the real space, in an arrangement corresponding to the position of the real space, and is equipped with a display control unit that displays, as the virtual object, a construction position image indicating a construction position whose position is determined based on a virtual construction surface, which is the construction surface recognized in the data for the overlay display, and a position acquisition unit that acquires the position in three-dimensional space of the actual construction surface, which is the actual construction surface of the real space, and the display control unit is configured to change the display position of the construction position image when there is a positional discrepancy between the position of the actual construction surface in three-dimensional space and the position of the virtual construction surface in three-dimensional space.

[0080] According to the above configuration, the display position of the construction position image can be changed so as to reduce the deviation between the position of the virtual construction surface and the position of the actual construction surface, thereby allowing the worker to more accurately recognize the construction position.

[0081] In the display device of aspect 2 of the present invention, in the above aspect 1, the display control unit may be configured to recognize the positional relationship between the construction location image and the virtual construction surface based on the data for the overlay display, and change the display position of the construction location image based on the positional relationship.

[0082] According to the above configuration, the display position of the construction position image can be appropriately changed whether or not the position of the virtual construction surface and the position of the construction position image coincide with each other.

[0083] A display device according to aspect 3 of the present invention may be configured such that, in aspect 1 above, the display control unit recognizes the amount of positional deviation by recognizing the position of the virtual construction surface in three-dimensional space and the position of the actual construction surface corresponding to the virtual construction surface in three-dimensional space, and changes the display position of the construction position image according to the recognized amount of positional deviation.

[0084] According to the above configuration, the position of the virtual construction surface and the position of the corresponding actual construction surface are automatically recognized, and the display position of the construction position image is changed according to the amount of positional deviation. Therefore, the worker can view the construction position image at a more accurate position without performing processing to correct the positional deviation.

[0085] In the display device of aspect 4 of the present invention, in the above aspect 1, the display control unit recognizes the amount of positional deviation by recognizing the position of the virtual construction surface in three-dimensional space and the position of the actual construction surface corresponding to the virtual construction surface in three-dimensional space, and displays the recognized amount of positional deviation, and may also be configured to change the display position of the construction position image in response to a worker's input of an instruction to change the display position of the construction position image.

[0086] According to the above configuration, the amount of positional deviation between the position of the virtual construction surface and the position of the corresponding actual construction surface is displayed, and the worker can change the display position of the construction position image accordingly. Therefore, the worker can view the construction position image at a more accurate position.

[0087] A display device according to aspect 5 of the present invention may be configured such that, in aspect 1 above, the display control unit changes the display position of the construction position image identified by the worker in accordance with the direction of movement instructed by the worker.

[0088] According to the above configuration, the worker can move the position of the identified construction position image in a desired direction, which makes it possible to fine-tune the display position of the specific construction position image, for example.

[0089] The display device according to aspect 6 of the present invention may be configured such that, in aspect 3 or 4 above, the display control unit recognizes the correspondence between the virtual construction surface and the actual construction surface through instruction input by the worker, and recognizes the amount of positional deviation based on the correspondence.

[0090] According to the above configuration, the correspondence between the virtual construction surface and the actual construction surface is indicated by the worker, so that the display position of the construction position image can be appropriately changed even in situations where the correspondence between the virtual construction surface and the actual construction surface is difficult to understand.

[0091] The display control method according to aspect 7 of the present invention is a display control method for overlaying a virtual object onto the field of view of a worker viewing a real space as a construction work target, or onto an image representing the real space, in an arrangement corresponding to the position of the real space. The display control method includes a display control step for displaying, as the virtual object, a construction position image indicating a construction position whose position is determined based on a virtual construction surface, which is the construction surface recognized in the data for the overlay display, and a position acquisition step for acquiring the position in three-dimensional space of an actual construction surface, which is the actual construction surface of the real space. In the display control step, if there is a positional discrepancy between the position in three-dimensional space of the actual construction surface and the position in three-dimensional space of the virtual construction surface, the method changes the display position of the construction position image.

[0092] A display control program according to an eighth aspect of the present invention is a display control program for causing a computer to function as the display device in the above-mentioned first aspect, and causes the computer to function as the display control unit and the position acquisition unit.

[0093] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0094] 1 Smart Glass (Display Device) 2 Control Unit 3 Projection Unit 4 Imaging Unit 5 Memory Unit 6 Communication Unit 7 Audio Input / Output Unit 21 Display Control Unit 22 Position Recognition Unit 23 Input Control Unit 24 Warning Control Unit 25 Spatial Distance Measurement Unit (Position Acquisition Unit) 101 Opening 102 Lift Shaft 103 Piano Wire 111 Safety Helmet 200 Platform 201 Floor Surface 202, 300, 300A, 300A1, 300A2, 300B Wall Surface 400A, 400B, 400A1, 400A2, 400X1·400X2·400Y1·400Z1·400Z2 Virtual Construction Surface M1·M2 Marker QR1· 2D Code

Claims

1. A display device that overlays and displays virtual objects in an arrangement according to positions in the real space in a field of view of a worker viewing the real space as a construction work target or an image representing the real space, a display control unit that displays, as the virtual object, a construction position image showing a construction position whose position is determined based on a virtual construction surface that is a construction surface recognized in the data for the overlay display; a position acquisition unit that acquires the position in a three-dimensional space of an actual construction surface that is an actual construction surface in the real space, The display control unit of the display device changes the display position of the construction position image when there is a positional discrepancy between the position of the actual construction surface in three-dimensional space and the position of the virtual construction surface in three-dimensional space.

2. The display device described in claim 1, wherein the display control unit recognizes the positional relationship between the construction location image and the virtual construction surface based on the data for the overlay display, and changes the display position of the construction location image based on the positional relationship.

3. The display device described in claim 1, wherein the display control unit recognizes the amount of positional deviation by recognizing the position of the virtual construction surface in three-dimensional space and the position of the actual construction surface corresponding to the virtual construction surface in three-dimensional space, and changes the display position of the construction position image according to the recognized amount of positional deviation.

4. The display device described in claim 1, wherein the display control unit recognizes the amount of positional deviation by recognizing the position of the virtual construction surface in three-dimensional space and the position of the actual construction surface corresponding to the virtual construction surface in three-dimensional space, displays the recognized amount of positional deviation, and changes the display position of the construction position image in response to a worker's input of an instruction to change the display position of the construction position image.

5. The display device according to claim 1 , wherein the display control unit changes a display position of the construction position image specified by a worker in accordance with a movement direction instructed by the worker.

6. 5. The display device according to claim 3, wherein the display control unit recognizes a correspondence relationship between the virtual construction surface and the actual construction surface through an instruction input by a worker, and recognizes the amount of the positional deviation based on the correspondence relationship.

7. A display control method for overlaying a virtual object on a field of view of a worker viewing a real space as a construction work target or on an image representing the real space, in an arrangement according to a position in the real space, comprising: a display control step of displaying, as the virtual object, a construction position image showing a construction position whose position is determined based on a virtual construction surface, which is a construction surface recognized in the data for the overlay display; a position acquisition step of acquiring a position in a three-dimensional space of an actual construction surface, which is an actual construction surface in the real space, A display control method in which, in the display control step, if there is a positional discrepancy between the position of the actual construction surface in three-dimensional space and the position of the virtual construction surface in three-dimensional space, the display position of the construction position image is changed.

8. 2. A display control program for causing a computer to function as the display device according to claim 1, the display control program causing the computer to function as the display control unit and the position acquisition unit.

Images