Display device, display control method, and display control program

Abstract

In the elevator construction work using a display device that overlays and displays virtual objects in an arrangement according to the position in the real space, a display device is provided that can clearly present to the worker which floor the displayed construction reference information corresponds to. 【Solution means】The smart glass (1) includes a position recognition unit (22) that recognizes the position of the real space including the elevator hoistway by image recognition of a marker installed at a predetermined position in the real space, and a display control unit (21) that overlays and displays a virtual object in the real space based on the marker. The display control unit (21) makes the display mode of the virtual object corresponding to a predetermined floor different from the display mode of the virtual object corresponding to a floor different from the predetermined floor.

Description

Technical Field

[0001] The present invention relates to a display device that overlays and displays virtual objects in an arrangement corresponding to the position in the real space with respect to the visual field of an operator viewing the real space or an image representing the real space.

Background Art

[0002] Conventionally, in construction work and the like, a method has been proposed in which construction work is performed in a state where information necessary for construction work is displayed in an overlapping manner with the real space on a display device such as smart glasses worn by an operator. For example, Patent Document 1 discloses a projection device that scans the real space to recognize the space shape, scales the drawing data according to the real space, and maps and displays it in the real space.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] During the construction stage of a building, there may be a situation where there is no door installed between the elevator shaft and the landing. In such a situation, when the display device installs markers for recognizing the position in the real space corresponding to each floor, for example, it is conceivable that a plurality of markers are recognized by the display device through the space of the elevator shaft. In this case, it is conceivable that it becomes unclear which floor the displayed construction reference information corresponds to.

[0005] One aspect of the present invention is to clearly present to an operator which floor the displayed construction reference information corresponds to in the elevator construction work using a display device that overlays and displays virtual objects in an arrangement according to the position in the real space. The object is to provide a display device capable of doing so.

Means for Solving the Problems

[0006] In order to solve the above problems, a display device according to the present invention is a display device that overlays and displays virtual objects in an arrangement according to the position in the real space with respect to the field of view of an operator viewing the real space including the hoistway of an elevator or an image representing the real space. The display device includes a position recognition unit that recognizes the position of the real space including the hoistway of the elevator by recognizing a marker installed at a predetermined position in the real space, and a display control unit that overlays and displays construction reference information related to the construction of the elevator as the virtual object in the real space based on the marker. The display control unit has a configuration in which the display mode of the construction reference information corresponding to a predetermined floor is made different from the display mode of the construction reference information corresponding to a floor different from the predetermined floor.

[0007] In order to solve the above problems, a display control method according to the present invention is a display control method that overlays and displays virtual objects in an arrangement according to the position in the real space with respect to the field of view of an operator viewing the real space including the hoistway of an elevator or an image representing the real space. The display control method includes a position recognition step of recognizing the position of the real space including the hoistway of the elevator by recognizing a marker installed at a predetermined position in the real space, and a display control step of overlaying and displaying construction reference information related to the construction of the elevator as the virtual object in the real space based on the marker. In the display control step, the display mode of the construction reference information corresponding to a predetermined floor is made different from the display mode of the construction reference information corresponding to a floor different from the predetermined floor.

[0008] The display device according to each aspect of the present invention may be realized by a computer. In this case, a display control program for the display device that realizes the display device by operating the computer as each part (software element) included in the display device, and a computer-readable recording medium on which the program is recorded also fall within the scope of the present invention.

Advantages of the Invention

[0009] According to the present invention, there is an effect that an operator can easily determine whether the displayed construction reference information corresponds to a predetermined floor or information corresponding to other floors.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Modes for Carrying Out the Invention

[0011] Hereinafter, an embodiment of the present invention will be described in detail.

[0012] (Outline of Smart Glass Application Example) FIG. 3 is a diagram showing a state in which the smart glasses (display device) 1 according to the present embodiment is mounted on the safety helmet 11. In the present embodiment, it is assumed that an operator who performs elevator construction during the construction of a building wears the smart glasses 1 together with the safety helmet 11.

[0013] The smart glasses 1 are an optical transmissive head-mounted display. The wearer of the smart glasses 1 can view the external real space and also view the projected image within the real space. That is, the smart glasses 1 display, in an arrangement corresponding to the position in the real space, a virtual object as construction reference information regarding elevator construction as an overlay on the field of view of an operator who views the real space including the elevator landing and the hoistway. Note that, in the present embodiment, elevator construction work is assumed, but the present invention is not limited thereto, and it is applicable to any construction work as long as it is a construction work for attaching an object to be attached to a predetermined position in the real space.

[0014] FIG. 1 shows an example of a state in which virtual objects are overlaid and displayed in the real space in a state of looking down on the hoistway of an elevator. As shown in the figure, rails RL1 to RL3 and rail brackets BR1 to BR3 as virtual objects are displayed in the three-dimensional space in the real space. Thereby, the operator can confirm the position in the three-dimensional space where the construction work should be performed in the real space. Therefore, operations such as measuring and confirming the position where the construction work should be performed with a measuring tool in the real space are no longer necessary, and the workability can be improved. Details of the rails RL1 to RL3 and the rail brackets BR1 to BR3 as virtual objects will be described later.

[0015] In the present embodiment, the smart glass 1, which is an optical transmissive head-mounted display, is used as the display device. However, the present invention is not limited to this. For example, a video transmissive head-mounted display that displays both an image of the external real space captured by a camera and an image of a virtual object may be used. Further, as the display device according to the present invention, instead of a head-mounted display, for example, a portable information display terminal such as a tablet PC or a notebook PC, or a portable display and a camera connected to an information processing device by wire or wirelessly may be used.

[0016] The virtual object as construction reference information is displayed based on the 3D CAD data or 2D CAD data generated based on the design specifications of the construction target. These CAD data may be generated, for example, based on the information of the 3D model generated as BIM (Building Information Modeling). In BIM, various information such as the quantity, part number, dimensions, material, performance, and price of each part is included as object information in the 3D model. Therefore, at least any one of these information may be displayed as construction reference information.

[0017] (Details of the Configuration of the Smart Glass) FIG. 2 is a block diagram showing an outline of the configuration of the smart glass 1. As shown in the figure, the smart glass 1 includes a control unit 2, a projection unit 3, an imaging unit 4, a storage unit 5, a communication unit 6, and an audio input / output unit 7. The control unit 2 is a block that performs various information processing in the smart glass 1, and includes a display control unit 21, a position recognition unit 22, an input control unit 23, a warning control unit 24, and a space distance measurement unit 25.

[0018] The display control unit 21 controls the projection unit 3 so as to overlay and display a virtual object as construction reference information regarding the construction of the elevator. By projecting an image onto the half mirror, the projection unit 3 enables the operator wearing the smart glasses 1 to visually recognize the external real space while also visually recognizing the projected image. The display control unit 21 can display the virtual object as being three-dimensionally arranged at a predetermined position within the real space by making the projected image for the right eye different from the projected image for the left eye. As a result, the operator can recognize the construction position in the three-dimensional position of the real space.

[0019] The display control unit 21 reads out construction reference information including 3D CAD data or 2D CAD data stored in the storage unit 5 and controls the display of the virtual object. Here, the 3D CAD data or 2D CAD data also includes information indicating the positional relationship with the objects existing in the real space. Further, the display control unit 21 controls the display position of the virtual object based on the position of the real space recognized by the position recognition unit 22.

[0020] The display control unit 21 performs control to make the display mode of the construction reference information corresponding to a predetermined floor different from the display mode of the construction reference information corresponding to a floor different from the predetermined floor. Details of this display control will be described later.

[0021] Note that the display control unit 21 may acquire construction reference information from the outside via the communication unit 6. The communication unit 6 may communicate with a PC or server on the local network, for example, via a wireless LAN, or communicate with an external server via the Internet. This enables the update of the data stored in the storage unit 5 and the acquisition of new data.

[0022] The position recognition unit 22 recognizes the captured images of markers M1 and M2 (details will be described later) arranged at predetermined positions in the real space captured by the imaging unit 4, thereby recognizing the three-dimensional position of the smart glasses 1 in the real space and the three-dimensional direction in which the smart glasses 1 are facing. Also, the position recognition unit 22 can perform position recognition in 6DoF (Degree of Freedom). As a result, once the operator has the position recognized by the markers M1 and M2 for the position recognition unit 22, even if the viewing direction or the position is changed later, the virtual object can be visually recognized while maintaining the relative positional relationship with the real space.

[0023] The input control unit 23 is a block that receives and processes various instruction inputs from the operator. Examples of the instruction input include an input to the input interface as a virtual object. That is, an image as an input interface is displayed in the real space as a virtual object by the display control unit 21, and the instruction input from the operator is received by recognizing that the operator has virtually touched it with a finger or the like.

[0024] Also, for example, the instruction input from the operator may be received by voice input from the voice input / output unit 7. That is, the instruction input from the operator is received by the input control unit 23 recognizing the voice received by the voice input / output unit 7 and recognizing the instruction content from the operator.

[0025] Also, the instruction input from the operator may be received by an external input device via the communication unit 6. For example, various input devices such as a Bluetooth (registered trademark)-compatible input controller and a wireless keyboard communicate with the communication unit 6, and the instruction input is received by the input to the input device by the operator.

[0026] The warning control unit 24 controls any warnings issued to the operator wearing the smart glasses 1. Examples of warning methods include projecting and displaying the warning content by overlay display by the display control unit 21, and outputting a warning sound or a voice indicating the warning content from the audio input / output unit 7.

[0027] As a warning condition, there is the case where the position recognition unit 22 recognizes that the position of the operator is within a predetermined area near the hoistway. As a result, a warning is issued at a timing when the possibility of the operator falling into the hoistway increases, so that the safety of the operator can be further enhanced.

[0028] Note that at least one of the functions of the display control unit 21, the position recognition unit 22, the input control unit 23, and the warning control unit 24 provided in the control unit 2 may be realized by an external computer via communication.

[0029] (Example of the real space to be the elevator construction target) FIG. 4 is a perspective view showing an overview of the elevator landing 200 and the hoistway 102 that are the construction targets of the elevator. The figure shows a state in which the elevator door is not attached during the construction stage of the building. In this state, an opening 101 as an elevator entrance exists between the landing 200 and the hoistway 102. The landing 200 has a floor surface 201 and a wall surface 202.

[0030] Two markers M1 and M2 are provided on the floor surface 201. As described above, the markers M1 and M2 are used for the position recognition unit 22 to recognize the position of the real space by recognizing the markers M1 and M2 in the captured image. Each of the markers M1 and M2 is constituted by a quadrangular planar member, and a predetermined pattern is formed on the surface. By arranging such markers M1 and M2 at predetermined positions near the opening 101 on the floor surface 201 of the landing 200, the position recognition unit 22 can accurately recognize the positions of the landing 200 and the hoistway 102 in the three-dimensional space.

[0031] In this embodiment, two markers are provided for one boarding area, but the present invention is not limited to this. One marker may be provided, or three or more markers may be provided. The more markers are provided, the higher the accuracy of position recognition. However, if two markers are provided, position recognition can be performed with sufficient accuracy.

[0032] Further, in this embodiment, markers M1 and M2 are provided on the floor surface 201, but the present invention is not limited to this. Markers M1 and M2 may be provided on the wall surface 202. Also, in this embodiment, markers M1 and M2 are provided in the boarding area 200, but markers may be provided in the hoistway 102. However, it is preferable to provide markers M1 and M2 on the floor surface 201 for the following reasons.

[0033] For example, during the construction stage of a building where an elevator is installed, various processes are added to the wall surface 202 of the boarding area 200 at any time, and the shape is likely to change significantly. For example, it is conceivable that the wall does not exist initially and only columns are provided, and then a wall is provided at a certain point. That is, if the markers are reinstalled according to the environmental changes, the positions of the markers will change significantly, and the reference for the position of the overlay display will also change significantly. In this case, it is necessary to significantly change the positions of the markers with respect to the data for the overlay display, and in some cases, it may be necessary to change the definition of the positional relationship between the markers and the virtual objects themselves. On the other hand, when markers M1 and M2 are provided on the floor surface 201, since the floor surface 201 changes little in shape during the construction stage of the building, for example, even if the height is slightly changed by beautifying the floor surface and the markers M1 and M2 are reinstalled, the change in the positions of the markers M1 and M2 can be relatively small. Therefore, it is possible to make corresponding adjustments by fine-tuning the data for the overlay display. Markers M1 and M2 are arranged at predetermined positions on the floor surface 201 defined as marker installation positions in construction reference information including 3D CAD data or 2D CAD data. Since high accuracy is required for the installation positions of these markers M1 and M2 to accurately match the positions in the real space and the display positions of the virtual objects, it is preferable that they are provided based on a predetermined position reference object with high accuracy in the installation position in the real space. As an example of the position reference object, the piano wire 103 provided in the elevator shaft 102 can be mentioned.

[0034] FIG. 5 is a perspective view showing an overview when the opening 101 is viewed from the inside of the elevator shaft 102. As shown in the figure, during the construction stage of the building, two piano wires 103 and 103 are suspended from a predetermined position at the upper part in the elevator shaft 102. That is, the piano wires 103 and 103 are arranged at fixed positions without being affected by the construction state of each floor. Therefore, by using this piano wire as the position reference object, the accuracy of the installation positions of the markers M1 and M2 can be kept high. Note that, not limited to the piano wire, any wire member with a sufficiently thin diameter and sufficient strength may be used.

[0035] Also, for example, laser light may be used as the position reference object. That is, by irradiating laser light vertically downward from a predetermined position at the upper part in the elevator shaft 102 or irradiating laser light vertically upward from a predetermined position at the lower part in the elevator shaft 102, the laser light can be arranged at a fixed position similar to the piano wire. However, in the case of laser light, as the distance from the light source increases, the thickness of the light beam tends to become thicker, so depending on the height of the building, there may be an influence of errors due to the thickness of the light beam. Also, as the position reference object, the reference points (construction marks) marked on the landing of each floor may be used. Also, for one elevator, a plurality of landings 200 corresponding to each floor are provided. However, at each landing 200, it is preferable to install the marker at the same location with respect to the elevator entrance, that is, at the same relative position with respect to the boarding entrance.

[0036] In many cases, for the same elevator, most landing areas 200 have a similar structural layout. In this case, at each landing area 200, markers M1 and M2 can be installed at the same position relative to the elevator's landing opening. Thus, the marker installation work can be carried out with the same operation based on the same criteria. Therefore, the marker installation work can be carried out efficiently. Also, the data for overlay display can have a common data structure among multiple landing areas 200.

[0037] (Data adjustment process associated with the position change of markers M1 and M2) As described above, markers M1 and M2 are provided on the floor surface 201 in the landing area 200. Here, during the construction stage of the building, it is conceivable that the surface of the floor surface 201 is beautified, and the markers M1 and M2 are reinstalled before and after the beautification process. Due to this beautification process before and after, the height of markers M1 and M2 will be changed. Here, in many cases, the change in the height of markers M1 and M2 before and after the beautification process is about several millimeters to several centimeters. Also, the horizontal position of markers M1 and M2 can be made not to change before and after the beautification process.

[0038] When the position of the marker is changed significantly, it is necessary to greatly change the positional relationship between the virtual object and the marker in the data for overlay display. On the contrary, even if markers M1 and M2 are reinstalled due to the floor beautification process as described above, the change in the position of the marker is limited to the height direction and the amount of change is also small. Therefore, the data correction is relatively easy.

[0039] Specifically, the input control unit 23 causes the display control unit 21 to display an interface for changing the height positions of the markers M1 and M2 as a virtual object for the operator to input. More specifically, an interface with input options is displayed, and from among them, the operator selects a virtual object of the option corresponding to the height change of the markers M1 and M2. Then, by the operator's input to a virtual object such as a numeric keypad, the amount of height change corresponding to the actual construction is input, and accordingly, the input control unit 23 corrects the data indicating the positional relationship between the virtual object and the marker. As described above, the input of the amount of height change may also be received by the operator's input to an input device through communication between various input devices such as a wireless keyboard and the communication unit 6. (Details of Virtual Object) Next, the virtual object shown in FIG. 1 will be described. As described above, FIG. 1 shows a state in which a virtual object is overlaid and displayed in the real space looking down on the elevator shaft. More specifically, it is assumed that the overlaid display is when the operator is on the third floor, where FL1 indicates the floor level of the first floor, FL2 indicates the floor level of the second floor, and FL3 indicates the floor level of the third floor.

[0040] Markers M31 and M32 are provided on the floor surface of the third floor, markers M21 and M22 are provided on the floor surface of the second floor, and markers M11 and M12 are provided on the floor surface of the first floor. Markers M31 and M32, markers M21 and M22, and markers M11 and M12 correspond to the markers M1 and M2 shown in FIG. 4.

[0041] Markers M31 and M32, markers M21 and M22, and markers M11 and M12 are each provided with a two-dimensional code. In FIG. 1, markers M31 and M32 are each provided with two-dimensional codes QR1 and QR2. The two-dimensional codes QR1 and QR2 indicate the information of the floor where the markers M31 and M32 provided with the two-dimensional codes QR1 and QR2 are installed. That is, when the position recognition unit 22 performs image recognition on the markers M31 and M32 in the captured image, by also recognizing the two-dimensional codes QR1 and QR2, the information of the floor where the read markers M31 and M32 are arranged can also be recognized.

[0042] Note that the floor information included in the marker is not limited to the two-dimensional code. For example, the marker may be displayed with characters such as "3F", and this may be acquired as the floor information by character recognition. Further, the floor may be recognized by image recognition by attaching a color unique to the floor or displaying an image with a unique shape to the marker.

[0043] Rails RL1 to RL3 are virtual objects as rails for the elevator car to move along when ascending and descending. Rail RL3 indicates the rail arranged in the third floor section, rail RL2 indicates the rail arranged in the second floor section, and rail RL1 indicates the rail arranged in the first floor section.

[0044] Rail brackets BR1 to BR3 are virtual objects as brackets for fixing the rails to the wall surface in the hoistway. Rail bracket BR3 indicates the rail bracket arranged in the third floor section, rail bracket BR2 indicates the rail bracket arranged in the second floor section, and rail bracket BR1 indicates the rail bracket arranged in the first floor section.

[0045] As described above, the display control unit 21 performs control to make the display mode of the virtual object as construction reference information corresponding to a predetermined floor different from the display mode of the virtual object as construction reference information corresponding to a floor different from the predetermined floor. In the example shown in FIG. 1, the rails RL3 and rail brackets BR3 corresponding to the third floor where the worker is present and the rails RL2·RL1 and rail brackets BR2·BR1 corresponding to the second and first floors where the worker is not present are displayed in different modes.

[0046] More specifically, while the rails RL3 and rail brackets BR3 are displayed as solid lines, the rails RL2·RL1 and rail brackets BR2·BR1 are displayed as dashed lines. Note that the difference in the display mode is not limited to solid lines and dashed lines, and it may be different depending on at least any one of density, color, and pattern. Thereby, it is possible for the worker to more easily determine whether the displayed virtual object corresponds to the virtual object corresponding to the floor where the worker is present or the virtual object corresponding to other floors.

[0047] Also, while the shapes of the rails RL3 and rail brackets BR3 are displayed in detail, the shapes of the rails RL2·RL1 and rail brackets BR2·BR1 are displayed in a simplified manner. This also enables the worker to more easily determine whether the displayed virtual object corresponds to the virtual object corresponding to the floor where the worker is present or the virtual object corresponding to other floors.

[0048] Note that in the example shown in FIG. 1, the state of looking down from the third floor is shown, but the same display control as described above may be performed even in the state of looking up. For example, the rails RL3 and rail brackets BR3 corresponding to the third floor where the worker is present and the rails and rail brackets corresponding to the fourth floor and above where the worker is not present may be displayed in different modes.

[0049] In addition, the virtual objects are not limited to rails and rail brackets, and various members existing in the hoistway or the landing of the elevator may be displayed as virtual objects. In this way, when a large number of members existing on multiple floors are displayed as virtual objects, a situation is assumed in which it becomes difficult for the operator to recognize which members exist on the floor where the operator is present. On the other hand, according to the above display control, the members existing on the floor where the operator is present can be easily recognized. Since the operator often performs construction work on the members existing on the floor where he / she is present, such display control can improve work efficiency and suppress the occurrence of mistakes.

[0050] Also, in the above example, display control is performed to make the display mode of the virtual object corresponding to the floor where the operator is present different from the display mode of the virtual object corresponding to a floor different from the floor where the operator is present, but it is not limited to this. For example, display control may be performed to make the display mode of the virtual object corresponding to the floor where the operator is present, and the display mode of the virtual object corresponding to at least one of the floor one floor above and the floor one floor below the floor where the operator is present, different from the display mode of the virtual object corresponding to the other floors.

[0051] The virtual objects existing on the floor one floor above and the floor one floor below the floor where the operator is present are highly likely to affect the work on the floor where the work is being performed. Therefore, the operator can be made to recognize the virtual objects related to the floors adjacent above and below, and further work efficiency improvement and mistake occurrence suppression can be realized.

[0052] Note that display control may be performed to make the display mode of the virtual object corresponding to the floor where the worker is present different from the display mode of the virtual object corresponding to at least one of the floor one level above and the floor one level below the floor where the worker is present. That is, the display mode of the floor where the worker is present, the display mode of at least one of the floor one level above and the floor one level below the floor where the worker is present, and the display mode of the other floors may be different from each other. As a result, since the display mode of the virtual object is changed in these three classifications, the virtual objects related to the work can be presented to the worker in a more organized and easily understandable state.

[0053] (Details of Construction Reference Data) As described above, the display control unit 21 controls the display of the virtual object based on the construction reference information including the 3D CAD data or the 2D CAD data. FIG. 6 is a diagram showing the data structure of the construction reference data DAT indicating the construction reference information. As shown in the figure, the construction reference data DAT has construction reference data corresponding to each floor of the building to be constructed. Specifically, the construction reference data DAT has the construction reference data DAT1 for the first floor, the construction reference data DAT2 for the second floor, and the construction reference data DATN for the Nth floor. Note that the Nth floor indicates either a floor of the third floor or higher or a floor of the first basement floor or lower.

[0054] The construction reference data DAT1 for the first floor is construction reference data that is overlaid and displayed by the display control unit 21 when an operator is present on the first floor. The construction reference data DAT1 for the first floor includes rail drawing data DAT11 and rail bracket drawing data DAT12, etc. The rail drawing data DAT11 includes data for drawing the rails of the part corresponding to the first floor and data for drawing the rails of the parts corresponding to the floors other than the first floor. As described above, the display modes of the data for drawing the rails of the part corresponding to the first floor and the data for drawing the rails of the parts corresponding to the floors other than the first floor are different. Similarly, the rail bracket drawing data DAT12 includes data for drawing the rail brackets of the part corresponding to the first floor and data for drawing the rail brackets of the parts corresponding to the floors other than the first floor. As described above, the display modes of the data for drawing the rail brackets of the part corresponding to the first floor and the data for drawing the rail brackets of the parts corresponding to the floors other than the first floor are different. Note that the construction reference data DAT1 for the first floor may include data for drawing various other members.

[0055] The construction reference data DAT2 for the second floor includes rail drawing data DAT21 and rail bracket drawing data DAT22, etc. Also, the construction reference data DATN for the Nth floor includes rail drawing data DATN1 and rail bracket drawing data DATN2, etc. That is, the construction reference data DAT2 for the second floor and the construction reference data DATN for the Nth floor also have the same data structure as the construction reference data DAT1 for the first floor.

[0056] Note that the rail drawing data and the rail bracket drawing data in the construction reference data for each floor may have three types of data: the data of the display mode of the corresponding floor, the data of the display mode of at least one of the floor above and the floor below the corresponding floor, and the data of the display mode of the other floors.

[0057] According to the construction reference data with the above data structure, the display control unit 21 can perform display control to make the display mode of the virtual object corresponding to the floor where the worker is present different from the display mode of the virtual object corresponding to the floor different from the floor where the worker is present by reading the construction reference data corresponding to the floor where the worker is present.

[0058] Note that the construction reference data is not limited to the data structure as described above. For example, the rail drawing data and the rail bracket drawing data may include two types of data with different display modes as the data corresponding to each floor. In this case, the display control unit 21 can perform display control to make the above display modes different by selecting the data to be read according to the floor where the worker is present. That is, for the drawing data of various members, two types of data with different display modes are prepared for each part corresponding to each floor, and the display control unit 21 can read the drawing data corresponding to the floor where the worker is present and the drawing data corresponding to the floor where the worker is not present so that their display modes are different from each other.

[0059] (Flow of processing of elevator construction method) Next, the flow of processing of the elevator construction method according to the present embodiment will be described with reference to FIG. 7. When the construction related to the elevator is started, first, in step 1 (hereinafter referred to as S1), markers are installed on the floor surface at predetermined positions in the landings of each floor. At this time, as described above, the markers are installed based on the position reference objects such as the piano wire 103.

[0060] Next, in S2, the worker wears the smart glasses 1 on the floor where the work is to be performed, and the imaging unit 4 images the marker corresponding to the floor where the worker is present, and causes the position recognition unit 22 to perform position recognition processing. Here, the position recognition unit 22 identifies the floor where the worker is present based on the information included in the marker.

[0061] Note that the position recognition unit 22 may determine whether a marker is installed on the floor where the operator is present according to the distance to the marker. Thereby, even if markers on other floors are included in the imaging range by the imaging unit 4, it is possible to appropriately recognize the marker installed on the floor where the operator is present. The distance to the marker may be determined based on, for example, the size of the marker in the captured image, or a distance sensor such as LiDAR (Light Detection And Ranging) may be used.

[0062] When position recognition is performed, in S3, the display control unit 21 performs an overlay display of virtual objects corresponding to the floor recognized by the position recognition unit 22. Once the operator has the position recognized by the position recognition unit 22 by the markers M1 and M2, even if the viewing direction or the position is changed thereafter, the virtual object can be visually recognized while maintaining the relative positional relationship with the real space.

[0063] In this way, the floor where the operator is working is specified based on the marker, and the virtual object corresponding to that floor is displayed. Therefore, the operator can check the information suitable for that floor without the trouble of inputting the floor information. Also, the marker corresponding to the floor where the work is being done is recognized by the display device for position recognition. Therefore, the accuracy of the display position of the overlay display can be increased compared to the case where position recognition is performed based on markers existing on other floors.

[0064] Note that in the above example, the floor where the operator is present is specified based on the information included in the marker, but it is not limited thereto. For example, the numbers indicating the floor numbers written in the landing or elevator shaft may be recognized.

[0065] Further, for example, the worker may input the floor where the worker is present by input means such as an input interface using virtual objects. Also, by such input means, it may be possible to display virtual objects related to floors where the worker is not present by inputting the floor that the worker wants to display.

[0066] In this state, in S4, the worker performs construction work while checking the construction reference information overlaid by the display control unit 21. Specifically, there are positioning operations (operations of marking the installation position of the rail bracket on the wall surface in the hoistway) for installing the rail bracket at a position corresponding to the floor where the worker is present. The actual installation work of the rail bracket (work of attaching anchor bolts, work of attaching the rail bracket, etc.) may be performed with the smart glass 1 removed by the worker, or may be performed as it is with the smart glass 1 worn. Also, the construction work is not limited to the above-described positioning work of the rail bracket, and other work may be performed. Note that the construction work includes not only the actual construction work but also the check work after the construction work is performed.

[0067] Next, in S5, it is confirmed whether or not the floor where the worker is present has been changed. If Yes in S5, that is, when the worker has moved to another floor, the process returns to S2, and position recognition is performed based on the marker set on the floor after the movement. Thereby, even when the floor where the worker performs work has been moved, the marker corresponding to the floor where the work is being performed can be re-recognized by the display device, so that the accuracy of the display position of the overlay display can be maintained high.

[0068] If No in S5, that is, when the floor where the worker is present has not been changed, in S6, it is confirmed whether the construction work has ended. If No in S6, that is, if the construction work is continuing, the process from S3 is repeated, and if Yes in S6, that is, if the construction work has ended, the process ends.

[0069] According to the above construction method and the smart glass 1, the construction of the elevator can be efficiently carried out. Such an effect also contributes to the achievement of, for example, Goal 11.c of the Sustainable Development Goals (SDGs) proposed by the United Nations, which states that "support the development of sustainable and resilient buildings using local materials in developing countries through financial and technical support, etc."

[0070] 〔Example of Realization by Software〕 The function of the smart glass 1 (hereinafter referred to as the "device") is a program for causing a computer to function as the device, and can be realized by a program for causing a computer to function as each control block of the device (especially each part included in the control unit 2).

[0071] In this case, the above device includes a computer having at least one control device (for example, a processor) and at least one storage device (for example, a memory) as hardware for executing the above program. By executing the above program with this control device and storage device, each function described in the above embodiments is realized.

[0072] The above program may be recorded on one or more computer-readable recording media, not temporarily. This recording medium may or may not be provided in the above device. In the latter case, the above program may be supplied to the above device via any wired or wireless transmission medium.

[0073] In addition, part or all of the functions of each of the above control blocks can also be realized by a logic circuit. For example, an integrated circuit in which a logic circuit functioning as each of the above control blocks is formed is also included in the scope of the present invention. In addition to this, for example, it is also possible to realize the functions of each of the above control blocks by a quantum computer.

[0074] In addition, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI may operate in the above control device, or may operate in another device (for example, an edge computer or a cloud server, etc.).

[0075] (Summary) The display device according to Aspect 1 of the present invention is a display device that overlays and displays a virtual object in an arrangement according to the position of a real space on the field of view of an operator viewing the real space including the hoistway of an elevator, or on an image representing the real space, and recognizes the position of the real space including the hoistway of the elevator by image recognition of a marker installed at a predetermined position in the real space. A position recognition unit, and a display control unit that overlays and displays construction reference information related to the construction of the elevator as the virtual object in the real space based on the marker, and the display control unit is different from the display mode of the construction reference information corresponding to a predetermined floor. And a configuration that makes the display mode of the construction reference information corresponding to a floor different from the predetermined floor different.

[0076] According to the above configuration, an operator can easily determine whether the displayed construction reference information corresponds to information for a predetermined floor or information for other floors.

[0077] The display device according to Aspect 2 of the present invention may be configured such that, in the above Aspect 1, the display control unit displays the floor where the operator is present as the predetermined floor.

[0078] According to the above configuration, it is possible to easily determine whether the displayed construction reference information corresponds to the floor where the operator is present or information corresponding to other floors.

[0079] In the display device according to Embodiment 3 of the present invention, in the above Embodiment 2, in addition to the floor where the worker is present, the display control unit may be configured to display at least one of the floor one level above and the floor one level below the floor where the worker is present as the predetermined floor.

[0080] According to the above configuration, it is possible to make the worker recognize the construction reference information regarding the floors adjacent above and below, which are highly likely to affect the floor where the work is being carried out.

[0081] In the display device according to Embodiment 4 of the present invention, in the above Embodiment 1, the display control unit may be configured to make the display mode of the construction reference information corresponding to a predetermined floor different from the display mode of the construction reference information corresponding to a floor different from the predetermined floor by at least one of density, color, and pattern.

[0082] According to the above configuration, it is possible for the worker to more easily determine whether the displayed construction reference information corresponds to information for a predetermined floor or information for another floor.

[0083] In the display device according to Embodiment 5 of the present invention, in the above Embodiment 1, the position recognition unit specifies the floor where the worker is present based on the information included in the recognized marker, and the display control unit may be configured to display the construction reference information corresponding to the floor recognized by the position recognition unit.

[0084] According to the above method, the floor where the worker is working is specified based on the marker recognized by the display device, and the construction reference information corresponding to that floor is displayed. Therefore, the worker can confirm the information suitable for that floor without the trouble of inputting floor information or the like.

[0085] The display control method according to aspect 6 of the present invention is a display control method for overlay-displaying virtual objects in a layout according to the position in a real space in the field of view of an operator viewing the real space including the elevator hoistway or an image representing the real space, the method including: a position recognition step of recognizing the position of the real space including the elevator hoistway by performing image recognition on a marker installed at a predetermined position in the real space; and a display control step of causing construction reference information regarding construction of the elevator to be overlay-displayed in the real space as the virtual object based on the marker, wherein, in the display control step, the display mode of the construction reference information corresponding to a predetermined floor is made different from the display mode of the construction reference information corresponding to a floor different from the predetermined floor.

[0086] The display control program according to aspect 7 of the present invention is a display control program for causing a computer to function as the display device in the above aspect 1, and causes the computer to function as the position recognition unit and the display control unit.

[0087] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims. Embodiments obtained by appropriately combining technical means disclosed in different embodiments are also included in the technical scope of the present invention.

Explanation of Reference Numerals

[0088] 1 Smart Glasses (display device) 2 Control Unit 3 Projection Unit 4 Imaging Unit 5 Storage Unit 6 Communication Unit 7 Audio Input / Output Unit 11 Safety Helmet 21 Display Control Unit 22 Position Recognition Unit 23 Input Control Unit 24 Warning Control Unit 25 Space Distance Measurement Unit 101 Opening 102 Elevator shaft 103 Piano wire 200 Landing 201 Floor surface 202, 300 Wall surface BR1, BR2, BR3 Rail brackets DAT1, DAT2 Construction reference data for each floor DAT11, DAT21, DATN1 Rail drawing data DAT12, DAT22, DATN2 Rail bracket drawing data M1·M2, M11·M12, M21·M22, M31·M32 Markers QR1·QR2 2D barcodes RL1, RL2, RL3 Rails

Claims

1. A display device that displays a virtual object in an overlay manner in accordance with a position in a real space in a field of view of a worker viewing a real space including an elevator shaft or an image representing the real space, the display device comprising: a position recognition unit that recognizes a position in the real space including the elevator shaft by performing image recognition on a marker installed at a predetermined position in the real space; a display control unit that overlays construction reference information regarding the construction of the elevator as the virtual object in the real space based on the marker, A display device in which the display control unit differentiates the display mode of the construction reference information corresponding to a specified floor from the display mode of the construction reference information corresponding to a floor other than the specified floor.

2. The display device according to claim 1 , wherein the display control unit displays a floor where the worker is present as the predetermined floor.

3. The display device according to claim 2 , wherein the display control unit displays, in addition to the floor on which the worker is present, at least one of a floor one level above and a floor one level below the floor on which the worker is present as the specified floor.

4. The display device described in claim 1, wherein the display control unit differentiates the display mode of the construction reference information corresponding to a specified floor from the display mode of the construction reference information corresponding to a floor other than the specified floor by at least one of density, color, and pattern.

5. The position recognition unit identifies the floor on which the worker is present based on information included in the recognized marker, The display device according to claim 1 , wherein the display control unit displays the construction reference information according to the floor recognized by the position recognition unit.

6. 1. A display control method for overlaying a virtual object on a field of view of a worker viewing a real space including an elevator shaft or on an image representing the real space, in an arrangement according to a position in the real space, comprising: a position recognition step of recognizing a position in the real space including the elevator shaft by performing image recognition on a marker installed at a predetermined position in the real space; and a display control step of overlaying and displaying construction reference information regarding the construction of the elevator as the virtual object in the real space based on the marker, A display control method, in which, in the display control step, the display mode of the construction reference information corresponding to a specified floor is made different from the display mode of the construction reference information corresponding to a floor other than the specified floor.

7. 2. A display control program for causing a computer to function as the display device according to claim 1, the display control program causing a computer to function as the position recognition unit and the display control unit.

Images