Augmented reality-based construction performance registration method

Abstract

Disclosed is an augmented reality-based construction performance registration method using a mobile device in which an application for construction performance registration is installed and a camera for photographing a construction performance registration object is mounted. The disclosed method comprises the steps of: implementing augmented reality related to a construction site so as to be displayed via a mobile device; displaying a start point on a virtual position of a registration object of the augmented reality when a first touch input of an object selection button is made by a user of the mobile device in a state where a cross pointer is located at an arbitrary point on the virtual position of the registration object in the augmented reality; displaying an end point when a second touch input of the object selection button is made by the user of the mobile device after the mobile device is panned and the cross pointer is moved by a predetermined distance along the virtual position of the registration object of the augmented reality, and displaying, via the mobile device, the registration object of the augmented reality from the start point to the end point on the virtual position of the registration object; and when an input is made by the user for opening a registration information input window, displaying, via the mobile device, the registration information input window together with the registration object of the augmented reality from the start point to the end point.

Description

Augmented Reality-Based Construction Performance Registration Method

[0001] The present invention relates to a construction performance registration method based on augmented reality, and more particularly, to a construction performance registration method based on augmented reality that uses a construction performance registration application installed on a mobile device for efficient management of construction performance.

[0002] High-tech industries are generally defined as highly technology-intensive and have a significant impact on related industries. For example, they encompass advanced technologies such as electronics, new materials, biotechnology, computers, and information and communications. Building facilities for these high-tech industries often involves the installation of numerous pipes and wiring. For example, building a semiconductor production line requires the installation of gas and chemical pipes, electrical wiring, and fire extinguishing pipes, all of which are essential for the various processes involved. Various applications are known to enable workers to register the construction status of various components, including these pipes, during the construction of semiconductor production lines, and to enable clients to clearly understand this status. One such method is utilizing augmented reality (AR) to register and update construction records.

[0003] Augmented reality (AR) is a technology that adds computer-generated information to the user's five senses of the real world. In other words, it superimposes virtual objects (VR) onto the real world the user sees. Unlike VR, which constructs reality solely within a computer, AR superimposes virtual information onto the real world.

[0004] Methods for managing plant construction sites or interior construction sites using such augmented reality have been disclosed in Korean Patent No. 10-2462229 (registered on October 28, 2022), etc. These conventional technologies, as well as conventional methods for managing construction sites or registering or modifying construction performance using augmented reality, have several drawbacks. Namely, in a construction site, especially when fire pipes, gas pipes, chemical pipes, and electrical wiring are located on the ceiling, there is a problem in that when using a mobile device (e.g., a tablet PC) to sense the actual reality with a camera mounted on the mobile device and match it with virtual reality to implement augmented reality, one must hold the mobile device with one hand and perform tasks such as touching, dragging, and entering information with the other hand. Due to the weight of the mobile device itself, it is difficult to perform tasks other than very short ones when held with one hand. Furthermore, when registering construction performance, designating or de-designating a section of pipe to be modified, or a specific area of ​​an entire tile, one must hold the mobile device with one hand while the other. This frequently results in designation errors, hindering smooth progress and significantly delaying work progress. Therefore, solutions to address these issues are urgently needed in the relevant technical field.

[0005] The problem to be solved by the present invention is to provide an improved method for registering construction performance using augmented reality to solve the problems of the conventional method for managing a construction site using augmented reality through a mobile device, namely, the fact that it is difficult to proceed with anything other than a very short task that can be held with one hand due to the weight of the mobile device itself, and also that when registering construction performance or designating or de-designating a certain section of pipe to be modified or a certain area of ​​the entire tile, etc., the mobile device must be held with one hand and the other hand must be used, which increases the frequency of designation errors and, as a result, lowers the accuracy of the work and prevents smooth progress or delays in the progress of the work.

[0006] According to one aspect of the present invention for solving the above problem, a method for registering construction performance based on augmented reality using a mobile device having a construction performance registration application installed and a camera for photographing a construction performance registration object, comprises the steps of: (a) executing the construction performance registration application to match the actual reality of the construction site being photographed by the camera with a virtual reality regarding the construction site, and implementing the augmented reality regarding the construction site to be displayed through the mobile device; (b) when a user of the mobile device makes a first touch input of an object selection button while a cross pointer is positioned at an arbitrary point on the virtual position of the registration object in the augmented reality, a step of displaying a starting point on the virtual position of the registration object of the augmented reality; (c) after the step (b), when the user of the mobile device pans the mobile device and the cross pointer moves along the virtual position of the registration object of the augmented reality by a predetermined distance, and when a user of the mobile device makes a second touch input of the object selection button, an end point is displayed on the virtual position of the registration object of the augmented reality. A step of displaying the registration target of the augmented reality from the starting point to the ending point through the mobile device, and (d) when there is an input by the user of the mobile device for opening the registration information input window, a step of displaying the registration information input window together with the registration target of the augmented reality from the starting point to the ending point through the mobile device.

[0007] According to one embodiment, the registration object may be any one of gas pipes, fire pipes, chemical pipes, and electrical wiring.

[0008] According to one embodiment, step (c) may include a step of displaying a selection assistance line of a straight line type from the starting point to a position on the virtual position of the registered object corresponding to a panning end position of the mobile device when the mobile device is panned by a user of the mobile device after step (b).

[0009] According to one embodiment, the method further comprises the step of displaying information about the registration object on top of the cross pointer.

[0010] In one embodiment, the registration object may be a tile member located on the floor or side.

[0011] In one embodiment, in step (b), a selection circle may be further displayed around the cross pointer with the cross pointer as the center.

[0012] According to one embodiment, in step (c), when the selection circle moves along the virtual position of the registration object of the augmented reality, tile members included within the selection circle can be selected in sequence.

[0013] According to one embodiment, the selection circle may have a diameter that can be increased or decreased by a touch input from a user of the mobile device to expand or reduce a selection area.

[0014] The present invention provides a construction performance registration method using an improved form of augmented reality, thereby solving problems in conventional construction site management methods using augmented reality, thereby increasing work duration, and has the effect of increasing work progress speed by increasing work accuracy in designating or de-designating a certain section of pipe or a certain area of ​​the entire tile for which construction performance is to be registered or modified.

[0015] FIG. 1a is a drawing illustrating a mobile device (1) used in a construction performance registration method using augmented reality according to one embodiment of the present invention and a display state of the mobile device (1) when an application for construction performance registration is executed on the mobile device (1).

[0016] FIG. 1b is a drawing showing an example of implementing augmented reality by executing an application for registering construction performance installed on a mobile device (1) shown in FIG. 1a to implement a method for registering construction performance using augmented reality according to one embodiment of the present invention.

[0017] Figures 2 to 10 are drawings for explaining the process of registering construction performance for piping or wiring according to one embodiment of the present invention.

[0018] FIGS. 11 to 17 are drawings for explaining the process of registering construction performance for floor tiles according to one embodiment of the present invention.

[0019] As mentioned above, the present invention is a method for increasing work efficiency by performing work by utilizing an application for registering construction performance in augmented reality installed in a mobile device when registering construction performance.

[0020] Hereinafter, preferred embodiments of the present invention will be described with reference to the attached drawings. It should be noted that the attached drawings and embodiments are simplified and illustrative in order to help those skilled in the art to understand the present invention. In addition, although the term "construction performance registration" is only mentioned in this specification, it should be noted that this can be interpreted comprehensively to include not only registration of construction performance but also modifications to construction details. Furthermore, in the present invention, the subject of construction performance registration may be any one of gas pipes, fire pipes, chemical pipes, and electrical wiring required when constructing facilities in the high-tech industrial field.

[0021] FIG. 1A is a drawing illustrating a display state of a mobile device (1) used in a construction performance registration method using augmented reality according to one embodiment of the present invention and a state in which a construction performance registration application is executed on the mobile device (1), FIG. 1B is a drawing illustrating an example of implementing augmented reality by executing a construction performance registration application installed on the mobile device (1) illustrated in FIG. 1A to implement a construction performance registration method using augmented reality according to one embodiment of the present invention, FIGS. 2 to 10 are drawings for explaining a process of registering construction performance for pipes according to one embodiment of the present invention, and FIGS. 11 to 17 are drawings for explaining a process of registering construction performance for floor tiles according to one embodiment of the present invention.

[0022] First, referring to FIG. 1A, a tablet PC is illustrated as an example of a mobile device (1), but it may be a smartphone or a dedicated computing device for implementing augmented reality, in addition to a tablet PC. The mobile device (1) is installed with an application for registering construction performance and is equipped with a camera for taking pictures of objects for registering construction performance. In addition to the camera, the mobile device (1) is basically equipped with a processing device and memory, and a detailed description of these basic components is omitted. When the application for registering construction performance installed on the mobile device (1) is executed, various buttons and pointers (8, 9, 11, 13, 16, 17, 19) are displayed on the screen (2) so that the user of the mobile device (1) can operate it by scrolling or touching with a finger, as shown in the drawing located below FIG. 1A. Various buttons (8, 9, 13, 15, 17, 19) are positioned at positions where the user can utilize the left thumb and the right thumb, taking into account the state in which the user holds the mobile device (1). In addition, the panning of the mobile device (1), which is one of the key elements in the augmented reality-based construction performance registration of the present invention, is symbolically represented by left and right arrows in the drawing located above Fig. 1a. Here, panning is used as a term meaning physically moving the mobile device (1) equipped with a camera up, down, left and right in a state of filming the actual reality where the construction performance registration object is located, i.e., the construction site, or in a state of implementing augmented reality. In other words, it is used as a term meaning the physical movement of the mobile device (1) in a state of implementing augmented reality.In the buttons and pointers (8, 9, 11, 13, 16, 17, 19), reference numeral 11 is a cross pointer, which is located in the center on the screen of the mobile device (1), and is a reference pointer used to designate a construction performance registration target by aiming the cross pointer (11) at a construction member by holding the mobile device (1) with both hands when performing a registration task. Reference numeral 8 is an initialization button, which is a button used to initialize the virtual reality implemented on the screen (2) of the mobile device (1). Reference numeral 9 is a button used to aim the construction member to be deleted with the cross pointer (11), and then use the construction object selection button (13) to input a touch (drag) and then delete the construction member. Reference numeral 13 is a construction object selection button (13), which is used to select the entire construction member, create an end point, or create a start point by touching or touching and then dragging upward or downward, thereby selecting a portion of the construction performance registration object. Reference numeral 16 is a button for adjusting the transparency of the screen (2) of the mobile device (1), and is configured so that, for example, the opacity can decrease when dragging downward, and the opacity can increase when dragging upward. Reference numeral 17 is a button for exiting the modification work screen when the construction performance deletion work is finished. Reference numeral 19 is a button for displaying a performance registration information input window on the screen. When touched by a user, the performance registration information input window is displayed together with augmented reality through the screen of the mobile device (1). The performance registration information input window includes a section for entering performance registration information and a section for performance management.

[0023] FIG. 1b is a drawing showing an example of implementing augmented reality by executing an application for registering construction performance installed on a mobile device (1) and matching it with virtual reality while taking a picture of the actual reality where a specific construction performance registration target is located. It exemplifies how various buttons and pointers (8, 9, 11, 13, 16, 17, 19) shown in the drawing located below FIG. 1a are displayed on the screen of a mobile device implementing augmented reality. It should be noted that the augmented reality illustrated in FIG. 1b is one example for explaining the features of the present invention.

[0024] According to one embodiment of the present invention, a method for registering construction performance based on augmented reality includes a step of executing a construction performance registration application to match the actual reality of a construction site captured by a camera with a virtual reality about the construction site, thereby implementing the augmented reality about the construction site to be displayed on a screen through a mobile device (1). The virtual reality about the construction site can be implemented to express various materials to be installed at the construction site, such as various pipes, floor or wall tiles, etc., in various colors. Next, in the augmented reality implemented through the mobile device, when a cross pointer (11) is positioned at an arbitrary point on the virtual position of the registration target (i.e., the construction performance registration target), that is, in the entire position where the construction performance registration target is installed, and there is a first touch input (e.g., downward drag) of an object selection button (13) by the user of the mobile device (1), the method includes a step of displaying a starting point (see 12 of FIG. 3) on the virtual position of the registration target in the augmented reality. Thereafter, when the user moves the mobile device left and right or up and down, i.e., pans, in a direction in which the registration target of the mobile device (1) can be designated at a certain section, and there is a second touch input (e.g., upward drag) by the user for the target selection button (13), an end point is indicated, and the registration target of augmented reality is displayed from the start point to the end point on the virtual position of the registration target of augmented reality through the screen of the mobile device (1). Through this process, when the construction performance of the registration target is designated using a hand or a pen as in the past, the accuracy is low, and especially, when the registration target is a pipe, the difficulty that has existed in the past can be solved in the task of holding the mobile device with one hand and drawing a line along the pipe with the other hand in augmented reality.In particular, in the present invention, when panning a mobile device, even if the panning is performed in the general direction of the line where the registration object is located, rather than precisely following the line, the vector component in the direction in which the registration object extends is followed so that the section of the registration object can be accurately designated. Then, when the user of the mobile device (1) touches the button of reference numeral 19 (Fig. 1a) to open the registration information input window, the registration information input window is displayed on the screen (2) of the mobile device (1) together with an augmented reality of a construction site including the designated registration object from the start point to the end point (as illustrated in Fig. 7, the screen can be displayed by splitting it into left and right sides).

[0025] Referring to FIGS. 2 to 10, a construction performance registration process in a case where the construction performance registration target is a pipe or a wiring is described according to one embodiment of the present invention.

[0026] Referring to FIG. 2, the construction performance registration method based on augmented reality of the present invention first executes a construction performance registration application to match the actual reality of the construction site captured by a camera mounted on a mobile device (1, FIG. 1a) with a related virtual reality, thereby implementing an augmented reality regarding the construction site to be displayed on the screen through the mobile device (1). In FIG. 2, (a) is an example of the implemented augmented reality, and (b) separately shows only the virtual line (10) of the construction performance registration object and the cross pointer (11) located at the center of the mobile device (1). That is, when the mobile device (1) is appropriately moved in a state where augmented reality is implemented and positioned at the part where the construction performance registration object is located, the virtual line (10) of the construction performance registration object is displayed as in FIG. 2.

[0027] Referring to FIG. 3, when a user drags the object selection button (13) downward (first touch input) while the cross pointer (11) is positioned on the virtual indication of the construction performance registration target object as in FIG. 2 in the augmented reality implemented through the mobile device (1), a starting point (12) is displayed on the virtual position (10) of the registration target object of the augmented reality. In FIG. 3, (a) is an example of the implemented augmented reality, and (b) separately shows only the virtual line (10), the cross pointer (11), and the starting point (12) of the construction performance registration target object.

[0028] Referring to FIG. 4, after the starting point (12) is displayed, when the user moves the mobile device (1) to the left (i.e., pans), a straight-line selection assistance line (15) is displayed on the augmented reality along the virtual position (10) of the registration target from the starting point (12). The selection assistance line (15) is displayed to a position on the virtual position of the registration target corresponding to that position until the end of the panning of the mobile device (1). Here, panning of the mobile device (1) means moving the registration target in a direction that can designate a certain section, i.e., from the left to the right on the drawing. In addition, information about the registration target can be displayed on the upper portion of the cross pointer (11) (reference numeral 14). The information about the registration target may be, for example, the number of selected member elements or the sum of the quantities of the selected member elements (e.g., area, length, etc.), but is not limited thereto. In Fig. 4, (a) is an example of augmented reality in which a selection assistance line (15) is displayed according to the panning of a mobile device (1), and (b) separately shows only the selection assistance line (15), the starting point (12), and the virtual line (10) of the construction performance registration target.

[0029] Referring to FIG. 5, if the user drags the object selection button (13) upward (i.e., second touch input) after the panning of the mobile device (1) is completed following FIG. 4, an end point (not separately indicated in the drawing, but may be the rightmost point of the registration object SE 13) is displayed, and the registration object of augmented reality is displayed from the start point to the end point on the virtual position of the registration object of augmented reality through the screen of the mobile device (1). In FIG. 5, the selected registration objects are exemplified as SE1, SE2, and SE3.

[0030] Referring to FIG. 6, steps for marking registered objects on the augmented reality screen, such as ① and ②, may be further included.

[0031] Referring to FIG. 7, when a user of a mobile device (1) touches the button of reference numeral 19 (FIG. 1a) to open a registration information input window (20), an augmented reality image (left screen) of a construction site including a registration target is displayed along with a registration information input window on the right side of the screen. The registration information input window (20) is configured to display information such as work packages, activities, selection elements (number of selected sub-elements), quantity (sum of quantities of selected sub-elements), etc. so that a worker or a purchaser can check them (reference numerals 21 to 25). In addition, reference numeral 26 is a file attachment function button, and reference numeral 27 is a confirmation and registration completion button.

[0032] Referring to Fig. 8, the registration information input window (20) further displays a preview button (52) and an edit button (51) for the attached file.

[0033] FIGS. 9 and 10 are drawings related to modification of performance details. Referring first to FIG. 8, in the registration information input window (20), previously registered performances can be checked by date, and can be sorted by date with the most recent performance at the top. For example, for one performance (registration target), it is possible to look up detailed registration information from the performance management section in the registration information input window (20), and an attached file can be checked. As shown in FIG. 8, the member selection button can be touched to select all or part of the member that needs to be modified. The selection of a part is substantially the same process as in the previous registration process. That is, as shown in FIG. 9, aim the cross pointer (11) in the center of the screen at the member (SE23) to be deleted, drag the object selection button (13) to select it, and then delete it. The deleted member can be displayed in red, for example.

[0034] Next, the process of registering construction performance for floor tiles according to one embodiment of the present invention will be described with reference to FIGS. 11 to 17. While floor tiles are exemplified, it should be noted that the process can also be applied to wall tiles.

[0035] Referring to Fig. 11, with the augmented reality screen implemented through a mobile device and the registration information input window (20) simultaneously displayed on the screen, floor construction is selected from the work package section (21) in the registration information input window (20). Then, the absence selection button (2) is touched. Reference symbol F1 represents the actual floor surface on which tiles are installed.

[0036] Referring to Fig. 12, first, the cross pointer (11) located at the center of the screen of the mobile device (1, Fig. 1a) is aimed at the floor tile area (SE31) where construction performance registration is required.

[0037] Referring to Fig. 13, as in Fig. 12, while aiming at the floor tile area (SE31) requiring construction performance registration with the cross pointer (11), the object selection button (13) is first touched or scrolled. Here, the starting point is displayed in the form of a predetermined rectangular area. Then, while panning the mobile device (1) along the virtual position of the registration target, tile members requiring construction performance registration are designated. For selection of floor tiles, a selection circle centered on the cross pointer (11) is displayed around the cross pointer (11). In addition, as illustrated in Fig. 13, the diameter of the selection circle is configured to be expanded in order to increase the selection speed by increasing the number or range of tiles designated at one time. In addition, information regarding the registration target, including the number of tiles that are registration targets and the sum of the areas of the tiles, may be further displayed inside the selection circle. All floor tiles requiring construction performance registration are selected by panning the mobile device (1), and then by second-touch input or scrolling the target selection button (13), all floor tile areas requiring construction performance registration are selected.

[0038] Referring to Fig. 14, a selection circle (41, 41') and a cross pointer (11) located at the center thereof are separately illustrated. As illustrated in Fig. 14, in order to increase the diameter of the selection circle (from 41 to 41'), a touch point (42) that a user can touch and drag with a hand may be present in the selection circle (41, 41').

[0039] Referring to Fig. 15, if a specific absence is selected in the performance management section within the registration information input window (20) and modifications are required, the modification work can be started by touching the modification button indicated by reference numeral 51.

[0040] Referring to FIG. 16, after the user touches the edit button (51) in FIG. 15, the augmented reality displayed on the screen of the mobile device is displayed, and after touching the absence selection button (30), the cross pointer is aimed at the area of ​​the floor tile that needs to be edited in the same manner as the previous registration process, the object selection button (13) is touched, and the mobile device is panned so that when all the floor tiles that need to be edited are selected, the object selection button (13) is touched again to finish the selection of the object (absence) that needs to be edited. At this time, too, information about the object to be edited, including the number of tiles and the sum of the tile areas, is reflected in real time and displayed inside the selection circle. In case of deletion, after selecting all the objects to be deleted, the delete button (9) is touched to delete them all. For example, the deleted object may be displayed in red. As shown in Fig. 17, after the deletion operation is completed, the current performance history modification screen is closed by touching the X button indicated by reference number 17, and as shown in Fig. 15, the performance registration information input window is opened again and the performance modification is registered by touching the modification button (51, Fig. 15) located at the bottom in the performance management section.

[0041] In the above, a method for registering construction performance based on augmented reality using a mobile device according to embodiments of the present invention has been described. By utilizing the method for registering construction performance based on augmented reality according to the present invention, it is possible to increase the duration of work using a mobile device, that is, the duration of work that a worker can perform at one time by holding the mobile device with both hands, as well as to improve the accuracy of the work and increase the speed of the work progress.

[0042] It should be noted that the scope of the present invention is defined not by the above description but by the claims appended hereto.

Claims

1. In a method for registering construction performance based on augmented reality using a mobile device equipped with a camera for taking pictures of the object of construction performance registration and an application for registering construction performance is installed, (a) A step of executing the application for registering the construction performance, matching the actual reality of the construction site captured by the camera with the virtual reality of the construction site, and implementing an augmented reality of the construction site to be displayed through the mobile device; (b) a step of displaying a starting point in the virtual position of the registration object of the augmented reality when a user of the mobile device makes a first touch input of an object selection button while a cross pointer is positioned at any point on the virtual position of the registration object of the augmented reality; (c) a step of displaying an end point when a second touch input of the object selection button is made by the user of the mobile device after the step (b) above is performed by the user of the mobile device to pan the mobile device so that the cross pointer moves along the virtual position of the augmented reality registration object by a predetermined distance, and displaying the augmented reality registration object from the start point to the end point on the virtual position of the augmented reality registration object through the mobile device; and (d) A method for registering construction performance based on augmented reality, comprising: a step of displaying the registration information input window together with the registration target of the augmented reality from the starting point to the ending point through the mobile device when there is an input by the user of the mobile device for opening the registration information input window; 2. In paragraph 1, the registration object is: An augmented reality-based construction performance registration method characterized by any one of gas pipes, fire pipes, chemical pipes, and electrical wiring.

3. In claim 2, step (c) An augmented reality-based construction performance registration method, characterized in that it comprises a step of displaying a selection assistance line of a straight line type from the starting point to a position on the virtual position of the registration object corresponding to the panning end position of the mobile device when the mobile device is panned by the user of the mobile device after the step (b).

4. In claim 2, An augmented reality-based construction performance registration method, characterized in that it further includes a step of displaying information about the registration target object on the upper part of the cross pointer.

5. In claim 1, the registration object is: An augmented reality-based construction performance registration method characterized by a tile absence located on the floor or side.

6. In claim 5, An augmented reality-based construction performance registration method, characterized in that in the step (b) above, a selection circle is further displayed around the cross pointer with the cross pointer as the center.

7. In claim 6, A method for registering construction performance based on augmented reality, characterized in that, in the step (c), when the selection circle moves along the virtual position of the registration target of the augmented reality, tile members included within the selection circle are selected in sequence.

8. In claim 6, An augmented reality-based construction performance registration method, wherein the above selection circle is characterized in that the diameter thereof increases or decreases by a touch input for expanding or reducing the selection area of ​​the user of the mobile device.

Images