Ar marker for inspection of opening, inspection system for opening, inspection method for opening and program
The AR marker and opening inspection system leverage MR/AR technology to accurately determine and verify the position of openings in structures, addressing the inefficiencies and costs of existing methods by using a pattern-fitted marker and a terminal device for precise alignment and error detection.
Patent Information
- Application Number
- JP2023087114
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-12-06
AI Technical Summary
Existing methods for determining the construction position of openings in structures, such as void pipes, are time-consuming, costly, and do not accurately verify the accuracy of the construction position, with technologies like LiDAR and AI image recognition being expensive and limited in verification capabilities.
An AR marker with a pattern indicating the core position of an opening is used, fitted into the opening, and combined with a terminal device that employs Mixed Reality (MR)/Augmented Reality (AR) technology to determine the opening's position in real space, utilizing a system that includes a terminal device with a camera, self-position estimation sensor, and processing unit to superimpose drawing data accurately on the real space, and compare it with the defined position.
The AR marker and opening inspection system enable precise determination of the opening's position in real space, reducing processing load and ensuring accurate alignment and error detection, thereby improving efficiency and reducing costs.
Smart Images

Figure 2024170124000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an AR marker for inspecting openings, an opening inspection system, an opening inspection method, and a program, and in particular to a technology for determining the position of a constructed opening in real space using MR (Mixed Reality) / AR (Augmented Reality) technology. [Background technology]
[0002] Openings are sometimes provided in structures and components. For example, in buildings made of reinforced concrete (including steel), through holes called sleeves are provided to pass pipes and ducts through concrete beams, floors, walls, foundations, etc. Usually, sleeves are formed by installing pipes called void pipes or sleeve pipes, or rectangular steel frames called box-cut openings (collectively referred to as void pipes, etc. below) in a specified position before pouring concrete. If a problem is discovered in a later process, such as the construction position of the void pipes being different from the construction drawings, huge costs will be incurred for repairs. For this reason, the construction position of the void pipes, etc. is checked many times using measures, measuring instruments, etc. This work process requires a considerable amount of time and effort.
[0003] To solve these problems, methods have been proposed, such as using LiDAR (Light Detection And Ranging) to determine the three-dimensional coordinates of void pipes, etc., and using AI image recognition to determine the central coordinates of the cross-sections of void pipes, etc.
[0004] Moreover, Patent Document 1 discloses a technology that uses a head mounted display (HMD) and an AR marker to support cable installation as designed. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2018-147440 A Summary of the Invention [Problem to be solved by the invention]
[0006] However, the method using LiDAR and AI image recognition requires a lot of preparation and is expensive. In addition, the technology described in Patent Document 1 only verifies the correspondence between the cable and the terminal, and does not verify the accuracy of the installation position (coordinates).
[0007] The present invention has been made to solve these problems, and aims to provide an AR marker for inspecting openings, an opening inspection system, an opening inspection method, and a program that utilize MR (Mixed Reality) / AR (Augmented Reality) technology to determine the position of constructed openings in real space. [Means for solving the problem]
[0008] According to one embodiment, an AR marker for inspecting an opening has a surface on which a pattern is drawn that indicates the center position of an opening provided in a structure or component, and is shaped to fit into the inner wall surface or end of the opening. According to one embodiment, the inspection AR marker for the opening is circular, and when the cross-sectional shape of the opening is circular, it can fit onto the inner wall surface or end of the opening without considering the rotation angle relative to the core. According to one embodiment, an opening inspection system includes an AR marker for inspecting openings, the marker having a surface on which a pattern indicating the core position of an opening provided in a structure or component is drawn and having a shape that fits into an inner wall surface or end of the opening, and a terminal device, wherein the terminal device includes a drawing placement unit that recognizes an AR marker for drawing placement installed in real space and superimposes and displays drawing data on the real space, an opening recognition unit that recognizes the AR marker for inspecting the opening that fits into the opening constructed in the real space and calculates the construction position of the opening, and an error determination unit that compares the position of the opening defined in the drawing data with the construction position of the opening constructed in the real space and displays the comparison result. According to one embodiment, the drawing arrangement unit re-arranges the drawing data in the real space when a position or an attitude of the terminal device changes. According to one embodiment, the opening inspection system further includes a total station, which measures the position or attitude of the terminal device, and the drawing placement unit repositions the drawing data in the real space based on the measurement results by the total station. According to one embodiment, a method for inspecting an opening includes a drawing arrangement step in which a terminal device recognizes an AR marker for drawing arrangement installed in real space and superimposes and displays drawing data on the real space; an opening recognition step in which an AR marker for inspection of an opening that fits into the opening constructed in the real space and calculates a construction position of the opening; and an error determination step in which a position of the opening defined in the drawing data is compared with a construction position of the opening constructed in the real space and a comparison result is displayed, wherein the AR marker for inspection of the opening has a surface on which a pattern indicating the core position of an opening provided in a structure or component is drawn, and has a shape that fits into an inner wall surface or end of the opening. According to one embodiment, a program causes a computer to carry out the above method. Effect of the Invention
[0009] The present invention makes it possible to provide an AR marker for inspecting openings, an opening inspection system, an opening inspection method, and a program that utilize MR (Mixed Reality) / AR (Augmented Reality) technology to determine the position of constructed openings in real space. [Brief description of the drawings]
[0010] [Figure 1] 1 is a diagram showing an example of an AR marker 10 for inspecting an opening. [Diagram 2] 1 is a diagram showing an example of an AR marker 10 for inspecting an opening. [Diagram 3] 1 is a block diagram showing a configuration of an inspection system 1 for an opening. [Figure 4] 2 is a block diagram showing the hardware configuration of a terminal device 20. FIG. [Diagram 5] 2 is a block diagram showing the functional configuration of a terminal device 20. FIG. [Figure 6] 4 is a flowchart showing the operation of the terminal device 20.
[0011] <Embodiment 1> (AR marker for inspecting openings) FIG. 1 is a diagram showing an example of an AR marker 10 for inspection of an opening according to a first embodiment of the present invention. The AR marker 10 for inspection of an opening has a contour shape that is substantially the same as the cross section of a void pipe or the like, and is formed to fit into the inner wall surface of a void pipe or the like. For example, the AR marker 10 for inspection of an opening can be fitted into the inside of a void pipe or the like without any gaps or loosely, and is retained in the void pipe by the frictional force acting between the marker and the inner wall surface. Alternatively, the AR marker 10 may be fitted into the inside of a void pipe or the like by a threaded fastener or other mechanism.
[0012] 2 is a diagram showing another example of the AR marker 10 for inspecting an opening. The AR marker 10 for inspecting an opening may be in the form of a lid that can be attached to the end of a void pipe or the like. For example, the AR marker 10 for inspecting an opening can be fitted externally or internally to the end of a void pipe or the like, and is locked by frictional force acting between the marker and the inner or outer wall surface. Alternatively, the AR marker 10 may be fitted to the end of a void pipe or the like by a screw thread or other mechanism.
[0013] The cross-sectional shapes of the void tube, sleeve tube, box-cut opening, etc. are not limited, but are typically circular or rectangular. It is preferable to prepare in advance several sizes of inspection AR markers 10 for openings according to the standard sizes (inner diameter, outer diameter, etc.) of the void tube, sleeve tube, etc.
[0014] A predetermined pattern indicating the center position (corresponding to the core of the void pipe, etc.) of the cross-sectional shape of the void pipe, etc. may be drawn on the inspection AR marker 10 of the opening. In addition, a pattern indicating the cross-sectional shape of the void pipe, etc. may be drawn. The terminal device 2 described later recognizes this pattern and calculates the center coordinates and shape of the cross-section of the void pipe, etc.
[0015] When the AR marker 10 for inspection of an opening is fitted into a void tube or the like, it is preferable that the surface on which the pattern is drawn is approximately perpendicular to the central axis of the void tube or the like. (Opening Inspection System)
[0016] 3 is a block diagram showing a configuration of the opening inspection system 1. The opening inspection system 1 includes an AR marker 10 for inspecting an opening, and a terminal device 20.
[0017] 4 is a block diagram showing a hardware configuration of the terminal device 20. The terminal device 20 is typically an information processing device such as an HMD such as Hololens (registered trademark), a tablet computer, a smartphone, etc. The terminal device 20 has a camera 21, a self-position estimation sensor 22, a processing unit 24, and a display 26.
[0018] The camera 21 captures an image of the real world and recognizes the AR marker 10 for inspection of the opening included in the image, as well as the AR marker for placement on the drawing, which will be described later.
[0019] The self-position estimation sensor 22 is a sensor for estimating the self-position in real space. For example, a depth sensor that measures the distance to an object by irradiating an existing object with infrared light or the like and measuring the time it takes for the light to be reflected, or an acceleration sensor that measures acceleration or angular velocity, etc., can be used as the self-position estimation sensor 22. It is also possible to use the camera 21 as the self-position estimation sensor 22. The self-position estimation sensor 22 calculates the amount of movement, the direction of movement, the speed of movement, the attitude (tilt, i.e., the direction of gaze), etc. of the terminal device 20 based on the acquired sensor information.
[0020] The processing unit 24 is an information processing device including a CPU (Central Processing Unit) 241, a memory 242, etc., and the CPU 241 executes a program stored in the memory 242 to realize a predetermined function.
[0021] The display 26 displays the information output by the processing unit 24. If the terminal device 20 is an HMD, the worker can visually recognize the real world through the transmissive display 26 and at the same time visually recognize various pieces of information displayed on the display 26. If the terminal device 20 is a tablet terminal, a smartphone, or the like, the worker can visually recognize various pieces of information superimposed on the real world projected on the non-transmissive display 26.
[0022] FIG. 5 is a block diagram showing the functional configuration of the terminal device 20. As shown in FIG.
[0023] The terminal device 20 includes a drawing arrangement unit 201 , an opening recognition unit 203 , and an error determination unit 205 .
[0024] The drawing arrangement unit 201 superimposes and displays the drawing on the real space using MR technology.
[0025] First, the drawing placement unit 201 uses the camera 21 to capture an AR marker for placing a drawing placed in real space. A pattern indicating an origin position is drawn on the AR marker for placing a drawing. Usually, one or more AR markers for placing a drawing are placed in real space.
[0026] Next, the drawing arrangement unit 201 recognizes the origin position (expressed in a world coordinate system specific to the terminal device 20) indicated by the drawing arrangement AR marker using the processing unit 24. The processing unit 24 reads out drawing data from a storage area (not shown). The drawing data has the same number of origins (expressed in a drawing coordinate system) as the drawing arrangement AR markers. The processing unit 24 adjusts the scale and angle of the drawing to match the origin position indicated by the drawing arrangement AR marker with the origin set in the drawing data. That is, the drawing coordinate system is converted into a world coordinate system. This makes it possible to superimpose and display drawing data with the correct position, size, and angle in the real space. If there are three drawing arrangement AR markers, the display position, scale, and angle of the drawing data can be uniquely determined. If there are two drawing arrangement AR markers, the elements (position, scale) of the drawing data excluding the angle can be determined. In this case, a horizontal plane based on the height of the first drawing arrangement AR marker is assumed, and the drawing data can be arranged based on this horizontal plane. Note that the horizontal plane is generally calculated using a gyro sensor of the device.
[0027] Then, the drawing arrangement unit 201 uses the processing unit 24 to display the drawing data adjusted according to the determined display position, scale, and angle on the display 26. This allows the worker to visually recognize the drawing data superimposed perfectly on the real space.
[0028] The drawing arrangement unit 201 sequentially and repeatedly executes a series of processes from photographing the drawing arrangement AR marker to arranging the drawing. As a result, even if the position of the terminal device 20 moves, as long as the camera 21 captures the drawing arrangement AR marker, the drawing data is displayed following the real space. Even if the camera 21 does not capture the drawing arrangement AR marker, the processing unit 24 sequentially updates the self-position and orientation in the world coordinate system based on the sensing data by the self-position estimation sensor 22. The position and angle of the drawing data, etc. displayed on the display 26 are adjusted according to the updated self-position and orientation. As a result, no matter how the worker moves, the drawing data is displayed following the real space.
[0029] The opening recognition unit 203 calculates the position of the installed sleeve in real space.
[0030] First, the opening recognition unit 203 captures an inspection AR marker 10 for an opening placed in real space using the camera 21. A pattern indicating the center position and shape of a cross section of a void pipe or the like is drawn on the inspection AR marker 10 for an opening.
[0031] Next, the opening recognition unit 203 uses the processing unit 24 to recognize the center position and shape (expressed in the world coordinate system) of the cross section of the void pipe or the like indicated by the inspection AR marker 10 of the opening.
[0032] Next, the opening recognition unit 203 uses the processing unit 24 to display the center position and shape of the recognized cross section of the void pipe, etc. on the display 26. This allows the worker to visually recognize the center position and shape of the cross section of the void pipe, etc., superimposed exactly on the real space.
[0033] In addition, the angle at which the AR marker 10 for inspection of an opening engages with a circular void pipe or the like cannot be uniquely determined because the void pipe or the like is circular. However, the opening recognition unit 203 can identify the center coordinates of the AR marker for inspection of the opening in the world coordinate system regardless of the angle at which the AR marker for inspection of the opening is engaged with the void pipe or the like, so there is no need to consider the angle at which the AR marker for inspection of the opening 10 engages with the opening. On the other hand, if the opening is other than circular (e.g., rectangular), the angle at which the AR marker for inspection of the opening 10 engages with the opening is automatically fixed.
[0034] Here, once the opening recognition unit 203 recognizes the center position and shape of the cross section of the void pipe or the like, it is preferable to store this in a temporary storage area (not shown). Thereafter, when drawing the center position and shape of the cross section of the void pipe or the like or performing error judgment (described later), the information in the temporary storage area is used. In other words, the recognition process is not performed every time the inspection AR marker 10 of the opening comes into the field of view of the camera 21. This reduces the processing load.
[0035] The error determination unit 205 compares the positions and shapes of void pipes and the like constructed in real space with the positions and shapes of void pipes and the like defined in the drawing, and displays the errors.
[0036] The opening recognition unit 203 acquires the center position (expressed in the world coordinate system) of the cross section of the void pipe, etc. defined in the drawing data (converted to the world coordinate system) being superimposed and displayed in the real space, using the processing unit 24. The processing unit 24 compares the center position of the cross section of the void pipe, etc. in the drawing data with the center position of the cross section of the void pipe, etc. in the real space recognized by the opening recognition unit 203, and calculates the difference.
[0037] The opening recognition unit 203 uses the processing unit 24 to display the comparison result on the display 26. For example, the difference is displayed in text such as "Error 30 mm." Also, an arrow is displayed between the center position of the cross section of the void pipe, etc. in the drawing data and the center position of the cross section of the void pipe, etc. in the real space recognized by the opening recognition unit 203. The size of the arrow indicates the size of the difference. Alternatively, an alert is displayed depending on whether the difference is within a preset tolerance. For example, the cross-sectional shape of the void pipe, etc. in the real space recognized by the opening recognition unit 203 is displayed in color A if the difference is within the tolerance, and in color B if it exceeds the tolerance.
[0038] FIG. 6 is a flowchart showing the operation of the terminal device 20.
[0039] S101: AR marker recognition for drawing placement The worker positions the AR marker for placing the drawing, which has been placed in advance in the real space, within the field of view of the camera 21 of the terminal device 20. The drawing placement unit 201 captures an AR marker for drawing placement and recognizes the origin position indicated by the AR marker for drawing placement.
[0040] S102: Overlay drawing display The drawing arrangement unit 201 reads out drawing data from the storage area. The drawing arrangement unit 201 adjusts the position, size, and angle of the drawing data so that the origin set in the drawing data coincides with the origin position indicated by the AR marker for drawing arrangement. The drawing arrangement unit 201 displays the adjusted drawing data on the display 26.
[0041] S103: AR marker 10 recognition for inspection of openings The worker positions the inspection AR marker 10 for an opening previously installed in real space within the field of view of the camera 21 of the terminal device 20. The opening recognition unit 203 captures an image of the inspection AR marker 10 of the opening, and recognizes the center position and shape of a cross section of a void pipe or the like indicated by the inspection AR marker 10 of the opening.
[0042] S104: Sleeve pipe installation position superimposed display The opening recognition unit 203 displays the center position and shape of the cross section of the void pipe or the like on the display 26.
[0043] S105: Error judgment The error determination unit 205 calculates the difference (error) between the actual position and shape of the void pipes etc. acquired in S104 and the position and shape of the void pipes etc. defined in the drawing acquired in S102.
[0044] S106: Result display The opening recognition unit 203 displays the error determination result on the display 26. For example, the determination result can be displayed by a method such as expressing the difference with a number, expressing the magnitude of the difference with an arrow, or expressing whether the difference is within a tolerance with a color.
[0045] According to the first embodiment, by using an AR marker that can be fitted into the inside or end of a completed void pipe or the like, the central position of the completed void pipe or the like can be accurately obtained. In addition, the recognition process of the AR marker 10 for inspecting the opening is not executed each time, and the recognition result is reused, thereby reducing the processing load.
[0046] <Embodiment 2> In the first embodiment, when the position of the terminal device 20 moves, the camera 21 captures the AR marker for placing the drawing again, thereby maintaining the drawing data superimposed on the real space. However, when the distance between the terminal device 20 and the AR marker for placing the drawing increases, the recognition accuracy decreases, and the accuracy of superimposing the drawing data may also decrease. Also, in the first embodiment, the superimposing of the drawing data on the real space can be maintained even by self-location estimation. However, when the moving distance of the terminal device 20 increases, errors in the self-location estimation accumulate, and the accuracy of superimposing the drawing data may also decrease.
[0047] Therefore, in the second embodiment, a total station is used in combination to maintain the accuracy of the drawing data superimposed on the real space even when the position of the terminal device 20 moves significantly.
[0048] The opening inspection system 1 according to the second embodiment further includes a total station 30.
[0049] The terminal device 20 further includes a target prism 28 and a communication unit 29 .
[0050] The total station 30 tracks the target prism 28 to measure the coordinates and attitude of the terminal device 20 in the world coordinate system, and transmits the measurement results to a communication unit 29 .
[0051] The communication unit 29 notifies the processing unit 24 of the measurement results by the total station 30. The processing unit 24 successively updates its own position and orientation in the world coordinate system based on the measurement results by the total station 30. The processing unit 24 adjusts the position and angle of the drawing data, etc. displayed on the display 26 according to the updated own position and orientation. As a result, no matter how the worker moves, the drawing data is displayed following the real space.
[0052] Although the embodiment of the present invention has been described above, the present invention is not limited to this, and the components and various processes can be changed, deleted, or added as appropriate without departing from the spirit of the present invention.
[0053] For example, the embodiments of the present invention have disclosed examples in which the present invention is mainly applied to sleeve inspection, but the present invention is not limited to this and can be applied to inspecting the positions of various openings provided in structures, components, etc.
[0054] Furthermore, each processing means constituting the present invention may be configured by hardware, or any process may be realized by making a CPU execute a computer program. Furthermore, the computer program may be stored and supplied to a computer using various types of temporary or non-temporary computer-readable media. A temporary computer-readable medium includes, for example, an electromagnetic signal supplied to a computer by wire or wirelessly. [Explanation of symbols]
[0055] 1 Opening Inspection System 10 AR marker for inspection of openings 20 Terminal Equipment 21 Camera 22 Self-location estimation sensor 24 Processing section 26 Display 28 Target Prism 29 Communications Department 201 Drawing Layout Section 203 Opening recognition part 205 Error judgment section 30 Total Station
Claims
1. A surface on which a pattern indicating the center position of an opening provided in a structure or member is drawn, A shape that fits into the inner wall surface or end of the opening AR marker for inspecting openings.
2. the inspection AR marker for the opening is circular; When the cross-sectional shape of the opening is circular, it can be fitted to the inner wall surface or end of the opening without considering the rotation angle with respect to the core. The AR marker for inspecting an opening according to claim 1 .
3. an AR marker for inspecting an opening, the marker having a surface on which a pattern indicating a core position of an opening provided in a structure or member is drawn and having a shape that fits into an inner wall surface or an end of the opening; A terminal device, The terminal device a drawing placement unit that recognizes an AR marker for placing drawings installed in a real space and displays drawing data in a superimposed manner on the real space; an opening recognition unit that recognizes an inspection AR marker for the opening that is fitted into the opening constructed in the real space and calculates a construction position of the opening; and an error determination unit that compares the position of the opening defined in the drawing data with the construction position of the opening constructed in the real space and displays the comparison result. Opening inspection system.
4. The drawing arrangement unit rearranges the drawing data in the real space when a position or an attitude of the terminal device changes. The opening inspection system according to claim 3 .
5. Further comprising a total station, The total station measures the position or attitude of the terminal device, The drawing arrangement unit rearranges the drawing data in the real space based on a measurement result by the total station. The opening inspection system according to claim 4 .
6. The terminal device a drawing arrangement step of recognizing an AR marker for drawing arrangement installed in a real space and superimposing and displaying drawing data on the real space; an opening recognition step of recognizing an inspection AR marker for an opening that is fitted into a constructed opening in the real space and calculating a construction position of the opening; an error determination step of comparing a position of the opening defined in the drawing data with a construction position of the opening constructed in the real space and displaying a comparison result; The AR marker for inspecting the opening has a surface on which a pattern indicating the center position of the opening provided in the structure or member is drawn, and has a shape that fits into the inner wall surface or end of the opening. How to inspect openings.
7. A program for causing a computer to execute the method according to claim 6.
Citation Information
Patent Citations
Cable construction management system
JP2018147440A
