Rebar Inspection Result Display System

Abstract

To display a bar arrangement inspection result that facilitates the correction work for a bar arrangement error of a worker in a site.SOLUTION: A bar arrangement inspection result display system S comprises: a measuring instrument 2 which has a communication unit 44 and a three-dimensional coordinate measurement unit 262; an eyewear display device 4 which has a display 41; and a system control unit 60 which manages a coordinate space of information about a position and orientation of the eyewear display device and a coordinate space of the measuring instrument in a space with a common reference point as an original point, and displays an image created by the system control unit on the display 41 to display the image so as to be superimposed on an actual stuff observed by wearing the eyewear display device 4. The system control unit generates a present three-dimensional model within a bar arrangement inspection range on the basis of three-dimensional point group data within the bar arrangement inspection range acquired in such a state that the position and orientation are known, generates three-dimensional inspection result display data by associating the bar arrangement inspection result for the bar arrangement inspection range with a three-dimensional model, displays the data on the display, and displays a bar arrangement error within the inspection range in a recognizable manner.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a reinforcement inspection result display system, and more particularly to a reinforcement inspection result display system using an eyewear display device.

Background Art

[0002] In the reinforcement inspection in steel bar work, it is inspected whether the type, number, position, interval, joint method, etc. of the steel bars are correctly arranged as designed. Conventionally, an inspector checks whether there are any errors by comparing the reinforcement drawing with the on-site reinforcement state, and takes pictures with a digital camera or the like for recording.

[0003] In recent years, techniques for obtaining reinforcement information such as the number, diameter, and pitch of steel bars using a photographed image have been proposed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the outputs of the reinforcement inspection systems of Patent Document 1 and Patent Document 2 were related to the display of data for creating an inspection report form and the display on a tablet or the like for an inspector to confirm the inspection results. For the work of correcting reinforcement errors, it was necessary to check this form or the display on the tablet, which was inconvenient for the operator. Therefore, the development of a technology that can display the reinforcement inspection results for use by the on-site workers who correct the reinforcement errors has been desired.

[0006] This invention has been made in view of the above circumstances, and aims to enable the display of rebar inspection results in a way that facilitates the work of correcting rebar placement errors by on-site workers. [Means for solving the problem]

[0007] To achieve the above objective, the reinforcement inspection result display system according to the first aspect of the present invention has the following configuration. 1. A rebar inspection result display system comprising: a measuring instrument having a communication unit and a 3D coordinate measurement unit; an eyewear display device having a display, a relative position sensor for detecting its own position, and a relative direction sensor for detecting its own direction; and a system control unit that manages the coordinate space of information regarding the position and direction of the eyewear display device and the coordinate space of the measuring instrument in a space with a common reference point as the origin; wherein the system control unit displays an image created by the system control unit on the display, thereby overlaying it on the actual object observed while wearing the eyewear display device, characterized in that the system control unit generates a current 3D model of the rebar inspection range based on 3D point cloud data of the rebar inspection range acquired with known position and direction, generates 3D inspection result display data relating the rebar inspection results for the rebar inspection range to the 3D model, displays it on the display, and displays rebar errors in the inspection range in a recognizable manner.

[0008] 2. In the configuration described in 1 above, it is also preferable that the system control unit generates correction support data for correcting the reinforcement error based on the 3D reinforcement design data within the inspection range, and displays a correction support image on the display to assist in the work of correcting the reinforcement error based on the correction support data.

[0009] 3. In the configuration of 1 or 2 above, it is preferable that the system control unit is able to recognize the worker's hand within the field of view of the eyewear display device, and that the reinforcing bar touched by the worker is displayed on the display in a recognizable manner. [Effects of the Invention]

[0010] According to the above embodiment, it becomes possible to display the results of reinforcement inspection in a way that facilitates the work of correcting reinforcement errors made by on-site workers. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram of the external appearance of a rebar inspection system that includes a rebar inspection result display system according to an embodiment of the present invention. [Figure 2] This is a block diagram of the system. [Figure 3] This is a block diagram of the scanners that make up the system. [Figure 4A] This is a perspective view of the external appearance of the eyewear display device that constitutes the system. [Figure 4B] This is a block diagram of the eyewear display device. [Figure 5] This is a block diagram of the data processing equipment that makes up the above system. [Figure 6] Figures (A) through (C) illustrate the differences between image data, point cloud data, and point cloud composite images. [Figure 7] This is a diagram illustrating the reinforcement bar condition inspection unit of the data processing device. [Figure 8] This diagram illustrates the method for generating the reinforcement bar arrangement identification model of the data processing device. [Figure 9] This is a schematic flowchart of the reinforcement inspection method using the system described above. [Figure 10] This is a flowchart of the processing performed by the data processing device in the same method. [Figure 11] This is a flowchart detailing the process for inspecting the reinforcement bar arrangement using the same method. [Figure 12] This is a flowchart illustrating the process for generating rebar inspection result display data using the same system. [Figure 13] This figure shows an example of how the reinforcement inspection results are displayed by the system. [Figure 14] This figure shows an example of how the reinforcement inspection results are displayed by the system. [Figure 15]This is a block diagram of a reinforcement inspection result display system according to the present embodiment. [Figure 16] This is a block diagram of a reinforcement inspection result display system according to a modified example of the system. [Figure 17] This is a diagram for explaining a motion capture device according to the system. [Figure 18] This is a diagram showing an example of displaying reinforcement inspection results by the system. [Figure 19] This is a schematic external view of another example of an inspection system for acquiring 3D point cloud data and 3D reinforcement inspection result data. [Figure 20] This is a block diagram of the configuration of the inspection system.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited thereto. Also, the same components common to each embodiment and modified example are denoted by the same reference numerals, and redundant descriptions are omitted as appropriate.

[0013] I Embodiment 1 Configuration of Reinforcement Inspection System 100 (Reinforcement Inspection Result Display System S) FIG. 1 is a block diagram showing an overview of the usage state of a reinforcement inspection system (hereinafter, simply referred to as an inspection system) 100 in which a reinforcement inspection result display system (hereinafter, simply referred to as a display system) S according to an embodiment of the present invention is incorporated. The inspection system 100 includes at least one scanner 2, an eyewear display device (hereinafter, referred to as an eyewear device) 4 including a camera 50, and a data processing device 6. The scanner 2, the eyewear device 4, and the data processing device 6 are wirelessly connected and can transmit and receive information to and from each other.

[0014] Scanner 2 is a ground-mounted 3D laser scanner. Scanner 2 is installed at any point within the foundation construction site, which is the inspection area. The instrument installation point is known by methods such as the resection method. Scanner 2 is installed via a leveling base mounted on a tripod and has a base portion 2α provided on the leveling base, a support portion 2β that rotates horizontally around axis H on the base portion 2α, and a light-emitting portion 2γ that rotates vertically at the center of the support portion 2β.

[0015] The eyewear device 4 is a so-called head-mounted display worn on the worker's head. A camera 50 is used to acquire images of the reinforcement bar arrangement within the inspection area. In a single shot, the camera 50 acquires images of the area within its field of view (referred to as the inspection area). The display 41 can also display the inspection results overlaid on the actual reinforcement bars.

[0016] In the illustrated example, the data processing device 6 is a laptop computer. The data processing device performs reinforcement inspection using the 3D point cloud data (hereinafter simply referred to as point cloud data) acquired by the scanner 2 and the images acquired by the camera 50. The details of each component will be described below.

[0017] 2. Scanner Figure 3 is a block diagram of the scanner 2. The scanner 2 comprises a distance measuring unit 21, a vertical rotation drive unit 22, a vertical angle detector 23, a horizontal rotation drive unit 24, a horizontal angle detector 25, a scanner control unit 26, a display unit 27, an operation unit 28, a storage unit 29, an external storage device 30, and a communication unit 31.

[0018] The distance measuring unit 21 comprises a light transmitting unit, a light receiving unit, a light transmitting optical system, a light receiving optical system that shares optical elements with the light transmitting optical system, and a rotating mirror 21α. The light transmitting unit is equipped with a light-emitting element such as a semiconductor laser and emits pulsed light, which is distance measuring light, as scanning light. The emitted distance measuring light enters the rotating mirror 21α via the light transmitting optical system, is deflected by the rotating mirror 21α, and irradiates the object to be measured. The rotating mirror 21α is driven by the vertical rotation drive unit 22 and rotates around axis V.

[0019] The reflected light retroreflected by the object being measured passes through the rotating mirror 21α and the light-receiving optical system before entering the light-receiving unit. The light-receiving unit is equipped with a light-receiving element such as a photodiode. In addition, a portion of the distance-measuring light is incident on the light-receiving unit as internal reference light, and the scanner control unit determines the distance to the illumination point based on the reflected light and the internal reference light.

[0020] The vertical rotation drive unit 22 and the horizontal rotation drive unit 24 are motors and are controlled by the scanner control unit. The vertical rotation drive unit 22 rotates the rotating mirror 21α vertically around axis V. The horizontal rotation drive unit 24 rotates the mounting unit 2β horizontally around axis H.

[0021] The vertical angle detector 23 and the horizontal angle detector 25 are rotary encoders. The vertical angle detector 23 measures the vertical rotation angle of the rotating mirror 21α. The horizontal angle detector 25 measures the horizontal rotation angle of the mounting section 2β. This allows the vertical and horizontal angles of the distance measuring optical axis to be detected.

[0022] The scanner control unit 26 includes at least one processor and at least one memory. The processor is, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The memory is, for example, SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory). The processor reads data and programs stored in the storage unit 29, etc., into the memory and executes processing to realize the functions of the scanner 2.

[0023] In this specification, a processor is not limited to one that performs software processing for all of the processes it performs. It may include dedicated hardware circuits (e.g., Application Specific Integrated Circuits: ASICs) that perform hardware processing for at least some of the processes it performs. That is, a processor may be configured as a circuit including a combination of at least one processor that operates according to a computer program (software) and one or more dedicated hardware circuits that perform at least some of the various processes.

[0024] The scanner control unit 26 calculates the distance to the irradiation point for each pulse of the ranging light based on the time difference (round-trip time of the pulsed light) between the light emission timing of the light transmitting unit and the light reception timing of the light receiving unit. It also calculates the irradiation angle of the ranging light at that time and calculates the angle of the irradiation point.

[0025] Furthermore, the scanner control unit 26 includes a point cloud data acquisition unit 261 and a 3D coordinate measurement unit 262 as functional units. The point cloud data acquisition unit 261 controls the distance measuring unit 21, the rotating mirror 21α, the vertical rotation drive unit 22, and the horizontal rotation drive unit 24 to scan the distance measuring light all around (360°) (full dome scan), acquire the coordinates of each irradiation point, and acquire point cloud data for the entire circumference. The point cloud data acquisition unit 261 acquires 3D point cloud data (hereinafter referred to as 3D point cloud data) 71 of the inspection area and sends it to the data processing unit 6. The 3D coordinate measurement unit 262 scans the area around the target at high density to measure the distance and angle of the target and realizes a target scan function to acquire the target's 3D coordinates.

[0026] The display unit 27 is, for example, a liquid crystal display. The operation unit 28 has a power key, number keys, decimal point key, + / - keys, execute key, cursor movement keys, etc. The operator can input operation instructions and information to the scanner 2 from the operation unit 28.

[0027] The storage unit 29 is, for example, a computer-readable storage medium such as an HDD (Hard Disk Drive) or flash memory. The storage unit 29 stores the program for executing the functions of the scanner control unit 26. The external storage device 30 is, for example, a memory card, and stores various data acquired by the scanner 2.

[0028] The communication unit 31 is a communication control device such as a network adapter, network interface card, LAN card, or Bluetooth® adapter, and connects the scanner 2 to the eyewear device 4 and the data processing device 6 by wire or wireless connection. The scanner control unit 26 can send and receive information to and from the eyewear device 4 and the data processing device 6 via the communication unit 31.

[0029] 3. Eyewear device 4 Figure 4A is an external perspective view of the eyewear device 4, and Figure 4B is a block diagram of the eyewear device 4. The eyewear device 4 includes a display 41, a camera 50, and a control unit 42. The control unit 42 includes an eyewear control unit 43, an eyewear communication unit 44, and a relative position detection sensor. (Hereafter referred to as the relative position sensor.) 45. Relative Direction Sensor (Hereafter referred to as the relative direction sensor.) It includes 46, an eyewear storage unit 47, and an operating switch 48.

[0030] The display 41 is a goggle-lens type transparent display that covers both of the worker's eyes when worn. For example, the display 41 is an optical see-through display using a half-mirror, and it displays the image received by the eyewear control unit 43 superimposed on the work site scenery.

[0031] Alternatively, the display 41 may be a video see-through display that shows an image obtained by superimposing the image received by the eyewear control unit 43 onto a front view image acquired in real time by the camera 50. Furthermore, the projection method may be a virtual image projection method or a retinal projection method.

[0032] Camera 50 is a digital camera equipped with a lens and an image sensor such as a CCD (Charge-Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 50 is located in the center of the front of the eyewear device 4 and captures an image of the front of the eyewear device 4 in real time. Camera 50 uses the field of view of the eyewear device 4 as the inspection target area and acquires image data 72 of the inspection target area. The acquired image data 72 is sent to the data processing device 6. The image sensor has a Cartesian coordinate system with the imaging center of camera 50 as the origin, and the local coordinates of each pixel are specified. The imaging center of camera 50 is the center of the eyewear device 4, and the imaging optical axis is in the line of sight direction of the eyewear device 4.

[0033] The eyewear communication unit 44 is the same communication control device as the communication unit 31. The eyewear communication unit 44 connects the eyewear device 4 by wire or wireless, preferably wirelessly. The eyewear control unit 43 can send and receive information with the scanner 2 and data processing device 6 via the eyewear communication unit 44.

[0034] The relative position sensor 45 performs wireless positioning using GNSS (Global Navigation Satellite System) antennas, Wi-Fi® access points, ultrasonic oscillators, etc., installed at the observation site, and detects the position (self-position) of the eyewear device 4 within the observation site.

[0035] The relative direction sensor 46 consists of a combination of a three-axis accelerometer or gyroscope and a tilt sensor. The relative direction sensor 46 detects the orientation (self-direction) of the eyewear device 4, with the vertical direction being the Z-axis direction, the horizontal direction being the Y-axis direction, and the front-to-back direction being the X-axis direction. As a result, the camera 50 can acquire images with its position and orientation known.

[0036] The eyewear storage unit 47 is a computer-readable storage medium, such as a memory card. The eyewear storage unit 47 stores the program for the eyewear control unit 43 to perform its functions.

[0037] The operation switch 48 is, for example, a push button located on the temple portion. The operation switch 48 includes, for example, a power button 48α for turning the power of the eyewear device 4 ON / OFF, a capture button 48β for taking a still image with the camera 50, and an image switching button 48γ for switching between images.

[0038] The eyewear control unit 43, like the scanner control unit 26 of the scanner 2, includes at least one processor (e.g., CPU) and at least one memory (e.g., SRAM, DRAM, etc.). The processor reads the program stored in the eyewear storage unit 47 and loads it into the memory, thereby realizing various functions of the eyewear device 4.

[0039] The eyewear control unit 43 outputs position and orientation information of the eyewear device 4 detected by the relative position sensor 45 and the relative direction sensor 46 to the data processing unit 6. The data received from the data processing unit 6 is then displayed on the eyewear device 4's display 41, overlaid on the scenery of the inspection site.

[0040] 4. Data processing device 6 Figure 5 is a block diagram of the data processing device 6. The data processing device 6 is a so-called computer, typically a personal computer, server computer, etc., but may also be a tablet terminal, smartphone, etc. The arithmetic processing unit 60 of the data processing device 6 corresponds to the system control unit in the claims. The system control unit may be a single computer, like the data processing device 6, but may also be a computer system in which multiple computers perform processing in a distributed manner. In this case, it may logically utilize a portion of the processing resources of one or more computers. The system control unit may be configured as part of the eyewear device 4, or as part of the scanner 2. The data processing device 6 may be configured to perform some of the processing on the eyewear device 4, or to perform some of the processing on the scanner 2.

[0041] The data processing device 6 comprises at least an arithmetic processing unit 60, a communication unit 63, a display unit 64, an input unit 65, and a storage unit 66.

[0042] The communication unit 63 is a communication control device equivalent to the communication unit 31 of the scanner 2, enabling the data processing unit 6 to communicate wirelessly with the scanner 2 and the eyewear device 4. The arithmetic processing unit 60 can send and receive information with the scanner 2 and the eyewear device 4 via the communication unit 63.

[0043] The display unit 64 is, for example, a liquid crystal display. The input unit 65 is, for example, a keyboard, mouse, etc., which allows the operator to input various commands, selections, decisions, etc.

[0044] The arithmetic processing unit 60, like the scanner control unit 26 of the scanner 2, is a control arithmetic unit comprising at least one processor (e.g., CPU, GPU) and at least one memory (e.g., SRAM, DRAM, etc.). The processor reads the program stored in the storage unit 66 and loads it into memory, thereby realizing various functions of the data processing device 6, in particular the functions of the following functional units.

[0045] The arithmetic processing unit 60 includes, as functional units, a synchronous measurement unit 601, a point cloud data reception unit 602, a reinforcement design data reading unit 603, an image data reception unit 604, a point cloud composite image generation unit 605, a reinforcement condition inspection unit 606, and a display data generation / output unit 607.

[0046] The synchronization measurement unit 601 synchronizes the scanner 2, the eyewear device 4 (camera 50), and the data processing device 6. Synchronization is the process of managing information including position coordinates, such as the position and orientation of the scanner 2 and the eyewear device 4 (camera 50), and the design data handled by the data processing device 6, in a common coordinate space with a common reference point as the origin. A suitable example is shown below, but synchronization can be achieved using any appropriate method based on the knowledge of those skilled in the art.

[0047] First, a reference point and reference direction are set at the inspection site for the inspection system 100, and the scanner 2 and data processing device 6 are synchronized. The reference point is selected as a known point (a point with known coordinates) or any point at the site. A feature point other than the reference point is selected, and the direction from the reference point toward this feature point is set as the reference direction. Using the target scan function of the scanner 2, the absolute coordinates of the scanner 2 are determined by observation using the resection method including the reference point and feature point, and transmitted to the data processing device 6. The synchronization measurement unit 601 recognizes the absolute coordinates of the reference point as (x,y,z)=(0,0,0) and the reference direction as a horizontal angle of 0°. As a result, the data processing device 6 can manage the relative position and relative direction of the scanner 2 in a space with the reference point as the origin, with respect to the information from the scanner 2.

[0048] Next, the eyewear device 4 and the data processing device 6 are synchronized. The eyewear device 4 is placed at the reference point, and with the eyewear device 4 in a horizontal position, the line of sight of the eyewear device 4 is aligned with the reference direction, and the (x,y,z) of the relative position sensor 45 is set to (0,0,0), and the (roll,pitch,yaw) of the relative direction sensor is set to (0,0,0). As a result, the data processing device 6 can then manage the relative position and relative direction of the eyewear device 4 (camera 50) in a space with the reference point as the origin, based on the information from the eyewear device 4 (camera 50).

[0049] The point cloud data receiving unit 602 receives 3D point cloud data of the entire inspection area, acquired by full dome scanning with scanner 2, via the communication unit 63.

[0050] The reinforcement design data reading unit 603 reads out the 3D (three-dimensional) reinforcement design data (hereinafter referred to as reinforcement design data) 73, which will be described later, stored in the storage unit 66.

[0051] The image data receiving unit 604 receives the image data 72 acquired by the camera 50 via the communication unit 63.

[0052] The point cloud composite image generation unit 605 generates a point cloud composite image by combining image data 72 with 3D point cloud data 71. Figure 6 shows images of image data (A), point cloud data (B), and point cloud composite image (C). Image data 72 is data that includes color information, but shadows and other elements may be included. On the other hand, point cloud data is not affected by shadows but does not include color information. Each point in the point cloud data has a specified coordinate, and its actual size can be determined. The point cloud composite image includes the color information contained in the image data and the actual size information obtained from the point cloud data. In other words, by using a point cloud composite image to identify the reinforcement arrangement, dimensions can be determined without setting up reference markers such as rulers.

[0053] The reinforcement condition inspection unit 606 inspects the reinforcement condition from the point cloud composite image and outputs 3D reinforcement inspection result data (hereinafter referred to as 3D reinforcement inspection result data or reinforcement inspection result data) 75. Details of the reinforcement condition inspection unit 606 will be described later.

[0054] The display data generation and output unit 607 generates display data to be displayed on the display 41 of the eyewear device 4 based on the reinforcement inspection result data.

[0055] The storage unit 66 is, for example, an HDD, SSD (Solid State Drive), etc. The storage unit 66 stores the 3D reinforcement design data (hereinafter also referred to as reinforcement design data) 73 mentioned above. In addition, the storage unit 66 stores programs for executing each function of the arithmetic processing unit 60, if each function is implemented as software.

[0056] The reinforcement design data 73 is a detailed reinforcement drawing created using 3D CAD data. The detailed reinforcement drawing is a diagram showing the detailed reinforcement configuration of a reinforced concrete member. The reinforcement configuration includes, for example, the type of reinforcement (material and thickness), the spacing and number of reinforcement bars, the placement of spacers to prevent insufficient concrete cover, the direction of the main reinforcement, the type of reinforcement splice, and the location of the tying points. The reinforcement design data 73 includes at least one of these pieces of information.

[0057] The reinforcement design data 73 is generated in advance by the 3D reinforcement design data generation device 70 and stored in the storage unit 66. The 3D reinforcement design data generation device 70 is a computer that includes at least one processor and at least one memory, which are equivalent to the data processing device 6.

[0058] Next, the details of the reinforcement condition inspection unit 606 will be explained with reference to Figures 7 and 8. 。 As shown in Figure 7, the reinforcement condition inspection unit 606 comprises a pre-processing unit 611, a reinforcement condition identification model 612, and a design data comparison unit 613.

[0059] The preprocessing unit 611 performs image processing on the point cloud composite image input as the object to be inspected to facilitate the recognition of reinforcing bars. For example, it performs known image processing such as grayscale conversion, edge and line segment extraction, and brightness value averaging. In addition, the point cloud composite image may be enlarged or reduced to a predetermined scale. Since the composite image data contains position coordinates, i.e., actual size information, it is possible to enlarge / reduce it to a predetermined scale without simultaneously photographing a reference marker such as a ruler.

[0060] The reinforcement state identification model 612 is a trained model obtained by training a large number of training point cloud composite images created by capturing point cloud data of numerous reinforcement states, as shown in Figure 8. The training point cloud composite images are labeled with information such as the type of reinforcing bar (material, shape, and thickness (diameter)), reinforcement spacing and number, placement of spacers to prevent insufficient concrete cover, main reinforcement direction, type of reinforcement splice, and location of tying points. As training data, a large number of point cloud composite images are used, generated in the same way as point cloud composite images by capturing various reinforcement states in general reinforcement methods. Image data may also be used.

[0061] The reinforcement bar identification model 612, upon input of a point cloud composite image of the inspection area, identifies the type of reinforcement (material and thickness), reinforcement spacing and number, placement of spacers to prevent insufficient concrete cover, main reinforcement direction, and type of reinforcement splice within the inspection area, and outputs these events in relation to their location. Learning is performed by a computer equipped with at least one processor and memory, similar to the data processing device 6. As a learning method, deep learning using convolutional neural networks (CNNs), recurrent neural networks (RNNs), Boltzmann machines, etc., is used.

[0062] Furthermore, the types of reinforcing bars are designated according to JIS standards based on their material, shape, and diameter, such as D3, D13, D38, SR295, etc. These designations may be used on labels indicating the type of reinforcing bar.

[0063] Furthermore, if the point cloud composite image is scaled up or down during preprocessing to match the scale of the point cloud composite image, using image data of the same predetermined scale for training will improve the accuracy of rebar arrangement identification.

[0064] When a point cloud composite image of the inspection area is input to the reinforcement condition identification model 612, for example, as shown in the lower right of Figure 8, the detected reinforcement condition information, namely the type of reinforcing bar (material, shape, and thickness (diameter)), the spacing and number of reinforcing bars, the placement of spacers to prevent insufficient concrete cover, the direction of the main reinforcement, the type of reinforcing bar splice, the location of the tying points, and other information, is output as reinforcement condition identification data 74.

[0065] The design data comparison unit 613 compares the point cloud composite image with the reinforcement design data 661 and outputs the differences (reinforcement errors) and their locations as reinforcement inspection result data 75. Specifically, it outputs differences in reinforcement type, reinforcement spacing and number, placement of spacers to prevent insufficient reinforcement cover, main reinforcement direction, reinforcement splice type, and tying locations, in association with location information within the inspection area.

[0066] 5. Reinforcement Inspection Method (Processing by Reinforcement Inspection System 100) Next, the reinforcement inspection method will be explained. Figure 9 is a schematic flowchart of the reinforcement inspection method using the inspection system 100 according to this embodiment.

[0067] In step S01, the scanner 2 is placed at a known point, and a full dome scan is performed to acquire point cloud data 71 of the inspection area (preferably the entire area). This entire inspection area does not strictly mean the entire inspection area, but rather the area within the inspection area that the operator needs. The acquired point cloud data 71 is sent to the data processing device 6. If possible, point cloud data 71 of the entire site is acquired in a single full dome scan. Take It is also possible to obtain point cloud data 71 using multiple scanners 2. Alternatively, point cloud data 71 may be obtained by performing multiple full dome scans at different installation points.

[0068] Next, in step S02, the operator uses the camera 50 to capture an image of a portion of the area set as the inspection area (the area to be inspected). Specifically, the portion of the area set as the inspection area is the field of view of the camera 50, which is the area that the operator intends to photograph for inspection. The acquired image data 72 is sent to the data processing device 6.

[0069] Next, in step S03, the data processing device 6 combines the received 3D point cloud data 71 and image data 72 to generate a point cloud composite image.

[0070] Next, in step S04, the data processing device 6 performs an inspection of the reinforcement state using the point cloud composite image and outputs reinforcement inspection result data 75.

[0071] Next, in step S05, the data processing device 6 generates and outputs display data based on the rebar inspection result data 75. The inspection results may be displayed on the display 41 of the eyewear device 4, on the display unit 64 of the data processing device 6, or output as a report from the data processing device 6 to an external device such as a printer.

[0072] 6. Reinforcement Inspection Method (Processing by Data Processing Device 6) Figure 10 is a flowchart of the processing of the data processing device 6 using the reinforcement inspection method described above. Figure 11 is a detailed flowchart of step S15.

[0073] When processing begins, first, in step S11, the point cloud data receiving unit 602 receives the point cloud data 71 covering the entire circumference from the scanner 2 and stores it in the storage unit 66.

[0074] Next, in step S12, the reinforcement design data reading unit 603 reads the reinforcement design data 73 from the storage unit 66. The order of steps S11 and S12 is not limited to this; the reinforcement condition inspection unit 606 may also read the data when performing the inspection.

[0075] Next, in step S13, the image data receiving unit 604 receives image data 72 of the inspection target area from the camera 50.

[0076] Next, in step S14, the point cloud composite image generation unit 605 combines the image data 72 of the inspection target area with the point cloud data 71 corresponding to the inspection target area to generate a point cloud composite image.

[0077] Next, in step S15, the reinforcement condition inspection unit 606 compares the point cloud composite image of the inspection target area with the corresponding 3D reinforcement design data 73 to inspect the reinforcement condition of the inspection target area.

[0078] Specifically, in step S21, the preprocessing unit 611 performs image processing on the point cloud composite image input as the inspection target to facilitate the recognition of reinforcing bars.

[0079] Next, in step S22, the reinforcement state identification model 612 receives a point cloud composite image of the inspection target area, identifies the reinforcement state of the inspection target area, and outputs it as reinforcement state identification data 74.

[0080] Next, in step S23, the 3D reinforcement design data 73 read in step S12 and the reinforcement status identification data 74 output in step S22 are compared to identify the parts with differences (reinforcement errors), and these are stored in the storage unit 66 as reinforcement inspection result data 75, associated with the location of those parts, and step S15 is terminated. The resulting reinforcement inspection results are accumulated for each inspection target area, and by inspecting the entire inspection site while moving through the inspection target area, it is possible to obtain 3D reinforcement inspection result data 75 for the entire inspection site.

[0081] Next, in step S16, the display data generation and output unit 607 generates display data to be displayed on the display of the eyewear device 4. Details will be described later.

[0082] 7. Details of display data generation Figure 12 is a detailed flowchart of the processing performed by the display data generation and output unit 607.

[0083] In step S21, the display data generation and output unit 607 generates a current 3D model based on the current 3D point cloud data 71 acquired from the scanner 2 and stored in the storage unit 66, and uses this as 3D current data.

[0084] Next, in step S22, the display data generation and output unit 607 generates 3D inspection result display data by associating the reinforcement inspection result data 75 output by the reinforcement condition inspection unit 606 and stored in the storage unit 66 with the current 3D model described above.

[0085] Next, in step S23, the display data generation and output unit 607 generates 3D modification support data based on the 3D reinforcement design data 73 that has been previously stored in the storage unit.

[0086] Then, in step S24, the display data generation and output unit 607 uses the 3D current data, 3D inspection result display data, and 3D correction support data to generate a display image corresponding to the field of view of the display 41, and displays it on the display 41 by overlaying it on the actual object.

[0087] Figures 13 and 14 illustrate the image of the display generated in this manner. In the figures, the dashed lines represent the actual object (reinforcement bars) visible within the field of view of the display 41, and the solid lines represent the display image generated by the display data generation / output unit 607. In reality, the display image is superimposed on the actual object, but for the sake of explanation, it is drawn with a slight offset.

[0088] Figure 13 shows an image of a worker wearing eyewear device 4 observing the reinforcement arrangement when facing the reinforcing bars corresponding to the wall surface. Figure 13(A) shows the current situation. ru3 D Current image 41a generated based on current data Display This is a model that is superimposed on the actual object. While not mandatory, 3D current state data may include the results of the reinforcement state identification by the reinforcement state identification model 612. Figure 13(A) shows an example of this, displaying the type of reinforcing bar (D13, D38) and the spacing between the reinforcing bars.

[0089] Figure 13(B) displays the inspection result image 41b, which is created based on the 3D inspection result display data. The inspection result shows that the reinforcing bar on the far right is incorrectly placed, and in addition to the 3D current data, Figure 13(B) highlights that the vertical reinforcing bar on the far right is incorrectly D13. Specifically, reinforcing bars with proper placement may be displayed in green, etc., while incorrectly placed reinforcing bars may be displayed in a conspicuous color such as red or yellow, or they may flash.

[0090] Figure 13(C) displays a correction support image 41c created based on 3D correction support data. The correction support image 41c shows the correct reinforcement state based on the design reinforcement data to assist in the work of correcting reinforcement errors. In Figure 13(C), it is shown that the reinforcement on the far right is correctly D38, and that reinforcement is highlighted to make it stand out.

[0091] In the eyewear device 4, the current image 41a, the inspection result image 41b, and the correction support image 41c may be switchable by the control of the calculation processing unit 60 or the eyewear control unit 43. In this case, for example, the operator may be able to switch by pressing the image switching button 48γ.

[0092] Figure 14 shows a display image when a worker wearing eyewear device 4 observes the reinforcement arrangement of reinforcing bars on the floor from an oblique angle above. Figure 14(A) displays a current state image 41a generated based on 3D current state data, showing the current situation, and is displayed superimposed on the actual object.

[0093] Figure 14(B) shows the inspection result image 41 created based on the 3D inspection result display data. bThis is what is displayed. Inspection result image 41b highlights the area with insufficient concrete cover by changing the color, etc., to show that the intersection of the central reinforcing bars is sagging and has insufficient concrete cover. At this time, although not required, it may be displayed in a way that makes it clear that the reinforcement error is "insufficient concrete cover," as shown in Figure 14(B).

[0094] Figure 14(C) displays a correction support image created based on 3D correction support data. In Figure 14(C), correction support image 41c shows that the error can be resolved by installing a spacer at the location so that the concrete cover is appropriate, thereby assisting in the work of correcting the reinforcement error.

[0095] In addition, the inspection result image 41b and correction support image 41c may show errors in reinforcement spacing and number, main reinforcement direction, type of reinforcement splice, location of tying points, and methods for correcting these errors, as well as the reinforcement state after correction. It is also possible to display only some of these, rather than all of them. Alternatively, the display may be switched for each type of reinforcement error.

[0096] Here, Figure 15 shows a block diagram of the display system S according to the embodiment. The display system S comprises a scanner (measuring instrument) 2, an eyewear device 4, and a data processing device 6. The scanner 2 functions as a measuring instrument for synchronizing absolute coordinate system data handled by the eyewear device 4 and the data processing device 6. From this perspective, the scanner 2 as a measuring instrument for the rebar inspection result display system S only needs to have a scanner control unit 26 equipped with a 3D coordinate measurement unit 262. The data processing device 6 also comprises a synchronization measurement unit 601 for synchronizing with the eyewear device 4 and the scanner 2, and a display data generation / output unit 607. Furthermore, the data processing device 6 only needs to have acquired 3D point cloud data for the inspection range, 3D rebar design data 73, and rebar inspection result data 75.

[0097] The measuring instrument is not limited to scanner 2; any surveying instrument equipped with a 3D coordinate measurement unit capable of acquiring the 3D position coordinates of an object to be measured is acceptable. For example, a total station with distance and angle measurement functions may be used. Alternatively, a camera equipped with two cameras, as exemplified in Japanese Patent Application Publication No. 2021-77127, capable of acquiring the 3D position coordinates of an object to be measured by photogrammetry may also be used.

[0098] Furthermore, the 3D point cloud data 71, 3D reinforcement design data 73, and 3D reinforcement inspection result data 75 do not necessarily have to be acquired by the inspection system 100 which includes the display system S, as in this embodiment; they may be acquired separately and stored in the memory unit in advance. However, if they are acquired by the inspection system 100 which includes the display system S, it is advantageous because the inspection results can be displayed in real time.

[0099] 8. Effects As described above, the display system S according to this embodiment displays the details and location of the rebar placement error on the display 41 of the eyewear display device 4 worn on the worker's head, superimposed on the actual object, making it identifiable. As a result, the worker can identify the rebar placement error simply by looking at the part they want to check, without having to compare it with a form or tablet display, thus reducing the burden on the worker.

[0100] Furthermore, the system displays the details of the reinforcement bar placement errors, making it easy to understand the necessary corrective actions.

[0101] In particular, the display system S further displays the correct reinforcement state to assist in correcting reinforcement errors, allowing workers to easily perform correction work to achieve the correct reinforcement state.

[0102] Furthermore, according to the display system S of this embodiment, the display system S is incorporated into the inspection system 100, and the eyewear display device 4 for displaying the rebar placement results is equipped with a camera 50 for acquiring images of the inspection area. Therefore, if the worker captures an image of the inspection area they want to check with the camera 50, they can conveniently check for rebar placement errors on the display 41 by overlaying the image onto the actual object in a series of processes.

[0103] 9. Variations Figure 16 is a block diagram of the display system SA according to a modified example.

[0104] Display system SA includes a motion capture device 5 in addition to the configuration of display system S. Furthermore, data processing device 6A includes a synchronous measurement unit 601A instead of the synchronous measurement unit 601.

[0105] The motion capture device 5 is a so-called magnetic motion capture device. The motion capture device 5 comprises a communication unit 51 that enables communication with the data processing device 6A, a plurality of magnetic three-dimensional position and attitude sensors 52 which are devices worn on the fingers of the worker, and a signal processing unit 53 that outputs the signals detected by the three-dimensional position and attitude sensors 52 to the data processing device in a time series as information about the worker's movements. As the three-dimensional position and attitude sensors 52, for example, the magnetic position and attitude sensors disclosed in Japanese Patent Application Publication No. 2007-236602 are preferred.

[0106] The motion capture device 5 has multiple three-dimensional position and orientation sensors 52 placed in a flexible glove, enabling it to detect subtle movements of the fingers.

[0107] The motion capture device 5 is shown in Figure 1. 7 As shown, for example, assuming the center of sensor 52a located at the tip of the index finger is the origin, information on their position coordinates (x, y, z) as seen from the fixed reference point of the signal processing unit 53, and Euler angles indicating the attitude determined from the rotation angles around the X, Y, and Z axes, can be obtained. Note that the Z axis is shown in Figure 1. 7It is an axis that passes through the origin of the XY plane and is perpendicular to the XY plane.

[0108] Furthermore, in addition to the functions of the synchronization measurement unit 601, the synchronization measurement unit 601A converts and manages the position and direction information received from the motion capture device A so that it matches the coordinate space of the synchronized scanner 2 and eyewear device 4. The synchronization of the motion capture device 5 is performed by placing the tip of the index finger of the worker wearing the motion capture device 5 at the reference point, pointing the tip of the index finger to coincide with the reference direction of the scanner 2, and setting the position coordinates and Euler angle to 0.

[0109] In this way, the eyewear device 4 can determine the location touched by the worker wearing the motion capture device 5.

[0110] Figure 18 shows an example of display on the display 41 by the display system SA. Figures 18(A) to (D) show the current state image 41a in Figure 18(A). Figure 18(B) shows the current state image 41a with the reinforcing bar on the far right grasped by the hand wearing the motion capture device 5. The reinforcing bar touched by the motion capture device 5 is highlighted. The reinforcement arrangement of the reinforcing bar touched by the motion capture device 5 (type of reinforcing bar and spacing in the figure) is also displayed.

[0111] Figures 18(C) and (D) show the correction support image 41c. In Figures 18(C) and (D), the process of correcting the reinforcement error is displayed for the reinforcing bar touched by the motion capture device 5.

[0112] This method allows workers to easily identify the reinforcing bar they have touched from among the many reinforcing bars displayed in the image, and to easily identify any errors in the placement of that reinforcing bar.

[0113] In this modified example, a motion capture device was used for hand recognition, but the system is not limited to this. Image processing could also be used to recognize the hand from its skin color and shape, and to identify the rebar that was touched by the hand.

[0114] 10. Another example of an inspection system The 3D point cloud data 71 and reinforcement inspection result data 75 used in the display system according to this embodiment do not necessarily have to be acquired by the inspection system 100 of the above form. The 3D point cloud data only needs to be acquired by a laser scanner with known position and direction within the inspection range, and the reinforcement inspection result data 75 can be used if reinforcement inspection results obtained as 3D data are available. For example, it may be acquired using the inspection system 200 described below as the inspection system.

[0115] Figure 19 is an external view showing the usage state of the rebar inspection system 200 according to the second embodiment, and Figure 20 is a block diagram.

[0116] System 200 comprises a scanner 2, a data processing device 206, a flying device 8 equipped with a camera 250, and a surveying instrument 9. The scanner 2, the data processing device 206, the flying device 8 equipped with a prism, and the surveying instrument 9 are wirelessly connected and can send and receive information from each other. The scanner 2 is the same as the scanner 2 according to Embodiment 1.

[0117] The flying device 8 is an autonomously flying UAV (Unmanned Air Vehicle). The flying device 8 is equipped with multiple propellers 8b extending radially from the main body 8a, a prism 8c as a target, and a camera 250 for capturing image data of the object to be inspected. The flying device 8 can fly along a predetermined flight path or fly freely by remote control. The flying device 8 is equipped with an IMU (Inertial Measuring Unit) and a timer (not shown) within the main body 8a. The positional relationship between the IMU and the camera 250 is known, and the direction and attitude of the camera 250 can be determined.

[0118] The surveying instrument 9 is a motor-driven total station equipped with an automatic tracking function. By flying the flight device 8 within the inspection area, the timing of the camera 250's shooting and the surveying instrument's shooting are synchronized to acquire image data of the inspection area.

[0119] The data processing device 206 has the same configuration as the data processing device 6 and can acquire rebar inspection result data 75 equivalent to that of the inspection system 100 using the 3D point cloud data 71 acquired by the scanner 2 and the image data acquired by the camera 250.

[0120] The 3D point cloud data and image data for the entire inspection area can be acquired using a scanner and a camera, respectively, whose positions and orientations are known, and are not limited to the examples above. For example, the camera may be replaced with an omnidirectional camera positioned so that its position and orientation (attitude) are known. Also, the scanner may be mounted on an aerial device instead of being a ground-based scanner.

[0121] Although preferred embodiments of the present invention have been described above, these embodiments are merely examples of the present invention, and it is possible to combine them based on the knowledge of those skilled in the art, and such forms are also included within the scope of the present invention. [Explanation of Symbols]

[0122] 4: Eyewear display device 31: Communications Department 41: Display 41a: Current image 41b: Examination result image 41c: Correction support image 44: Communications Department 45: Relative position detection sensor 46: Relative direction detection sensor 51: Communications Department 52a: Sensor 60: Processing Unit 63: Communications Department 70: 3D reinforcement design data generation device 71: 3D point cloud data 73: Reinforcement Design Data 262: 3D Coordinate Measurement Unit S: Rebar Inspection Result Display System

Claims

1. A measuring instrument having a three-dimensional coordinate measurement unit; An eyewear display device comprising a display, a relative position detection sensor for detecting its own position, and a relative direction detection sensor for detecting its own direction; and A rebar inspection result display system comprising: at least one processor configured to manage the coordinate space of the eyewear display device's information relating to its own position and direction, and the coordinate space of the measuring instrument, in a space with a common reference point as the origin; The aforementioned processor, Based on the three-dimensional point cloud data of the inspection area acquired by the measuring instrument with known position and orientation, a current three-dimensional model of the inspection area is generated. The three-dimensional reinforcement inspection result data, which associates reinforcement errors and their locations within the aforementioned inspection range, is associated with the three-dimensional model to generate three-dimensional inspection result display data. A rebar placement inspection result display system characterized in that the inspection result image is superimposed on the actual object observed by the eyewear display device on the display, thereby displaying the rebar placement error in a recognizable manner.

2. The processor generates correction support data for correcting the reinforcement error based on the three-dimensional reinforcement design data within the inspection range. The rebar inspection result display system according to claim 1, characterized in that it displays on the display a correction support image that assists in the work of correcting the rebar error based on the correction support data.

3. Furthermore, comprising a motion capture device, The rebar inspection result display system according to claim 1, characterized in that the processor is capable of recognizing the worker's hand within the field of view of the eyewear display device using the motion capture device, and displays the rebar arrangement status of the rebar touched by the worker on the display.

4. Further comprising a motion capture device, The rebar inspection result display system according to claim 2, characterized in that the processor is capable of recognizing the worker's hand within the field of view of the eyewear display device, and displays, as the correction support image, the work of correcting the rebar error with respect to the rebar touched by the worker.

5. The rebar inspection result display system according to Claim 1, characterized in that the processor is capable of recognizing the worker's hand within the field of view of the eyewear display device by image processing, and displays the rebar arrangement status of the rebar touched by the worker on the display.

6. The rebar inspection result display system according to claim 2, wherein the processor is capable of recognizing the worker's hand within the field of view of the eyewear display device by image processing, and the correction support image on the display shows the work of correcting the rebar error with respect to the rebar touched by the worker.

7. The three-dimensional reinforcement inspection result data is generated as a result of identifying the reinforcement state included in a point cloud composite image obtained by combining the three-dimensional point cloud data of the inspection range and the image data of the inspection range using a reinforcement state identification model obtained by learning a large number of training point cloud composite images created for various reinforcement states, and comparing it with the three-dimensional reinforcement design data of the inspection range, as described in Claim 1.

8. Further comprising at least one camera that acquires image data of the inspection range with known coordinates and direction, The measuring instrument is a three-dimensional laser scanner. The processor generates a point cloud composite image by combining the three-dimensional point cloud data of the inspection area and the image data of the inspection area. The reinforcement inspection result display system according to claim 1, characterized in that it identifies the reinforcement arrangement state and position of the reinforcement bars included in the point cloud composite image, compares the point cloud composite image with the three-dimensional reinforcement design data of the inspection range, and generates the three-dimensional reinforcement inspection result data.

Images