Measuring device, creating device, inspection result creating system, and program

Abstract

To save the power consumption and simplify the process in creating design drawings for reporting.SOLUTION: A measurement device 11 comprises a first camera 13, a distance measuring sensor 14, a direction designation unit 16, an input unit 15 and a first controller 18. The distance measuring sensor 14 detects the distance to objects in a detection range for each direction. The direction designation unit 16 detects one discretionary direction in the detection range as a designated direction. The input unit 15 detects input of the state of an object existing in the designated direction. The first controller 18 calculates the three-dimensional coordinates of a target object on the basis of the distance to the target object for which state input was detected in the input unit 15. The first controller 18 stores original information in combination with the three-dimensional coordinates and state of the target object. The original information is used for the calculation of the three-dimensional coordinates of a marker. The marker is equivalent to a marker image included in the images imaged by the first camera.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a measurement device, a creation device, an inspection result creation system, and a program. [Background technology]

[0002] In buildings such as condominiums, the condition of the exterior wall tiles is regularly inspected. In the inspection of the condition of the exterior wall tiles, tiles that require a report are identified on-site using a tapping rod, and the location of the tiles is recorded. After the on-site inspection, a document is created that describes the condition of the tiles at the location of the tiles that require a report on the building drawings.

[0003] Such condition investigation and document creation are cumbersome, and simplification is required. For example, it has been proposed to record the positions of specific tiles using a range finder (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-174390 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the recording method described in Patent Document 1, the positions of two objects located along a specific direction such as the vertical direction relative to a coordinate system based on a range finder are calculated, which facilitates conversion from the coordinate system to, for example, a ground surface coordinate system. However, even if the method described in Patent Document 1 is used, the preparation of documents is complicated, and there has been a demand for labor-saving and simplification in the process from investigation to preparation of documents.

[0006] Therefore, in consideration of the problems with the conventional technology as described above, an object of the present disclosure is to provide a measuring device, a creation device, an inspection result creation system, and a program that reduce the labor and simplify the creation of a report design drawing in which the condition of an object at any position in a building or the like is recorded at a corresponding position in a design drawing for the building. [Means for solving the problem]

[0007] In order to solve the above problems, a measurement apparatus according to a first aspect comprises: a first camera for capturing an image of an object and generating an image; A distance measuring sensor that detects the distance of an object within a detection range in each direction; a direction designation unit that detects any one direction within the detection range as a designated direction; an input unit for detecting an input of a state of an object present in the specified direction; The distance measuring sensor detects a distance to a target object when a state input is detected by the input unit. The three-dimensional coordinates of the target object based on the first camera are calculated, and the three-dimensional coordinates are added to the image captured by the first camera. When a predetermined marker image is included, the distance measuring sensor of the marker corresponding to the marker image is used as a reference. The raw information used to calculate the three-dimensional coordinates of the target object is combined with the three-dimensional coordinates and state of the target object. a first controller for storing the a first interface capable of reading a blueprint of a building together with location information on said blueprint of at least one origin; Equipped with 、 The first controller calculates three-dimensional coordinates of the marker based on the raw information, and associates the three-dimensional coordinates and state of the target object with the corresponding position on the design drawing based on the three-dimensional coordinates and position information on the design drawing. do.

[0009] A creating device according to a third aspect of the present invention comprises: a first camera which captures an object and generates an image; a distance measuring sensor which detects the distance of an object within a detection range by direction; a direction designation unit which detects any one direction within the detection range as a designated direction; an input unit which detects an input of a state of an object existing in the designated direction; and a calculation unit which calculates three-dimensional coordinates of the object based on the distance measuring sensor based on the distance to the object whose state input was detected by the input unit, and which, when a predetermined marker image is included in an image captured by the first camera, combines raw information used to calculate the three-dimensional coordinates of a marker corresponding to the marker image based on the distance measuring sensor with the three-dimensional coordinates and state of the object. a first controller that calculates three-dimensional coordinates of the marker based on the original information, and a first interface that can read a blueprint of a building together with position information of at least one origin on the blueprint, the first controller having a second interface that can read association of the three-dimensional coordinates and state of the target object to the corresponding position on the blueprint from a measuring device that calculates three-dimensional coordinates of the marker based on the original information and associates the three-dimensional coordinates and state of the target object with the corresponding position on the blueprint, together with the three-dimensional coordinates and state, and the blueprint; and a second controller that creates a report design drawing showing the state at a corresponding position in the design drawing based on the association.

[0011] In addition, the test result creation system according to the fifth aspect is as follows: a first camera that captures an object to generate an image, a distance measuring sensor that detects the distance of an object within a detection range by direction, a direction designation unit that detects any one direction within the detection range as a designated direction, an input unit that detects an input of a state of an object existing in the designated direction, a first controller that calculates three-dimensional coordinates of the object with the distance measuring sensor as a reference based on the distance to the object whose state input was detected by the input unit, and stores original information used in calculating the three-dimensional coordinates of a marker corresponding to a marker image with the distance measuring sensor as a reference in a case where the image captured by the first camera includes a predetermined marker image, in combination with the three-dimensional coordinates and state of the object, and a first interface that is capable of reading a blueprint of a building together with position information of at least one origin on the blueprint, wherein the first controller calculates the three-dimensional coordinates of the marker based on the original information, and associates the three-dimensional coordinates and state of the object with a corresponding position on the blueprint based on the three-dimensional coordinates and the position information on the blueprint; and a creation device having a second interface capable of reading, from the measurement device, an association of the three-dimensional coordinates and state of the target object with the corresponding position on the design drawing, together with the three-dimensional coordinates and the state and the design drawing, and a second controller that creates a report design drawing showing the state at the corresponding position on the design drawing based on the association.

[0012] In addition, the sixth perspective program is as follows: a first camera that captures an object and generates an image; a distance measuring sensor that detects the distance of an object within a detection range by direction; a direction designation unit that detects any one direction within the detection range as a designated direction; and an input unit that detects an input of a state of an object existing in the designated direction; a first interface capable of reading a blueprint of a building together with location information on said blueprint of at least one origin; A computer of a measuring device comprising: calculating three-dimensional coordinates of a target object based on the distance to the target object for which a state input is detected by the input unit; and When a predetermined marker image is included in the image captured by the first camera, raw information used for calculating three-dimensional coordinates of a marker corresponding to the marker image with respect to the distance measuring sensor as a reference is stored in combination with the three-dimensional coordinates and a state of the target object; Calculating three-dimensional coordinates of the marker based on the raw information; The Associating the three-dimensional coordinates and states of the target object with corresponding positions on the blueprint based on the three-dimensional coordinates and position information on the blueprint; and executing an operation including:

[0014] In addition, the program based on the eighth viewpoint is as follows: The present invention relates to a method for detecting a state of an object by imaging an object, the method comprising: a first camera for capturing an image of an object; a distance measuring sensor for detecting the distance of an object within a detection range by direction; a direction designation unit for detecting any one direction within the detection range as a designated direction; an input unit for detecting an input of a state of an object existing in the designated direction; a first interface capable of reading a design drawing of a building together with position information of at least one origin in the design drawing; and a method for calculating three-dimensional coordinates of the object based on the distance measuring sensor based on the distance to the object whose state input was detected in the input unit, and a method for detecting a state of the object when a predetermined marker image is included in an image captured by the first camera. a first controller that stores original information used to calculate the three-dimensional coordinates of a marker corresponding to the marker image based on the distance measuring sensor in combination with the three-dimensional coordinates and state of the target object, calculates the three-dimensional coordinates of the marker based on the original information, and associates the three-dimensional coordinates and state of the target object with the corresponding positions on the design drawing based on the three-dimensional coordinates and position information on the design drawing, together with the three-dimensional coordinates and the state, and the design drawing, from a measurement device having the above-mentioned measurement device; and generating a report design drawing showing the condition at a corresponding position on the design drawing based on the association. Effect of the Invention

[0015] The measuring device, creation device, inspection result creation system, and program disclosed herein configured as described above reduce the labor required to create a report design drawing in which the state of an object at any position in a building or the like is recorded at a corresponding position in a design drawing for the building. [Brief description of the drawings]

[0016] [Figure 1] 1 is an external view showing a schematic configuration of an inspection result creation system including a measurement device and a creation device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing a schematic configuration of the measurement device shown in FIG. [Diagram 3] 13 is an example of a list of options for the state of a target object. [Figure 4] 13 is an example of an image captured when a marker is placed in a predetermined position. [Diagram 5] 13 is an example of an attention area displayed on a display. [Figure 6] FIG. 2 is a block diagram showing a schematic configuration of the creating device shown in FIG. 1. [Figure 7] FIG. 1 is an example of a plan view for reporting. [Figure 8] FIG. 8 is a partial enlarged view of the part surrounded by the dashed line in FIG. 7. [Figure 9] This is an example of a report elevation. [Figure 10] 13 is an example of a report image. [Figure 11] 13 is an example of a summary table. [Figure 12] 4 is a flowchart for explaining a measurement process executed by a first controller included in the measurement device of FIG. [Figure 13] 4 is a flowchart for explaining a warning process executed by a first controller of the measurement device of FIG. [Figure 14] 4 is a flowchart for explaining a creating process executed by a second controller of the creating device of FIG. 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, embodiments of a measuring device and a creating device to which the present disclosure is applied will be described with reference to the drawings.

[0018] 1, an inspection result creation system 10 according to an embodiment of the present disclosure includes a measuring device 11 and a creation device 12. As will be described in detail below, the three-dimensional position of an object to be measured is measured at a location where measurement is performed by the measuring device 11, and is recorded in association with the state of the object. In addition, based on the three-dimensional position and state of the object acquired from the measuring device 11, a report design drawing showing the state of the object at a corresponding position on a drawing of the location where measurement is performed is created by the creation device 12.

[0019] In this embodiment, the measuring device 11 is a see-through head-mounted display type device. The measuring device 11 may be a terminal device such as a smartphone or tablet having the structure and function described below. The measuring device 11 has a front-rear direction, a left-right direction, and an up-down direction. In this embodiment, the up-down direction, the front-rear direction, and an up-down direction of the measuring device 11 are determined so that the front-rear direction, the left-right direction, and the up-down direction of the user approximately match the front-rear direction, the left-right direction, and the up-down direction of the measuring device 11 when the measuring device 11 is worn by the user.

[0020] 2, the measuring device 11 includes a first camera 13, a distance measuring sensor 14, an input unit 15, a direction designation unit 16, a memory 17, and a first controller 18. The measuring device 11 may further include a display 19, a first interface 20, a gravity direction sensor 21, a tilt sensor 22, and a displacement sensor 23.

[0021] The first camera 13 has an optical system and an imaging element, captures an image of an object, and generates an image as a signal. In this embodiment, the first camera 13 is fixed to the measuring device 11 at a predetermined position and in a predetermined attitude so that the field of view of the user through the transmissive display is included in the imaging range when the measuring device 11 is worn by the user. The first camera 13 is fixed to the measuring device 11, for example, near the center position in the left-right direction of the measuring device 11, in an attitude in which the optical axis is parallel to the front-rear direction of the measuring device 11 and the front is included in the imaging range.

[0022] The distance measuring sensor 14 detects the distance of an object within a detection range by direction. The distance measuring sensor 14 is, for example, an infrared sensor, a millimeter wave radar, a Lidar, etc. The distance measuring sensor 14 has a detection axis, and for example, the direction of the object to be detected by distance is distinguished by two directions based on the detection axis.

[0023] The distance measuring sensor 14 may be fixed to the measuring device 11 such that at least a part of the detection range is included within the imaging range of the first camera 13. Furthermore, in this embodiment, the distance measuring sensor 14 is fixed to the measuring device 11 at a predetermined position and in a predetermined attitude such that at least a part of the user's field of vision via the transmissive display is included in the detection range when the measuring device 11 is worn by the user. The distance measuring sensor 14 is fixed to the measuring device 11, for example, near the center position in the left-right direction of the measuring device 11, in an attitude in which the detection axis is parallel to the front-rear direction of the measuring device 11 and the front is included in the detection range.

[0024] The input unit 15 detects an input of the state of an object existing in a specified direction detected by a direction designation unit 16 described later. The input unit 15 is, for example, at least one of a combination of a second camera and a first controller 18, a combination of a switch mechanism capable of detecting whether it is in one of a plurality of states such as a button switch and the first controller 18, and a microphone. In this embodiment, the input unit 15 is a combination of the second camera and the first controller 18.

[0025] The second camera may be the same as the first camera 13 in a configuration in which at least a part of the detection range of the distance measuring sensor 14 is included within the imaging range of the first camera 13. The second camera may be provided in the measuring device 11 as a separate body of the first camera 13 in a configuration in which the second camera is separate from the first camera 13, at least a part of the detection range of the distance measuring sensor 14 is included within the imaging range of the second camera.

[0026] In a configuration in which the input unit 15 is a combination of the second camera and the first controller 18, when the first controller 18 detects a predetermined gesture from an image captured by the second camera, the first controller 18 detects the gesture as an operation input from the user. As will be described in detail later, the first controller 18 detects an input of the state of an object existing in the specified direction based on the operation input when the direction designation unit 16 described later detects the designated direction.

[0027] In a configuration in which the input unit 15 is a combination of a switch mechanism and a first controller 18, as will be described in detail later, the first controller 18 detects an input of the state of an object existing in a specified direction based on an operation input to the switch mechanism when the direction designation unit 16 described below detects a specified direction.

[0028] The direction designation unit 16 detects any one direction within the detection range of the distance measurement sensor 14 as the designated direction. The direction designation unit 16 is, for example, at least one of a line-of-sight sensor, a third camera alone, and a combination of the third camera and a pointer such as a percussion rod. In this embodiment, the direction designation unit 16 is the third camera alone.

[0029] The gaze sensor detects, as a specified direction, the gaze direction of a user of measuring device 11. In this embodiment, when measuring device 11 is worn by a user, the gaze sensor is fixed at a position and attitude in measuring device 11 that allows the gaze of the user to be detected.

[0030] The third camera may be the same as the second camera or may be separate from the second camera. In a configuration in which the third camera is separate from the second camera, at least a part of the detection range of the distance measuring sensor 14 is included within the imaging range of the third camera.

[0031] In a configuration in which the direction designation unit 16 is the third camera alone, the direction designation unit 16 detects a specific direction that overlaps with the detection range of the distance measurement sensor 14 within the imaging range of the third camera as the designated direction.

[0032] In a configuration in which the direction designation unit 16 is a combination of a third camera and a pointer, the position of the pointer is detected in an image captured by the third camera by image recognition, etc. A direction in real space corresponding to the position detected in the image is detected as the designated direction.

[0033] The memory 17 includes any storage device, such as a random access memory (RAM) and a read only memory (ROM), etc. The memory 17 stores various programs that cause the first controller 18 to function and various information used by the first controller 18.

[0034] The display 19 displays a desired image under the control of the first controller 18. The display 19 is a transmissive display or a non-transmissive display. In this embodiment, the display 19 is a transmissive display. The display 19 is provided on the measuring device 11 so as to be located in front of the user's eyes when the measuring device 11 is worn by the user. When wearing the measuring device 11, the user can view the scenery in front of them, with the desired image superimposed, through the display 19.

[0035] The first interface 20 communicates information with at least one of an external device and a storage medium. The first interface 20 may communicate with the external device in a wired or wireless manner. For example, as described below, the first interface 20 can read a blueprint of a building together with position information of at least one origin on the blueprint. The position information of the origin may be, for example, a point drawn at a specific position on the blueprint, or a coordinate in the coordinate system of the blueprint.

[0036] The design drawing includes, for example, at least one of a floor plan, an elevation, a 3D model, and a 2D model of a building. The floor plan may be a floor plan of any single or multiple floors. The 3D model is, for example, data in which a large number of points constituting a wall surface, such as CAD data, are converted into three-dimensional coordinates. The 2D model is data in which a large number of points constituting a wall surface, etc. are converted into two-dimensional coordinates by fixing the position of one coordinate axis of the 3D model. The origin is a point added to a predetermined position on the design drawing in order to associate the three-dimensional coordinates of a specific position detected by the measuring device 11 with the position of the above-mentioned design drawing. In this embodiment, the design drawing is not limited to a drawing such as the above-mentioned floor plan in which the shape, structure, and dimensions of the designed building, etc. are written according to a certain rule, but may be a drawing showing the structure of the building, etc., and may be, for example, a floor plan used in a pamphlet.

[0037] The gravity direction sensor 21 detects the direction of gravity on the ground.

[0038] The tilt sensor 22 detects the tilt of the measuring device 11. The tilt sensor 22 detects the tilt from the reference attitude, which is the attitude of the measuring device 11 with respect to the ground at the start of tilt detection. The tilt sensor 22 detects, for example, the tilt angle about three directions that are not parallel to each other as the tilt of the measuring device 11. In this embodiment, the tilt sensor 22 detects the tilt angle about three axes of the forward / backward direction, left / right direction, and up / down direction of the measuring device 11. The tilt sensor 22 is, for example, a gyro sensor.

[0039] The displacement sensor 23 detects the amount of displacement of the measuring device 11. The displacement sensor 23 detects the amount of displacement from a reference position, which is the position of the measuring device 11 with respect to the ground at the start of detecting the amount of displacement. The displacement sensor 23 detects the amount of displacement of the measuring device 11 along, for example, three mutually orthogonal directions. In this embodiment, the displacement sensor 23 detects the amount of displacement along three directions of the measuring device 11, the forward / backward direction, the left / right direction, and the up / down direction. The displacement sensor 23 is, for example, an acceleration sensor. In a configuration in which the displacement sensor 23 is an acceleration sensor, the amount of displacement detected by the displacement sensor 23 may be supplemented by high-precision GNSS data, a two-point survey of a 3D real space model, or the like, depending on the usage environment.

[0040] The first controller 18 includes one or more processors and memories. The processor may include a general-purpose processor that loads a specific program to execute a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include a field-programmable gate array (FPGA). The first controller 18 may be either a system-on-a-chip (SoC) or a system in a package (SiP) in which one or more processors work together. The first controller 18 controls the operation of the entire measurement device 11.

[0041] The first controller 18 functions as a part of the input unit 15 as described above, and detects an input of the state of an object existing in a specified direction as described below. In a configuration in which the input unit 15 includes a second camera, the first controller 18 determines whether or not a position designation gesture is present in a plurality of images captured continuously by the second camera. The position designation gesture is a predetermined action such as a finger flicking motion.

[0042] The first controller 18 determines an object located in the specified direction detected by the direction designation unit 16 when detecting the presence of a position designation gesture as a target object to be measured. When determining the target object, the first controller 18 presents options for the state of the target object to the user. The presentation to the user may be by displaying an image or outputting sound. In this embodiment, the first controller 18 controls the display 19 to display a list of options that are assumed in advance as the state of the target object. The first controller 18 may present a list of options that are assumed in advance as the target object to the user prior to or simultaneously with the presentation of the list of state options.

[0043] For example, as shown in Fig. 3, the options included in the list 24 of state options include "rebar explosion / exposure," "missing / spaling," "efflorescence / rust," "crack," "floating," "crack (area)," and "floating (area)." Also, for example, the list 25 of target object options includes "paint," "concrete / mortar," "tile," and "sheet."

[0044] The target object assumed in advance is not limited to an object existing on an exterior wall, but may be an object existing inside a building. Target objects present indoors include the stone floor, floor stones, ceiling, wall covering, and downlights in the entrance hall; the flooring, storage doors, sliding doors, interior walls, ceiling, and outlet plates in the Western-style room; the flooring and baseboards, upper frames and flooring, wall covering, flooring, and door frames and coverings in the hallway; the toilet paper holder, ceiling ventilation fan, inspection hatch, hanging door, and door; the ceiling, faucet, lighting, counter, and bathtub and walls in the bathroom; the sink, faucet, sink, linen closet door, and waterproof pan ventilation fan in the washroom; the counter, lighting, cupboards and interior walls, alarm, and garbage disposer in the kitchen; the curtain rail, glass, sashes, wall doors, and sleeves in the living and dining rooms; the clothes drying hardware, handrails, piping covers, ventilation sleeves, and partitions on the balcony; and the ceiling resin, meter box door, entrance door, and grilles in the open corridor.

[0045] When the first controller 18 detects an input for selecting one of a plurality of options for a state, the first controller 18 stores the state specified by the option in the memory 17 as the state of the target object. The first controller 18 may detect the input for selection by, for example, detecting a gesture for confirming the selection in a state in which the direction designation unit 16 detects a direction corresponding to the display position of one of the plurality of options displayed on the display 19. Alternatively, the first controller 18 may detect the input for selection by, for example, voice analysis detected by a microphone. The first controller 18 may detect an input for selecting one of a plurality of options for specifying a target object in the same manner as the plurality of options for a state, and store the input in the memory 17.

[0046] In a configuration in which the input unit 15 includes a switch mechanism, the first controller 18 uses an operation input to the switch mechanism instead of detecting a gesture in an image from the second camera described above to detect an input of the state of an object existing in a specified direction.

[0047] When an input of a state of a target object is detected, the first controller 18 acquires the distance to the target object and the direction based on the distance measurement sensor 14 from the distance measurement sensor 14. The first controller 18 calculates three-dimensional coordinates of the target object based on the distance measurement sensor 14 based on the acquired distance and direction. The first controller 18 may store the three-dimensional coordinates of the target object in the memory 17.

[0048] 4, when a predetermined marker image 27 is included in an image 26 captured by the first camera 13, the first controller 18 acquires raw information. The raw information is information that allows calculation of the three-dimensional coordinates of the marker corresponding to the marker image 27 with respect to the distance measuring sensor 14.

[0049] The marker may be a point indicating the origin, a line segment including the origin, or a predetermined pattern such as a QR code (registered trademark). The origin as a marker may be a point recognizable in the image 26 by the first controller 18 or a second controller of the creation device 12 described later, for example, a cross intersection drawn with ink at an arbitrary location at a construction site, or a point printed on a sheet or the like. The line segment including the origin as a marker may be a line segment recognizable in the image 26 by the first controller 18 or the second controller, for example, a line segment that is part of the boundary between the entryway step and the baseboard, or a line segment printed on a sheet or the like. The predetermined pattern as a marker may be printed on a rectangular sheet or board, and may be determined to be placed so that two sides perpendicular to each other are in close contact with two walls of a building when the board surface is viewed in a plan view. Information on the origin position is associated with the predetermined pattern in advance.

[0050] The two markers may be located on-site and may be preliminarily arranged to be captured in a single image 26 or multiple images 26. When the first controller 18 recognizes two marker images 27 in the image 26, the first controller 18 may obtain raw information based on the distance measuring sensor 14 of the markers corresponding to the respective marker images.

[0051] The markers may be placed on a plurality of floors, which are at least a portion of the floors of the entire building, and may be imaged by the first camera 13. When the markers are placed on a plurality of floors, the markers may have different patterns defined for the floors on which they are placed.

[0052] In a configuration in which the second camera is the same as the first camera 13, the raw information may be the distance of the marker detected by the distance measuring sensor 14 and the direction in which the marker is located. Based on the distance and the direction, the three-dimensional coordinates of the marker based on the distance measuring sensor 14 can be calculated.

[0053] Alternatively, the original information may be the distance of an object detected by the distance measuring sensor 14 when the marker image 27 is detected, the direction in which the object is located, the relative position and orientation of the distance measuring sensor 14 with respect to the first camera 13, and the position and size of the marker image 27 in the image 26. Based on the distance of the object and the direction in which the object is located, the three-dimensional coordinates of the object based on the distance measuring sensor 14 can be calculated. Based on the position and size of the marker image 27 in the entire image 26, the three-dimensional coordinates of the marker based on the first camera 13 can be calculated. Based on the relative position and orientation of the distance measuring sensor 14 with respect to the first camera 13, the three-dimensional coordinates of the object based on the distance measuring sensor 14, and the three-dimensional coordinates of the marker based on the first camera 13, the three-dimensional coordinates of the marker based on the distance measuring sensor 14 can be calculated.

[0054] Furthermore, the raw information may include a marker image 27 .

[0055] The first controller 18 combines the original information with the three-dimensional coordinates and state of the target object, and stores the combined information in the memory 17. The first controller 18 may associate the three-dimensional coordinates and state of the target object with a position in an image captured by the second camera that corresponds to the three-dimensional coordinates, and store the image in the memory 17.

[0056] The first controller 18 stores in the memory 17 the first tilt detected by the tilt sensor 22 when the distance measuring sensor 14 detects the distance to the target object, in combination with the three-dimensional coordinates of the target object. The first controller 18 includes in the original information the second tilt detected by the tilt sensor 22 when the first camera 13 generates the image 26 including the marker image 27. Even if the attitude of the measuring device 11 with respect to the ground is different when the target object is determined and when the image 26 including the marker image 27 is captured, the three-dimensional coordinates of the target object and the marker can be arranged in the same coordinate space based on the tilt at each time, as described below. Note that if the attitude of the measuring device 11 with respect to the ground is the same when the target object is determined and when the image 26 including the marker image 27 is captured, the three-dimensional coordinates of the target object and the marker can be arranged in the same coordinate space even without tilt information.

[0057] The first controller 18 may calculate the three-dimensional coordinates of the target object based on the attitude of the distance measuring sensor 14 with respect to the ground at the time of acquiring the original information, in other words, at the time of capturing the image of the marker, based on the first tilt and the second tilt. The first controller 18 may store the calculated three-dimensional coordinates of the target object at the time of acquiring the original information as a combination of the original information and the three-dimensional coordinates of the target object.

[0058] The first controller 18 stores the amount of displacement from the reference position detected by the displacement sensor 23 when the distance measuring sensor 14 detects the distance to the target object as a first displacement amount in the memory 17 in combination with the three-dimensional coordinates of the target object. The first controller 18 includes the amount of displacement from the reference position detected by the displacement sensor 23 when the first camera 13 generates the image 26 including the marker image 27 as a second displacement amount in the original information. Even if the measuring device 11 is displaced between the time when the target object is determined and the time when the image 26 including the marker image 27 is captured, the three-dimensional coordinates of the target object and the marker can be arranged in the same coordinate space based on the amount of displacement at each time, as described later. Note that, if the measuring device 11 is in the same position between the time when the target object is determined and the time when the image 26 including the marker image 27 is captured, the three-dimensional coordinates of the target object and the marker can be arranged in the same coordinate space even without information on the amount of displacement.

[0059] The first controller 18 may calculate the three-dimensional coordinates of the target object based on the position of the distance measuring sensor 14 relative to the ground at the time of acquiring the original information, in other words, at the time of capturing the image of the marker, based on the first displacement amount and the second displacement amount. The first controller 18 may store the calculated three-dimensional coordinates of the target object at the time of acquiring the original information as a combination of the original information and the three-dimensional coordinates of the target object.

[0060] The first controller 18 may control the tilt sensor 22 and the displacement sensor 23 to start detection when the raw information is first acquired. By starting detection when the raw information is first acquired, the attitude and position of the measuring device 11 with respect to the ground when the raw information is first acquired become the reference attitude and reference position, respectively.

[0061] The first controller 18 may generate a 3D model of a structure within the detection range of the distance measuring sensor 14 based on three-dimensional coordinates calculated based on the distances to objects located in various directions, in other words, object points, of the distance measuring sensor 14. Furthermore, the first controller 18 generates a 3D model of at least a part of the structure based on the distances to a number of objects detected while the user is moving in various directions while wearing the measuring device 11, as well as the inclination of the measuring device 11 and the amount of displacement from a reference position. The first controller 18 may calculate the three-dimensional coordinates of the target object as three-dimensional coordinates in the coordinate system of the created 3D model, and store them in the memory 17.

[0062] When a marker image 27 different from a marker image 27 detected in an image 26 is detected at a position displaced from the position at which the image 26 was captured, the first controller 18 may combine the original information that enables calculation of the three-dimensional coordinates of the marker corresponding to each marker image 27 with the three-dimensional coordinates and state of the target object and store it in the memory 17.

[0063] The first controller 18 may control the first interface 20 to transmit the combined raw information, as well as the three-dimensional coordinates and state of the target object to an external server.

[0064] When the first controller 18 reads the design drawing through the first interface 20, the first controller 18 calculates the three-dimensional coordinates of the marker based on the original information. Furthermore, the first controller 18 associates the three-dimensional coordinates and state of the target object with the corresponding position on the design drawing based on the three-dimensional coordinates of the marker and the position information of the origin of the design drawing, as described below, and stores them in the memory 17.

[0065] The blueprint includes at least one origin and a building, as described above. The blueprint may further include at least one of an exterior wall, an interior wall, a ceiling, and a floor of the building. The position of the origin in the blueprint is associated with the position of the marker on the design, by the pattern of the marker. The position of the origin associated with the marker is not limited to one, and a plurality of positions of the origin may be associated with a single marker. The position of the origin from the position of the marker, which corresponds to the pattern of the marker, is stored in advance in the memory 17. When the marker image 27 is detected, the first controller 18 further reads out the three-dimensional coordinates of the origin relative to the three-dimensional coordinates of the marker from the memory 17 based on the pattern of the marker image 27. The first controller 18 associates the three-dimensional coordinates and state of the target object with the corresponding position in the blueprint, as described above, based on the calculated three-dimensional coordinates of the marker, the three-dimensional coordinates of the origin relative to the three-dimensional coordinates of the marker, and the position of the origin in the coordinate system of the blueprint.

[0066] The first controller 18 may associate the three-dimensional coordinates and state of the target object with a position corresponding to the blueprint, also based on the direction of gravity detected by the gravity direction sensor 21.

[0067] The first controller 18 may control the first interface 20 to transmit the design drawing associated with the three-dimensional information and state of the target object to an external server.

[0068] When an attention area is detected in the image captured by the second camera, the first controller 18 controls the display 19 to display the attention area. As shown in FIG. 5, the attention area CA may be, for example, an area indicating a boundary with an area where the height changes to a higher or lower position than the floor surface where the user wearing the measuring device 11 is located, such as a staircase, a step, a hole, or a groove, an area where an obstacle such as a beam exists at a predetermined height, such as 170 cm, from the floor surface, and an area where entry is prohibited. The first controller 18 detects the attention area CA by detecting an area where the height changes with respect to a surface recognized as the floor surface in the generated 3D model, an area where an obstacle is located at a height less than a predetermined height with respect to the floor surface in the generated 3D model, and an area behind where a no entry sign is detected in the image captured by the second camera. Alternatively, the attention area CA may be predetermined in the loaded design drawing, and the first controller 18 detects the attention area CA when the area of ​​the image captured by the second camera includes the attention area CA.

[0069] The first controller 18 controls the display 19 to display a warning when detecting the approach of the user wearing the measuring device 11 to the attention area CA detected in the image captured by the second camera. The displayed warning may be, for example, a simple wording of "warning", a wording meaning a warning such as "wear a helmet from here" and "use a safety belt", or a symbol meaning a warning such as "!". The first controller 18 may detect the approach to the attention area CA when the user enters within a predetermined range, such as 2 m, of the attention area CA recognized in the generated 3D model. The warning is not limited to being displayed on the display 19, and may be configured to be recognized by the user by outputting a warning sound from a speaker or the like, for example.

[0070] In this embodiment, the creation device 12 may be a general-purpose device or a dedicated device such as a personal computer, a terminal device such as a smartphone, a tablet, etc. As shown in FIG. 6, the creation device 12 includes a second interface 28, a memory 29, and a second controller 30.

[0071] The second interface 28 communicates information with the measurement device 11, a storage medium, and / or an external device such as a scanner. The second interface 28 may communicate with the external device and the measurement device 11 in a wired or wireless manner.

[0072] The second interface 28 can read a combination of the three-dimensional coordinates and state of the target object and the original information from the measurement device 11 directly or indirectly via a storage medium. Alternatively, the second interface 28 can read associations of the three-dimensional coordinates and state of the target object to corresponding positions in a design drawing, together with the three-dimensional coordinates and state, and the design drawing, from the measurement device 11. Furthermore, the second interface 28 can read associations of the three-dimensional coordinates and state of the target object to corresponding positions in an image captured of the target object, together with the image, from the measurement device 11.

[0073] The second interface 28 is capable of reading, directly or indirectly via at least one of a storage medium, a design drawing of a building together with position information on the design drawing of at least one origin from an external device.

[0074] The memory 29 includes any storage device, such as, for example, a RAM, a ROM, etc. The memory 29 stores various programs that cause the second controller 30 to function, and various information used by the second controller 30.

[0075] The second controller 30 includes one or more processors and memories. The processor may include a general-purpose processor that loads a specific program to execute a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application specific IC (ASIC). The processor may include a programmable logic device (PLD). The PLD may include an FPGA. The second controller 30 may be either an SoC in which one or more processors work together, or an SiP. The second controller 30 controls the operation of the entire creation device 12.

[0076] When the second interface 28 reads the original information and the three-dimensional coordinates and state of the target object from the measuring device 11, the second controller 30 calculates the three-dimensional coordinates of the marker based on the original information. The second controller 30 associates the three-dimensional coordinates and state of the target object with the corresponding position on the blueprint based on the three-dimensional coordinates of the marker and position information of at least one origin on the blueprint. The association method is the same as the association method by the first controller 18 of the measuring device 11. The second controller 30 stores in the memory 29 the association of the three-dimensional coordinates and state of the target object with the corresponding position on the blueprint by the second interface 28.

[0077] The second controller 30 stores in the memory 29 the association of the three-dimensional coordinates and state of the target object to the corresponding position in the design drawing when the second interface 28 reads the association.

[0078] The second controller 30 creates a report design drawing showing the state at the corresponding position in the design drawing based on the association of the three-dimensional coordinates and state of the target object with the corresponding position in the design drawing. When the design drawing is a plan drawing, for example, as shown in FIG. 7, a report design drawing showing the state ST is created at the position of the target object whose state is recorded in the plan drawing PD. As shown in FIG. 8, in addition to the state of the target object itself, for example, a position not expressed in the plan drawing PD, such as a distance from the floor or ceiling, the size of the state, etc. may be described as the state ST. When the design drawing is an elevation drawing, for example, as shown in FIG. 9, a report elevation drawing showing the state ST is created at the position of the target object whose state is recorded in the elevation drawing ED.

[0079] The second controller 30 may create a report design drawing in which states ST are written separately for multiple target objects at different positions at corresponding positions in the design drawing, as shown in Fig. 7. When multiple target objects at different positions are included within a range of a predetermined size in the report design drawing, the second controller 30 may collect the states of the multiple target objects and write them on the report design drawing. For example, as shown in Fig. 8, the states ST1 of multiple target objects within a predetermined range in the plan view PD are collected and written in one place, unlike the state ST2 of a single target object.

[0080] When the report design drawing includes multiple types of design drawings, such as a report plan and an elevation drawing, the second controller 30 associates the descriptions of the states of the target object at the same three-dimensional coordinates in the multiple types of design drawings with each other between the multiple types of design drawings. In other words, for example, descriptions of the states of the target object at the same position in the report plan, elevation drawing, 3D model, and 2D model are linked with each other. Therefore, when viewing with an electronic device such as the creation device 12, when a state described in any design drawing is selected, another type of design drawing in which the state of the target object in that state is described is opened.

[0081] When the second controller 30 reads in the association of the three-dimensional coordinates and state of the target object with the corresponding position in the image captured of the target object together with the image, the second controller 30 generates a report image. As shown in Fig. 10, in the report image, a mark MK for identifying the target object whose state has been input is drawn at the corresponding position in the image 31. In addition, the report image may have the state ST of the target object drawn next to the image 31.

[0082] The second controller 30 generates the report image and associates the drawing of the mark MK in the report image and the description of the state in the report design drawing for the target object at the same three-dimensional coordinates with each other. In other words, for example, for the target object at the same position, the description of the state in the report plan or the like and the drawing of the mark MK in the report image are linked with each other. Therefore, when the mark MK drawn in the report image is selected in a state in which the report image is viewed on an electronic device such as the creation device 12, a report design drawing is opened in which the state of the target object whose position is indicated by the mark MK is described. Alternatively, when the symbol CM drawn in the report drawing, which indicates that the report image is being captured, is selected in a state in which the report drawing is viewed on an electronic device such as the creation device 12, a report image including an image captured from the position indicated by the symbol CM is opened.

[0083] The second controller 30 creates a summary table regarding the state of the target object in conjunction with the creation of the design drawing for report. As shown in Fig. 11, the summary table is a table classified by state type, and is a table summarizing the details of the state by location of the structure.

[0084] The second controller 30 stores a report that includes the report design drawings, report images, and a summary table in the memory 29. The second controller 30 controls the second interface 28 to transmit the report to a printer that prints the report, or transmit the report to an external personal computer, terminal device, or the like, based on an operational input of a transmission request from the user of the creation device 12.

[0085] Next, the measurement process executed by the first controller 18 in this embodiment will be described with reference to the flowchart of Fig. 12. The measurement process starts when the input unit 15 detects an input from the measuring device 11 to start measurement.

[0086] In step S100, the first controller 18 activates the first camera 13, the distance measurement sensor 14, the gravity direction sensor 21, the tilt sensor 22, and the displacement sensor 23. The tilt sensor 22 detects the subsequent tilt of the measuring device 11, using as a reference attitude the attitude of the measuring device 11 with respect to the ground at the time when the tilt detection starts upon activation. The displacement sensor 23 detects the subsequent displacement amount of the measuring device 11, using as a reference the position of the measuring device 11 with respect to the ground at the time when the detection of the displacement amount starts upon activation. After activation, the process proceeds to step S101.

[0087] In step S101, the first controller 18 starts creating a 3D model of the structure based on the distances of the object in multiple directions detected by the distance measuring sensor 14, the direction of gravity detected by the gravity direction sensor 21, the inclination detected by the inclination sensor 22, and the amount of displacement detected by the displacement sensor 23. After starting the creation, the process proceeds to step S102.

[0088] In step S102, the first controller 18 determines whether or not the image 26 captured by the first camera 13 includes a marker image 27. If the image 26 includes a marker image 27, the process proceeds to step S103. If the image 27 does not include a marker image 27, the process proceeds to step S105.

[0089] In step S103, the first controller 18 acquires the raw information. After acquiring the raw information, the process proceeds to step S104.

[0090] In step S104, the first controller 18 stores the raw information acquired in step S103 in the memory 17. After storage, the process proceeds to step S105.

[0091] In step S105, the first controller 18 determines whether the input unit 15 detects an input of a state of an object existing in the specified direction. If it has been detected, the process proceeds to step S106. If it has not been detected, the process proceeds to step S108.

[0092] In step S106, the first controller 18 regards the object for which the state input was detected in step S102 as the target object, and calculates the three-dimensional coordinates of the target object in the coordinate system of the 3D model based on the distance and direction to the target object, the direction of gravity of the measuring device 11, the inclination of the measuring device 11, and the displacement amount of the measuring device 11. After the calculation, the process proceeds to step S107.

[0093] In step S107, the first controller 18 stores the three-dimensional coordinates of the target object calculated in step S106 together with the state in the memory 17. After storage, the process proceeds to step S108.

[0094] In step S108, the first controller 18 determines whether the raw information in step S104 and the three-dimensional coordinates and state in step S107 have been stored in the memory 17. If at least one of them has not been stored, the process proceeds to step S109. If both have been stored, the process proceeds to step S110.

[0095] In step S109, the first controller 18 determines whether or not the input unit 15 detects an input for ending the measurement. If not, the process returns to step S102. If yes, the measurement process ends.

[0096] In step S110, the first controller 18 stores the raw information confirmed to have been stored in step S107, and the three-dimensional coordinates and state of the target object as a combination in the memory 17. After storage, the process proceeds to step S111.

[0097] In step S111, the first controller 18 determines whether the design drawing has been loaded. If the design drawing has been loaded, the process proceeds to step S112. If the design drawing has not been loaded, the process returns to step S102.

[0098] In step S112, the first controller 18 calculates the three-dimensional coordinates of the marker based on the original information stored in the memory 17 in step S104. The first controller 18 further associates the three-dimensional coordinates and state of the target object with the corresponding position in the design drawing based on the calculated three-dimensional coordinates and the position information of the origin of the design drawing confirmed to have been read in step S111. The first controller 18 stores the association in the memory 17. After storage, the process returns to step S102.

[0099] Next, the warning process executed by first controller 18 in this embodiment will be described with reference to the flowchart in Fig. 13. The warning process is executed in parallel with the measurement process. In other words, the warning process starts when input unit 15 detects an input to start measurement by measuring device 11.

[0100] In step S200, the first controller 18 detects an attention area CA for the current position based on the 3D model created in the measurement process. After detection, the process proceeds to step S201.

[0101] In step S201, the first controller 18 determines whether or not the attention area CA is included in the field of view of the user wearing the measurement device 11. In a configuration in which the display 19 is a transmissive display, the user's field of view is a range that can be seen from the position of the user's pupil through the transmissive display. Alternatively, in a configuration in which the display 19 is a non-transmissive display, the user's field of view is a range of the field of view that is blocked by the non-transmissive display. If the attention area CA is included, the process proceeds to step S202. If the attention area CA is not included, the process proceeds to step S203.

[0102] In step S202, the first controller 18 controls the display 19 to display the attention area CA. After displaying, the process proceeds to step S203.

[0103] In step S203, the first controller 18 determines whether the user of the measurement device 11 is approaching the attention area CA. If the user is approaching, the process proceeds to step S204. If the user is not approaching, the process proceeds to step S205.

[0104] In step S204, the first controller 18 controls the display 19 to display a warning. After the warning, the process proceeds to step S205.

[0105] In step S205, the first controller 18 determines whether or not the input unit 15 detects an input indicating the end of measurement. If not, the process returns to step S200. If detected, the warning process ends.

[0106] Next, the creation process executed by the second controller 30 in this embodiment will be described with reference to the flowchart of Fig. 14. The creation process starts when the input unit of the creation device 12 detects an operation input for creating a report in the creation device 12.

[0107] In step S300, the second controller 30 determines whether or not a combination of raw information and the three-dimensional coordinates and state of the target object has been read from the measurement device 11. If the combination has been read, the process proceeds to step S301. If the combination has not been read, the process proceeds to step S305.

[0108] In step S301, the second controller 30 calculates the three-dimensional coordinates of the marker based on the raw information whose reading has been confirmed in step S300. After the calculation, the process proceeds to step S302.

[0109] In step S302, the second controller 30 reads out the origin position corresponding to the marker from the memory 29 based on the raw information whose reading was confirmed in step S300. After reading, the process proceeds to step S303.

[0110] In step S303, the second controller 30 determines whether the design drawing has been read in. If the design drawing has been read in, the process proceeds to step S304. If the design drawing has not been read in, the process proceeds to step S305.

[0111] In step S304, the second controller 30 associates the three-dimensional coordinates and state of the target object with the corresponding position in the design drawing based on the three-dimensional coordinates of the marker calculated in step S301, the origin position corresponding to the marker read in step S302, and at least one position information of the origin in the design drawing whose reading is confirmed in step S303. Furthermore, the second controller 30 stores the association of the three-dimensional coordinates and state of the target object with the corresponding position in the design drawing in the memory 29. After the association, the process proceeds to step S305.

[0112] In step S305, the second controller 30 determines whether or not the association of the three-dimensional coordinates and state of the target object with the corresponding position in the design drawing together with the three-dimensional coordinates and state has been read from the measurement device 11. If the association has been read, the process proceeds to step S306. If the combination has not been read, the process proceeds to step S307.

[0113] In step S306, the second controller 30 stores the association of the three-dimensional coordinates and state of the target object confirmed to have been read in step S305 with the corresponding position on the design drawing in the memory 29. After storage, the process proceeds to step S307.

[0114] In step S307, the second controller 30 determines whether the association of the three-dimensional coordinates and state of the target object with the corresponding position in the design drawing has been stored in the memory 29. If not, the process returns to step S300. If it has been stored, the process proceeds to step S308.

[0115] In step S308, the second controller 30 creates a report design based on the association of the three-dimensional coordinates and states of the target object to the corresponding positions in the design. After creation, the process proceeds to step S309.

[0116] In step S309, the second controller 30 determines whether the design drawings confirmed to have been read in step S303 include multiple types of design drawings. If multiple types of design drawings are included, the process proceeds to step S310. If multiple types of design drawings are not included, the process proceeds to step S311.

[0117] In step S310, the second controller 30 associates the descriptions of the states of the target object at the same three-dimensional coordinates in the multiple design drawings among the multiple types of design drawings. Furthermore, the second controller 30 stores the association of the descriptions of the states among the multiple types of design drawings in the memory 29. After the association, the process proceeds to step S311.

[0118] In step S311, the second controller 30 determines whether an image associated with the three-dimensional coordinates of the target object has been read from the measurement device 11. If the image has been read, the process proceeds to step S312. If the image has not been read, the process proceeds to step S313.

[0119] In step S312, the second controller 30 creates an image for report using the image whose reading has been confirmed in step S311. After the image is created, the process proceeds to step S313.

[0120] In step S313, the second controller 30 associates the description of the state in the report image created in step S308 with the drawing of the mark MK in the report image created in step S312. After the association, the process proceeds to step S314.

[0121] In step S314, the second controller 30 creates a summary table regarding the state of the target object. After creation, the process proceeds to step S315.

[0122] In step S315, the second controller 30 stores the report, which includes at least the report design drawing created in step S308 and the summary table created in step S14, and also includes the report image if one was created in step S312, in the memory 29. After storing, the creation process ends.

[0123] The measuring device 11 of the present embodiment configured as described above calculates the three-dimensional coordinates of the target object based on the distance measuring sensor 14 based on the distance to the target object whose state input is detected by the input unit 15. Furthermore, when a predetermined marker image 27 is included in the image 26 captured by the first camera 13, the measuring device 11 stores the original information used to calculate the three-dimensional coordinates of the marker corresponding to the marker image 27 based on the distance measuring sensor 14 in combination with the three-dimensional coordinates and state of the target object. With this configuration, the measuring device 11 can provide information to an external device such as the creation device 12, which has information on the design drawing including the position information of the origin corresponding to the marker, for associating the three-dimensional coordinates and state of the target object with the corresponding position in the design drawing. Therefore, the measuring device 11 configured as described above can cause an external device to execute the association of the three-dimensional coordinates and state of the target object with the design drawing, which is a cumbersome task in creating a design drawing for reporting, thereby reducing the labor and simplifying the creation of the design drawing for reporting.

[0124] In addition, the measuring device 11 of this embodiment calculates the three-dimensional coordinates of the marker based on the original information, and associates the three-dimensional coordinates and state of the target object with the corresponding position in the blueprint based on the three-dimensional coordinates and the position information of at least one origin in the blueprint. With this configuration, the measuring device 11 can provide an external device such as a creation device 12 that creates a report blueprint with the association of the three-dimensional coordinates and state of the target object with the corresponding position in the blueprint. Therefore, the measuring device 11 having the above configuration associates the three-dimensional coordinates and state of the target object with the blueprint, which is a cumbersome task in creating a report blueprint, and therefore can reduce and simplify the creation of the report blueprint.

[0125] In addition, the measuring device 11 of this embodiment associates the three-dimensional coordinates and state of the target object with a position corresponding to the blueprint based on the direction of gravity detected by the gravity direction sensor 21. The three-dimensional coordinates of the target object are based on the distance measuring sensor 14, and are not associated with a coordinate system on the earth's surface. The measuring device 11 having the above-mentioned configuration can convert the three-dimensional coordinates based on the distance measuring sensor 14 into three-dimensional coordinates in a coordinate system based on the earth's surface by using the direction of gravity. With this configuration, the measuring device 11 improves the accuracy of associating the three-dimensional coordinates and state of the target object with a position corresponding to the blueprint. With this configuration, even if the measuring device 11 does not generate a 3D model, the measuring device 11 can associate the three-dimensional coordinates and state of the target object with a position corresponding to the blueprint as long as it has position information of two origins.

[0126] Moreover, the measuring device 11 of this embodiment includes a second camera, which is the same as or separate from the first camera 13 and includes at least a part of the detection range within its imaging range, and associates the three-dimensional coordinates and state of the target object with the corresponding position in the image captured by the second camera. With this configuration, the measuring device 11 can provide an external device, such as a creation device 12 that creates a report image, with the association of the three-dimensional coordinates and state of the target object with the image 31. Therefore, the measuring device 11 having the above configuration can cause an external device to perform the association of the three-dimensional coordinates and state of the target object with the image 31, which is a cumbersome task in creating a report image, and therefore can reduce and simplify the creation of the report image.

[0127] Moreover, the measuring device 11 of this embodiment includes a second camera that is the same as or separate from the first camera 13 and includes at least a part of the detection range within the imaging range, and detects the input of the state of the target object based on a gesture detected in an image captured by the second camera. With this configuration, the measuring device 11 can detect the input of the state without using a device that requires finger operation, such as a keyboard or joystick. Therefore, in a configuration in which the measuring device 11 can perform measurement while moving, the measuring device 11 can reduce the amount of hardware to be held in the hand, thereby improving the safety of the user of the measuring device 11.

[0128] Furthermore, the measuring device 11 of this embodiment includes a second camera that is the same as or separate from the first camera 13 and includes at least a part of the detection range within an imaging range, and a display 19 that displays an attention area CA detected in an image captured by the second camera. With this configuration, in a configuration in which the measuring device 11 is a head-mounted display type, the measuring device 11 can suppress a decrease in awareness of dangers in an actual site due to concentration on the measurement work by displaying the attention area CA.

[0129] Moreover, the measuring device 11 of this embodiment is equipped with a second camera, which is the same as or separate from the first camera 13 and includes at least a part of the detection range within its imaging range, and outputs a warning when it detects approach to the attention area CA detected in the image captured by the second camera. With this configuration, the measuring device 11 can further suppress a decrease in awareness of danger in the actual field.

[0130] Furthermore, the measuring device 11 of this embodiment stores raw information for at least a portion of the floors of the entire building in combination with the three-dimensional coordinates and state of the target object. With this configuration, the measuring device 11 can suppress a decrease in accuracy of the raw information due to movement between floors even in a configuration in which the displacement sensor 23 detects the amount of displacement of the measuring device 11. Alternatively, the measuring device 11 can combine raw information corresponding to a plurality of floors with the three-dimensional coordinates and state of the target object in a configuration in which the displacement sensor 23 does not detect the amount of displacement of the measuring device 11.

[0131] The measuring device 11 of this embodiment is equipped with an inclination sensor 22, and stores a combination of a first inclination detected by the inclination sensor 22 and the three-dimensional coordinates of the target object when the distance measuring sensor 14 detects the distance to the target object, and includes a second inclination detected by the inclination sensor 22 when the first camera 13 generates an image 26 including a marker image 27 in the original information. With this configuration, the measuring device 11 can cancel the deviation of the coordinate system between the three-dimensional coordinates of the marker calculated from the original information and the three-dimensional coordinates of the target object even if the attitude of the measuring device 11 with respect to the ground surface is different when the state of the multiple target objects is input and when the image including the marker image 27 is captured. Therefore, the measuring device 11 can store information that can associate the three-dimensional coordinates and state of the target object with the corresponding position in the design drawing even if the first camera 13 cannot capture the image of the marker when the state of the target object is input.

[0132] In addition, the measuring device 11 of this embodiment includes a displacement sensor 23, and stores a combination of the displacement amount from a reference position detected by the displacement sensor 23 when the distance measuring sensor 14 detects the distance to the target object and the three-dimensional coordinates of the target object, and includes the displacement amount from a reference position detected by the displacement sensor 23 when the first camera 13 generates an image 26 including a marker image 27 in the original information. With this configuration, even if the measuring device 11 moves between the time when the state of multiple target objects is input and the time when the image including the marker image 27 is captured, the measuring device 11 can cancel the deviation of the coordinate system between the three-dimensional coordinates of the marker calculated from the original information and the three-dimensional coordinates of the target object. Therefore, even if the first camera 13 cannot capture an image of the marker when the state of the target object is input, the measuring device 11 can store information that can associate the three-dimensional coordinates and state of the target object with the corresponding position in the design drawing.

[0133] In addition, the measuring device 11 of the present embodiment may transmit the combination of the original information and the three-dimensional coordinates and state of the target object to an external server. Alternatively, the measuring device 11 may transmit the design drawing associated with the three-dimensional coordinates and state of the target object to an external server. With this configuration, the measuring device 11 may also store information for creating the report design drawing in the cloud server without requiring manual operation by the user.

[0134] Furthermore, the measuring device 11 of this embodiment generates a 3D model of the structure within the detection range of the distance measuring sensor 14. With this configuration, the measuring device 11 generates a 3D model that can be compared in size and the like with the blueprint of the building to be inspected, so that once it can be aligned with a single origin, the three-dimensional coordinates and state of the target object can be associated with the corresponding position on the blueprint by adjusting the rotation and magnification.

[0135] In addition, the creation device 12 of this embodiment can read the blueprint of the building together with the position information of at least one origin in the blueprint, and reads the original information and the combination of the three-dimensional coordinates and state of the target object from the measurement device 11, and calculates the three-dimensional coordinates of the marker based on the original information. Furthermore, the creation device 12 associates the three-dimensional coordinates and state of the target object with the corresponding position in the blueprint based on the three-dimensional coordinates of the marker and the position information of the origin in the blueprint. Furthermore, the creation device 12 creates a report blueprint showing the state at the corresponding position in the blueprint based on the association of the three-dimensional coordinates and state of the target object with the corresponding position in the blueprint. With this configuration, the creation device 12 associates the three-dimensional coordinates and state of the target object with the blueprint, which is a cumbersome task in creating a report blueprint, and therefore can reduce and simplify the creation of the report blueprint.

[0136] In addition, the creation device 12 of this embodiment reads the association of the three-dimensional coordinates and state of the target object with the corresponding position on the design drawing from the measurement device 11, together with the three-dimensional coordinates and state of the target object and the design drawing, and creates a report design drawing showing the state at the corresponding position on the design drawing based on the association. With this configuration, the creation device 12 automatically creates the report design drawing, which can reduce and simplify the creation of the report design drawing.

[0137] Moreover, the creation device 12 of this embodiment acquires an image of a target object, and creates a report image in which a mark MK for identifying the target object whose state has been input is drawn at a corresponding position in the image of the target object. With this configuration, the creation device 12 automatically creates a report image, which can reduce the labor required for creating the report image and simplify the process.

[0138] In addition, the creation device 12 of this embodiment associates the description of the state of the target object in the report design drawing with the drawing of the mark MK in the report image for the same three-dimensional coordinates. With this configuration, when checking the report design drawing using a terminal device, for example, when it is desired to check the state of the target object described in characters in the report design drawing with a photograph, the creation device 12 can easily display the image 31 of the target object.

[0139] In the creation device 12 of this embodiment, the report design drawing includes multiple types of design drawings, and the descriptions of the state of the target object at the same three-dimensional coordinates in each of the multiple types of design drawings are associated with each other among the multiple types of design drawings. With this configuration, when checking the report design drawing using a terminal device, for example, when it is desired to check the state of the target object described in the report plan drawing using the elevation drawing ED, the creation device 12 can easily display the elevation drawing ED including the description of the state of the target object.

[0140] Although the present invention has been described based on the drawings and embodiments, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present invention.

[0141] For example, a general-purpose measuring device having a first camera 13, a distance measuring sensor 14, a direction designation unit 16, and an input unit 15 can be configured to function as the measuring device 11 of the above-mentioned embodiment. Specifically, a program describing the processing contents for realizing each function of the measuring device 11 of the above-mentioned embodiment is stored in a memory of the general-purpose measuring device, and the program is read and executed by a processor of the measuring device. Therefore, the invention according to this embodiment can also be realized as a program executable by a processor. The program may be downloaded via communication, or may be stored in a portable, non-transient recording / storage medium readable by an electronic device, and read from the medium by the measuring device.

[0142] In this embodiment, at least a part of the functions of the creation device 12 may be performed by the measurement device 11. For example, the measurement device 11 may be configured to be capable of performing all the functions of the creation device 12 and to create a design drawing for report. [Explanation of symbols]

[0143] 10. Test result creation system 11 Measuring equipment 12 Creation Device 13 First Camera 14 Distance sensor 15 Input section 16 Direction designation section 17 Memory 18 Display 19 First Controller 20 First Interface 21 Gravity direction sensor 22 Tilt sensor 23 Displacement Sensor 24 List of state options 25 List of object options 26 Image captured by the first camera 27 Marker Statue 28 Second Interface 29 Memory 30 Second Controller 31 Images in Reporting Images CA attention area CM: A symbol indicating that a report image is being taken. ED Elevation PD Plan View ST condition ST1 Multiple target object state ST2 Single target object state

Claims

1. a first camera for capturing an image of an object; A distance measuring sensor that detects the distance of an object within a detection range in each direction; a direction designation unit that detects any one direction within the detection range as a designated direction; an input unit for detecting an input of a state of an object present in the specified direction; a first controller that calculates three-dimensional coordinates of a target object based on the distance measuring sensor based on a distance to the target object for which a state input has been detected by the input unit, and that, when a predetermined marker image is included in an image captured by the first camera, stores raw information used in calculating the three-dimensional coordinates of a marker corresponding to the marker image based on the distance measuring sensor in combination with the three-dimensional coordinates and the state of the target object; a first interface capable of reading a design drawing of a building together with position information on the design drawing of at least one origin; The first controller calculates three-dimensional coordinates of the marker based on the raw information, and associates the three-dimensional coordinates and a state of the target object with a corresponding position on the design drawing based on the three-dimensional coordinates and position information on the design drawing. Measuring equipment.

2. 2. The measuring device according to claim 1, A gravity direction sensor for detecting a direction of gravity is further provided, The first controller associates the three-dimensional coordinates and the state of the target object with a position corresponding to the blueprint based on the direction of gravity as well. Measuring equipment.

3. 3. The measuring device according to claim 1 or 2, The design drawings include at least one of a floor plan, an elevation, and a 3D model of the building. Measuring equipment.

4. The measuring device according to any one of claims 1 to 3, The present invention further includes a second camera, which is the same as or separate from the first camera and has an imaging range that includes at least a part of the detection range. Measuring equipment.

5. The measuring device according to claim 4, The first controller associates the three-dimensional coordinates and state of the target object with a corresponding position in an image captured by the second camera. Measuring equipment.

6. The measuring device according to claim 4 or 5, The input unit includes the second camera and the first controller that detects an input of a state of the target object based on a gesture detected in an image captured by the second camera. Measuring equipment.

7. 7. The measuring device according to claim 4, further comprising: The display further includes a display that displays an attention area detected in an image captured by the second camera. Measuring equipment.

8. The measuring device according to any one of claims 4 to 7, When the second camera detects an approach to a caution area in an image captured by the second camera, the second camera outputs a warning. Measuring equipment.

9. The measuring device according to any one of claims 4 to 8, The direction designation unit is a gaze sensor for detecting a gaze direction of a user of the measuring device; A combination of a percussion rod and a third camera that is the same as or separate from the second camera, includes at least a part of the detection range within an imaging range, and detects the position of the percussion rod in the captured image as a direction; and a third camera that is the same as or different from the second camera, includes at least a part of the detection range within an imaging range, and detects a specific direction of the imaging range. Measuring equipment.

10. 10. The measuring device according to claim 1 , The first controller stores the original information for at least a part of a plurality of floors in the entire building in combination with three-dimensional coordinates and a state of the target object. Measuring equipment.

11. The measuring device according to any one of claims 1 to 10, The raw information is the distance of the marker detected by the distance measuring sensor and the direction in which the marker is located. Measuring equipment.

12. The measuring device according to any one of claims 1 to 10, The raw information includes the distance to an object detected by the distance measuring sensor when capturing an image including the marker image, the direction in which the object is located, the relative position and relative orientation of the distance measuring sensor with respect to the first camera, and the position and size of the marker image in the image. Measuring equipment.

13. 13. The measuring device according to claim 1 , A tilt sensor for detecting a tilt of the measuring device is further provided. The first controller stores a combination of a first tilt detected by the tilt sensor when the distance measuring sensor detects the distance to the target object and the three-dimensional coordinates of the target object, and includes a second tilt detected by the tilt sensor when the first camera generates an image including the marker image in the original information. Measuring equipment.

14. 14. The measuring device according to claim 1 , A displacement sensor for detecting a displacement amount of the measuring device is further provided. The first controller stores a combination of a displacement amount from a reference position detected by the displacement sensor when the distance measuring sensor detects the distance to the target object and a three-dimensional coordinate of the target object, and includes the displacement amount from the reference position detected by the displacement sensor when the first camera generates an image including the marker image in the original information. Measuring equipment.

15. 2. The measuring device according to claim 1, The raw information and the combination of the three-dimensional coordinates and state of the target object may be transmitted to an external server. Measuring equipment.

16. a second interface capable of reading, from the measurement device according to claim 1, association of the three-dimensional coordinates and state of the target object with the corresponding position on the design drawing, together with the three-dimensional coordinates and state, and the design drawing; and a second controller that creates a report design drawing showing the state at a corresponding position in the design drawing based on the association. Creation device.

17. 17. The creation device according to claim 16, The second interface acquires an image of the target object; The second controller creates a report image in which a mark for identifying the target object for which the state has been input is drawn at a corresponding position in an image of the target object. Creation device.

18. 18. The creation device according to claim 17, The second controller correlates the description of the state of the target object in the report design drawing with the drawing of the mark in the report image of the target object having the same three-dimensional coordinates. Creation device.

19. 19. The creation device according to any one of claims 16 to 18, The report blueprint includes a plurality of types of blueprints; The second controller associates descriptions of states of the target object at the same three-dimensional coordinates in each of the plurality of types of design drawings with each other among the plurality of types of design drawings. Creation device.

20. 20. The creation device according to any one of claims 16 to 19, The second controller creates a summary table regarding the state of the target object. Creation device.

21. 21. The creation device according to any one of claims 16 to 20, The report design drawing describes the states of the target objects separately for the plurality of target objects at different positions. Creation device.

22. 22. The creation device according to any one of claims 16 to 21, In the case where the report design drawing includes a plurality of target objects at different positions within a range of a predetermined size in the design drawing, the states of the plurality of target objects are summarized and described. Creation device.

23. The measuring device according to claim 1 ; a creation device having a second interface capable of reading, from the measurement device, association of the three-dimensional coordinates and the state of the target object with the corresponding position on the design drawing, together with the three-dimensional coordinates and the state and the design drawing, and a second controller for creating a report design drawing showing the state at the corresponding position on the design drawing based on the association. Test result creation system.

24. a computer of a measuring device including a first camera that captures an object and generates an image, a distance measuring sensor that detects the distance of an object within a detection range by direction, a direction designation unit that detects an arbitrary direction within the detection range as a designated direction, an input unit that detects an input of a state of an object existing in the designated direction, and a first interface that can read a designation of a building together with position information on at least one origin on the designation, calculating three-dimensional coordinates of a target object based on the distance to the target object for which a state input is detected by the input unit; and when a predetermined marker image is included in the image captured by the first camera, raw information used for calculating three-dimensional coordinates of a marker corresponding to the marker image with respect to the distance measuring sensor as a reference is stored in combination with the three-dimensional coordinates and a state of the target object; Calculating three-dimensional coordinates of the marker based on the raw information; and associating the three-dimensional coordinates and a state of the target object with a corresponding position on the design drawing based on the three-dimensional coordinates and position information on the design drawing. program.

25. The present invention relates to a method for detecting a state of an object by imaging an object, the method comprising: a first camera for capturing an image of an object; a distance measuring sensor for detecting the distance of an object within a detection range by direction; a direction designation unit for detecting any one direction within the detection range as a designated direction; an input unit for detecting an input of a state of an object existing in the designated direction; a first interface capable of reading a design drawing of a building together with position information of at least one origin in the design drawing; and a method for calculating three-dimensional coordinates of the object based on the distance measuring sensor based on the distance to the object whose state input was detected in the input unit, and a method for detecting a state of the object when a predetermined marker image is included in an image captured by the first camera. a first controller that combines and stores original information used to calculate the three-dimensional coordinates of a marker corresponding to the marker image based on the distance measuring sensor when the marker image is not captured, calculates the three-dimensional coordinates of the marker based on the original information, and associates the three-dimensional coordinates and state of the target object with the corresponding position on the design drawing based on the three-dimensional coordinates and position information on the design drawing, reading the association of the three-dimensional coordinates and state of the target object with the corresponding position on the design drawing together with the three-dimensional coordinates and the state, and the design drawing from a measurement device having the above-mentioned measurement device; and generating a report design drawing showing the state at a corresponding position in the design drawing based on the association. program.

Images