Fault storage device, fault storage method, and fault storage program

Abstract

The defect memory device allows a user to input a defect on an image captured by a camera, and then automatically input the defect on the drawing of the object.In this case, instead of using only image recognition technology, sensors such as LiDAR are used to grasp the target structure, making it possible to remove noise and accurately grasp the structure, and this aims to input and store the position of the defect on the drawing with high accuracy. [Solution] The defect memory device is a device that stores defect locations in a structure, and has a defect input unit that accepts input of defect locations in an image acquired from a camera, and a mapping unit that estimates where the defect locations are located on a pre-selected drawing.

Description

[Technical field]

[0001] The present disclosure relates to a defect storage device, a defect storage method, and a defect storage program. [Background technology]

[0002] In the fields of architecture and civil engineering, when handling buildings and structures, defects such as cracks in concrete and peeling paint film may occur. In both new construction and repair fields, defects that exist after construction is completed must be repaired to within a specified range. Previously, these defects were all checked manually, and defective areas were noted on drawings, etc., and it was common to check that they had been repaired upon completion. However, with the advancement of information technology, it is expected that this series of tasks will be assisted by computers, reducing the burden on workers as much as possible and improving work efficiency.

[0003] For example, Patent Document 1 discloses a technique for detecting cracks from image data obtained by photographing the surface of a structure. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2024-80934 A Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, machine learning is used to detect cracks from image data, making it possible to detect defects without human intervention. This reduces the burden on workers. On the other hand, going further, if it becomes possible to automatically determine where defects such as cracks are located on the drawing when they are detected, it will be possible to further reduce the burden on workers and increase work efficiency.

[0006] The present disclosure has been proposed in consideration of the above-mentioned problems, and aims to provide a defect memory device that, when a defect in a structure is detected, captures the shape of the structure around the defect and maps where on the drawing the defect is located. [Means for solving the problem]

[0007] In order to achieve the above objective, the defect memory device of the present disclosure has a defect input unit that accepts input of a location where a defect is present in an image acquired from a camera, and a mapping unit that estimates where the location of the defect is located on a pre-selected drawing.

[0008] In order to achieve the above-mentioned objective, the defect storage method disclosed herein includes a defect input step of accepting input of an area where a defect is present in an image acquired from a camera, and a mapping step of estimating where the area where the defect is located on a preselected drawing.

[0009] In order to achieve the above objective, the defect memory program of the present disclosure includes a defect input step for accepting input of an area where a defect is present in an image acquired from a camera, and a mapping step for estimating where the area where the defect is located on a preselected drawing. Effect of the Invention

[0010] According to the present disclosure, by using a sensor that uses reflected laser light to capture the shape of a structure around a defect, it is possible to grasp the shape of the structure more accurately than is possible using image recognition, and by accurately matching it with the drawing, it is possible to accurately map where the defect is located on the drawing. [Brief description of the drawings]

[0011] [Figure 1] FIG. 2 is a diagram showing a scene in which the defect storage device 10 is used. [Diagram 2] FIG. 2 is a diagram showing a hardware configuration of a defect storage device 10. [Diagram 3]FIG. 2 is a block diagram showing basic functions of a defect storage device 10. [Figure 4] FIG. 2 is a diagram showing an elevation view as an example of a drawing to be read by the defect storage device 10. [Diagram 5] 4 is a diagram showing an example of image data acquired by the defect storage device 10 from a camera. FIG. [Figure 6] FIG. 13 is a diagram showing an example of a defective portion extracted by the defect storage device 10. [Figure 7] 3 is a diagram showing an example of point cloud data acquired by the defect storage device 10. FIG. [Figure 8] 10 is a flowchart showing an example of a control process performed by the defect storage device 10. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present disclosure will now be described with reference to the drawings.

[0013] (Scenes in which the defect storage device 10 is used) 1 is a diagram showing a scene in which the defect storage device 10 is used, and shows a scene in which defects such as cracks in a structure are stored. The structure 20 is shown as an example, but it may be a structure such as a building, or a structure such as a bridge. It may also be a movable property (workpiece) such as a machine, and any building, structure, or movable property (workpiece) for which a drawing exists may be a target.

[0014] The defect storage device 10 is directed toward a structure 20 that is to be searched for in the drawings, and obtains an image in the direction of the structure 20 as input data using a sensor that detects reflected laser light and a camera or the like.

[0015] (Hardware configuration of the defect storage device 10) 2 is a diagram showing a hardware configuration of the defect storage device 10. The defect storage device 10 can be configured by a general-purpose computer.

[0016] As shown in FIG. 2, the defect storage device 10 includes a processor 11, a memory 12, a storage 13, a communication IF 14 (note that IF is written as an abbreviation of Interface. The same applies below), and an input / output IF 15.

[0017] The processor 11 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, peripheral circuits, and the like.

[0018] The memory 12 is for temporarily storing programs and data to be processed by the programs, and is realized by a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0019] The storage 13 is a storage device for saving data such as programs, and is realized by, for example, a flash memory, an SSD (Solid State Drive), or an HDD (Hard Disk Drive).

[0020] The communication IF 14 is an interface for transmitting and receiving signals so that the defect storage device 10 can communicate with other devices. For example, when drawing data is stored in a database server on a network, the defect storage device 10 may obtain the drawing data via the communication IF 14. However, for example, the drawing data can also be stored in the storage 13, and the communication IF 14 is not an essential component of the defect storage device 10.

[0021] The input / output IF 15 functions as an interface for an image input device such as a camera, a sensor for detecting reflected laser light, input devices such as a keyboard, mouse, and numeric keypad for receiving input from a user, and an output device such as a display for presenting information to a user. The defect storage device 10 obtains image data 32 from a camera or the like, and obtains data from a sensor for detecting reflected laser light, so the input / output IF 15 is connected to these devices to transmit and receive data.

[0022] 3 is a block diagram for explaining basic functions of the defect storage device 10. The defect storage device 10 includes a communication unit 110, a storage unit 120, and a control unit .

[0023] The communication unit 110 performs processing for communicating with other devices. For example, when a database exists outside via a network, the communication unit 110 may acquire data such as drawings via the communication unit 110. Note that the communication unit 110 is not an essential component.

[0024] The storage unit 120 stores data held by the defect storage device 10. For example, the storage unit 120 includes a drawing database 121 and stores drawings to be searched. The storage unit 120 may also store a program to be executed by the defect storage device 10.

[0025] The control unit 130 controls the operation of the defect storage device 10. Specifically, the control unit 130 includes a drawing selection unit 131, a defect input unit 132, an extraction unit 133, a distance estimation unit 134, a mapping unit 135, and a compilation unit 136.

[0026] The drawing selection unit 131 selects a drawing of the structure 20 that is the target of defect detection.

[0027] The plan selection unit 131 may select a plan of the structure 20 by acquiring information on the plan selected by the user via the input / output IF 15, for example.

[0028] For example, the plan selection unit 131 may automatically select a plan of the structure 20. For example, the plan may include location information, and the plan may be selected using location information from a Global Positioning System (GPS).

[0029] The plan selection unit 131 may be configured to, for example, acquire an image of the structure 20 and then automatically select a plan that matches the image. This may be achieved by using existing technology such as machine learning.

[0030] The plan selection unit 131 may be configured to, for example, acquire the shape of the structure 20 using a sensor and then automatically select a plan that matches the shape.

[0031] The defect input unit 132 receives and acquires input of a location where a defect exists in an image acquired by a camera.

[0032] The defect input unit 132 acquires, for example, a portion of an image acquired by a camera that is input by a user as a defect. The user may input the portion of a defect into the image using a pointing device such as a mouse, a touch pad, or a touch panel, and the defect input unit 132 may acquire the portion (position) of the defect.

[0033] The defect input unit 132 may acquire the location of the defect after the user inputs the location of the defect by eye-gaze input, for example.

[0034] The extraction unit 133 extracts the range of the defect from the defect location acquired by the defect input unit 132. For example, the extraction unit 133 uses machine learning to extract the range of the defect from the defect location acquired by the defect input unit 132. The defect corresponds to a crack in concrete or peeling of a paint film, and generally, the defect is not a point but has a certain range. The extraction unit 133 may extract the range of the defect from image data including the defect and the surrounding image data of the defect, using a machine learning model that has learned the range of the defect in advance using image data, etc.

[0035] The extraction unit 133 may extract the range of the defect by using not only machine learning but also a method generally used in the field of cognitive engineering.

[0036] The extraction unit 133 may function in conjunction with the defect input unit 132, and when a user inputs the range of a defect using a pointing device or the like, the defect input unit 132 may acquire the location of the defect and the extraction unit 133 may acquire the range of the defect.

[0037] The distance estimation unit 134 extracts a plurality of feature points from the structure 20 and estimates the distance between the plurality of feature points. For example, the distance estimation unit 134 uses LiDAR (Light Detection And Ranging) to irradiate a laser in the direction of a defect in the structure 20 and detect the reflected light with a sensor, thereby acquiring point cloud data.

[0038] The distance estimation unit 134 performs processing to detect the contour and shape of an object by enhancing the contour from the point cloud data using image processing technology or the like. Then, feature points are extracted from the shape of the object. When an object has a contour, feature points are points that are the end points of each straight line or curve that constitutes the contour. In other words, feature points are points where multiple straight lines intersect, points where a straight line and a curve intersect, and points where a curve intersect with another curve. These include, for example, points that form corners where a straight line or curve intersects with another straight line or curve.

[0039] After extracting the feature points as described above, the distance estimation unit 134 measures the distance and angle to the extracted feature points using a sensor that detects reflected light using LiDAR as a base point.

[0040] The distance estimation unit 134 measures the distance and angle between the base point and each feature point using LiDAR, and then calculates the distance between the feature points using that information.

[0041] When extracting the distances between feature points, the distance estimation unit 134 does not need to calculate the distances between all combinations of feature points, but only needs to calculate the distances between adjacent feature points along the contour (outer shape), in other words, the distances of the straight lines or curves that make up the contour (outer shape).

[0042] The distance estimation unit 134 is expected to achieve higher accuracy by extracting feature points and contours of an object using an optical sensor such as LiDAR, compared to extracting contours using image recognition technology.

[0043] First, the mapping unit 135 reads the drawing selected by the drawing selection unit 131 from the drawing database 121. At this time, the distance between the feature points may be stored as a numerical value in advance in the drawing data, and the distance between the feature points may be acquired at the same time. When the distance between the feature points is not stored as a numerical value in the drawing data, the distance may be calculated from the distance between the feature points drawn on the drawing, for example, by the following method.

[0044] The feature points grasped by the mapping unit 135 from the drawing data are points that can be grasped as the end points of the straight lines or curved lines constituting the drawing, i.e., points at which a plurality of straight lines intersect to form corners, points at which straight lines and curved lines intersect, and points at which curved lines intersect are grasped as feature points.

[0045] The mapping unit 135 selects any two adjacent feature points for one drawing read from the drawing database 121, and calculates the distance on the drawing. In other words, the mapping unit 135 calculates the distance of the straight lines or curves that make up the drawing. Then, for the read drawing, the mapping unit 135 calculates the lengths of all the straight lines and curves that make up the drawing.

[0046] Secondly, the mapping unit 135 selects any one straight line or curve from the loaded drawing, and calculates the ratio between the selected straight line or curve and the adjacent straight line or curve.

[0047] Thirdly, the mapping unit 135 performs alignment by calibration of a space consisting of feature points generated based on the image data acquired from the camera and the point cloud data read from the sensor, and maps the location of the defect onto the space consisting of feature points (on the point cloud data read from the sensor). At this time, the mapping unit 135 may calculate the scale ratio between the image data acquired from the camera and the image data read from the sensor.

[0048] Fourthly, the mapping unit 135 determines whether or not the ratio of adjacent straight lines or curves selected from the drawing matches the ratio of two adjacent straight lines or curves formed by the feature points calculated by the distance estimation unit 134.

[0049] When the ratio of adjacent lines or curves selected from the drawing matches the ratio of two adjacent lines or curves calculated by distance estimation unit 134, mapping unit 135 further determines whether the ratio of adjacent lines or curves in the drawing matches the ratio of further adjacent lines or curves calculated by distance estimation unit 134, continues the search as long as there is a match, and finally calculates the number of matches. At this time, when determining whether the ratio of adjacent lines or curves selected from the drawing matches the ratio of two adjacent lines or curves calculated by distance estimation unit 134, a threshold may be set and an error within the threshold may be allowed.

[0050] When the ratio of adjacent straight lines or curves selected from the drawing does not match the ratio of two adjacent straight lines or curves formed by the feature points calculated by the distance estimation unit 134, the mapping unit 135 shifts any straight line or curve that starts the search on the drawing and sets it to a straight line or curve, and then performs a similar search.

[0051] The mapping unit 135 changes the line or curve at which the search starts on the drawing, and determines whether it matches the ratio of the line or curve formed by the feature points calculated by the distance estimation unit 134, and the value with the highest number of matches is determined as the matching degree of that starting point.

[0052] Fifthly, the mapping unit 135 regards the straight line or curve constituting the line with the highest degree of matching in the selected drawing as representing the configuration of the drawing around the defect inputted by the defect input unit 132. Then, using the coincidence between the distance ratio of the straight line or curve constituting the structure on the drawing and the distance ratio of the straight line or curve estimated by the distance estimation unit 134, the mapping unit 135 matches the shape around the defect acquired by the sensor with the drawing. That is, the mapping unit 135 compares the distance ratio between the feature points calculated by the distance estimation unit 134 with the distance ratio of the straight line or curve constituting the object on the selected drawing, and matches the part with the most suitable straight line or curve as the part captured by the sensor on the drawing. Since the mapping unit 135 aligns the image data with the point cloud data in the third process, the part with the defect can be grasped on the drawing.

[0053] The mapping unit 135 may calculate a scale ratio from the ratio between a straight line or curve on the drawing and the straight line or curve estimated by the distance estimation unit 134. That is, since the scale ratio is stored in the drawing, the scale ratio of the straight line or curve estimated by the distance estimation unit 134 may be calculated from the scale ratio of the drawing using the ratio between a straight line or curve on the drawing and the straight line or curve estimated by the distance estimation unit 134.

[0054] The mapping unit 135 aligns the location of the defect with the drawing in an area based on the point cloud data acquired by the sensor, and can perform matching while eliminating information that becomes noise in the image data, such as background and color. In addition, since the matching of the image data and the point cloud data acquires data of almost the same area, the alignment can be performed accurately.

[0055] The counting unit 136 may store data obtained by trimming images of the defective portions extracted by the extraction unit 133 in the defect database 122. Also, the counting unit 136 may count the number of defective portions and store the total number.

[0056] The counting unit 136 may calculate the area of ​​the defect from the data obtained by trimming the image of the portion having the defect extracted by the extraction unit 133, and store the area of ​​the defect in the defect database 122.

[0057] To calculate the area of ​​the location where the defect extracted by extraction unit 133 is present, counting unit 136 calculates the area of ​​the location where the defect extracted by extraction unit 133 is present on the image input from the camera. Then, if counting unit 136 can calculate the ratio between the scale on the image input from the camera and the scale on the drawing, it can calculate the actual area of ​​the location where the defect is present.

[0058] The counting unit 136 calculates the scale of the image data acquired from the camera from the scale of the drawing, using the ratio of the scale of the image data acquired from the camera calculated in the mapping unit 135 to the scale of the point cloud data read from the sensor, and the ratio of the straight line or curve on the drawing calculated in the mapping unit 135 to the straight line or curve estimated in the distance estimation unit 134. In this way, the counting unit 136 calculates the area of ​​the location where a defect exists in the image input from the camera.

[0059] At construction sites, manual tallying up the locations (positions) of defects, their number, and their areas is often required, so if the tallying unit 136 can perform the tallying automatically, it is possible to reduce the burden of manual work.

[0060] (Specific example of processing performed by control unit 130) 4 shows an elevation 31 of a certain structure 20. The drawing selection unit 131 selects a drawing of the structure 20 for which defects are to be stored, and selects, for example, a drawing as shown in FIG.

[0061] 5 is a diagram showing an example of an image acquired from a camera by the defect storage device 10. The defect input unit 132 inputs a defective portion (defect 323 in FIG. 5) to image data 32 as shown in FIG.

[0062] 6 is a diagram showing an example in which the extraction unit 133 extracts a defective portion 331 from a defect 323 in the image data 32. The extraction unit 133 extracts the defective portion 331 by using a frame such as a rectangle for the defect in the image data.

[0063] Fig. 7 shows an example of acquiring point cloud data by using a technology such as LiDAR (Light Detection And Ranging) to irradiate a laser in the direction of a defect in a structure 20 and detect the reflected light with a sensor. For example, when LiDAR is used on a structure as shown in Fig. 4, Fig. 7 shows the detection of the periphery of the frame 314. In reality, 341-342-344-343, 345-346-348-347 are detected as a point cloud, but when the end points of the straight lines or curves are extracted, they become 341, 342, 343, 344, 345, 346, 347, and 348.

[0064] Distance estimation unit 134 extracts end points from the point cloud data and calculates the distance between adjacent end points, as shown in Fig. 7. In the example of Fig. 7, distance estimation unit 134 calculates the distances of 341-342, 342-344, 344-343, 343-341, 345-346, 346-348, 348-347, and 347-345.

[0065] The mapping unit 135 estimates and maps the position on the elevation 31 of the defect 323 in the image data 32. The mapping unit 135 reads a drawing such as the elevation 31 selected by the drawing selection unit 131, obtains or calculates distance information of the straight lines or curves constituting the features such as 312 to 316, and calculates their ratios.

[0066] The mapping unit 135 calibrates the image data 32 acquired from the camera and the point cloud data 34 acquired from the LiDAR to grasp the correspondence relationship. At this time, the ratio of the image data 32 to the point cloud data 34 (for example, image data 32:point cloud data 34=1:R1) is calculated.

[0067] The mapping unit 135 detects that the ratio of the sides of 341-342-344-343 in the point cloud data 34 is similar to the ratio of the sides of 314 in the elevation 31, but does not match those of 312-313 and 315-316. Also, the mapping unit 135 detects that the ratio of the sides of 345-346-348-347 is similar to the ratio of the sides of 316 in the elevation 31, but does not match those of 312-315. From this, it is understood that the point cloud data 34 captures the areas around 314 and 316 on the elevation 31, and mapping of the point cloud data 34 and the elevation 31 is performed.

[0068] By the above processing, the image data 32 and the point cloud data 34 are calibrated to allow the position of the defect 323 to be grasped on the point cloud data 34, and the point cloud data 34 and the elevation 31 are mapped to allow the position of the defect 323 to be grasped. This makes it possible to grasp the position of the defect 323 on the image data 32 on the elevation 31.

[0069] The mapping unit 135 calculates the ratio of the point cloud data 34 to the elevation data 31 (for example, the point cloud data 34: the elevation data 31 = 1:R2) from the distance of the point cloud data 34 and the distance of the elevation data 31. As a result, the ratio of the image data 32: the elevation data 31 becomes 1:R1R2.

[0070] The counting unit 136 may count the number of defective locations extracted by the extraction unit 133, and may perform a count of the total number of defective locations in the structure 20, etc.

[0071] The counting unit 136 may calculate the area of ​​the defect. For example, the area of ​​the defect 323 in the image data 32 is calculated on the image data, and the actual area can be calculated by multiplying it by the square of the ratio R1R2 (because it is an area).

[0072] By carrying out the above-described processing, it is possible to incorporate information about defects in the image data captured by the camera onto the drawing.

[0073] In addition, when attempting to match (map) the location of defects between image data captured by a camera and the drawing, there is a risk of a loss of accuracy because the image data contains a lot of information that can become noise. On the other hand, by mapping the shape of the object based on point cloud data acquired by sensors such as LiDAR with the drawing, it is possible to map the location of the detected defect on the drawing with greater accuracy.

[0074] (Processing flow) Hereinafter, an example of a control process in which the defect storage device 10 executes defect detection will be described with reference to FIG.

[0075] The control unit 130 of the defect storage device 10 selects a drawing of the structure 20 for which defects are to be stored based on a user input or by using information from a camera or a sensor (step S101).

[0076] The control unit 130 of the defect storage device 10 acquires the location of the defect on the image data input from the camera using a pointing device, eye gaze input, etc. Also, the location of the defect may be acquired automatically using machine learning, etc. (Step S102).

[0077] The control unit 130 of the defect storage device 10 extracts the range of the defect. For example, examples of defects in a structure include cracks in concrete and peeling of a paint film, and the range of such defects is extracted (step S103).

[0078] The control unit 130 of the defect storage device 10 estimates the distance between adjacent feature points (points that are the end points of a line or a curve) from point cloud data acquired using LiDAR or the like (step S104).

[0079] The control unit 130 of the defect memory device 10 calibrates the drawing data and the point cloud data, and further matches the point cloud data with the drawing data using the degree of agreement between the ratio of distances between feature points of the point cloud data and the ratio of distances between feature points of the drawing data as an index, and maps defects on the image data onto the drawing data (step S105).

[0080] The control unit 130 of the defect storage device 10 calculates and tallies the total number and area of ​​the defect locations (step S106).

[0081] (Explanation of effect) The configuration of the defect storage device 10 has been described above, but by using data acquired using LiDAR or the like, it is possible to project the position of defects on image data onto a drawing. If one were to simply map from image data to drawing data, it would be difficult to perform accurate mapping due to the effects of noise and distance distortion, so the method disclosed herein makes it possible to improve accuracy.

[0082] Although the preferred embodiment of the present disclosure has been described above, the present disclosure is not limited to such a specific embodiment, and the present disclosure includes the disclosure described in the claims and its equivalents. In addition, the configurations of the devices described in the above embodiments and modifications can be appropriately combined as long as no technical contradiction occurs. [Explanation of symbols]

[0083] 10...defect storage device, 11...processor, 12...memory, 13...storage, 14...communication IF, 15...input / output IF, 20...structure, 30...input device, 31...elevation, 32-33...image data, 34...point cloud data, 110...communication unit, 120...storage unit, 121...drawing database, 130...control unit, 131...measurement unit, 132...defect input unit, 133...extraction unit, 134...distance estimation unit, 135...mapping unit, 136...counting unit, 311-316...frame, 321-322...frame, 323...defect, 331...defective area, 341-348...feature point

Claims

1. A device for storing locations of defects in a structure, a defect input unit that receives an input of a location where a defect is present in an image acquired from the camera; a distance estimation unit that acquires reflected light of a laser directed toward a location of the structure where the defect is present from a sensor, extracts a plurality of feature points, and calculates the distances between the feature points; a mapping unit that aligns the image acquired from the camera with a space consisting of the feature points by calibration, maps the location where the defect is located onto a section consisting of the feature points, determines whether the location where the defect is located on a preselected drawing using a ratio of distances between straight lines or curves on the preselected drawing and a ratio of distances between the feature points calculated by the distance estimation unit, and estimates the location where the defect is located on the drawing by matching the shape of the area around the location where the defect is located with the drawing.

2. A device for storing locations of defects in a structure, a defect input unit that receives an input of a location where a defect is present in an image acquired from the camera; a distance estimation unit that acquires, from a sensor, reflected light of a laser directed toward a location of the structure where the defect is present, extracts an outline of the structure, extracts a plurality of points that are end points of straight lines or curves as feature points, and calculates the distances between each of the feature points; a mapping unit that aligns the image acquired from the camera with a space consisting of the feature points by calibration, maps the location where the defect is located onto a section consisting of the feature points, determines whether the location where the defect is located on a preselected drawing using a ratio of distances between straight lines or curves on the preselected drawing and a ratio of distances between the feature points calculated by the distance estimation unit, and estimates the location where the defect is located on the drawing by matching the shape of the area around the location where the defect is located with the drawing.

3. A device for storing locations of defects in a structure, a defect input unit that receives an input of a location where a defect is present in an image acquired from the camera; an extraction unit that extracts an area where the defect exists from the image acquired by the camera; a distance estimation unit that acquires reflected light of a laser directed toward a location of the structure where the defect is present from a sensor, extracts a plurality of feature points, and calculates the distances between the feature points; a mapping unit that aligns the image acquired from the camera with a space consisting of the feature points by calibration, maps the location where the defect is located onto a section consisting of the feature points, determines whether the location where the defect is located on a preselected drawing using a ratio of distances between straight lines or curves on the preselected drawing and a ratio of distances between the feature points calculated by the distance estimation unit, and estimates the location where the defect is located on the drawing by matching the shape of the area around the location where the defect is located with the drawing.

4. A device for storing locations of defects in a structure, a defect input unit that receives an input of a location where a defect is present in an image acquired from the camera; an extraction unit that extracts an area where the defect exists from the image acquired by the camera using a model that has been trained in advance by machine learning; a distance estimation unit that acquires reflected light of a laser directed toward a location of the structure where the defect is present using a sensor to extract a plurality of feature points and calculates the distances between the feature points; a mapping unit that aligns the image acquired from the camera with a space consisting of the feature points by calibration, maps the location where the defect is located onto a section consisting of the feature points, determines whether the location where the defect is located on a preselected drawing using a ratio of distances between straight lines or curves on the preselected drawing and a ratio of distances between the feature points calculated by the distance estimation unit, and estimates the location where the defect is located on the drawing by matching the shape of the area around the location where the defect is located with the drawing.

5. A device for storing locations of defects in a structure, a defect input unit that receives an input of a location where a defect is present in an image acquired from the camera; an extraction unit that extracts the range of the defect and the type of the defect from the image acquired by the camera; a distance estimation unit that acquires reflected light of a laser directed toward a location of the structure where the defect is present using a sensor to extract a plurality of feature points and calculates the distances between the feature points; a mapping unit that aligns the image acquired from the camera with a space consisting of the feature points by calibration, maps the location where the defect is located onto a section consisting of the feature points, determines whether the location where the defect is located on a preselected drawing using a ratio of distances between straight lines or curves on the preselected drawing and a ratio of distances between the feature points calculated by the distance estimation unit, and estimates the location where the defect is located on the drawing by matching the shape of the area around the location where the defect is located with the drawing.

6. 6. The defect storage device according to claim 3, further comprising a counting unit that stores data obtained by trimming images of the defect locations extracted by the extraction unit and a total number of the defect locations.

7. The defect storage device according to any one of claims 3 to 5, further comprising a calculation unit that calculates an area of ​​the location where the defect is present using the range of the location where the defect is present extracted by the extraction unit and the scale of the drawing mapped by the mapping unit.

8. A method for storing locations of defects in a structure, comprising: a defect input step of receiving an input of a location where a defect exists in an image acquired from a camera; a distance estimation step of acquiring, from a sensor, reflected light of a laser directed toward a location of the structure where the defect is present, extracting a plurality of feature points, and calculating distances between the feature points; a mapping step of aligning the image acquired from the camera with a space consisting of the feature points by calibration, mapping the location where the defect is located onto a section consisting of the feature points, determining whether the location where the defect is located on a preselected drawing is consistent with the ratio of the distances of a straight line or curve on the preselected drawing and the ratio of the distances between the feature points calculated by the distance estimation unit, and estimating the location where the defect is located on the drawing by matching the shape of the area around the location where the defect is located with the drawing.

9. A program for storing locations of defects in a structure, a defect input step of receiving an input of a location where a defect exists in an image acquired from a camera; a distance estimation step of acquiring, from a sensor, reflected light of a laser directed toward a location of the structure where the defect is present, extracting a plurality of feature points, and calculating distances between the feature points; and a mapping step of aligning the image acquired from the camera with a space consisting of the feature points by calibration, mapping the location where the defect is located onto a section consisting of the feature points, determining whether the location where the defect is located on a preselected drawing is consistent with the ratio of the distances of straight lines or curves on the preselected drawing and the ratio of the distances between the feature points calculated by the distance estimation unit, and estimating the location where the defect is located on the drawing by matching the shape of the area around the location where the defect is located with the drawing.

Images