Estimation device, estimation method, and program

The estimation device accurately calculates the scale of concrete or asphalt structure images without a reference object by dividing the image into regions, extracting relevant areas, and using two distinct scale estimation methods, addressing the inefficiency of existing methods by enhancing accuracy and eliminating the need for reference objects.

WO2024247236A1PCT designated stage expired Publication Date: 2024-12-05NT T INC
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2023/020546
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-06-01
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Existing methods for estimating the scale of a photographed concrete structure image require the installation of a reference object, such as a crack or marker, which is time-consuming and inefficient, and cannot perform scale estimation on images without such objects.

Method used

An estimation device and method that divides the image into regions, extracts areas with only concrete or asphalt, and uses two distinct scale estimation units to calculate the scale without a reference object, utilizing shadows and unevenness for accurate scale determination.

Benefits of technology

Enables accurate scale estimation in captured images of concrete or asphalt structures without installing a reference object, improving efficiency by determining suitable areas for scale calculation based on the difference between two estimation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023020546_05122024_PF_FP_ABST
    Figure JP2023020546_05122024_PF_FP_ABST
Patent Text Reader

Abstract

An estimation device (100) according to the present disclosure comprises: an image division unit (110) that divides a captured image, which was obtained by imaging a structure, into a plurality of divided regions; an extraction unit (120) that extracts a target region from among the plurality of divided regions; a scale estimation unit (131) that performs scale estimation on the target region; a scale estimation unit (132) that has a feature different from that of the scale estimation unit (131) and performs scale estimation on the target region; an estimation result determination unit (140) that determines whether the target region is a region suitable for scale estimation or a region that is not suitable for scale estimation, on the basis of the estimation results from the scale estimation unit (131) and the scale estimation unit (132); and a scale calculation unit (150) that calculates the scale of each of the plurality of divided regions on the basis of the scale of the region suitable for scale estimation.
Need to check novelty before this filing date? Find Prior Art

Description

Estimation device, estimation method, and program

[0001] The present disclosure relates to an estimation device, an estimation method, and a program.

[0002] In order to improve the efficiency of inspections of concrete structures, there is a demand for technology that uses image recognition to measure the length and area of ​​deformation. In order to measure the length and area of ​​deformation from a photographed image of a concrete structure, it is necessary to understand the scale of the photographed image (the number of pixels in the image per actual unit length, for example, pixels / cm). Non-Patent Document 1 describes a technology that estimates the scale from a photographed image by photographing cracks in the concrete structure together with a ruler (crack scale). Furthermore, Non-Patent Document 2 describes a technology that estimates the scale from a photographed image by photographing the concrete structure with markers of known sizes attached to the concrete structure.

[0003] Takeda et al.: Crack Measurement in Concrete Structures Using Image Processing, Obayashi Corporation Technical Research Institute Bulletin, No. 79, 2015. Jurado et al., “Automatic generation and detection of highly reliable fiducial markers under occlusion,” Pattern Recognition, pp. 2280-2292, 2014.

[0004] The techniques described in Non-Patent Documents 1 and 2 require that a concrete structure be photographed with a reference object (crack scale, marker) installed. This requires time and effort to install the reference object. Furthermore, scale estimation cannot be performed on images photographed without installing a reference object.

[0005] The purpose of the present disclosure, made in consideration of the above-mentioned problems, is to provide an estimation device, an estimation method, and a program that can perform scale estimation with higher accuracy in images of structures without installing a reference object.

[0006] In order to solve the above problem, the estimation device according to the present disclosure is an estimation device that performs scale estimation in a captured image of a structure, and includes an image division unit that divides the captured image into a plurality of divided areas, an extraction unit that extracts a target area from the plurality of divided areas in which only concrete or asphalt is captured, a first scale estimation unit that performs scale estimation in the extracted target area, a second scale estimation unit that has characteristics different from those of the first scale estimation unit and performs scale estimation in the extracted target area, an estimation result determination unit that determines whether the target area is a region that is suitable for scale estimation or a region that is not suitable for scale estimation based on the estimation results by the first scale estimation unit and the second scale estimation unit, and a scale calculation unit that calculates the scale in each of the plurality of divided areas based on the scale in the region that is suitable for scale estimation.

[0007] In addition, in order to solve the above-mentioned problems, the estimation method disclosed herein is an estimation method using an estimation device that performs scale estimation in a captured image of a structure, and includes the steps of dividing the captured image into a plurality of divided areas, extracting a target area from the plurality of divided areas in which only concrete or asphalt is captured, performing a first scale estimation in the extracted target area, performing a second scale estimation in the extracted target area having characteristics different from those of the first scale estimation, determining whether the target area is a region suitable for scale estimation or a region unsuitable for scale estimation based on the estimation results of the first scale estimation and the second scale estimation, and calculating the scale in each of the plurality of divided areas based on the scale in the region suitable for scale estimation.

[0008] In order to solve the above problem, a program according to the present disclosure causes a computer to operate as the above-described estimation device.

[0009] According to the estimation device, estimation method, and program of the present disclosure, scale estimation can be performed with higher accuracy in a captured image of a structure without installing a reference object.

[0010] 10 is a diagram illustrating an example configuration of an estimation device according to a first embodiment of the present disclosure. FIG. 11 is a diagram for explaining division of a captured image by the image division unit shown in FIG. 1. FIG. 12 is a diagram for explaining extraction of a concrete region by the extraction unit shown in FIG. 1. FIG. 13 is a diagram for explaining an example configuration of a scale estimation unit 131 shown in FIG. 1. FIG. 14 is a diagram for explaining an example configuration of a scale estimation unit 132 shown in FIG. 1. FIG. 14 is a diagram for explaining a difference between an estimation result by the scale estimation unit 131 shown in FIG. 1 and an estimation result by the scale estimation unit 132. FIG. 15 is a diagram for explaining an example of calculation of a scale in a captured image by the scale calculation unit shown in FIG. 1. FIG. 16 is a diagram for explaining another example of calculation of a scale in a captured image by the scale calculation unit shown in FIG. 1. A flowchart for an example operation of the estimation device shown in FIG. 1. A flowchart for an example operation of the estimation result determination unit shown in FIG. 1. FIG. 16 is a diagram for explaining an example configuration of an estimation device according to a second embodiment of the present disclosure. A flowchart for an example operation of the exclusion unit shown in FIG. 10. FIG. 17 is a diagram for explaining an example configuration of an estimation device according to a third embodiment of the present disclosure. FIG. 18 is a diagram for explaining division of a captured image by the image division unit shown in FIG. 12. FIG. 19 is a diagram for explaining extraction of a concrete region by the extraction unit shown in FIG. 12. FIG. 19 is a diagram for explaining determination by the estimation result determination unit shown in FIG. 12. A diagram showing a state in which a first pattern and a second pattern are superimposed. 10 is a diagram for explaining selection of a scale estimation unit from a plurality of candidate scale estimation units.FIG. 11 is a diagram illustrating an example configuration of a computer that functions as an estimation apparatus according to the present disclosure.

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] First Embodiment FIG. 1 is a diagram illustrating an example configuration of an estimation device 100 according to a first embodiment of the present disclosure. The estimation device 100 according to this embodiment performs scale estimation in a captured image of a structure made of concrete or asphalt. In the following description, the structure is assumed to be a concrete structure made of asphalt. In addition, in the following description, the estimation device 100 is assumed to estimate the number of pixels per unit length (pixel / cm) in the captured image.

[0013] As shown in FIG. 1 , the estimation device 100 according to this embodiment includes an image dividing unit 110, an extraction unit 120, a scale estimation unit 131 serving as a first scale estimation unit, a scale estimation unit 132 serving as a second scale estimation unit, an estimation result determination unit 140, and a scale calculation unit 150.

[0014] An image of a concrete structure is input to the image segmentation unit 110. The image is an image of the concrete structure captured without the installation of reference objects such as a crack scale or markers. The image segmentation unit 110 segments the input image into a plurality of divided regions, as shown in FIG. 2 .

[0015] Referring again to FIG. 1 , the extraction unit 120 extracts a target area in which only concrete or asphalt is photographed from among the plurality of divided areas obtained by dividing the captured image. As described above, in this embodiment, the structure is a concrete structure made of concrete. In this case, the extraction unit 120 extracts, as the target area, an area in which only concrete is photographed (hereinafter referred to as a concrete area), as shown in FIG. 3 . The extraction unit 120 extracts the concrete area from among the plurality of divided areas, for example, using any image recognition technology. Since there are various methods for such image recognition technology, detailed description thereof will be omitted.

[0016] The scale estimation unit 131 performs scale estimation in the target region extracted by the extraction unit 120 (in this embodiment, the concrete region).

[0017] The scale estimation unit 132 performs scale estimation in the concrete region extracted by the extraction unit 120. The scale estimation unit 132 has different features from the scale estimation unit 131 in terms of scale estimation.

[0018] The scale estimation unit 131 and the scale estimation unit 132 can estimate the scale of a concrete region, for example, using shadows formed by voids or irregularities on the concrete surface of the concrete region. Typically, when the shooting distance to a concrete structure is short, the shadows formed by voids or irregularities on the concrete surface appear larger in the captured image. Furthermore, when the shooting distance to a concrete structure is long, the shadows formed by voids or irregularities on the concrete surface appear smaller in the captured image. The scale estimation unit 131 and the scale estimation unit 132 can estimate the scale of a concrete region, for example, using a learning model trained on pairs of captured images with known shooting distances and true scale values ​​for the captured images as training data. Even when a structure is made of asphalt, the scale estimation unit 131 and the scale estimation unit 132 can estimate the scale using, for example, shadows formed by voids or irregularities on the asphalt surface of an asphalt region extracted as a target region and in which only asphalt is photographed.

[0019] As described above, the scale estimation units 131 and 132 have different features regarding scale estimation. The scale estimation units 131 and 132 can be constructed using, for example, ensemble learning and non-defective learning. The following describes an example in which the scale estimation unit 131 is constructed using ensemble learning and the scale estimation unit 132 is constructed using non-defective learning. However, this example is not limiting, and the scale estimation unit 131 may be constructed using non-defective learning and the scale estimation unit 132 may be constructed using ensemble learning. Furthermore, the scale estimation units 131 and 132 may be constructed using techniques other than ensemble learning and non-defective learning.

[0020] FIG. 4A is a diagram showing an example of the configuration of the scale estimation unit 131 when it is constructed using ensemble learning.

[0021] As shown in FIG. 4A , the scale estimation unit 131 includes multiple weak estimators 131-1 to 131-n. Each of the weak estimators 131-1 to 131-n is an estimator constructed by learning different data sets (pairs of concrete images of concrete and true values ​​of the scale in the concrete images). Each of the weak estimators 131-1 to 131-n receives an image of a concrete region as input, performs scale estimation for the input image, and outputs the estimation result. The scale estimation unit 131 outputs one estimation result based on the estimation results of each of the weak estimators 131-1 to 131-n. For example, the scale estimation unit 131 outputs the average or median of the estimation results of each of the weak estimators 131-1 to 131-n as the estimation result.

[0022] FIG. 4B is a diagram showing an example of the configuration of the scale estimation unit 132 when constructed using non-defective product learning.

[0023] As shown in FIG. 4B , the scale estimation unit 132 includes an estimator 132-1. The estimator 132-1 is constructed by learning only images (good-quality data) suitable for scale estimation, and is capable of more accurate scale estimation than the weak estimators 131-1 to 131-n described above. For example, a concrete image with a known scale is input to the weak estimators 131-1 to 131-n described above, and the estimation results of each of the weak estimators 131-1 to 131-n are compared with the true value of the scale. If the number of weak estimators whose errors between the estimation results and the true value are equal to or less than a predetermined value is equal to or greater than a certain number, the concrete image is determined to be good-quality data. The estimator 132-1 is constructed by learning such good-quality data.

[0024] Among concrete images, there are images that are suitable for scale estimation and images that are not suitable for scale estimation. As shown in Fig. 5, the scale estimation unit 131 and the scale estimation unit 132 are selected so that the difference in the estimation results is small for areas that are suitable for scale estimation and the difference in the estimation results is large for areas that are not suitable for scale estimation. The scale estimation unit 131 and the scale estimation unit 132 having such characteristics can be selected experimentally using, for example, images that are known to be suitable for scale estimation and images that are known to be not suitable for scale estimation.

[0025] Referring again to Figure 1, the estimation result determination unit 140 determines, for each target region (in this embodiment, a concrete region), based on the estimation results by the scale estimation unit 131 and the scale estimation unit 132, whether the concrete region is suitable for scale estimation or not.

[0026] As described with reference to Figure 5, the scale estimation unit 131 and the scale estimation unit 132 have the characteristic that the difference in the estimation results is small for areas suitable for scale estimation, and the difference in the estimation results is large for areas not suitable for scale estimation.

[0027] The estimation result determination unit 140 determines whether a concrete region is suitable for scale estimation or not, based on the difference between the scale estimated by the scale estimation unit 131 and the scale estimated by the scale estimation unit 132 for a target region. For example, if the difference between the scale estimated by the scale estimation unit 131 and the scale estimated by the scale estimation unit 132 for a concrete region is equal to or smaller than a predetermined threshold, the estimation result determination unit 140 determines that the concrete region is suitable for scale estimation. Furthermore, if the difference is greater than the predetermined threshold, the estimation result determination unit 140 determines that the concrete region is not suitable for scale estimation.

[0028] In general, whether a concrete image is suitable for scale estimation or not is difficult to determine in general because it depends on various factors. In this embodiment, by using the difference between the estimation results of the scale estimation unit 131 and the scale estimation unit 132, which have different characteristics, it is determined whether a target area extracted from a captured image is suitable for scale estimation or not.

[0029] The scale calculation unit 150 calculates the scale of each of the multiple divided regions based on the scale of the region suitable for scale estimation. Specifically, the scale calculation unit 150 calculates the scale of the divided region based on the scale of the region suitable for scale estimation surrounding the divided region. The scale calculation unit 150 performs scale estimation for the entire captured image and outputs the estimation result.

[0030] As shown in Fig. 6 , the scale calculation unit 150 sets one of the multiple divided regions as the region of interest. The scale calculation unit 150 calculates the scale of the region of interest based on the scales of the regions determined to be suitable for scale estimation among the surrounding regions, which are divided regions adjacent to the region of interest in four directions (up, down, left, and right). For example, if the surrounding regions include multiple regions suitable for scale estimation, the scale calculation unit 150 calculates the average value of the scales of the multiple regions suitable for scale estimation as the scale of the region of interest. After calculating the scale for one region of interest, the scale calculation unit 150 moves the region of interest as shown in Fig. 6 . The scale calculation unit 150 performs the above-mentioned scale calculation process for all divided regions.

[0031] After calculating the scale of the region of interest, the scale calculation unit 150 determines that the region of interest is suitable for scale estimation, and calculates the scale of subsequent regions of interest. Depending on the arrangement of the regions suitable for scale estimation, there may be no regions suitable for scale estimation among the peripheral regions surrounding the region of interest. In this case, the scale calculation unit 150 moves the region of interest without calculating the scale of the region of interest. If, after performing the above-described scale calculation process for all divided regions, there remain divided regions for which the scale has not been calculated, the scale calculation unit 150 performs the above-described scale calculation process again for all divided regions. By repeating this process, the scale calculation unit 150 can calculate the scale for all divided regions, i.e., for the entire captured image.

[0032] 6, an example has been described in which divided regions in four directions around the region of interest are defined as the peripheral regions, but this is not limiting. For example, as shown in FIG. 7, divided regions adjacent to the region of interest in eight directions, namely, upper right, lower right, upper left, and lower left, in addition to the above, below, left, and right directions, may be defined as the peripheral regions.

[0033] Next, the operation of the estimation device 100 according to this embodiment will be described.

[0034] FIG. 8 is a flowchart showing an example of the operation of the estimation device 100 according to this embodiment, and is a diagram for explaining the estimation method performed by the estimation device 100.

[0035] The image dividing unit 110 divides a photographed image of a concrete structure into a plurality of divided regions (step S11).

[0036] The extraction unit 120 extracts a target area (in this embodiment, a concrete area) in which only concrete or asphalt is photographed from among the plurality of divided areas (step S12).

[0037] The scale estimation unit 131 performs scale estimation (first scale estimation) in the extracted concrete region (step S13).

[0038] The scale estimation unit 132, which has different characteristics from the scale estimation unit 131, performs scale estimation (second scale estimation) in the extracted concrete region (step S14).

[0039] The estimation result determination unit 140 determines whether the concrete region is suitable for scale estimation or not based on the estimation results of the first and second scale estimations (step S15). The determination by the estimation result determination unit 140 will be described in more detail with reference to FIG. 9.

[0040] The estimation result determination unit 140 calculates the difference between the estimation result obtained by the first scale estimation and the estimation result obtained by the second scale estimation for one concrete region (step S151).

[0041] The estimation result determination unit 140 determines whether the calculated difference is equal to or less than a predetermined threshold (step S152). As described with reference to Fig. 5, the difference between the estimation result of the first scale estimation (the estimation result by the scale estimation unit 131) and the estimation result of the second scale estimation (the estimation result by the scale estimation unit 132) is small for an image suitable for scale estimation and is large for an image unsuitable for scale estimation.

[0042] If it is determined that the difference is equal to or smaller than the predetermined threshold (step S152: Yes), the estimation result determination unit 140 determines that the concrete region is an region suitable for scale estimation (step S153).

[0043] If it is determined that the difference is not less than the predetermined threshold, i.e., that the difference is greater than the predetermined threshold (step S152: No), the estimation result determination unit 140 determines that the concrete region is an area that is not suitable for scale estimation.

[0044] The estimation result determination unit 140 performs the above-described process for all extracted concrete regions.

[0045] Referring again to FIG. 8, the scale calculation unit 150 calculates the scale for each of the plurality of divided regions based on the scale for the region suitable for scale estimation (step S16).

[0046] As described above, the estimation device 100 according to this embodiment includes an image segmentation unit 110, an extraction unit 120, a scale estimation unit 131 as a first scale estimation unit, a scale estimation unit 132 as a second scale estimation unit, an estimation result determination unit 140, and a scale calculation unit 150. The image segmentation unit 110 divides a captured image of a structure into multiple divided regions. The extraction unit 120 extracts, from the multiple divided regions, a target region in which only concrete or asphalt is captured. The scale estimation unit 131 performs scale estimation for the extracted target region. The scale estimation unit 132 has characteristics different from those of the scale estimation unit 131 and performs scale estimation for the extracted target region. The estimation result determination unit 140 determines whether the target region is suitable for scale estimation or is not suitable for scale estimation based on the estimation results from the scale estimation units 131 and 132. The scale calculation unit 150 calculates the scale for each of the multiple divided regions based on the scale for the region suitable for scale estimation.

[0047] By comparing the estimation results of the scale estimation unit 131 and the scale estimation unit 132, it is possible to determine whether the target region is a region suitable for scale estimation or a region unsuitable for scale estimation. Then, by using the scale in the region suitable for scale estimation to calculate the scale in each of the multiple divided regions, it is possible to perform scale estimation in the captured image with higher accuracy without installing a reference object.

[0048] Second Embodiment FIG. 10 is a diagram illustrating an example of the configuration of an estimation device 100A according to a second embodiment of the present disclosure.

[0049] 10 , the estimation device 100A according to this embodiment includes an image dividing unit 110, an extracting unit 120, a scale estimating unit 131, a scale estimating unit 132, an estimation result determining unit 140, a scale calculating unit 150, and an excluding unit 160. The estimation device 100A according to this embodiment differs from the estimation device 100 according to the first embodiment in that the excluding unit 160 is added.

[0050] The exclusion unit 160 counts the number of pixels of a specific color (for example, white or black) in the target region. If the number of pixels of the specific color in the target region is equal to or greater than a predetermined threshold, the exclusion unit 160 excludes the target region from the targets of scale estimation by the scale estimation units 131 and 132.

[0051] 11 is a flowchart showing an example of the operation of the exclusion unit 160. The exclusion unit 160 performs the process described with reference to FIG. 11 between the process of step S12 and the process of step S13 in FIG.

[0052] The exclusion unit 160 counts the number of pixels of a specific color in one target region (in this embodiment, a concrete region) (step S161).

[0053] The exclusion unit 160 determines whether the counted number of pixels of the specific color is equal to or greater than a predetermined threshold value (step S162).

[0054] If it is determined that the number of pixels of a specific color counted for one concrete region is equal to or greater than a predetermined threshold (step S162: Yes), the exclusion unit 160 excludes that concrete region from the scale estimation targets (step S163).

[0055] If it is determined that the number of pixels of a specific color counted for a concrete region is less than a predetermined threshold (step S162: No), the exclusion unit 160 determines that the concrete region is a target for scale estimation (step S164).

[0056] If the target area contains a certain number or more of pixel groups of a specific color, such as white or black, there is a possibility that shadows, noise, dirt, etc. are captured. If shadows, noise, dirt, etc. are captured, the scale estimation units 131 and 132 may not be able to accurately estimate the scale.

[0057] In this embodiment, if the number of pixels of a specific color in a target region is equal to or greater than a predetermined threshold, the target region is excluded from the scale estimation target, thereby improving the accuracy of scale estimation.

[0058] Third Embodiment FIG. 12 is a diagram illustrating an example configuration of an estimation device 100B according to a third embodiment of the present disclosure.

[0059] 12 , the estimation device 100B according to this embodiment includes an image division unit 110B, an extraction unit 120B, a scale estimation unit 131, a scale estimation unit 132, an estimation result determination unit 140B, a scale calculation unit 150B, and an identification unit 170. The estimation device 100B according to this embodiment differs from the estimation device 100 according to the first embodiment in that the image division unit 110, the extraction unit 120, the estimation result determination unit 140, and the scale calculation unit 150 are changed to the image division unit 110B, the extraction unit 120B, the estimation result determination unit 140B, and the scale calculation unit 150B, respectively, and in that an identification unit 170 is added.

[0060] Image dividing unit 110B divides the input photographed image into a plurality of divided areas using a first pattern and a second pattern different from the first pattern, as shown in Fig. 13. Fig. 13 shows an example in which the division size is the same for the first pattern and the second pattern, but the division positions are shifted by half the size of one divided area in the x and y directions.

[0061] Referring again to Fig. 12, the extraction unit 120B extracts a target region (in this embodiment, a concrete region) from among the plurality of divided regions divided by the first pattern, as shown in Fig. 14. Also, the extraction unit 120B extracts a concrete region from among the plurality of divided regions divided by the second pattern, as shown in Fig. 14.

[0062] 12 again, the scale estimation unit 131 performs scale estimation in the concrete regions extracted for each of the first pattern and the second pattern, and the scale estimation unit 132 performs scale estimation in the concrete regions extracted for each of the first pattern and the second pattern.

[0063] 15 , the estimation result determination unit 140B determines whether a concrete region extracted from a plurality of divided regions divided according to the first pattern is a region suitable for scale estimation or a region unsuitable for scale estimation, based on the estimation results of the scale estimation units 131 and 132. Furthermore, the estimation result determination unit 140B determines whether a concrete region extracted from a plurality of divided regions divided according to the second pattern is a region suitable for scale estimation or a region unsuitable for scale estimation, based on the estimation results of the scale estimation units 131 and 132.

[0064] As shown in FIG. 16 , the identification unit 170 identifies, from among the multiple regions formed by superimposing the first pattern and the second pattern, a region that is included in at least one of the regions suitable for scale estimation in the first pattern and the regions suitable for scale estimation in the second pattern. As described above, in this embodiment, the division sizes are the same for the first pattern and the second pattern, but the division positions are shifted by half the size of one divided region in the x and y directions. Therefore, by superimposing the first pattern and the second pattern, a region whose size in the x and y directions is half that of the original divided region is formed. The identification unit 170 identifies, from among the regions whose size in the x and y directions is half that of the original divided region, a region that is included in at least one of the regions suitable for scale estimation in the first pattern and the regions suitable for scale estimation in the second pattern.

[0065] Based on the scale in the region identified by the identifying unit 170, the scale calculating unit 150B estimates the scale in each of the multiple regions formed by superimposing the first pattern and the second pattern.

[0066] By doing this, the captured image can be divided into smaller areas than the original divided areas, and an area suitable for scale estimation can be identified, and the scale of the entire captured image can be estimated based on the scale of the identified area, thereby improving estimation accuracy.

[0067] The first and second patterns are not limited to the above examples, and any patterns can be used. However, it is preferable to adopt a pattern in which rectangular regions are arranged in the x and y directions by superimposing the first and second patterns, as in this embodiment.

[0068] In the first to third embodiments described above, an example has been described in which the scale estimation unit 131 and the scale estimation unit 132 are predetermined, but this is not limiting. As shown in Fig. 17 , the estimating devices 100, 100A, and 100B may include a plurality of candidate scale estimation units 133-1 to 133-n that are candidates for the scale estimation unit 131 and the scale estimation unit 132, and a selection unit 134 that selects the scale estimation unit 131 and the scale estimation unit 132 from the plurality of candidate scale estimation units 133-1 to 133-n. Note that Fig. 17 only shows the configuration related to the selection of the scale estimation unit 131 and the scale estimation unit 132, and omits the description of other configurations included in the estimating devices 100, 100A, and 100B.

[0069] Candidate scale estimation units 133-1 to 133-n each have different characteristics and perform scale estimation. Candidate scale estimation units 133-1 to 133-n each receive an image suitable for scale estimation and an image unsuitable for scale estimation, and output the results of scale estimation for each. Note that the images suitable for scale estimation and the images unsuitable for scale estimation can be experimentally identified using, for example, an estimator different from candidate scale estimation units 133-1 to 133-n.

[0070] The selection unit 134 selects scale estimation unit 131 and scale estimation unit 132 from among the plurality of candidate scale estimation units 133-1 to 133-n based on the estimation results of each of the plurality of candidate scale estimation units 133-1 to 133-n for the input image (an image suitable for scale estimation and an image unsuitable for scale estimation). Specifically, the selection unit 134 identifies a combination of two candidate scale estimation units 133 from among the candidate scale estimation units 133-1 to 133-n that will produce a small error in the estimation result for an image suitable for scale estimation and a large error in the estimation result for an image unsuitable for scale estimation. The selection unit 134 then selects the two identified candidate scale estimation units 133 as scale estimation unit 131 and scale estimation unit 132.

[0071] In this way, even when the target of scale estimation changes, it is possible to select the candidate scale estimation unit 133 suitable for the target of scale estimation as the scale estimation unit 131 and the scale estimation unit 132 .

[0072] The above-described estimation devices 100, 100A, and 100B can each be realized by a computer 10 shown in FIG. 18 . A program for causing the computer 10 to function as the estimation devices 100, 100A, and 100B may be provided. The program may be stored in a storage medium or provided via a network. FIG. 18 is a block diagram showing a schematic configuration of a computer 10 functioning as the estimation devices 100, 100A, and 100B. The computer 10 may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, or the like. The program instructions may be program code, code segments, or the like for executing necessary tasks.

[0073] 18, a computer 10 includes a processor 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is communicably connected to one another via a bus 19. The processor 11 is specifically a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a SoC (System on a Chip), or the like, and may be configured with multiple processors of the same or different types.

[0074] The processor 11 is a control unit that controls each component and performs various arithmetic processing. That is, the processor 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The processor 11 controls each component and performs various arithmetic processing in accordance with the program stored in the ROM 12 or the storage 14. In this embodiment, the ROM 12 or the storage 14 stores a program for operating the computer 10 as the estimation devices 100, 100A, and 100B according to the present disclosure. The processor 11 reads and executes the program, thereby realizing each component of the estimation devices 100, 100A, and 100B described above.

[0075] The program may be provided in a form stored on a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), a USB (Universal Serial Bus) memory, etc. The program may also be provided in a form downloaded from an external device via a network.

[0076] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0077] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to input various types of information.

[0078] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may be a touch panel type and function as the input unit 15. The display unit 16 is an example of the result output unit 124.

[0079] The communication interface 17 is an interface for communicating with other devices, for example, an interface for a LAN.

[0080] The following additional notes are provided regarding the above-described embodiments.

[0081] [Supplementary Item 1] An estimation device that performs scale estimation in a captured image of a structure, comprising a control unit, wherein the control unit divides the captured image into a plurality of divided regions, extracts from the plurality of divided regions a target region in which only concrete or asphalt is captured, performs a first scale estimation in the extracted target region, performs a second scale estimation in the extracted target region having characteristics different from those of the first scale estimation, determines whether the target region is a region suitable for scale estimation or a region unsuitable for scale estimation based on the estimation results of the first scale estimation and the second scale estimation, and calculates the scale in each of the plurality of divided regions based on the scale in the region suitable for scale estimation.

[0082] [Supplementary Item 2] In the estimation device described in Supplementary Item 1, the control unit determines, for one target region, whether the one target region is a region suitable for the scale estimation or a region unsuitable for the scale estimation, based on a difference between a scale estimated by the first scale estimation and a scale estimated by the second scale estimation.

[0083] [Supplementary Item 3] The estimation device according to Supplementary Item 1 or 2, wherein the control unit calculates the scale of the divided region based on a scale in a region suitable for the scale estimation surrounding the divided region.

[0084] [Supplementary Item 4] In the estimation device according to any one of Supplementary Items 1 to 3, the control unit counts the number of pixels of a specific color in the target region, and if the number of pixels of the specific color is equal to or greater than a predetermined threshold, excludes the target region from the targets of the first scale estimation and the second scale estimation.

[0085] [Supplementary Item 5] In the estimation device according to any one of Supplementary Items 1 to 4, the control unit divides the captured image into a plurality of divided areas using a first pattern and a second pattern different from the first pattern, extracts the target area from among the plurality of divided areas divided using the first pattern, and extracts the target area from each of the plurality of divided areas divided using the second pattern, performs the first scale estimation and the second scale estimation for the target areas extracted for each of the first pattern and the second pattern, and determines, based on the results of the first scale estimation and the second scale estimation, whether the target area extracted from the plurality of divided areas divided using the first pattern is a region suitable for the scale estimation or a region not suitable for the scale estimation, and determines whether the target area extracted from the plurality of divided areas divided using the second pattern is a region suitable for the scale estimation or a region not suitable for the scale estimation, an estimation device that identifies, from among a plurality of regions formed by superimposing the first pattern and the second pattern, a region included in at least one of a region in the first pattern that is suitable for the scale estimation and a region in the second pattern that is suitable for the scale estimation, and estimates the scale of each of the plurality of regions formed by superimposing the first pattern and the second pattern based on the scale of the identified region.

[0086] [Supplementary Item 6] The estimation device according to any one of Supplementary Items 1 to 5, wherein the estimation device comprises a plurality of candidate scale estimation units having different characteristics, and the control unit selects a candidate scale estimation unit to perform the first scale estimation and the second scale estimation from among the plurality of candidate scale estimation units based on estimation results of each of the plurality of candidate scale estimation units for the input image.

[0087] [Supplementary Item 7] An estimation method using an estimation device that performs scale estimation in a captured image of a structure, comprising: dividing the captured image into a plurality of divided regions; extracting a target region from the plurality of divided regions in which only concrete or asphalt is captured; performing a first scale estimation in the extracted target region; performing a second scale estimation in the extracted target region having characteristics different from those of the first scale estimation; determining whether the target region is a region suitable for scale estimation or a region unsuitable for scale estimation based on the estimation results of the first scale estimation and the second scale estimation; and calculating the scale in each of the plurality of divided regions based on the scale in the region suitable for scale estimation.

[0088] [Supplementary Item 8] A non-transitory storage medium storing a program executable by a computer, the program causing the computer to operate as the estimation device according to Supplementary Items 1-6.

[0089] Although the above-described embodiments have been described as typical examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present disclosure. Therefore, the present invention should not be interpreted as being limited by the above-described embodiments, and various modifications and alterations are possible without departing from the scope of the claims. For example, multiple building blocks shown in the block diagrams of the embodiments can be combined into one, or one building block can be divided.

[0090] 10 Computer 11 Processor 12 ROM 13 RAM 14 Storage 15 Input unit 16 Display unit 17 Communication I / F 19 Path 100, 100A, 100B Estimation device 110, 110B Image division unit 120, 120B Extraction unit 131 Scale estimation unit (first scale estimation unit) 131-1 to 131-n Weak estimator 132 Scale estimation unit (second scale estimation unit) 132-1 Estimator 133-1 to 133-n Candidate scale estimation unit 134 Selection unit 140, 140B Estimation result determination unit 150, 150B Scale calculation unit 160 Exclusion unit 170 Identification unit

Claims

1. An estimation device that performs scale estimation in a captured image of a structure, comprising: an image division unit that divides the captured image into a plurality of divided areas; an extraction unit that extracts a target area from the plurality of divided areas in which only concrete or asphalt is captured; a first scale estimation unit that performs scale estimation in the extracted target area; a second scale estimation unit that has characteristics different from those of the first scale estimation unit and performs scale estimation in the extracted target area; an estimation result determination unit that determines whether the target area is a region suitable for scale estimation or a region unsuitable for scale estimation based on estimation results by the first scale estimation unit and the second scale estimation unit; and a scale calculation unit that calculates the scale in each of the plurality of divided areas based on the scale in the region suitable for scale estimation.

2. An estimation device as described in claim 1, wherein the estimation result determination unit determines, for a given target region, whether the given target region is an area suitable for the scale estimation or an area unsuitable for the scale estimation, depending on the difference between the scale estimated by the first scale estimation unit and the scale estimated by the second scale estimation unit.

3. An estimation device according to claim 1, wherein the scale calculation section calculates the scale of the divided region based on the scale of a region suitable for the scale estimation surrounding the divided region.

4. The estimation device according to claim 1, further comprising an exclusion unit that counts the number of pixels of a specific color in the target area, and if the number of pixels of the specific color is equal to or greater than a predetermined threshold, excludes the target area from being subject to scale estimation by the first scale estimation unit and the second scale estimation unit.

5. An estimation device according to claim 1, wherein the image division unit divides the captured image into a plurality of divided regions using a first pattern and a second pattern different from the first pattern; the extraction unit extracts the target region from among the plurality of divided regions divided using the first pattern, and also extracts the target region from each of the plurality of divided regions divided using the second pattern; the first scale estimation unit and the second scale estimation unit perform scale estimation for the target regions extracted for each of the first pattern and the second pattern, respectively; the estimation result determination unit determines, based on the estimation results of the first scale estimation unit and the second scale estimation unit, whether the target region extracted from the plurality of divided regions divided using the first pattern is an area suitable for the scale estimation or an area not suitable for the scale estimation, and determines whether the target region extracted from the plurality of divided regions divided using the second pattern is an area suitable for the scale estimation or an area not suitable for the scale estimation; the estimation device further comprising an identification unit that identifies an area included in at least one of an area in the first pattern suitable for the scale estimation and an area in the second pattern suitable for the scale estimation, among a plurality of areas formed by superimposing the first pattern and the second pattern, wherein the scale calculation unit estimates the scale in each of the plurality of areas formed by superimposing the first pattern and the second pattern, based on the scale in the area identified by the identification unit.

6. An estimation device as described in claim 1, further comprising: a plurality of candidate scale estimation units having different characteristics; and a selection unit that selects the first scale estimation unit and the second scale estimation unit from the plurality of candidate scale estimation units based on the estimation results of each of the plurality of candidate scale estimation units for an input image.

7. An estimation method by an estimation device that performs scale estimation in a captured image of a structure, comprising the steps of: dividing the captured image into a plurality of divided areas; extracting a target area from the plurality of divided areas in which only concrete or asphalt is captured; performing a first scale estimation in the extracted target area; performing a second scale estimation in the extracted target area having characteristics different from the first scale estimation; determining whether the target area is a region suitable for scale estimation or a region unsuitable for scale estimation based on the estimation results of the first scale estimation and the second scale estimation; and calculating the scale in each of the plurality of divided areas based on the scale in the region suitable for scale estimation.

8. A program for causing a computer to operate as the estimation device according to claim 1.

Citation Information

Patent Citations

  • Measurement device

    JP2014185996A

  • Information processing device, information processing method and program

    JP2019194562A

  • Information processing apparatus, information processing method and program

    JP2021163190A

  • Learning model construction device, estimation device, learning model construction method, estimation method, and program

    WO2023100336A1