Learning model construction device, estimation device, learning model construction method, estimation method, and program

Abstract

A learning model construction device (3A) according to the present disclosure includes: a plurality of learning units (354-k) that uses, as a basis, training data in which image data indicating a learning image is associated with a true value of a scale of the learning image, to construct a plurality of learning models using a plurality of mutually different loss functions; a plurality of verification units (355-k) that calculates a plurality of estimation values for images to be verified for optimality; a plurality of correlation calculation units (356-k) that calculates correlations between the estimation values and the true values of the plurality of scales; and an optimum learning model selection unit (357) that selects an optimum learning model that is a learning model having the highest correlation.

Description

[Technical field]

[0001] The present disclosure relates to a learning model construction device, an estimation device, a learning model construction method, an estimation method, and a program. [Background technology]

[0002] In recent years, it has been known to detect an image of an object from an image using image processing technology. In particular, it has become possible to detect an image of an object from an image on a pixel-by-pixel basis by using a technique such as segmentation. In this way, a technique for detecting an image of an object has been researched and developed to improve the efficiency of inspection and diagnosis of the object, that is, a structure. For example, when the object is a concrete structure, it is known to diagnose deterioration of the concrete structure by detecting damage such as cracks and exposed rebar that occurs on the surface of the concrete structure.

[0003] Furthermore, it is known that the size (area, length, etc.) of the damaged portion of a structure can be quantitatively evaluated by determining the scale (unit: "cm / pixel" or "pixel / cm") of an image showing an image of the structure in which damage has been detected. For example, Non-Patent Document 1 proposes estimating the scale of an image of a concrete structure's surface using deep learning with a Convolutional Neural Network (CNN). Specifically, it proposes estimating the scale of an image based on texture features formed by unevenness, shadows, voids, etc. on the surface of concrete structures such as footbridges and concrete walls. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Ju An Park, et al., “Learning-based image scale estimation using surface texture for quantitative visual inspection of regions-of interest”, Computer-Aided Civil and Infrastructure engineering, Vol.36, pp.227-241, 2020 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in an environment where the surface of a concrete structure is installed indoors, such as inside a communication tunnel, the amount of light is less than in an outdoor environment. Therefore, noise may occur in an image of a concrete structure captured by a camera indoors. Such noise may make it difficult to estimate the scale of an image of the surface of a concrete structure with high accuracy.

[0006] In consideration of the above circumstances, the object of the present disclosure is to provide a learning model construction device, an estimation device, a learning model construction method, an estimation method, and a program that can estimate the surface scale of a concrete structure with high accuracy. [Means for solving the problem]

[0007] In order to solve the above problems, the learning model construction device of the present disclosure includes a plurality of learning units that construct a plurality of learning models using a plurality of different loss functions based on teacher data that corresponds image data showing a learning image, the true value of which is known and is an image of a concrete surface, to the true value of the scale of the learning image, a plurality of verification units that calculate a plurality of estimated values ​​of the scale for an optimal verification image, different from the learning image, the true value of which is known, using a plurality of learning models, a plurality of correlation calculation units that calculate a correlation between the plurality of estimated values ​​of the scale and the true value of the scale for the optimal verification image, and an optimal learning model selection unit that selects an optimal learning model from the plurality of learning models, which is a learning model with the highest correlation.

[0008] In addition, in order to solve the above problem, the estimation device according to the present disclosure includes a learning model memory unit that stores the optimal learning model selected by the learning model construction device, and an estimation unit that uses the optimal learning model to calculate an estimate of the scale of an unknown image, the true value of which is unknown.

[0009] In addition, in order to solve the above-mentioned problems, the learning model construction method disclosed herein includes the steps of constructing a plurality of learning models using a plurality of different loss functions based on teacher data that corresponds image data showing a learning image, the true value of which is known and is an image of a concrete surface, to the true value of the scale of the learning image; calculating a plurality of estimated values ​​of the scale for an optimal verification image, different from the learning image and the true value of the scale, using each of the plurality of learning models; calculating a correlation between the plurality of estimated values ​​of the scale and the true value of the scale for the optimal verification image; and selecting an optimal learning model from the plurality of learning models, which is the learning model with the highest correlation.

[0010] In addition, in order to solve the above problem, the estimation method disclosed herein is an estimation method executed by an estimation device having a learning model memory unit that stores the optimal learning model selected by the above-mentioned learning model construction device, and includes a step of calculating, using the optimal learning model, an estimate of the scale of an unknown image, the true value of which is unknown.

[0011] In addition, in order to solve the above problem, a program according to the present disclosure causes a computer to function as the above-mentioned learning model construction device. Effect of the Invention

[0012] According to the learning model construction device, estimation device, learning model construction method, estimation method, and program disclosed herein, the scale of the surface of a concrete structure can be estimated with high accuracy. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of an estimation system according to a first embodiment. [Diagram 2] 2 is a schematic diagram illustrating an example of a model construction unit illustrated in FIG. 1. [Diagram 3] 2 is a schematic diagram illustrating an example of a scale estimation unit illustrated in FIG. 1. [Figure 4] FIG. 4 is a schematic diagram illustrating another example of the estimation system according to the first embodiment. [Diagram 5] 1. FIG. 4 is a sequence diagram showing an example of an operation for storing teacher data in the estimation device shown in FIG. [Figure 6] 2 is a sequence diagram showing an example of an operation for constructing a learning model in the estimation device shown in FIG. 1. [Figure 7] 7 is a sequence diagram showing a first example of details of the operation shown in FIG. 6. FIG. [Figure 8] 7 is a sequence diagram showing a second example of details of the operation shown in FIG. 6. FIG. [Figure 9] 1. FIG. 4 is a sequence diagram showing an example of an operation for calculating an estimated value of a scale in the estimation device shown in FIG. [Figure 10] FIG. 11 is a schematic diagram illustrating an example of an estimation system according to a second embodiment. [Figure 11] FIG. 11 is a schematic diagram illustrating another example of the estimation system according to the second embodiment. [Figure 12] 11 is a sequence diagram showing an example of an operation for storing teacher data in the estimation device shown in FIG. [Figure 13] 11 is a sequence diagram showing an example of an operation for calculating an estimated value of a scale in the estimation device shown in FIG. [Figure 14] FIG. 13 is a schematic diagram illustrating an example of an estimation system according to a third embodiment. [Figure 15] 15 is a schematic diagram illustrating an example of a noise image removing unit illustrated in FIG. 14. [Figure 16A] FIG. 13 is a diagram showing an example of a frequency distribution of pixel values ​​of the a* component in an image containing color noise. [Figure 16B] FIG. 13 is a diagram showing an example of a frequency distribution of pixel values ​​of the a* component in an image that does not contain color noise. [Figure 17A] FIG. 13 is a diagram showing an example of a frequency distribution of pixel values ​​of the b* component in an image containing color noise. [Figure 17B] FIG. 13 is a diagram showing an example of a frequency distribution of pixel values ​​of the b* component in an image that does not contain color noise. [Figure 18] FIG. 11 is a schematic diagram illustrating another example of the estimation system according to the third embodiment. [Figure 19] 15 is a sequence diagram showing an example of an operation for storing teacher data in the estimation device shown in FIG. 14. [Figure 20] FIG. 15 is a sequence diagram showing an example of an operation for calculating an estimated value of a scale in the estimation device shown in FIG. [Figure 21] FIG. 2 is a hardware block diagram of the estimation device and the learning data construction device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] <<First embodiment>> The overall configuration of the first embodiment will be described with reference to FIGS.

[0015] As shown in FIG. 1, the estimation system 100 according to the first embodiment includes an image capturing device 1, a data storage device 2, an estimation device 3, and a data retention device 4.

[0016] <Configuration of the imaging device> The image capturing device 1 may be configured by a camera including an optical element, an image capturing element, and an output interface. The output interface is an interface for outputting image data representing an image captured by the image capturing element.

[0017] The image capturing device 1 captures an image of a subject and generates an image. The subject may be a concrete surface. The concrete may form, for example, a footbridge, a wall, a paved road, or the like. The surface of the subject may have a surface pattern and may have exposed aggregate. The image may be in any format, for example, a JPG format or a PNG format.

[0018] Moreover, the image capturing device 1 outputs image data representing an image to the data storage device 2.

[0019] <Data storage device configuration> The data storage device 2 may be configured by a computer including a memory, a controller, an input interface, and an output interface. The memory may be configured by a hard disk drive (HDD), a solid state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a random access memory (RAM), or the like. The controller may be configured by dedicated hardware such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), or may be configured by a processor, or may be configured to include both. The input interface may be a pointing device, a keyboard, a mouse, or the like. The input interface may also be an interface that accepts input of information received by a communication interface. For the communication interface, standards such as Ethernet (registered trademark), Fiber Distributed Data Interface (FDDI), Wi-Fi (registered trademark), or the like may be used.

[0020] The data storage device 2 receives the input of image data output by the image capturing device 1, and stores the image data. The data storage device 2 also outputs the image data to the estimation device 3.

[0021] <Configuration of the estimation device> The estimation device 3 includes an input unit 31, a teacher data storage unit 32, a learning model storage unit 33, a loss function storage unit 34-k (k=1 to n, n is an integer equal to or greater than 2), a model construction unit 35, a scale estimation unit 36, and an output unit 37. In the example shown in FIG. 1, n=2. The input unit 31 is configured by an input interface. The teacher data storage unit 32, the learning model storage unit 33, and the loss function storage unit 34-k are configured by memories. The model construction unit 35 and the scale estimation unit 36 ​​are configured by a controller. The output unit 37 is configured by an output interface.

[0022] The input unit 31 accepts input of image data output from the data storage device 2. The input unit 31 may accept input of image data from the image capturing device 1 without going through the data storage device 2.

[0023] Specifically, the input unit 31 receives image data indicating an image whose true value of the scale is known, and a true value y true The images for which the true value of the scale is known include a learning image, a learning verification image, and an optimum verification image, and the learning image, the learning verification image, and the optimum verification image are different from each other. The input unit 31 also receives the true value y true The input of image data representing an unknown image, the size of which is unknown, is accepted.

[0024] Scale is a value that indicates the ratio between the length of one pixel in an image and the length in real space, and may be the length in real space relative to the length of one pixel (cm / pixel), or the length of one pixel relative to the length in real space (pixel / cm).

[0025] The teacher data storage unit 32 stores the true value y true Image data showing an image with known scale and the true value y trueSpecifically, the teacher data storage unit 32 may store teacher data whose input is accepted by the input unit 31. The teacher data storage unit 32 also stores the image data whose input is accepted by the input unit 31 and the true value y of the scale calculated by the scale calculation unit 351, which will be described later in detail. true The teacher data storage unit 32 may store teacher data in which the image data representing an image input by the input unit 31 and processed by a data processing unit 352, which will be described later in detail, and the true value y true The teacher data in which the above and the above are associated with each other may be stored.

[0026] The images stored by the teacher data storage unit 32 include a learning image, a learning verification image, and an optimum verification image.

[0027] When image data is input, the learning model storage unit 33 stores an estimated value y pred The learning model storage unit 33 stores a learning model that calculates one estimated value y for scale when a learning model is constructed by each of the learning units 354-k, which will be described later in detail. The learning model storage unit 33 stores an optimal learning model, which will be described later in detail. The learning model may be any model, and may be, for example, a deep learning model. When one image data is input, the learning model calculates one estimated value y for scale. pred The device is configured to output:

[0028] The multiple loss function storage units 34-k each store a different loss function (parameters set for the loss function). The loss function is a function for calculating a loss value L for N pieces of image data i (i=1 to N, N is an integer). The loss value L is calculated by dividing the true value y true,i The scale estimate y for pred,iThe loss function is a value for evaluating the accuracy of a learning model based on the error of the loss function. The loss function can be, for example, the functions shown in formulas (1) to (4). In formulas (1) and (2), α is a coefficient that is set in advance.

[0029]

number

[0030] The loss value L calculated by the loss function shown in equation (1) is the true value y true,i Estimated value for y pred,i The mean absolute error (MAE) of the true value y true,i Estimated value for y pred,i The mean absolute percentage error (MAPE) of the true value y true,i and the estimated value y pred,i In addition to the mean absolute percentage error with , the value of the mean absolute error is taken into account. Therefore, the true value of the scale y true,i The larger the loss value L, the larger the scale estimate y pred,i is the true value y true,i If the outlier is significantly different from the target, the loss value L becomes large.

[0031]

number

[0032] The loss value L calculated by the loss function shown in equation (2) is the true value y true,i Estimated value for y pred,i and the true value y true,i Estimated value for y pred,iBy using the loss function shown in formula (2), the loss value L can show the same tendency as when the loss function shown in formula (1) is used, and furthermore, it becomes larger for outliers than when the loss function shown in formula (1) is used. Also, the first term of formula (2) is the true value y true,i Estimated value for y pred,i The variance of the estimated value y pred,i and the true value y true,i The more the image data with a large difference from the estimated value y pred,i is the true value y true,i If the guess value y is significantly off, pred,i The second term in formula (2) can reflect the size of the outlier in the loss value L to a greater extent than the first term in formula (2).

[0033]

number

[0034] The loss value L calculated by the loss function shown in equation (3) is the true value y true,i Estimated value for y pred,i By using the loss function shown in equation (3), the loss value L is expressed as the mean absolute percentage error. Therefore, the loss value L is expressed as the mean absolute percentage error of the scale estimate y pred,i The true value y true,i The true value y true,i and the guess value y pred,i Therefore, the loss function shown in formula (3) can be used without the need for setting the coefficient α as in the loss functions shown in formulas (1) and (2).

[0035]

number

[0036] The loss value L calculated by the loss function shown in equation (4) is calculated by multiplying the denominator of the mean absolute percentage error in the loss function shown in equation (1) by the true value y true,i From the estimated value y pred,i By using the loss function shown in equation (4), the estimated value y pred,i and the true value y true,i The loss value L can be changed depending on the magnitude relationship with the estimated value y pred,i is the true value y true,i If it is smaller, the estimate y pred,i is the true value y true,i The loss value L changes significantly compared to the larger case.

[0037] As shown in FIG. 2, the model construction unit 35 includes a scale calculation unit 351, a data processing unit 352, a learning model reading unit 353, a learning unit 354-k, a verification unit 355-k, a correlation calculation unit 356-k, and an optimal learning model selection unit 357.

[0038] The scale calculation unit 351 calculates the true value y of the scale of the image represented by the image data received by the input unit 31. true,i Calculate.

[0039] Specifically, the scale calculation unit 351 calculates the true value y of the scale based on the length in image space of an image of a tape measure whose dimensions in real space are known and which is included in the image represented by the image data, and the length of the tape measure in real space. true,i In addition, the scale calculation unit 351 may calculate the true value y of the scale by using a known library of AR (Augmented Reality) markers and the like included in the image represented by the image data. true,i The scale calculation unit 351 may calculate the true value y of the scale by any method other than the above. true,i can be calculated.

[0040] The data processing unit 352 calculates the true value y true,iSpecifically, the data processing unit 352 may change the size or format of the image data.

[0041] For example, the data processing unit 352 can generate multiple images by dividing an image represented by image data. This allows the learning unit 354-k, which will be described in detail later, to learn from multiple pieces of image data representing the multiple images as training data, and can generate a learning model with higher accuracy more efficiently than when learning from one piece of image data as training data.

[0042] Furthermore, the data processing unit 352 can convert the shape of the image represented by the image data into a square, and create a rotated image by rotating the square image converted into a square, an inverted image by inverting the square image, etc. This allows the learning unit 354-k to learn a plurality of image data respectively representing a plurality of patterns of images as teacher data, and can generate a learning model with higher accuracy more efficiently than when learning with one image data as teacher data. Furthermore, when the image represented by the image data includes an image of a subject other than a concrete structure, the data processing unit 352 may process the image data so as to remove pixels representing the image.

[0043] The data processing unit 352 compares the true value y of the scale of the image represented by the processed image data and the image data before processing. true,i The teacher data in which the above is associated with the above is stored in the teacher data storage unit 32.

[0044] The learning model reading unit 353 reads the learning model stored in the learning model storage unit 33. The learning model reading unit 353 also reads the image data and the true value y of the scale of the image represented by the image data, which are associated as teacher data stored in the teacher data storage unit 32. true,i and load it.

[0045] The multiple learning units 354-k image the surface of the concrete and calculate the true value y true,iImage data showing a learning image whose scale is known, and a true value y true,i Based on the teacher data in which n and n are associated with each other, a plurality of learning models are constructed using a plurality of mutually different loss functions. For example, in a configuration where n=2, the learning unit 354-1 constructs a learning model using the loss function stored in the loss function storage unit 34-1. The learning unit 354-2 constructs a learning model using the loss function stored in the loss function storage unit 34-2.

[0046] Specifically, each of the multiple learning units 354-k learns multiple learning models and calculates the true value y true,i For training and validation images different from the training images and the optimal validation image, the scale estimate y pred,i Then, the multiple learning units 354-k calculate the estimated values ​​y pred,i , and the true scale value y true,i A learning model is constructed based on the loss value L calculated using the loss function.

[0047] For example, in a configuration in which the learning model is a CNN, each of the learning units 354-k receives the deep learning model read by the learning model reading unit 353, the learning image indicated by the image data, and the true value y true,i Then, the learning unit 354-k constructs a learning model based on the convolution of the learning and verification image to obtain the estimated value y pred,i Then, the learning unit 354-k calculates the true value y true,i and the estimated value y pred,i The learning unit 354-k constructs a learning model (weight file) in which the weight parameters are adjusted so that the loss value L calculated based on the above is the minimum value. In addition, the learning unit 354-k stores the constructed learning model in the learning model storage unit 33.

[0048] The plurality of verification units 355-k verify the true value y true,iFor an optimal validation image different from the training image, for which y is known, multiple estimates of the scale y are calculated using multiple learning models. pred,i For example, in a configuration where n=2, the verification unit 355-1 calculates the estimated value y pred,i The verification unit 355-2 calculates the estimated value y pred,i The verification unit 355-k calculates the estimated values ​​y pred,i It is preferable to calculate

[0049] The multiple correlation calculation units 356-k calculate the true value y true,i Multiple scale estimates of y pred,i The correlations can be expressed as indices such as a correlation coefficient and a coefficient of determination. For example, in a configuration where n=2, the correlation calculation unit 356-1 calculates the true value y true,i The estimated value y calculated by the verification unit 355-1 for pred,i The correlation calculation unit 356-2 calculates the correlation between the true value y true,i The estimated value y calculated by the verification unit 355-2 for pred,i Calculate the correlation.

[0050] The optimal learning model selection unit 357 selects an optimal learning model that is a learning model with the highest correlation among multiple learning models. For example, in a configuration where n=2, the optimal learning model selection unit 357 selects a learning model with the highest correlation among the correlation calculated by the correlation calculation unit 356-1 and the correlation calculated by the correlation calculation unit 356-2 as the optimal learning model.

[0051] As shown in FIG. 3, the scale estimation unit 36 ​​includes a data processing unit 361, a learning model reading unit 362, an estimation unit 363, and a data restoration unit 364.

[0052] The data processing unit 361 processes image data representing an unknown image input by the input unit 31. Details of the process executed by the data processing unit 361 are similar to the details of the process executed by the data processing unit 352.

[0053] The learning model reading unit 362 reads the optimum learning model selected by the optimum learning model selection unit 357 and stored in the learning model storage unit 33 .

[0054] The estimation unit 363 uses the optimal learning model to estimate the true value y true,i The scale estimate of the unknown image, y, is unknown. pred,i Calculate.

[0055] The data restoration unit 364 restores the image data processed by the data processing unit 361 .

[0056] For example, if the unknown image represented by the image data input by the input unit 31 has been divided by the data processing unit 361, the data restoration unit 364 restores the image data by processing the divided images so as to return them to the unknown images before division. In this case, the data restoration unit 364 restores the image data by using the estimated value y pred,i The representative value of is used as the estimated value y pred,i The representative value may be a statistical value such as an average value or a median value.

[0057] In addition, if the size of the unknown image indicated by the image data accepted as input by the input unit 31 has been changed by the data processing unit 361, the data restoration unit 364 restores the image data by processing the image whose size has been changed so as to return it to the unknown image before it was divided.

[0058] The output unit 37 outputs image data and an estimated value y of the scale of the image represented by the image data. pred,iand outputs scale estimation information including the above. Specifically, the output unit 37 may output the scale estimation information to the data storage device 4 via a communication network. The output unit 37 may output the scale estimation information to a display device configured with an organic EL (Electro Luminescence) panel, a liquid crystal panel, or the like.

[0059] <Data storage device configuration> 1 is configured by a computer equipped with a memory, a controller, and an input interface. The data storage device 4 stores the scale estimation information output from the estimation device 3.

[0060] As shown in FIG. 4, an estimation system 100A of another example of the first embodiment includes an image capturing device 1, a data storage device 2, a learning model construction device 3A, an estimation device 3B, and a data storage device 4. In the estimation system 100A, the learning model construction device 3A includes an input unit 31, a teacher data storage unit 32, a learning model storage unit 33, a loss function storage unit 34-k, and a model construction unit 35. In addition, the estimation device 3B includes an input unit 31, a learning model storage unit 33, a scale estimation unit 36, and an output unit 37. In the estimation system 100A, the functional units with the same reference numerals as those in the estimation system 100 have the same functions. However, in the estimation system 100A, the learning model storage unit 33 included in the estimation device 3B stores the optimal learning model selected by the learning model construction device 3A.

[0061] <Operation of the estimation device> Here, the operation of the estimation device 3 according to the first embodiment will be described with reference to Fig. 5 to Fig. 9. Fig. 5 to Fig. 9 are flowcharts showing an example of the operation of the estimation device 3 according to the first embodiment. The operation of the estimation device 3 described with reference to Fig. 5 to Fig. 9 corresponds to an example of an estimation method of the estimation device 3 according to the first embodiment.

[0062] (Memorizing teacher data) A method in which the estimation device 3 stores training data will be described with reference to FIG.

[0063] In step S11, the input unit 31 accepts the input of image data output from the data storage device 2.

[0064] In step S12, the scale calculation unit 351 calculates the true value y true,i Calculate.

[0065] In step S13, the data processing unit 352 processes the image data.

[0066] In step S14, the teacher data storage unit 32 stores the image data processed in step S13 and the true value y of the image scale calculated in step S12. true,i The teacher data that corresponds to the above is stored.

[0067] (Building a learning model) A method in which the estimation device 3 constructs a learning model will be described with reference to FIG.

[0068] In step S21, the learning model reading unit 353 reads the teacher data stored in the teacher data storage unit 32. In addition, the learning model reading unit 353 reads the learning model from the learning model storage unit 33.

[0069] In step S22, the multiple learning units 354-k acquire the true scale value y true,i Image data showing a learning image whose scale is known, and a true value y true,i Based on the training data in which the above-mentioned correlations are established, a plurality of learning models are constructed using a plurality of mutually different loss functions.

[0070] In step S23, a plurality of verification units 355-k verify the true value y true,i For an optimal validation image different from the training image, for which y is known, multiple estimates of the scale y are calculated using multiple learning models. pred,i Calculate each one.

[0071] In step S24, the multiple correlation calculation units 356-k calculate the true value y true,i Multiple scale estimates for y pred,i The correlations of each are calculated.

[0072] In step S25, the optimum learning model selection unit 357 selects an optimum learning model, which is a learning model having the highest correlation, from among the multiple learning models.

[0073] Here, a first example of a method in which the estimation device 3 constructs a learning model in a configuration where n=2 will be described in detail with reference to FIG.

[0074] 7, in step S21, the learning model reading unit 353 reads the teacher data stored in the teacher data storage unit 32. In addition, the learning model reading unit 353 reads the learning model from the learning model storage unit 33.

[0075] In step S22-11, the learning unit 354-1 acquires the true value y true,i A learning image whose scale is known and the true scale y true,i A first learning model is constructed using a first loss function based on the training data in which the and are associated with each other.

[0076] In step S23-11, the verification unit 355-1 verifies the true value y true,i For an optimal validation image different from the training image, for which y is known, a first estimate of the scale y is obtained using a first learning model. pred,i Calculate.

[0077] In step S24-11, the correlation calculation unit 356-1 calculates the true value y true,i First estimate of the scale for pred,i Calculate the correlation.

[0078] Next, in step S22-12, the learning unit 354-2 calculates the true value y true,i A learning image whose scale is known and the true scale y true,i A second learning model is constructed using a second loss function based on the training data in which the and are associated with each other.

[0079] In step S23-12, the verification unit 355-2 verifies the true value y true,i For an optimal validation image different from the training image, for which y is known, a second estimate of the scale y is obtained using a second training model. pred,i Calculate.

[0080] In step S24-12, correlation calculation unit 356-2 calculates the true value y true,i The second estimate of the scale, y pred,i Calculate the correlation.

[0081] In step S25, the optimum learning model selection unit 357 selects an optimum learning model, which is a learning model having the highest correlation, from among the multiple learning models.

[0082] Next, a second example of a method in which the estimation device 3 constructs a learning model in a configuration where n=2 will be described in detail with reference to FIG.

[0083] 8, in step S21, the learning model reading unit 353 reads the teacher data stored in the teacher data storage unit 32. In addition, the learning model reading unit 353 reads the learning model from the learning model storage unit 33.

[0084] In step S22-21, the learning unit 354-1 acquires the true value y true,i A learning image whose scale is known and the true scale y true,i A first learning model is constructed using a first loss function based on the training data in which the and are associated with each other.

[0085] In step S23-21, the verification unit 355-1 verifies the true value y true,i For an optimal validation image different from the training image, for which y is known, a first estimate of the scale y is obtained using a first learning model. pred,i Calculate.

[0086] In step S24-21, the correlation calculation unit 356-1 calculates the true value y true,i First estimate of the scale for pred,i Calculate the correlation.

[0087] At the same time as the process of step S22-21, in step S22-22, the learning unit 354-2 acquires the true value y true,i A learning image whose scale is known and the true scale y true,i A second learning model is constructed using a second loss function based on the training data in which the and are associated with each other.

[0088] In step S23-22, the verification unit 355-2 verifies the true value y true,i For an optimal validation image different from the training image, for which y is known, a second estimate of the scale y is obtained using a second training model. pred,i Calculate.

[0089] In step S24-22, the correlation calculation unit 356-2 calculates the true value y true,i The second estimate of the scale, y pred,i Calculate the correlation.

[0090] In step S25, the optimum learning model selection unit 357 selects an optimum learning model, which is a learning model having the highest correlation, from among the multiple learning models.

[0091] (Calculating the scale estimate) Referring to FIG. 9, the estimation device 3 estimates the image scale y pred,i A method for calculating the above will be described.

[0092] In step S31, the input unit 31 accepts input of image data output from the data storage device 2, the image data indicating an unknown image with an unknown scale.

[0093] In step S32, the data processing unit 361 processes the image data representing the unknown image.

[0094] In step S33, the learning model reading unit 362 reads the optimal learning model selected by the optimal learning model selection unit 357.

[0095] In step S34, the estimation unit 363 calculates the true value y true,i The scale estimate of the unknown image, y, is unknown. pred,i Calculate.

[0096] In step S 35 , the data restoration unit 364 restores the image data representing the unknown image processed by the data processing unit 361 .

[0097] In step S36, the output unit 37 outputs the image data restored in step S35 and the estimated value y pred,i and outputting scale estimation information including:

[0098] In the estimation system 100A, the method by which the learning model construction device 3A stores the teacher data and constructs the learning model is the same as the method by which the estimation device 3 stores the teacher data and constructs the learning model. pred,i The method of calculating the estimated value y pred,i This is the same as the method for calculating

[0099] As described above, according to the first embodiment, the multiple learning units 354-k acquire the true value y true,i A learning image whose scale is known and the true scale y true,i Based on the teacher data in which the scale and the true value y true,i For an optimal validation image, which is different from the training image and for which y is known, we use multiple learning models to estimate the scale y pred,i The correlation calculation unit 356-1 also calculates the true value y true,i Multiple scale estimates of y pred,i Furthermore, the optimal learning model selection unit 357 selects the optimal learning model, which is the learning model with the highest correlation, from among the multiple learning models. In this way, the estimation device 3 and the learning model construction device 3A estimate the scale of the surface of the concrete structure with high accuracy (the estimated value y pred,i It is possible to construct a learning model that can calculate the

[0100] In particular, images of the surface of a concrete structure installed in a dark indoor place, such as a communication tunnel, contain a lot of noise. Therefore, for example, when only one loss function is used, it may be difficult to construct a learning model taking into account noise that becomes an outlier. In contrast, by using multiple loss functions as in the estimation device 3 of this embodiment, a learning model is constructed using other loss functions that are significantly affected by outliers that are difficult to consider with a single loss function. Therefore, the estimation device 3 of this embodiment can extract the true value y true,i By selecting the optimal learning model with the highest correlation with, it is possible to estimate the scale with high accuracy, taking outliers into account.

[0101] According to the first embodiment, each of the learning units 354-k constructs a plurality of learning models. true,i For training and validation images different from the training images and the optimal validation image, the scale estimate y pred,i The multiple learning units 354-k calculate the estimated values ​​y pred,i , and the true scale value y true,i A learning model is constructed based on the loss value L calculated by the loss function using the above. This enables the estimation device 3 and the learning model construction device 3A to construct a learning model that can estimate the scale of the surface of a concrete structure with even higher accuracy.

[0102] Furthermore, according to the first embodiment, the learning model storage unit 33 stores the optimal learning model selected by the optimal learning model selection unit 357. The estimation unit 363 calculates an estimated value of the scale of an unknown image, the true value of which is unknown, using the optimal learning model. This allows the estimation device 3 and the estimation device 3A to estimate the scale of the surface of a concrete structure with high accuracy.

[0103] According to the first embodiment, as described with reference to FIG. 7, the estimation device 3 constructs a first learning model and calculates a first estimated value y pred,i Calculate the true value of the scale y true,i First estimate for y pred,i After calculating the correlation between the two, we build a second learning model and estimate the second value y pred,i Calculate the true value of the scale y true,i The second estimate for y pred,i In a configuration where n is greater than or equal to 3, the estimation device 3 may similarly perform the construction of a learning model, the estimation value y pred,iThe calculation of the correlation and the calculation of the correlation may be repeated. As a result, the estimation device 3 does not need to have multiple processors, but only needs to have one processor, and can be configured simply. For the same reason, the learning model construction device 3A can also be configured simply.

[0104] According to the first embodiment, the estimation device 3 constructs a first learning model and calculates a first estimated value y pred,i Calculate the true value of the scale y true,i First estimate of y pred,i The correlation between the two is calculated, and at the same time, a second learning model is constructed and the second estimated value y pred,i Calculate the true value of the scale y true,i The second estimate for y pred,i In a configuration where n≧3, the estimation device 3 may calculate the correlation between the estimated value y pred,i The calculation of the correlation and the calculation of the correlation may be performed at the same timing. This allows the estimation device 3 to reduce the calculation time compared to the example described with reference to Fig. 7. For the same reason, the learning model construction device 3A can also reduce the calculation time.

[0105] <<Second embodiment>> The overall configuration of the second embodiment will be described with reference to Fig. 10. In the second embodiment, the same functional units as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

[0106] As shown in FIG. 10, an estimation system 100-1 according to the second embodiment includes an image capturing device 1, a data storage device 2, an estimation device 3-1, and a data retention device 4.

[0107] <Configuration of the estimation device> The estimation device 3-1 includes an input unit 31, a teacher data storage unit 32, a learning model storage unit 33, a loss function storage unit 34-k, a model construction unit 35, a scale estimation unit 36, an output unit 37, and a focus correction unit 38. The focus correction unit 38 is configured by a controller.

[0108] The focus correction unit 38 corrects image data representing an image including a learning image and an optimal verification image so that the image does not include a blurred portion. The focus correction unit 38 may correct an image further including an unknown image so that the image does not include a blurred portion. The focus correction unit 38 may correct an image further including a learning verification image so that the image does not include a blurred portion.

[0109] Specifically, the focus correction unit 38 determines whether or not there is an out-of-focus portion in the image represented by the image data received by the input unit 31. For example, the focus correction unit 38 may determine whether or not there is an out-of-focus portion in the image by using an image processing method such as edge detection using a fast Fourier transform (FFT) or Laplacian derivative, a deep learning method, or the like.

[0110] Furthermore, when the focus correction unit 38 determines that the image has an out-of-focus part, it corrects the image so that the out-of-focus part is not included. For example, the focus correction unit 38 may remove the out-of-focus part from the image, or may convert the out-of-focus part so that it is not out-of-focus. Furthermore, the focus correction unit 38 may perform a sharpening process using an unsharp mask or the like in the conversion.

[0111] The model constructing unit 35 uses the image corrected by the focus correcting unit 38 to execute the same processing as in the first embodiment.

[0112] The scale estimation unit 36 ​​uses the image corrected by the focus correction unit 38 to execute the same processing as in the first embodiment.

[0113] As shown in FIG. 11, an estimation system 100-1A according to another example of the second embodiment includes an image capturing device 1, a data storage device 2, a learning model construction device 3-1A, an estimation device 3-1B, and a data storage device 4. In the estimation system 100-1A, the learning model construction device 3-1A includes an input unit 31, a teacher data storage unit 32, a learning model storage unit 33, a loss function storage unit 34-k, a model construction unit 35, and a focus correction unit 38. In addition, the estimation device 3-1B includes an input unit 31, a learning model storage unit 33, a scale estimation unit 36, an output unit 37, and a focus correction unit 38. In the estimation system 100-1A, functional units with the same reference numerals as those in the estimation system 100-1 have the same functions. However, in the estimation system 100-1A, the focus correction unit 38 of the learning model construction device 3-1A corrects the learning image, the optimum verification image, and the learning verification image, and the focus correction unit 38 of the estimation device 3-1B corrects the unknown image.

[0114] <Operation of the estimation device> Here, the operation of the estimation device 3-1 according to the second embodiment will be described with reference to Fig. 12 and Fig. 13. Fig. 12 and Fig. 13 are flowcharts showing an example of the operation of the estimation device 3-1 according to the second embodiment. The operation of the estimation device 3-1 described with reference to Fig. 12 and Fig. 13 corresponds to an example of an estimation method of the estimation device 3-1 according to the second embodiment.

[0115] (Memorizing teacher data) A method in which the estimation device 3-1 stores training data will be described with reference to FIG.

[0116] The estimation device 3-1 executes the process of step S41. The process of step S41 is the same as the process of step S11 in the first embodiment.

[0117] In step S42, the focus correction unit 38 corrects the images including the learning image and the optimum verification image so that the images do not include out-of-focus portions.

[0118] In step S43, the scale calculation unit 351 calculates the true value y of the scale of the image corrected by the focus correction unit 38. true,i Calculate.

[0119] Thereafter, the estimation device 3-1 executes the processes of steps S44 and S45. The processes up to step S45 and step S46 are the same as the processes of steps S13 and S14 in the first embodiment.

[0120] (Building a learning model) A method in which the estimation device 3-1 constructs a learning model will be described.

[0121] The method by which the estimation device 3-1 constructs a learning model is the same as the method by which the estimation device 3 in the first embodiment constructs a learning model.

[0122] (Calculating the scale estimate) Referring to FIG. 13, the estimation device 3-1 estimates the image scale y pred,i A method for calculating the above will be described.

[0123] The estimation device 3-1 executes the process of step S51. The process of step S51 is the same as the process of step S31 in the first embodiment.

[0124] In step S52, the focus correction unit 38 corrects the unknown image so that the unknown image does not include any out-of-focus portions.

[0125] In step S53, the data processing unit 352 processes the image data representing the image corrected in step S52.

[0126] Thereafter, the estimation device 3-1 executes the processes from step S54 to step S57. The processes from step S54 to step S57 are the same as the processes from step S33 to step S36 in the first embodiment.

[0127] The method by which the learning model construction device 3-1A stores the teacher data and constructs the learning model is the same as the method by which the learning model construction device 3-1 stores the teacher data and constructs the learning model. pred,i The method of calculating the estimated value y pred,i This is the same as the method for calculating

[0128] As described above, according to the second embodiment, the estimation device 3-1 and the learning model construction device 3-1A correct images including a learning image and an optimal verification image so that the images do not include out-of-focus parts. This allows the estimation device 3-1 and the learning model construction device 3-1A to suppress a decrease in accuracy of the learning model caused by an image showing a gap, a shadow, or the like formed on a concrete surface such as a communication tunnel being out of focus. In addition, the estimation device 3-1 and the estimation device 3-1B can estimate the scale of an out-of-focus image with high accuracy.

[0129] <<Third embodiment>> The overall configuration of the third embodiment will be described with reference to Figures 14 and 15. In the third embodiment, the same functional units as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

[0130] As shown in FIG. 14, an estimation system 100-2 according to the third embodiment includes an image capturing device 1, a data storage device 2, an estimation device 3-2, and a data retention device 4.

[0131] <Configuration of the estimation device> The estimation device 3-2 includes an input unit 31, a teacher data storage unit 32, a learning model storage unit 33, a loss function storage unit 34-k, a model construction unit 35, a scale estimation unit 36, an output unit 37, and a noise image removal unit 39. The noise image removal unit 39 is configured by a controller.

[0132] As shown in FIG. 15, the noise image removal unit 39 includes a color space conversion unit 391, a noise determination unit 392, and an image removal unit 393.

[0133] The color space conversion unit 391 converts the color space of an image represented by image data received by the input unit 31. For example, if the color space of an image represented by image data received by the input unit 31 is an RGB color space, the color space conversion unit 391 converts the color space of the image from the RGB color space to the L * a * b * Convert to color space. * a * b * The L* component in the color space is a component that represents lightness, and the a* (green-red) component and the b* (blue-yellow) component are chromaticity components. In this way, the color space conversion unit 391 converts the color space of an image captured of a concrete structure as a subject into an L * a * b * By using this color space, the color of the color noise and the color of the concrete can be clearly distinguished from each other. For the same reason, the color space conversion unit 391 may convert the color space of the image into the Luv color space.

[0134] The noise determination unit 392 determines whether or not color noise is included in the image based on the chromaticity components in the color space of the image including the learning image and the optimum verification image. The noise determination unit 392 may determine whether or not color noise is included in the image based on the chromaticity components in the color space of an image further including the learning verification image. The noise determination unit 392 may determine whether or not color noise is included in the image based on the chromaticity components in the color space of an image further including an unknown image. For example, the color space conversion unit 391 converts the color space of the image into L * a * b * When converted to color space, the noise determination unit 392 * a * b * A color component in the color space, a * Component or b *Based on the pixel values ​​of the components, it is determined whether the image contains color noise.

[0135] When color noise occurs in an image, the pixel values ​​of the a* component are distributed both in a range below the first threshold (120 in this example) and in a range above a second threshold (140 in this example) that is greater than the first threshold, as shown in Fig. 16A. In contrast, when color noise does not occur in an image, the pixel values ​​of the a* component are hardly distributed in a range below the first threshold or in a range above the second threshold, as shown in Fig. 16B. Therefore, the noise determination unit 392 can determine whether or not color noise occurs in an image based on the pixel values ​​of the a* component of the pixels that constitute the image and the first and second thresholds.

[0136] Therefore, as a first example, the noise determination unit 392 determines that color noise has occurred in an image when the pixel value of the a* component of any pixel constituting the image is less than the first threshold value or greater than the second threshold value (when formula (5) is not satisfied). Also, the noise determination unit 392 determines that color noise has not occurred in an image when the pixel values ​​of the a* component of all pixels constituting the image are greater than or equal to the first threshold value and less than or equal to the second threshold value (when formula (5) is satisfied). Note that, i,j is the pixel value of the a* component of a pixel located at coordinates (i,j) in an image, where i is the x coordinate of the pixel and j is the y coordinate of the pixel.

[0137]

number

[0138] As a second example, the noise determination unit 392 may determine noise based on whether the variance of the a* component of pixel values ​​in an image is greater than a third threshold value β, as shown in equation (6). Specifically, when the variance of the a* component pixel values ​​in an image is greater than the third threshold value β (when equation (6) is satisfied), the noise determination unit 392 determines that color noise has occurred in the image. When the variance of the a* component pixel values ​​in an image is equal to or less than the third threshold value β (when equation (6) is not satisfied), the noise determination unit 392 determines that color noise has not occurred in the image. Note that, ave may be the average value of the a* components for multiple pixels constituting each of multiple images, or may be the average value of the a* components for multiple pixels constituting one image.

[0139]

number

[0140] Furthermore, when color noise occurs in an image, the pixel values ​​of the b* component are also distributed in a range below the fourth threshold (120 in this example), as shown in Fig. 17A. In contrast, when color noise does not occur in an image, the pixel values ​​of the b* component are hardly distributed in a range below the fourth threshold, as shown in Fig. 17B. For this reason, the noise determination unit 392 can determine noise in the image based on the pixel values ​​of the b* component of the pixels constituting the image and the fourth threshold.

[0141] Therefore, as a third example, when the pixel value of the b* component of any pixel constituting the image is less than the fourth threshold value (when formula (7) is not satisfied), the noise determination unit 392 determines that color noise has occurred in the image. Also, when the b* component of the pixel values ​​of all pixels constituting the image is equal to or greater than the fourth threshold value (when formula (7) is satisfied), the noise determination unit 392 determines that color noise has not occurred in the image. Note that, when the b* i,j is the pixel value of the b* component at coordinate (i,j) of the image.

[0142]

number

[0143] Furthermore, the noise determining section 392 may determine noise by using two or more of the methods described in the first to third examples above.

[0144] The image removal unit 393 removes image data representing an image determined by the noise determination unit 392 to have color noise from the set of image data. For example, the image removal unit 393 may erase image data representing an image determined to have color noise. The image removal unit 393 may also move image data representing an image determined to have color noise to a folder different from a folder (logical area in memory) that stores a plurality of image data.

[0145] The model constructing unit 35 uses the images that have not been removed by the noise image removing unit 39 to execute the same processing as in the first embodiment.

[0146] For example, the learning unit 354-k constructs a learning model in the same manner as in the first embodiment, based on teacher data in which image data indicating learning images determined not to contain color noise are associated with true values ​​of the scales of the learning images. Also, the verification unit 355-k constructs an estimated value y pred,i is calculated in the same manner as in the first embodiment.

[0147] Moreover, the scale estimation unit 36 ​​uses the image that has not been removed by the noise image removal unit 39 to execute the same process as in the first embodiment.

[0148] For example, the scale estimation unit 36 ​​may estimate the scale y pred,i is calculated in the same manner as in the first embodiment.

[0149] As shown in Fig. 18, an estimation system 100-2A according to another example of the third embodiment includes an image capturing device 1, a data storage device 2, a learning model construction device 3-2A, an estimation device 3-2B, and a data storage device 4. The learning model construction device 3-2A includes an input unit 31, a teacher data storage unit 32, a learning model storage unit 33, a loss function storage unit 34-k, a model construction unit 35, and a noise image removal unit 39. The estimation device 3-2B includes an input unit 31, a learning model storage unit 33, a scale estimation unit 36, an output unit 37, and a noise image removal unit 39. In the estimation system 100-2A, functional units with the same reference numerals as those in the estimation system 100-2 have the same functions. However, in the estimation system 100-2A, the noise judgment unit 392 of the learning model construction device 3-1A judges whether or not color noise is contained in the learning image, the optimal verification image, and the learning verification image, and the noise judgment unit 392 of the estimation device 3-2B judges whether or not color noise is contained in the unknown image.

[0150] <Operation of the estimation device> Here, the operation of the estimation device 3-2 according to the third embodiment will be described with reference to Fig. 19 and Fig. 20. Fig. 19 and Fig. 20 are flowcharts showing an example of the operation of the estimation device 3-2 according to the third embodiment. The operation of the estimation device 3-2 described with reference to Fig. 19 and Fig. 20 corresponds to an example of an estimation method of the estimation device 3-2 according to the third embodiment.

[0151] (Memorizing teacher data) A method in which the estimation device 3-2 stores training data will be described with reference to FIG.

[0152] The estimation device 3-2 executes the process of step S61. The process of step S61 is the same as the process of step S11 in the first embodiment.

[0153] In step S 62 , the color space conversion unit 391 converts the color space of the image input by the input unit 31 .

[0154] In step S63, the noise determination unit 392 determines whether or not color noise is included in the images including the learning image, the optimum verification image, and the unknown image, based on the chromaticity components in the color space of the images.

[0155] If it is determined in step S63 that color noise is included, then in step S64, the image remover 393 removes the image data from the set of multiple image data.

[0156] If it is determined in step S63 that color noise is included, then in step S65, the image remover 393 does not remove the image data from the set of multiple image data.

[0157] After step S64 is executed, the estimation device 3-2 executes the processes of steps S66 to S68. The processes of steps S65 to S68 are the same as the processes of steps S12 to S14 in the first embodiment.

[0158] When the process of step S65 or step S68 is executed, the estimation device 3-2 ends the process of storing the teacher data.

[0159] (Building a learning model) A method in which the estimation device 3-2 constructs a learning model will be described.

[0160] The method by which the estimation device 3-2 constructs a learning model is the same as the method by which the estimation device 3 in the first embodiment constructs a learning model.

[0161] (Scale Estimation) Referring to FIG. 20, the estimation device 3-2 estimates the image scale y pred,i A method for calculating the above will be described.

[0162] The estimation device 3-2 executes the process from step S71 to step S75. The process from step S71 to step S75 is the same as the process from step S61 to step S65 in the method in which the estimation device 3-2 stores training data. However, in step S71, the image data input by the input unit 31 is a scale true value y true,i is image data representing an unknown image whose

[0163] When the image data is removed in step S74, the estimation unit 3-2 calculates the estimated value y pred,i The process of calculating is then completed.

[0164] If the image data is not removed in step S75, the estimation device 3-2 executes the processes of steps S76 to S80. The processes of steps S76 to S80 are the same as the processes of steps S32 to S36 in the first embodiment.

[0165] The method by which the learning model construction device 3-2A stores the teacher data and constructs the learning model is the same as the method by which the learning model construction device 3-2 stores the teacher data and constructs the learning model. pred,i The method of calculating the estimated value y pred,i This is the same as the method for calculating

[0166] As described above, according to the third embodiment, the estimation device 3-2 and the learning model construction device 3-2A judge whether or not color noise is included in an image including a learning image, an optimum verification image, and an unknown image, based on the chromaticity components in the color space of the image. The estimation device 3-2 and the learning model construction device 3-2A estimate the learning image that is determined not to include color noise and the true value y of the scale of the learning image. true,i A learning model is constructed based on the teacher data in which the estimated value y pred,iCalculate.

[0167] When capturing an image of the surface of a concrete structure installed indoors, the shutter speed of the camera is adjusted to a slow speed so that the camera can receive more light. This can cause color noise (false colors), such as red, blue, and green, which are colors that the subject does not have in real space, to occur in the pixels that compose the image captured by the camera of the indoor concrete structure. As a result, the texture features formed on the surface of the concrete structure can be lost or broken in the image. This can result in a decrease in accuracy in estimating the scale of the image captured of the surface.

[0168] It is known to use a median filter to suppress such color noise. By using the median filter, the pixel values ​​of pixels expected to have color noise are converted based on the relationship between the central pixel in the image and neighboring pixels located in the vicinity of the central pixel. At this time, there is a possibility that the pixel values ​​of pixels included in the image that show unevenness, shadows, and the like formed on the surface of the concrete structure are converted. For this reason, even by using the median filter, it may not be possible to estimate the scale of the image with high accuracy.

[0169] In contrast, the estimation device 3-2 and the learning model construction device 3-2A in the third embodiment can suppress a decrease in accuracy of the learning model caused by using an image containing color noise. That is, the estimation device 3-2 and the learning model construction device 3-2A can estimate the scale estimate y with high accuracy by not using an image containing color noise. pred,i In addition, the estimation device 3-2 and the estimation device 3-2B can construct a learning model that can calculate the estimated value y pred,i In the process of calculating the scale, by removing images containing color noise, it is possible to prevent the scale from being estimated with low accuracy.

[0170] <First Modification> In the above-described first embodiment, the estimation device 3 and the learning model construction device 3A do not need to include the scale calculation unit 351. In such a configuration, the estimation device 3 and the learning model construction device 3A receive image data from the input unit 31 together with the true value y true,i In this way, the estimation device 3 and the learning model construction device 3A can receive the true value y true,i Since there is no need to calculate the scale, the processing load can be reduced. In addition, in the configuration without the scale calculation unit 351, the estimation device 3 and the learning model construction device 3A do not execute step S12 in the process of storing the teacher data described above.

[0171] Similarly, the estimation device 3-1 and the learning model construction device 3-1A of the second embodiment, and the estimation device 3-2 and the learning model construction device 3-2A of the third embodiment may not include the scale calculation unit 351. In such a configuration, the estimation device 3-1, the learning model construction device 3-1A, the estimation device 3-2, and the learning model construction device 3-2A do not execute steps S43 and S66 in the process of storing teacher data.

[0172] <Second Modification> In the first embodiment described above, the estimation device 3 and the learning model construction device 3A may not include the data processing unit 352, the data processing unit 361, and the data restoration unit 364. In such a configuration, as described above, the teacher data storage unit 32 stores unprocessed image data and the true value y true,i In addition, in a configuration in which the estimation device 3 and the learning model construction device 3A do not include the data processing unit 352, the data processing unit 361, and the data restoration unit 364, step S13 is not executed in the process of storing the above-mentioned teacher data. In addition, the estimation device 3 and the learning model construction device 3A do not store the above-mentioned estimated value y pred,i In the process of calculating, steps S32 and S35 are not executed.

[0173] Similarly, the estimation device 3-1 and the learning model construction device 3-1A of the second embodiment, and the estimation device 3-2 and the learning model construction device 3-2A of the second embodiment may not include the data processing unit 352, the data processing unit 361, and the data restoration unit 364. In such a configuration, the estimation device 3-1, the learning model construction device 3-1A, the estimation device 3-2, and the learning model construction device 3-2A do not execute steps S44 and S67, respectively, in the process of storing the teacher data. The estimation device 3-1 and the estimation device 3-2 do not execute steps S44 and S67, respectively, in the process of storing the teacher data. In addition, the estimation device 3-1, the learning model construction device 3-1A, the estimation device 3-2, and the learning model construction device 3-2A do not execute the estimated value y pred,i In the process of calculating, steps S53, S56, S76, and S79 are not executed.

[0174] <Program> The above-mentioned estimation devices 3, 3-1, and 3-2 can be realized by a computer 101. A program for functioning as the estimation devices 3, 3-1, and 3-2 may be provided. The program may be stored in a storage medium or provided through a network. FIG. 21 is a block diagram showing a schematic configuration of a computer 101 functioning as each of the estimation devices 3, 3-1, and 3-2. Here, the computer 101 may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, or the like. The program instructions may be program codes, code segments, or the like for performing necessary tasks. The same applies to the learning model construction devices 3A, 3-1A, and 3-2A and the estimation devices 3B, 3-1B, and 3-2B.

[0175] 21, the computer 101 includes a processor 110, a read only memory (ROM) 120, a random access memory (RAM) 130, a storage 140, an input unit 150, a display unit 160, and a communication interface (I / F) 170. Each component is connected to each other so as to be able to communicate with each other via a bus 180. The processor 110 is specifically a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), a digital signal processor (DSP), a system on a chip (SoC), or the like, and may be composed of multiple processors of the same type or different types.

[0176] The processor 110 controls each component and executes various arithmetic processing. That is, the processor 110 reads a program from the ROM 120 or the storage 140, and executes the program using the RAM 130 as a working area. The processor 110 controls each component and executes various arithmetic processing according to the program stored in the ROM 120 or the storage 140. In the above-described embodiment, the program according to the present disclosure is stored in the ROM 120 or the storage 140.

[0177] The program may be stored in a storage medium readable by the computer 101. By using such a storage medium, the program can be installed in the computer 101. Here, the storage medium in which the program is stored may be a non-transitory storage medium. The non-transitory storage medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, or a USB (Universal Serial Bus) memory. In addition, the program may be in a form that is downloaded from an external device via a network.

[0178] The ROM 120 stores various programs and various data. The RAM 130 temporarily stores programs or data as a working area. The storage 140 is configured with a hard disk drive (HDD) or a solid state drive (SSD), and stores various programs including an operating system and various data.

[0179] The input unit 150 includes one or more input interfaces that receive an input operation from a user and acquire information based on the user's operation. For example, the input unit 150 is a pointing device, a keyboard, a mouse, etc., but is not limited to these.

[0180] The display unit 160 includes one or more output interfaces that output information. For example, the display unit 160 is a display that outputs information as a video or a speaker that outputs information as a sound, but is not limited to these. Note that, if the display unit 160 is a touch panel type display, it also functions as the input unit 150.

[0181] The communication interface (I / F) 170 is an interface for communicating with an external device.

[0182] The following supplementary notes are further disclosed regarding the above embodiment.

[0183] (Additional note 1) Memory, at least one controller coupled to the memory; The controller: constructing a plurality of learning models using a plurality of mutually different loss functions based on training data in which image data representing learning images in which a concrete surface is imaged and the true value of the scale is known corresponds to the true value of the scale of the learning images; calculating a plurality of estimated values ​​of the scale for an optimal verification image different from the learning image, the true value of the scale being known, using each of the plurality of learning models; Calculating a correlation between the true value of the scale and each of the estimated values ​​of the scale for the optimal verification image; selecting an optimal learning model from among the plurality of learning models, the optimal learning model being the learning model having the highest correlation; Learning model construction device. (Additional note 2) The learning model construction device described in Appendix 1, wherein the controller trains a plurality of learning models, calculates an estimated value of the scale using each of the plurality of learning models for a learning verification image different from the learning image and the optimal verification image, and whose true value of scale is known, and constructs the learning model based on a loss value calculated by the loss function using the plurality of estimated values ​​of the scale for the learning verification image and the true value of the scale for the learning verification image. (Additional note 3) The learning model construction device according to claim 1 or 2, wherein the controller corrects images including the learning image and the optimal verification image so that the images do not include out-of-focus portions that are out of focus. (Additional note 4) The controller: determining whether or not color noise is included in an image including the learning image and the optimum verification image based on chromaticity components in a color space of the image; The learning model construction device according to any one of claims 1 to 3, which constructs the learning model based on the teacher data that associates image data indicating the learning image that is determined to not contain the color noise with a true value of the scale of the learning image. (Additional note 5) A memory that stores the optimal learning model constructed by the learning model construction device according to any one of claims 1 to 4; a controller that uses the optimal learning model to calculate an estimate of the scale of an unknown image, the true value of which is unknown; An estimation device comprising: (Additional note 6) A step of constructing a plurality of learning models using a plurality of mutually different loss functions based on training data in which image data representing learning images in which a concrete surface is captured and the true value of the scale is known is associated with the true value of the scale of the learning images; calculating, for an optimal verification image different from the learning image and having a known true value of the scale, a plurality of estimated values ​​of the scale using each of the plurality of learning models; calculating correlations of the estimated values ​​of the scales of the optimal verification image with respect to the true values ​​of the scales; selecting an optimal learning model from among the plurality of learning models, the optimal learning model being the learning model having the highest correlation; A learning model construction method comprising: (Additional note 7) An estimation method executed by an estimation device having a memory that stores the optimal learning model selected by the learning model construction method according to supplementary claim 6, The estimation method includes a step of calculating an estimate of the scale of an unknown image, the true value of which is unknown, using the optimal learning model. (Additional note 8) A non-transitory storage medium storing a program executable by a computer, the non-transitory storage medium storing the program causing the computer to function as the estimation device according to any one of claims 1 to 4.

[0184] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0185] Although the above-mentioned embodiment has been described as a representative example, it is obvious to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present disclosure. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiment, and various modifications or changes can be made without departing from the scope of the claims. For example, it is possible to combine multiple building blocks shown in the configuration diagram of the embodiment into one, or to divide one building block. [Explanation of symbols]

[0186] 1. Imaging device 2 Data storage device 3, 3-1, 3-2 Estimation device 3A, 3-1A, 3―2A Learning model construction device 3B, 3-1B, 3-2B estimation device 4 Data storage devices 31 Input section 32 Teacher data storage unit 33 Learning model memory unit 34-k Loss function memory 35 Model Construction Department 36 Scale Estimation Unit 37 Output section 38 Focus correction section 39 Noise image removal section 100, 100-1, 100-2 Estimation Systems 100A, 100-1A, 100-2A Estimation System 101 Computer 110 Processor 120 ROM 130 RAM 140 Storage 150 Input section 160 Output section 170 Communication Interface 180 Bus 351 Scale Calculation Unit 352 Data Processing Department 353 Learning model reading part 354-k Learning Department 355-k Verification Department 356-k Correlation calculation section 357 Optimal Learning Model Selection Unit 361 Data Processing Department 362 Learning model reading part 363 Estimation Department 364 Data Recovery Department 391 Color Space Conversion Unit 392 Noise Judgment Unit 393 Image Removal Unit

Claims

1. a plurality of learning units that construct a plurality of learning models using a plurality of mutually different loss functions based on teacher data that associates image data representing learning images in which a true value of the scale is known, the learning images being images of a concrete surface; a plurality of verification units that calculate a plurality of estimated values ​​of the scale by using the plurality of learning models for an optimal verification image different from the learning image, the true value of the scale being known; a plurality of correlation calculation units that calculate correlations of the plurality of estimated values ​​of the scale with respect to a true value of the scale for the optimal verification image; an optimal learning model selection unit that selects an optimal learning model, which is a learning model having the highest correlation, from among the plurality of learning models; A learning model construction device comprising:

2. 2. The learning model construction device according to claim 1, wherein the multiple learning units each learn a multiple learning model, calculate an estimated value of the scale for a learning verification image different from the learning image and the optimal verification image, and whose true value of scale is known, using each of the multiple learning models, and construct the learning model based on a loss value calculated by the loss function using the multiple estimated values ​​of the scale for the learning verification image and the true value of the scale for the learning verification image.

3. The learning model construction device according to claim 1 or 2, further comprising a focus correction unit that corrects images including the learning image and the optimal verification image so that the images do not include out-of-focus portions that are out of focus.

4. a noise determination unit that determines whether or not color noise is included in an image including the learning image and the optimum verification image based on chromaticity components in a color space of the image, 4. The learning model construction device according to claim 1, wherein the learning unit constructs the learning model based on the teacher data that associates image data indicating the learning image that is determined to not contain the color noise with a true value of the scale of the learning image.

5. a learning model storage unit that stores the optimal learning model selected by the learning model construction device according to any one of claims 1 to 4; an estimation unit that calculates an estimated value of the scale of an unknown image, the true value of which is unknown, using the optimal learning model; An estimation device comprising:

6. A step of constructing a plurality of learning models using a plurality of mutually different loss functions based on training data in which image data representing learning images in which a concrete surface is captured and the true value of the scale is known is associated with the true value of the scale of the learning images; calculating a plurality of estimated values ​​of the scale for an optimal verification image different from the learning image, the true value of the scale being known, using each of the plurality of learning models; calculating correlations of the estimated values ​​of the scales of the optimal verification image with respect to the true values ​​of the scales; selecting an optimal learning model from the plurality of learning models, the optimal learning model being the learning model having the highest correlation; A learning model construction method comprising:

7. An estimation method executed by an estimation device having a learning model storage unit that stores the optimal learning model selected by the learning model construction method according to claim 6, comprising: The estimation method includes a step of calculating an estimate of the scale of an unknown image, the true value of which is unknown, using the optimal learning model.

8. A program for causing a computer to function as the learning model construction device according to any one of claims 1 to 4.

Images