Image classification determination system, image classification determination method, and program therefor

Abstract

To provide an image classification determination system capable of determining, by using an image, a classification to which the image belongs, from among a plurality of classifications defined in advance, and also to provide an image classification determination method and a program therefor.SOLUTION: An image classification determination system makes a computer 10 execute processing of determining whether a photographic image 3a belongs to a class D in a bedrock classification. A first teacher data output part 2a outputs similarity determination teacher data 4b. A second teacher data output part 2b outputs non-similarity determination teacher data 4c. A model generation part 2c generates, according to the similarity determination teacher data 4b and the non-similarity determination teacher data 4c, a separate hyperplane P1 by a support vector machine. A determination part 2d uses the separate hyperplane P1 and outputs a determination result showing whether or not the photographic image 3a belongs to the class D in the classification.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image classification determination system, an image classification determination method, and a program therefor, which are used to determine to which of a plurality of predefined classifications an image captured by a camera belongs, for example. [Background technology]

[0002] Traditionally, in the construction and civil engineering industries, rock mass classification observations are conducted based on the condition of borehole cores and tunnel faces as part of the investigations when constructing buildings or excavating tunnels. In these observations, engineers use a hammer to strike the area to be investigated, or observe the shape and condition of cracks to qualitatively determine which of multiple predefined classifications the rock mass belongs to. Since there are many judgment items and it requires skilled training, there are problems with judgments being easily inconsistent depending on the experience and technical ability of the engineers, or judgments are often based on the subjective judgments of the engineers.

[0003] In order to avoid such qualitative judgment based on the subjective judgment of the engineer, for example, Patent Document 1 discloses a method of quantitatively classifying rocks by image processing of geological images. This method grasps the weathering and crack conditions underground by numerical analysis values ​​obtained from the drilling energy obtained when forming a borehole in a rock mass and numerical analysis values ​​based on various data obtained by photographing the inside of the borehole with a hyperspectral camera and processing the geological images in the borehole obtained by the photographing. Specifically, the engineer first classifies the geological images by a predetermined depth, creates a frequency distribution of the area of ​​the rock types occupying the divided images, and then performs an evaluation work in which the most occupying rock type is set as the representative value for the depth. Next, an evaluation work is performed in which the degree of weathering and the spacing of the cracks are also numerically classified by depth based on various data obtained from the geological images. Furthermore, after an evaluation process is performed in which the average value of drilling energy for each specified depth is calculated and the average value is numerically classified, a rock class classification is determined based on a combination of these evaluation results, which makes it possible to properly grasp the underground situation and accurately determine the rock class. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2022-169055 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Patent Document 1, although it is considered that the rock classification can be quantitatively determined, there is a problem that a lot of processing is required before the rock classification can be determined from the acquired geological image, and a lot of time is spent on evaluating each geological image. In addition, there is a problem that a lot of processing is required to make the determination, and the work is complicated. Furthermore, in the case of Patent Document 1, a hyperspectral camera needs to be used to obtain a lot of data from the geological image, and there is also a problem that the cost of the determination work is high. This problem is not limited to rock classification as described above, but also applies to other technical fields where an experienced technician cannot determine which of multiple defined classifications a rock belongs to without looking at the actual object, when it is desired to make an efficient judgment using acquired images. The present invention has been made in consideration of the above points, and its object is to provide an image classification determination system, an image classification determination method, and a program therefor that are capable of using an image to determine which of a plurality of pre-defined classifications an image belongs to. [Means for solving the problem]

[0006] In order to achieve the above object, the present invention is characterized in that it determines whether or not a target image belongs to a predetermined classification using a trained model through machine learning. Specifically, the present invention targets an image classification determination system that causes a computer to execute a process for determining to which of a plurality of defined classifications an image to be determined belongs, and the following measures have been taken.

[0007] That is, the image classification determination system according to the first invention is characterized in that it comprises a first teacher data output unit that outputs, as similarity determination teacher data, a data group of similarity data calculated respectively between at least one first learning image belonging to a predetermined classification and at least one first sample image belonging to the predetermined classification, and between at least one first learning image belonging to the predetermined classification and at least one second sample image belonging to another classification; a second teacher data output unit that outputs, as dissimilarity determination teacher data, a data group of similarity data calculated respectively between at least one second learning image belonging to the other classification and at least one first sample image belonging to the predetermined classification, and between at least one second learning image belonging to the other classification and at least one second sample image belonging to the other classification; a model generation unit that generates a trained model by machine learning based on the similarity determination teacher data and the dissimilarity determination teacher data; and a determination unit that uses the trained model to output a determination result as to whether the image to be determined belongs to the predetermined classification or not. The image classification determination system configured in this manner operates so that the learned model can determine the boundaries between a region of a given classification and regions of other classifications with the same accuracy as a skilled technician. It also operates to reduce the number of processes required for the determination. Furthermore, it operates so that the determination can be made using images captured by a general camera.

[0008] The present invention also targets an image classification determination method for determining to which of a plurality of defined classifications an image to be determined belongs, and provides the following solution. That is, the image classification determination method of the second invention is characterized in that it uses as similarity determination teacher data a data group of similarity data calculated between at least one first learning image belonging to a predetermined classification and at least one first sample image belonging to the predetermined classification, and between at least one first learning image belonging to the predetermined classification and at least one second sample image belonging to another classification, while using as dissimilarity determination teacher data a data group of similarity data calculated between at least one second learning image belonging to the other classification and at least one first sample image belonging to the predetermined classification, and between at least one second learning image belonging to the other classification and at least one second sample image belonging to the other classification, and then generates a trained model by machine learning based on the similarity and dissimilarity determination teacher data, and then causes a computer to execute a process of outputting a determination result as to whether the image to be determined belongs to the predetermined classification using the trained model. The image classification judgment method configured in this manner operates so that the judgment of the boundary between the area of ​​a predetermined classification and the area of ​​other classifications by the trained model is as accurate as that of a skilled technician, and also operates to reduce the number of processes required for judgment.Furthermore, it operates so that judgment can be made using images captured by a general camera, etc.

[0009] The present invention also targets an image classification determination program that causes a computer to execute a process for determining to which of a plurality of defined classifications an image to be determined belongs, and provides the following solution. That is, the image classification determination program according to the third invention determines, as similarity determination teacher data, data groups of similarity data calculated between at least one first learning image belonging to a predetermined classification and at least one first sample image belonging to the predetermined classification, and between at least one first learning image belonging to the predetermined classification and at least one second sample image belonging to another classification, while determining, as dissimilarity determination teacher data, data groups of similarity data calculated between at least one second learning image belonging to the other classification and at least one first sample image belonging to the predetermined classification, and between at least one second learning image belonging to the other classification and at least one second sample image belonging to the other classification, The method is characterized in that the computer is caused to execute a process of outputting a judgment result as to whether or not the judgment target image belongs to the predetermined classification using a trained model generated by machine learning based on the similarity and dissimilarity judgment teacher data. The image classification judgment program configured in this manner operates so that the trained model judges the boundary between the area of ​​the predetermined classification and the area of ​​other classifications with the same accuracy as that of a skilled technician. It also operates so that the number of processes required for making the judgment is reduced. Furthermore, it operates so that the judgment can be made using images captured by a general camera, etc.

[0010] The image classification determination system according to a fourth invention is characterized in that, in the first invention, the similarity data calculated by the first and second teacher data output units is configured to be calculated by a color histogram method. The image classification determination system configured in this manner operates by utilizing the color differences and characteristics of the images to create a similarity that indicates whether or not the images are similar to each other.

[0011] The image classification determination system according to a fifth invention is characterized in that, in the fourth invention, the model generation unit performs machine learning to generate the trained model using a support vector machine. The image classification determination system configured in this manner operates to make it possible to create a separating hyperplane that accurately separates an area of ​​a predetermined classification from areas of other classifications.

[0012] The image classification determination system according to a sixth aspect of the present invention is the image classification determination system according to the fifth aspect of the present invention, characterized in that the classification is a bedrock classification. The image classification determination system configured in this manner, for example, acts to eliminate the need for a skilled engineer to directly look at borehole cores or tunnel faces and perform subjective, qualitative evaluations of rock mass classification. Effect of the Invention

[0013] In the first to third inventions, the learning image used to create the trained model is provided with a set of similarity data showing how similar the image is using a similar sample image, and similarity data showing how dissimilar the image is using a dissimilar sample image, and the trained model is created by machine learning. This allows the trained model to determine the boundary between a predetermined classification area and other classification areas with the same accuracy as a skilled technician. In addition, when creating a trained model, not only similar sample images but also dissimilar sample images are used for one training image, so that the amount of information that one training image has when creating the trained model is increased, and even if the image quality of the training image or sample image is low resolution, a versatile and highly accurate trained model is created. Therefore, when a computer is used to determine an image to be determined using the above-mentioned trained model, the classification can be determined accurately in a short time, and the time spent on image evaluation can be shortened. In addition, the number of processes required for determination is reduced compared to the process of Patent Document 1, so that the determination work can be easily performed. Furthermore, since it is now possible to make a judgment based on images captured by a general camera, and there is no need to capture the images to be judged using a hyperspectral camera as in Patent Document 1, the costs associated with the judgment work can be kept low.

[0014] The image classification determination system of the fourth invention can create a similarity that indicates whether images are similar or not by utilizing the color differences and characteristics of the images, so that the determination process can be performed with high accuracy, especially in cases where color differences in images are important in determining classification.

[0015] The image classification determination system of the fifth invention is capable of creating a separating hyperplane that accurately separates areas of a specified classification from areas of other classifications, and can therefore accurately determine whether an image to be determined belongs to a specified classification.

[0016] The image assessment system of the sixth invention, for example, eliminates the need for experienced engineers to directly view borehole cores or tunnel faces and perform subjective, qualitative evaluations to classify rock mass, and can perform objective, quantitative rock mass classification without visiting the site. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram of an image classification determination system according to a first embodiment of the present invention. [Diagram 2] 4 is a first half of a flowchart showing the machine learning operation of the image classification determination system according to the first embodiment of the present invention. [Diagram 3] 11 is a second half of a flowchart showing the machine learning operation of the image classification determination system according to the first embodiment of the present invention. [Figure 4] 4 is a flowchart showing an image classification determination operation performed by the image classification determination system according to the first embodiment of the present invention. [Diagram 5] 11 is an explanatory diagram regarding a generation process of similarity and dissimilarity determination teaching data. FIG. [Figure 6] FIG. 11 is an explanatory diagram relating to a method of calculating similarity data. [Figure 7] FIG. 11 is an explanatory diagram relating to a generation process of classification determination teaching data. [Figure 8] FIG. 11 is an explanatory diagram regarding the generation process of a separating hyperplane P1. [Figure 9] FIG. 11 is an explanatory diagram regarding the generation process of a separating hyperplane P2. [Figure 10] FIG. 11 is a diagram showing a comparison between the results of rock mass classification performed by the image classification determination system according to the first embodiment of the present invention and the results of rock mass classification performed by an engineer. [Figure 11] FIG. 1 is a conceptual diagram illustrating the theory of the present invention in which a separating hyperplane capable of performing highly accurate judgment is generated in machine learning using a support vector machine. [Figure 12] FIG. 11 is a diagram showing a state in which an unmanned aerial vehicle of the image classification determination system according to the second embodiment of the present invention is acquiring an image to be determined. [Figure 13] (a) is a diagram showing a worker performing repair work on the impermeable wall of a reservoir where the unmanned aerial vehicle has acquired an image to be judged, (b) is a diagram showing an unsound condition of the impermeable wall called blistering, and (c) is a diagram showing an unsound condition of the impermeable wall called mud curling. [Figure 14] FIG. 11 is a diagram comparing the results of classification of a reservoir's impermeable wall surface into healthy and unsound parts by the image classification determination system according to Example 2 of the present invention with the results of classification of a reservoir's impermeable wall surface into healthy and unsound parts by an engineer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, in which: It should be noted that the following description of the preferred embodiments is merely exemplary in nature. EXAMPLES

[0019] FIG. 1 shows a block diagram of an image classification determination system 1 according to a first embodiment of the present invention. The image classification determination system 1 uses a photographed image 3a (a determination target image) to classify rock masses of borehole cores and tunnel faces (for example, a D-class classification for the Denken system, a C-class classification for the Denken system ... L Classification: C M This classifies rock mass into D and C classes. L Grade and C M The computer 10 is caused to execute a process of determining which of the classes the object belongs to. The computer 10 includes an information processing unit 2 that performs machine learning processing and classification judgment processing, an input database 3 that stores data input to the information processing unit 2 via a keyboard, mouse, etc., a teacher database 4 that stores various data used in machine learning, and an output database 5 that stores classification judgment processing result data 5a.

[0020] The input database 3 includes a photographed image 3a to be judged and a sample image Z to be used for machine learning. i (i is a natural number) and training image X k (k is a natural number) and is designed to be preserved. Sample image Z i In the first embodiment of the present invention, as shown in FIG. 5, three D-class sample images Z1 to Z3 belonging to the D-class classification and C L The three C's in the classification L Sample images of grades Z4 to Z6 and C M The three C's in the classification M It consists of grade sample images Z7 to Z9. On the other hand, training image X k In the first embodiment of the present invention, D-class learning images X1, X2, ... X 24 And, C L Class C L Class learning image X 25 , X 26 , …X 60 And, C M Class C M Class learning image X 61 , X 62 , …X 90 It is composed of:

[0021] As shown in FIG. 1, the teacher database 4 stores similarity data 4a, similarity determination teacher data 4b, dissimilarity determination teacher data 4c, and separating hyperplanes P1 and P2 (trained models). The similarity data 4a is the learning image X k and sample image Z i The larger the value, the more similar the two are judged to be. As shown in FIGS. 5 and 7, the similarity judgment training data 4b includes learning images X1, X2, . . . X belonging to the D-class classification. 24 between sample images Z1 to Z3 belonging to the D-class classification and learning images X1, X2, ... X 24 and C. M Grade and C L The data groups Y1, Y2, ... Y 24 It is composed of:

[0022] Dissimilarity judgment training data 4c is C M Grade and C L Learning image X belongs to the class classification 25 , X 26 , …X 90 Between sample images Z1 to Z3 belonging to the D class classification and C M Grade and C L Learning image X belongs to the class classification 25 , X 26 , …X 90 and C. M Grade and C L A data group Y of each similarity data 4a calculated between sample images Z4 to Z9 belonging to the same classification level 25 , Y 26 , … Y 90 It is composed of: As shown in Figure 8, the separating hyperplane P1 is a trained model trained by machine learning using a support vector machine (SVM), and is a model that is trained by dividing the data set Y1, Y2, ... Y 24 And the other similarity data 4a data group Y 25 , Y 26 , … Y 90 This shows the boundary between As shown in Figure 9, the separating hyperplane P2 is a trained model trained by machine learning using a support vector machine (SVM), and C L Data group Y of class similarity data 4a 25 , Y 26 , … Y 60 And, C M Data group Y of class similarity data 4a 61 , Y62 , … Y 90 This shows the boundary between In the first embodiment, in order to create a general-purpose trained model, the data groups Y1, Y2, ... Y 90 The optimal separating hyperplanes P1 and P2 are derived using a soft margin method that allows for a certain degree of misjudgment.

[0023] As shown in FIG. 1, the information processing unit 2 is made up of a first teacher data output unit 2a, a second teacher data output unit 2b, a model generation unit 2c, and a judgment unit 2d. The first teacher data output unit 2a outputs training images X1, X2, ... X 24 Similarity data 4a of sample images Z1 to Z3 similar to and learning images X1, X2, ... X belonging to the D-class classification 24 A data set Y1, Y2, ... Y 24 The similarity data 4a is calculated using a color histogram method as shown in FIG. 6. k and sample image Z i For each pixel of RGB, the frequency of the values ​​for each pixel is examined and made into a histogram, and for example, the difference in relative frequency between the R histogram of learning image X and the R histogram of sample image Z is examined, and if the value is large, it is determined that they are similar, and if the value is small, it is determined that they are not similar. When determining whether an image belongs to the D-class classification or another classification, the first teacher data output unit 2a outputs the learning images X1, X2, ...X1 belonging to the D-class classification. 24 between sample images Z1 to Z3 belonging to the D-class classification and learning images X1, X2, ... X 24 and the sample images Z4 to Z9 belonging to the other classifications, data groups Y1, Y2, ... Y 24 is output as the similarity judgment teacher data 4b. In this case, the learning images X1, X2, ... X 24constitutes the first learning image of the present invention, sample images Z1 to Z3 constitute the first sample image of the present invention, and sample images Z4 to Z9 constitute the second sample image of the present invention. Also, C L Does it belong to the C grade? M When determining whether a product belongs to the C class, L Learning image X belongs to the class classification 25 , X 26 , …X 60 and C. L Between sample images Z4 to Z6 belonging to the class classification and C L Learning image X belongs to the class classification 25 , X 26 , …X 60 A data group Y of each similarity data 4a obtained by calculating the similarity between the sample images Z1 to Z3 and Z7 to Z9 belonging to the other classifications 25 , Y 26 , … Y 60 is output as the similarity judgment training data 4b. In this case, the learning image X 25 , X 26 , …X 60 constitute the first learning image of the present invention, sample images Z4 to Z6 constitute the first sample image of the present invention, and sample images Z1 to Z3 and Z7 to Z9 constitute the second sample image of the present invention.

[0024] The second teacher data output unit 2b is C L Grade and C M Learning image X belongs to the class classification 25 , X 26 , …X 90 Similarity data 4a of sample images Z1 to Z3 that are not similar to C L Grade and C M Learning image X belongs to the class classification 25 , X 26 , …X 90 Data group Y of combinations of sample images Z4 to Z9 similar to each other and similarity data 4a 25 , Y 26 , … Y 90The second teacher data output unit 2b is adapted to calculate each similarity data 4a using a color histogram method, similar to the first teacher data output unit 2a. When determining whether the data belongs to the D-class classification or another classification, the second teacher data output unit 2b outputs C L Grade and C M Learning image X belongs to the class classification 25 , X 26 , …X 90 Between sample images Z1 to Z3 belonging to the D class classification and C L Grade and C M Learning image X belongs to the class classification 25 , X 26 , …X 90 and C. L Grade and C M A data group Y of each similarity data 4a calculated between sample images Z4 to Z9 belonging to the same classification level 25 , Y 26 , … Y 90 is output as the dissimilarity judgment training data 4c. In this case, the learning image X 25 , X 26 , …X 90 constitutes the second learning image of the present invention, sample images Z1 to Z3 constitute the first sample image of the present invention, and sample images Z4 to Z9 constitute the second sample image of the present invention. Also, C L Does it belong to the C grade? M When determining whether a product belongs to the C class, M Learning image X belongs to the class classification 61 , X 62 , …X 90 and C. L Between sample images Z4 to Z6 belonging to the class classification and C M Learning image X belongs to the class classification 61 , X 62 , …X 90 and D and C M A data group Y of each similarity data 4a calculated between the sample images Z1 to Z3 and Z7 to Z9 belonging to the class classification 61 , Y 62 , … Y 90is output as the dissimilarity judgment training data 4c. In this case, the learning image X 61 , X 62 , …X 90 constitute the second learning image of the present invention, sample images Z4 to Z6 constitute the first sample image of the present invention, and sample images Z1 to Z3 and Z7 to Z9 constitute the second sample image of the present invention.

[0025] As shown in Figures 7 and 8, when determining whether an object belongs to the D-class classification or another classification, the model generation unit 2c is configured to generate a separating hyperplane P1, which is a learning model based on machine learning, using a support vector machine based on the similarity judgment teacher data 4b and the dissimilarity judgment teacher data 4c. In addition, as shown in FIG. 9, the model generating unit 2c L Does it belong to the C grade? M When determining whether an object belongs to a class of classification, a separating hyperplane P2, which is a learning model based on machine learning, is generated by a support vector machine based on the similarity determination teacher data 4b and the dissimilarity determination teacher data 4c. Note that a specific processing method using a support vector machine is described in detail in, for example, Patent Document No. 2007-35017, and therefore a description thereof will be omitted here.

[0026] As shown in Figures 1 and 4, when judging whether a captured image 3a belongs to the D-class classification or another classification, the judgment unit 2d is configured to output a judgment result as to whether the captured image 3a belongs to the D-class classification or not using a separating hyperplane P1, which is a learning model. As shown in FIG. 1 and FIG. 4, the determination unit 2d L Does it belong to the C grade? M When determining whether the captured image 3a belongs to the C class, the separating hyperplane P2, which is a learning model, is used to determine whether the captured image 3a belongs to the C class. L Does it belong to the C grade? M The system is designed to output the result of the judgment as to whether the object belongs to a certain grade classification. The output database 5 stores the classification judgment processing result data 5a outputted by the judgment section 2d, and is capable of displaying the data on a display or the like, for example.

[0027] Next, an operation of creating a trained model by machine learning using a support vector machine in the information processing unit 2 of the image classification determination system 1 according to the first embodiment of the present invention will be described in detail. i There are a maximum of m images, and the training images X k 2 to 4 also show the steps executed by computer 10 for the image classification determination program of the present invention, and explaining the flow of data processing in image classification determination system 1 with reference to these figures is equivalent to explaining an embodiment of the image classification determination program.

[0028] As shown in FIG. 2, first, in step SA1, 1 is substituted for the number k of the learning image X, and then the process proceeds to step SA2 where 1 is substituted for the number i of the sample image Z, and then the process proceeds to step SA3. In step SA3, sample image Z i After creating histograms for each of the RGB values ​​of the learning image X k Then, a histogram is created for each of the RGB values, and the process proceeds to step SA5.

[0029] In step SA5, the similarity data 4a is calculated using a color histogram method. k and sample image Z i The frequency of the values ​​for each pixel is examined for RGB for each pixel, and a histogram is created. The value is then calculated by looking at the difference in relative frequency between the histograms for each RGB. After calculating the similarity data 4a in step SA5, the process proceeds to step SA6, where the learning image X k Determine whether it belongs to the D class classification. If the determination in step SA6 is YES, that is, if the learning image X kIf it is determined that the learning image X is classified as D class, the process proceeds to step SA7. k A predetermined teacher value indicating that the similarity data 4a corresponding to the above is classified as D class is assigned to the similarity data 4a to be used as similarity judgment teacher data 4b, and the process proceeds to step SA9. On the other hand, if the determination in step SA6 is NO, that is, if the learning image X k C L Grade or C M If it is determined that the classification is of the class, the process proceeds to step SA8, and the learning image X k A predetermined teacher value indicating that the similarity data 4a corresponding to this is not classified as Class D is assigned to the similarity data 4a to be used as dissimilarity judgment teacher data 4c, and the process proceeds to step SA9.

[0030] In step SA9, sample image Z i If the determination in step SA9 is NO, that is, if the sample image Z i If the mth image has not yet been reached, proceed to step SA10, substitute i+1 for i, and then return to step SA3 to select the next sample image Z i The process is repeated for . On the other hand, if the determination in step SA9 is YES, that is, if the sample image Z i When the number of sheets reaches the mth sheet, the process proceeds to step SA11. In step SA11, the learning image X k If the determination in step SA11 is NO, that is, if the learning image X k If the nth sample image has not yet been reached, proceed to step SA12, substitute k+1 for k, and then return to step SA2 to select the next sample image Z i The process is repeated for . On the other hand, if the determination in step SA11 is YES, that is, if the learning image X k When the number of sheets reaches n, the process proceeds to step SA13.

[0031] In step SA13, the similarity judgment teacher data 4b and the dissimilarity judgment teacher data 4c are input to the model generation unit 2c, and then the process proceeds to step SA14. In step SA14, a separating hyperplane P1 between the D-class classification and other classifications is created by machine learning using a support vector machine, and the process proceeds to step SA15. In step SA15, as shown in FIG. 3, a learning image X belonging to the D-class classification is k For example, the data shown in FIG. 8 has similarity data 4a of learning images X1, X2, . . . belonging to the D-class classification, but as shown in FIG. 9, the data shown in FIG. 9 has similarity data 4a of learning images X1, X2, . . . belonging to the D-class classification, but as shown in FIG. 9, the data shown in FIG. L Grade and C M Learning image X belongs to the class classification 25 , X 26 Continue with step SA16.

[0032] Returning to FIG. 3, in step SA16, the number k of the learning image X is reset to 1, and then the process proceeds to step SA17, where the number i of the sample image Z is reset to 1, and the process proceeds to step SA18. In step SA18, the learning image X k C L If the determination in step SA18 is YES, that is, if the learning image X k C L If it is determined that the classification is of the class, the process proceeds to step SA19, and the learning image X k The similarity data 4a corresponding to C L A predetermined teacher value indicating that it is a grade classification is assigned to the similarity judgment teacher data 4b, and the process proceeds to step SA21. On the other hand, if the determination in step SA18 is NO, that is, if the learning image X k C M If it is determined that the classification is of the class, the process proceeds to step SA20, and the learning image X k The similarity data 4a corresponds to C LA predetermined teacher value indicating that the data is not a class classification is assigned to the data, which is treated as dissimilarity determination teacher data 4c, and the process proceeds to step SA21.

[0033] In step SA21, sample image Z i It is determined whether the mth image has been reached. If the determination in step SA21 is NO, that is, if the sample image Z i If the mth sample image has not yet been reached, the process proceeds to step SA22, where i+1 is substituted for i, and the process returns to step SA18 to select the next sample image Z i The process is repeated for . On the other hand, if the determination in step SA21 is YES, that is, if the sample image Z i When the number of sheets reaches the mth sheet, the process proceeds to step SA23.

[0034] In step SA23, the learning image X k If the determination in step SA23 is NO, that is, if the learning image X k If the nth sample image Z has not yet been reached, the process proceeds to step SA24, where k+1 is substituted for k, and the process returns to step SA17. i The process is repeated for . On the other hand, if the determination in step SA23 is YES, that is, if the learning image X k When the number of sheets reaches n, the process proceeds to step SA25. In step SA25, the similarity judgment teacher data 4b and the dissimilarity judgment teacher data 4c are input to the model generation unit 2c, and then the process proceeds to step SA26. In step SA26, C is generated by machine learning using a support vector machine. L Classification and C M A separating hyperplane P2 between the class classification is created, and the processing of the information processing unit 2 is terminated.

[0035] Next, an image classification determination operation by the determination unit 2d of the image classification determination system 1 according to the first embodiment of the present invention will be described in detail. In step SB1, the borehole core, tunnel face, etc., which are the objects of observation, are photographed to obtain a photographed image 3a, and then the process proceeds to step SB2. After creating histograms for each of the RGB of the photographed image 3a in step SB2, the process proceeds to step SB3, where similarity data Q is calculated using a color histogram method. i The frequency of the values ​​for each pixel is examined for RGB for each pixel, and a histogram is created. The value is then calculated by looking at the difference in relative frequency between the histograms for each RGB.

[0036] After calculating the similarity data Q in step SB3, the process proceeds to step SB4, where the separating hyperplane P1 created by the model generating unit 2c is received, and the process proceeds to step SB5. In step SB5, the similarity data Q is compared with the separating hyperplane P1, and the process proceeds to step SB6. In step SB6, it is determined whether or not the captured image 3a belongs to the classification of class D. If the determination in step SB6 is YES, that is, if it is determined that the captured image 3a is classified as class D, the process proceeds to step SB7, where it is determined that the captured image 3a is classified as class D, and the process of the determination unit 2d is terminated.

[0037] On the other hand, if the determination in step SB6 is NO, that is, if it is determined that the captured image 3a is not classified as class D, the process proceeds to step SB8, where the separating hyperplane P2 created by the model generating unit 2c is received. After that, the process proceeds to step SB9, where the similarity data Q is compared with the separating hyperplane P2, and the process proceeds to step SB10. In step SB10, the captured image 3a is L If the determination in step SB10 is YES, that is, the photographed image 3a is classified as C L If it is determined that the captured image 3a is classified as class C, the process proceeds to step SB11. L The process of the determination unit 2d is terminated as a result of classifying the object into classes. On the other hand, if the determination in step SB10 is NO, that is, the photographed image 3a is C M If it is determined that the captured image 3a is classified as class C, the process proceeds to step SB12. M The process of the determination unit 2d is terminated as a result of classifying the object into classes.

[0038] Next, a detailed description will be given of the results of observing the photographed image 3a of a predetermined boring core using the image classification determination system 1 according to the first embodiment of the present invention. Figure 10 shows the results of observing a certain borehole core using different rock classification methods. L Grade and C M The results were then judged according to the grade classification, color-coded and plotted in a columnar diagram, and the accuracy rate was calculated for each classification result compared with that of a technician using the conventional method.

[0039] The result RA1 shows the result of rock classification from the photographed image 3a by creating a separating hyperplane by machine learning using a support vector machine. When making a D-class classification judgment, each learning image X k Using the separating hyperplane obtained by adopting the similarity data 4a between only sample images Z1 to Z3 belonging to the D-class classification as features, C L When classifying the class, each learning image X k and C. L A separating hyperplane was used that was obtained by using only sample images Z4 to Z6 that belong to the class classification as features. As a result, as shown in RA1, the overall accuracy rate was 33%, and the learning image X used for machine learning was k In order to create a separating hyperplane that determines a given classification, the necessary information in i It has been found that it is difficult to create boundary data (separating hyperplane) for determining the classification of the photographed image 3a based only on the similarity data 4a.

[0040] The result RA2 shows the result obtained by performing rock classification from the photographed image 3a using the image classification determination system 1 of the embodiment 1 of the present invention. When performing the D-class classification determination, each learning image X k Not only sample images Z1 to Z3 that belong to the D class, but also C L Grade and C MThe similarity data 4a for sample images Z4 to Z9 belonging to the class classification is also adopted as a feature, and the separating hyperplane obtained is used to L When classifying the class, each learning image X k and C. L Not only sample images Z4 to Z6 belonging to the grade classification, but also C M The similarity data 4a for the sample images Z7 to Z9 belonging to the class classification was also adopted as a feature quantity, and a separating hyperplane was obtained. As a result, the overall accuracy rate was 69%, and it was found that the image classification determination system 1 of the first embodiment of the present invention can perform rock classification with high determination accuracy from the captured image 3a.

[0041] Here, the theory (idea) that enables the image classification determination system 1 of the present invention to generate a trained model (separating hyperplanes P1, P2) capable of performing rock mass classification with high determination accuracy as described above will be described. As shown in FIG. 11(a), when a separating hyperplane is to be generated for determining whether a person in the captured image 3a belongs to Asian or non-Asian, if a separating hyperplane capable of accurate classification is W1, for example, if sample images of only Japanese people are used and a separating hyperplane is generated using similarity data 4a of only these sample images of Japanese people as features, a separating hyperplane will be generated between Japanese and Chinese, as in W2, and there is a risk that the separating hyperplane will determine that even a captured image 3a of a Chinese person, which any normal person would determine to belong to the Asian category, does not belong to the Asian category. However, if a separating hyperplane is generated using similarity data 4a for sample images of Caucasian and Black people as features in addition to sample images of Japanese people, the separating hyperplane W3 will be generated closer to W1, as shown in FIG. 11(b), and it is considered that this will be a separating hyperplane that can determine that Chinese people belong to the Asian category.

[0042] In this way, according to the first embodiment of the present invention, the learning image X used to create a trained model such as the separating hyperplane P1 or the separating hyperplane P2 is k About Similar Sample Images Z i The similarity data 4a shows how similar the images are, and the dissimilar sample image Z i The separating hyperplanes P1 and P2 are created by machine learning with the similarity data 4a, which indicates how similar the images are. Therefore, the separating hyperplane P1 can determine the boundary between the D-class classification area and the other classification areas, and the separating hyperplane P2 can determine the C-class classification area. L Classification area and C M The judgment of the boundary between the classification areas of the different classes is as accurate as that of a skilled technician. Therefore, when the computer 10 judges the captured image 3a, which is the image to be judged, using the above-mentioned separating hyperplanes P1 and P2, the classification judgment can be performed accurately in a short time, and the time spent on image evaluation can be shortened. In addition, since the number of processes for making the judgment is reduced compared to the process of Patent Document 1, the judgment work can be performed easily. Furthermore, since the judgment can be performed using images captured by a general camera, and there is no need to capture the image to be judged by a hyperspectral camera as in Patent Document 1, the cost of the judgment work can be kept low.

[0043] Furthermore, since the similarity data 4a is calculated using the color histogram method, it is possible to create similarity data 4a that indicates whether images are similar or not by utilizing the color differences and characteristics of the images. This makes it possible to perform the classification judgment with high accuracy, especially when color differences in images are important. Furthermore, since the machine learning that generates the trained model is performed using a support vector machine, a separating hyperplane P1 that accurately separates the D-class classification area from the other classification areas can be created, or C L Classification area and C M It is now possible to create a separating hyperplane P2 that accurately separates the regions of the D and C classes. L Grade and C M It is possible to accurately determine which classification of the class the object belongs to. The image classification determination system 1 according to the first embodiment of the present invention is capable of determining whether the rock mass is classified as Class D or Class C from the captured image 3a. L Grade and C M Although it is possible to judge the three classifications of grades, the present invention is not limited to this and it is also possible to judge the four or more classifications, for example. EXAMPLES

[0044] 12 shows a partial configuration of an image classification determination system 1 according to a second embodiment of the present invention. In this second embodiment, the observation object and the method of acquiring the photographed image 3a are different from those in the first embodiment, so the same parts as those in the first embodiment are denoted by the same reference numerals, and only the different parts will be described. The image classification determination system 1 of the second embodiment includes an unmanned aerial vehicle 7 equipped with a camera 6. In the second embodiment, as shown in Fig. 13(a), the observation target is a water impermeable wall surface 8 of a reservoir, and the image classification determination system 1 is used to determine whether the water impermeable wall surface 8 is in an unhealthy state, such as blistering shown in Fig. 13(b) or mud curling shown in Fig. 14(c). The captured image 3a, which is the image to be determined, is obtained by photographing the water impermeable wall surface 8 from above with the camera 6 while flying the unmanned aerial vehicle 7.

[0045] FIG. 14 shows the results of observing a watertight wall surface 8 using the image classification judgment system 1 of Example 2. Result RB1 is a color-coded display of areas classified as unsound by an engineer's judgment, and area A is shown surrounded by a dashed line for convenience. Meanwhile, result RB2 is a color-coded display of areas classified as unsound by the image classification judgment system 1 of Example 2, and area B is shown surrounded by a dashed line for convenience. Note that result RB2 is the result of processing by the image classification judgment system 1 using a captured image 3a with a resolution of 300 dpi. Comparing results RB1 and RB2, the accuracy rate of classification judgment by the image classification judgment system 1 was 79.3%, which shows that by using the image classification judgment system 1 of Example 2, it is possible to identify unhealthy areas on the water-impermeable wall surface 8 with high accuracy.

[0046] Result RB3 is a color-coded display of the area classified as unhealthy by the image classification determination system 1 using the captured image 3a with the resolution reduced to 72 dpi, and for convenience, the area C is shown surrounded by a dashed line. Comparing result RB1 with result RB3, the accuracy rate of classification determination by the image classification determination system 1 was 77.0%, which was almost the same as result RB2. Thus, it was found that the image classification determination system 1 of Example 2 can obtain a result equivalent to the result obtained based on the learning analysis using the captured image 3a with a high resolution, even when the resolution of the captured image 3a is low.

[0047] In this way, according to the second embodiment of the present invention, when creating a trained model such as the separating hyperplane P1 or the separating hyperplane P2, the training image X k Similar sample images for 1 Z i Not only dissimilar sample images Z i Since it is created using the training image X when creating a trained model, k The amount of information that each image has increases. For example, learning image X k Or sample image Z i Even if the image quality is low, it is possible to create a versatile, highly accurate trained model.

[0048] In the first and second embodiments of the present invention, the image classification determination system 1 is applicable to classification determination work related to civil engineering and architecture. However, the image classification determination system 1 is also applicable to other fields. In the first and second embodiments of the present invention, the color histogram method is used to output the similarity data 4a, but machine learning may be performed in the model generating unit 2c using the similarity data 4a output by another method. Furthermore, in Examples 1 and 2 of the present invention, a support vector machine is used as machine learning to generate a trained model that outputs a determination result that a sample image Zi belonging to the D-class classification when the sample image Zi is input and that the image belongs to the D-class classification. However, the trained model may be generated using other machine learning methods as long as the correct correlation between the input and the output can be maintained. For example, a trained model constructed using another algorithm, such as a convolutional neural network (CNN), may be used. [Industrial Applicability]

[0049] The present invention is suitable for an image classification determination system, an image classification determination method, and a program thereof, for use in determining to which of a plurality of predefined classifications an image captured by a camera belongs, for example. [Explanation of symbols]

[0050] 1...Image classification judgment system 2...Information processing unit 2a...First teacher data output unit 2b...Second teacher data output unit 2c...Model generation unit 2d...Judgment unit 3...Input database 3a...Photographed image (image to be judged) 4...Teacher database 4a...Similarity data 4b...Similarity judgment teacher data 4c...Dissimilarity judgment teacher data 5...Output database 6...Camera 7...Unmanned aerial vehicle 8...Dam wall 10...Computer P1...Separating hyperplane (trained model) P2...Separating hyperplane (trained model) Q...Similarity data W1~W3...Separating hyperplane X k …Learning image Y k …Data group Z i … Sample image

Claims

1. An image classification determination system that causes a computer to execute a process of determining to which of a plurality of defined classifications an image to be determined belongs, comprising: a first teacher data output unit that outputs, as teacher data for similarity determination, a group of similarity data calculated between at least one first learning image belonging to a predetermined classification and at least one first sample image belonging to the predetermined classification, and between at least one first learning image belonging to the predetermined classification and at least one second sample image belonging to another classification; a second teacher data output unit that outputs, as teacher data for dissimilarity determination, a set of similarity data calculated between at least one second learning image belonging to the other classification and at least one first sample image belonging to the predetermined classification, and between at least one second learning image belonging to the other classification and at least one second sample image belonging to the other classification; A model generation unit that generates a trained model by machine learning based on the similarity determination teacher data and the dissimilarity determination teacher data; and a judgment unit that uses the trained model to output a judgment result as to whether the target image belongs to the specified classification.

2. An image classification determination method for determining to which of a plurality of defined classifications an image to be determined belongs, comprising the steps of: A group of similarity data calculated between at least one first learning image belonging to a predetermined classification and at least one first sample image belonging to the predetermined classification, and between at least one first learning image belonging to the predetermined classification and at least one second sample image belonging to another classification, is set as similarity judgment teacher data, a group of similarity data calculated between at least one second learning image belonging to the other classification and at least one first sample image belonging to the predetermined classification, and between at least one second learning image belonging to the other classification and at least one second sample image belonging to the other classification, is set as dissimilarity determination teacher data; Then, a trained model is generated by machine learning based on the similarity and dissimilarity judgment teacher data, Thereafter, the image classification determination method is characterized in that the computer is caused to execute a process of using the trained model to output a determination result as to whether the target image belongs to the specified classification or not.

3. An image classification determination program for causing a computer to execute a process for determining to which of a plurality of defined classifications an image to be determined belongs, comprising: A group of similarity data calculated between at least one first learning image belonging to a predetermined classification and at least one first sample image belonging to the predetermined classification, and between at least one first learning image belonging to the predetermined classification and at least one second sample image belonging to another classification, is set as similarity judgment teacher data, a group of similarity data calculated between at least one second learning image belonging to the other classification and at least one first sample image belonging to the predetermined classification, and between at least one second learning image belonging to the other classification and at least one second sample image belonging to the other classification, is set as dissimilarity determination teacher data; An image classification determination program that causes a computer to execute a process of outputting a determination result as to whether or not the target image belongs to the specified classification using a trained model generated by machine learning based on the similarity and dissimilarity determination teacher data.

4. 2. The image classification determination system according to claim 1, 2. An image classification determination system, comprising: a first training data output unit and a second training data output unit, the first training data output unit and the second training data output unit being configured to calculate the similarity data by a color histogram method.

5. 5. The image classification determination system according to claim 4, The image classification determination system is characterized in that the model generation unit performs machine learning to generate the trained model using a support vector machine.

6. 6. The image classification determination system according to claim 5, The image classification determination system, wherein the classification is a rock classification.

Images