Muscle atrophy prediction device, machine learning device, muscle atrophy prediction method, and machine learning method

Abstract

To provide a rock fall prediction device capable of predicting a portion having a risk of rock fall with higher accuracy.SOLUTION: A rock fall prediction device 5 for predicting rock fall in a tunnel face, comprises: a determination data acquisition unit 500 that acquires determination data including prediction images in which the tunnel face is imaged; a learned model storage unit 52 that stores a learning model 2 in which a correlation between input data including learning images in which the tunnel face is imaged, and output data including data on indication locations of rock fall prediction and indication locations of rock fall factor patterns included in the learning images, is machine-learned; and an inference unit 501 that inputs the determination data acquired by the determination data acquisition unit 500 to the learning model, and infers rock fall prediction locations on the tunnel face imaged in the prediction images.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to a peeling prediction device for predicting the possibility of peeling at a tunnel face, a machine learning device for generating a learning model used in the peeling prediction device, a peeling prediction method for predicting the possibility of peeling at a tunnel face, and a machine learning method for generating a learning model used in the peeling prediction method.

Background Art

[0002] At the forefront (face) of excavation in mountain tunnel construction, it is crucial to suppress as much as possible major disasters such as rock falls (peeling). For this reason, face observation is carried out, the locations where peeling is predicted are estimated, and measures such as knocking down the locations in advance or spraying concrete on the locations are taken. Since face observation depends on the visual judgment of skilled engineers, oversights and individual differences are likely to occur. Therefore, in Non-Patent Document 1, a technique for predicting peeling at the face using artificial intelligence technology has been proposed.

Prior Art Documents

Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the prior art described in Non-Patent Document 1, a face image, a weathering alteration degree value, and a joint intersection density value are subjected to feature extraction using Resnet50, and a neural network is used to detect the binary choice of the presence or absence of the possibility of peeling. There is a problem that the positions where there is a risk of peeling cannot be predicted.

[0005] In addition, the prior art has a problem in that, in addition to the face image, it evaluates the possibility of skin peeling only from the weathering alteration degree value and the joint density value of the cracks, and has low accuracy and versatility. Furthermore, there is also a problem that, as in the prior art, if only feature quantities are extracted by Resnet50, the factors for predicting skin peeling become a black box.

[0006] The present invention has been made in view of the above-described problems, and an object thereof is to provide a skin peeling prediction device capable of predicting a position at which there is a risk of skin peeling, a machine learning device for generating a learning model used in the skin peeling prediction device, a skin peeling prediction method capable of predicting a position at which there is a risk of skin peeling, and a machine learning method for generating a learning model used in the skin peeling prediction method.

Means for Solving the Problems

[0007] In order to achieve the above object, a skin peeling prediction device according to the present invention is a skin peeling prediction device for predicting skin peeling in a tunnel face, including a determination data acquisition unit that acquires determination data including a prediction image in which the tunnel face is imaged, input data including a learning image in which the tunnel face is imaged, and a learned model storage unit that stores a learning model obtained by machine learning of the correlation between the input data and output data including data related to a skin peeling prediction point and a skin peeling factor pattern point included in the learning image, and an inference unit that inputs the determination data acquired by the determination data acquisition unit into the learning model and infers a skin peeling prediction point of the tunnel face imaged in the prediction image. The prediction image is cut into a plurality of image data and used, and each of the plurality of image data has an overlapping part with other image data. Each of the plurality of image data has a weight coefficient, and the weight coefficient of the image data is input into the learning model together with the image data. The overlapping part includes a plurality of images divided in a grid pattern, and the weight coefficient is higher at the peripheral part than at the central part of the tunnel face It is characterized by the above.

[0010] In addition, the skin peeling prediction device according to the present invention is characterized in that the skin peeling prediction point is labeled in the prediction image.

[0011] In addition, the skin peeling prediction device according to the present invention is characterized in that the basis for predicting the skin peeling prediction point is visualized in the prediction image.

[0012] Moreover, the machine learning device according to the present invention is a machine learning device that generates a learning model used in a peeling prediction device for predicting peeling at a tunnel face, and includes an input data including learning images of the tunnel face being imaged, and an output data including data related to peeling prediction points and peeling factor pattern points included in the learning images. A learning data storage unit that stores a plurality of sets of learning data; a machine learning unit that causes the learning model to perform machine learning on the correlation between the input data and the output data by inputting a plurality of sets of the learning data into the learning model; and a learned model storage unit that stores the learning model machine-learned by the machine learning unit. The learning image is cut into a plurality of image data and used, and each of the plurality of image data has an overlapping part with other image data. Each of the plurality of image data has a weight coefficient, and the weight coefficient of the image data is input into the learning model together with the image data. The overlapping part includes a plurality of images divided in a grid pattern, and the weight coefficient is higher at the peripheral part than at the central part of the tunnel face It is characterized by this.

[0015] In addition, the peeling prediction method according to the present invention is a peeling prediction method for predicting peeling at a tunnel face, and includes a determination data acquisition step of acquiring determination data including a prediction image of the tunnel face being imaged, and the determination data acquired in the determination data acquisition step is input into a learning model that has performed machine learning on the correlation between input data including learning images of the tunnel face being imaged and output data including data related to peeling prediction points and peeling factor pattern points included in the learning images, and the peeling prediction points of the tunnel face imaged in the prediction image and An inference step of inferring. The prediction image is cut into a plurality of image data and used, and each of the plurality of image data has an overlapping part with other image data. Each of the plurality of image data has a weight coefficient, and the weight coefficient of the image data is input into the learning model together with the image data. The overlapping part includes a plurality of images divided in a grid pattern, and the weight coefficient is higher at the peripheral part than at the central part of the tunnel face It is characterized by this.

[0016] Moreover, the machine learning method according to the present invention is a machine learning method for generating a learning model used in a peeling prediction method for predicting peeling in a tunnel face, including input data including learning images of the tunnel face captured, and output data including data related to peeling prediction pointed parts and peeling factor pattern pointed parts included in the learning images. A learning data storage step of storing a plurality of sets of learning data in a learning data storage unit; a machine learning step of inputting a plurality of sets of the learning data into the learning model to machine-learn the correlation between the input data and the output data in the learning model; and a learned model storage step of storing the learning model machine-learned by the machine learning step in a learned model storage unit The learning image is cut into a plurality of image data and used, and each of the plurality of image data has an overlapping part with other image data. Each of the plurality of image data has a weight coefficient, and the weight coefficient of the image data is input into the learning model together with the image data. The overlapping part includes a plurality of images divided in a grid pattern, and the weight coefficient is higher at the peripheral part than at the central part of the tunnel face It is characterized by this.

Effects of the Invention

[0017] According to the peeling prediction device according to the present invention, in addition to the peeling prediction pointed parts, the peeling prediction parts are inferred based on the peeling factor pattern pointed parts, so that it is possible to predict more accurately the parts where there is a risk of peeling. In addition, since the inference is made based on the peeling factor pattern pointed parts, it is possible to prevent the peeling prediction factors from being black-boxed.

[0018] Moreover, according to the machine learning device according to the present invention, it is possible to generate a learning model used in a peeling prediction device that can predict more accurately the parts where there is a risk of peeling.

[0019] According to the peeling prediction method according to the present invention, in addition to the peeling prediction pointed parts, the peeling prediction parts are inferred based on the peeling factor pattern pointed parts, so that it is possible to predict more accurately the parts where there is a risk of peeling. In addition, since the inference is made based on the peeling factor pattern pointed parts, it is possible to prevent the peeling prediction factors from being black-boxed.

[0020] Moreover, according to the machine learning method according to the present invention, it is possible to generate a learning model used in a peeling prediction method that can predict more accurately the parts where there is a risk of peeling.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0022] Hereinafter, embodiments for implementing the present invention will be described with reference to the drawings. In the following, the scope necessary for the description to achieve the object of the present invention is schematically shown, and mainly the scope necessary for the description of the relevant part of the present invention will be described, and the parts where the description is omitted shall be based on known techniques. (Embodiment of the Present Invention) FIG. 1 is an overall configuration diagram showing an example of a skin peeling prediction system 1 according to an embodiment of the present invention. The skin peeling prediction system 1 is a system that acquires information regarding the cutting face 10 at the forefront of excavation in mountain tunnel construction and predicts skin peeling at the cutting face 10 based on this information. For this purpose, the skin peeling prediction system 1 includes a machine learning device 4 using artificial intelligence technology and a skin peeling prediction device 5 that makes inferences based on the learning data learned by this machine learning device 4.

[0023] The skin peeling prediction system 1 includes an imaging device 3 that images the cutting face 10, a machine learning device 4 that operates as the main body in the learning phase of machine learning, a skin peeling prediction device 5 that operates as the main body in the inference phase of machine learning, and a terminal device 6 used by the administrator or operator of the skin peeling prediction system 1. Each device 3 to 6 of the skin peeling prediction system 1 is communicably connected to each other by a network 7. Note that the system configuration of the skin peeling prediction system 1 shown in FIG. 1 is an example and can be changed as appropriate. For example, in the present embodiment, the skin peeling prediction device 5, the machine learning device 4, and the terminal device 6 are each configured by an independent information processing device, but they can also be configured by a single information processing device.

[0024] The imaging device 3 includes, for example, a camera composed of an image sensor such as a CMOS sensor or a CCD sensor, and images the entire face of the tunnel heading 10. Note that the imaging device 3 is a handheld or fixedly installed camera, and for example, it may be an imaging device that an operator operates to capture an image, or it may be an imaging device that automatically captures an image when predetermined imaging conditions are satisfied. In the present embodiment, the entire face of the tunnel heading 10 is imaged by one imaging device 3, but the face of the tunnel heading 10 may be imaged by a plurality of imaging devices 3.

[0025] The imaging device 3 images the face of the tunnel heading 10 within the angle of view of the imaging device 3 and outputs an image as digital data based on a predetermined image format. Note that, as shown in FIG. 1, the imaging device 3 connected to the machine learning device 4 and the imaging device 3 connected to the peeling prediction device 5 may be provided separately, or one imaging device 3 may be connected to and shared by both the machine learning device 4 and the peeling prediction device 5. Also, in FIG. 1, for simplicity, one imaging device 3 is connected to the machine learning device 4 and the peeling prediction device 5 respectively, but a plurality of imaging devices 3 may be connected respectively.

[0026] The machine learning device 4 is composed of a general-purpose or dedicated computer or the like, and operates as the main body in the learning phase of machine learning. The machine learning device 4 performs machine learning of the learning model 2 using learning data including an image (learning image) captured by the imaging device 3. The machine learning device 4 provides the learned learning model 2 to the peeling prediction device 5 via a network 7, a recording medium, or the like. Note that an example of the configuration of the computer will be described later.

[0027] The peeling prediction device 5 is composed of a general-purpose or dedicated computer or the like, and operates as the main body in the inference phase of machine learning. The peeling prediction device 5 performs prediction of the face of the tunnel heading 10 from an image (prediction image) captured by the imaging device 3 using the learning model 2 learned by the machine learning device 4.

[0028] The terminal device 6 is composed of a general-purpose or dedicated computer or the like. In the skin peeling prediction system 1, the terminal device 6 receives various operation inputs via an input screen and displays various information via a display screen such as an application or a browser, in order to prepare learning data, perform machine learning on the learning model 2, and predict skin peeling at the tunnel face 10. Also, in FIG. 1, for simplicity, one terminal device 6 is illustrated, but there may be a plurality of terminal devices 6. (Machine learning device 4) FIG. 2 is a block diagram showing an example of the machine learning device 4 according to an embodiment of the present invention. The machine learning device 4 includes a control unit 40, a communication unit 41, a learning data storage unit 42, and a learned model storage unit 43.

[0029] The imaging device 3 captures a learning image 30 and transmits it to the machine learning device 4. The control unit 40 of the machine learning device 4 functions as a learning data acquisition unit 400 and a machine learning unit 401. The communication unit 41 is connected to external devices (for example, the imaging device 3, the skin peeling prediction device 5, the terminal device 6, etc.) via the network 7 and functions as a communication interface for transmitting and receiving various data.

[0030] The learning data acquisition unit 400 is connected to external devices via the communication unit 41 and the network 7, and acquires learning data in which input data and output data are associated.

[0031] The learning data storage unit 42 is a database that stores a plurality of sets of learning data acquired by the learning data acquisition unit 400. Note that the specific configuration of the database constituting the learning data storage unit 42 may be designed as appropriate.

[0032] The machine learning unit 401 performs machine learning using the learning data stored in the learning data storage unit 42. That is, the machine learning unit 401 generates the learning model 2 by inputting a plurality of sets of learning data into the learning model 2 and causing the learning model 2 to machine-learn the correlation between the input data and the output data constituting the learning data.

[0033] The learned model storage unit 43 is a database that stores the learned learning model 2 machine-learned by the machine learning unit 401. The learning model 2 stored in the learned model storage unit 43 is provided to the actual system (for example, the hair loss prediction device 5) via the network 7, a recording medium, or the like. Note that the learning model 2 may be provided to an external computer (for example, a server-type computer or a cloud-type computer) and stored in the storage unit of the external computer. Also, in FIG. 2, the learning data storage unit 42 and the learned model storage unit 43 are shown as separate storage units, but they may be configured as a single storage unit.

[0034] FIG. 3 is a diagram for explaining an example of a learning data creation process in the hair loss prediction system 1 according to an embodiment of the present invention. FIG. 3(A) is an image of the entire face of the tunnel face 10 captured by the imaging device 3. FIG. 3(B) is a diagram showing the labeling of the hair loss prediction pointed parts pointed out by a skilled technician for the image, and the part surrounded by the dotted line in the figure indicates the labeling part of the hair loss prediction pointed parts. Such labeling becomes the learning data in the hair loss prediction system 1 according to the embodiment of the present invention.

[0035] FIG. 3(C) is a diagram showing the labeling of the hair loss factor pattern pointed parts pointed out by a skilled technician for the image, and the part surrounded by the frame in the figure indicates the labeling part of the hair loss factor pattern pointed parts. FIG. 4 shows a classification table of the hair loss factor pattern pointed parts. Based on such classification, a skilled technician designates, together with the classification, the parts likely to be the causes of hair loss in the learning image of the tunnel face 10.

[0036] In the skin loss prediction system 1 according to the embodiment of the present invention, as the skin loss factor patterns, there are eight factor patterns caused by geology, namely, (1) the trace of the falling off of rock masses, (2) the face unevenness along the cracks, (3) the fine entry of cracks, (4) clay and inclusions in the cracks, (5) weathering discoloration more than the surroundings, significant weathering discoloration, (6) fractured zones, fractured state, sandification, (7) seepage water, presence of gushing water, and (8) others. By classifying them into these eight factor patterns and using them as learning data based on this, it is possible to prevent the factors of skin loss prediction from being black-boxed. The classification of the skin loss factor patterns is not limited to these eight factor patterns, and can be appropriately selected. The key is that if at least two or more skin loss factor patterns are set, the skin loss prediction system 1 according to the present invention can be configured. In FIG. 3(C), as the skin loss factor patterns, a case is shown in which (2) the face unevenness along the cracks and (3) the fine entry of cracks are pointed out by a skilled technician, and these skin loss factor patterns become the learning data in the skin loss prediction system 1.

[0037] The learning image 30 included in the input data is an image in which the entire tunnel face 10 is imaged by the imaging device 3. Here, the learning image 30 in which the entire tunnel face 10 is imaged is divided and used as data for learning, and is also divided and used as a prediction image at the time of inference of the skin loss prediction system 1 according to the present invention for inference.

[0038] FIG. 5 is a diagram for explaining a processing example of the learning image 30 in which the tunnel face 10 is imaged. The learning image 30 or the prediction image 31 in which the tunnel face 10 is imaged is divided into a plurality of grid-like square meshes after the tunnel face 10 is recognized based on a well-known image analysis technique or the like. The image divided into such a grid, as shown in FIG. 5(A), for example, an image G1 composed of four grids in the upper left corner is cut out, and this image data G1 is used for learning and the like. Subsequently, as shown in FIG. 5(B), the image data G2 is cut out, and this image data is used for learning and the like.

[0039] Here, as can also be seen from FIGS. 5(A) and 5(B), the image data G1 and the image data G2 are cut out such that two grids (the right two grids in the image data G1 and the left two grids in the image data G2) overlap. Similarly hereinafter, each image data is sequentially cut out so as to scan from the upper left to the lower right (as shown in FIGS. 5(C) → 5(D)) and includes overlapping grids with the adjacent image data above and below and left and right, and is used for learning, determination, etc. In the hair loss prediction system 1 according to the present invention, since the image data including such overlapping grids is used for learning, determination, etc., there is no omission, the accuracy is high, and it is possible to predict the dangerous locations of hair loss.

[0040] In the example shown in FIG. 5, one piece of cut-out image data is configured with four grids, but the number of grids constituting the cut-out image data is arbitrary.

[0041] The data cut out based on the grids from the learning image and the prediction image as described above is referred to as "image data" in this specification.

[0042] Now, the machine learning device 4 performs learning using the image data (hereinafter generally referred to as Gn. n is a natural number) cut out from the learning image 30 as described above. FIG. 6 is a diagram showing an example of a data set of learning data. Here, the input data is the image data Gn, and the output data is, for example, what is called a correct label in supervised learning.

[0043] In the image data Gn, when the pointed hair loss prediction location has an area equal to or larger than a predetermined area, a correct label of "present" for the hair loss prediction location is set as the output data. Also, in the image data Gn, when the pointed hair loss prediction location is smaller than the predetermined area, a correct label of "absent" for the hair loss prediction location is set as the output data.

[0044] Also, the correct label is set according to the ratios of (1) to (8) of the hair loss factor patterns in the image data Gn. In the example of FIG. 6, the ratio (for example, the area ratio) of (2) as the hair loss factor pattern and the ratio (for example, the area ratio) that does not correspond to any of (1) to (8) as the hair loss factor pattern are set as the correct labels. The sum of the respective correct labels based on the classification of the hair loss factor patterns is 1.

[0045] Also, when the hair loss prediction pointed-out part in the image data Gn has an area equal to or larger than a predetermined area, the correct label "1" is set as the hair loss possibility. Also, in the image data Gn, when the hair loss prediction pointed-out part is smaller than the predetermined area, the correct label of "0" is set as the hair loss possibility.

[0046] Next, the weight coefficients used in the hair loss prediction system 1 according to the present invention will be described. FIG. 7 is a diagram for explaining the weight coefficients used in the learning model 2 of the hair loss prediction system 1 according to the present invention. It is empirically known that the probability of hair loss at the tunnel face 10 is higher at the peripheral part than at the central part of the tunnel face 10, and further higher at the upper part than at the lower part of the tunnel face 10.

[0047] Therefore, when processing the image data Gn with the learning model 2, weighting is performed using weight coefficients according to which four grids the image data Gn is composed of. In the example shown in FIG. 7, three types of grids are set: a grid having a weight coefficient of 1.3, a grid having a weight coefficient of 1.0, and a grid having a weight coefficient of 0.7. The values and types of the weight coefficients are arbitrary, but the values of the weight coefficients are set so that they are higher at the peripheral part than at the central part of the tunnel face 10, and further higher at the upper part than at the lower part of the tunnel face 10.

[0048] For image data composed of grids with different weight coefficients, for example, the average value of four weight coefficients can be used as the weight coefficient for that image data. For example, as the weight coefficient of the image data Gn shown in FIG. 7, (1.0 + 1.0 + 1.0 + 1.3) / 4 = 1.075 can be used.

[0049] FIG. 8 is a schematic diagram showing an example of a neural network model 20 applied to the learning model 2 according to an embodiment of the present invention. The neural network model 20 employs a convolutional neural network (CNN) as a specific method of machine learning. The neural network model 20 includes an input layer 21, a first intermediate layer 22a, a second intermediate layer 22b, a first output layer 23a, a second output layer 23b, and a third output layer 23c.

[0050] In the neural network model 20, in the first intermediate layer 22a and the first output layer 22 a, data processing regarding the skin peeling prediction pointed-out location is performed, and in the second intermediate layer 22b and the second output layer 23b, data processing regarding the skin peeling factor pattern pointed-out location is performed. The output from the first output layer 23a is weighted by the weight coefficient described above and then input to the fully-connected layer 222c having the third output layer 23c. Also, the output from the second output layer 23b is input to the fully-connected layer 222c having the third output layer 23c.

[0051] The input layer 21 has a number of neurons corresponding to the number of pixels of the image data Gn as input data, and the pixel value of each pixel is input to each neuron respectively.

[0052] The first intermediate layer 22a and the second intermediate layer 22b are each composed of a convolutional layer 220a, 220b, a pooling layer 221a, 221b, and a fully connected layer 222a, 222b. A common one can be used as each convolutional layer and pooling layer. The convolutional layers 220a, 220b and the pooling layers 221a, 221b are provided, for example, with a plurality of layers alternately. The convolutional layers 220a, 220b and the pooling layers 221a, 221b extract feature amounts from the image input via the input layer 21.

[0053] Each of the fully connected layers 222a, 222b converts the feature amounts of the two-dimensional array extracted from the image by the respective convolutional layers 220a, 220b and pooling layers 221a, 221b, for example, by an activation function, and outputs them as feature vectors of a one-dimensional array. Note that a plurality of fully connected layers 222a, 222b may be provided.

[0054] The output layer 23a outputs output data including the determination result of the skin loss prediction pointed part included in the image data Gn based on the feature vector output from the fully connected layer 222a. Further, the output layer 23b outputs output data including the determination result of the skin loss factor pattern pointed part included in the image data Gn based on the feature vector output from the fully connected layer 222b.

[0055] FIG. 9 is a diagram for explaining the inference results obtained by each output layer. As shown in FIG. 9, the output layer 23a outputs "There is a predicted skin loss location" and "There is a predicted skin loss location". Further, from the output layer 23b, "Ratio of classification (1)", "Ratio of classification (2)", "Ratio of classification (3)", "Ratio of classification (4)", "Ratio of classification (5)", "Ratio of classification (6)", "Ratio of classification (7)", "Ratio of classification (8)", and "Ratio not corresponding to any of classifications (1)-(8)" are output.

[0056] The output from the first output layer 23a is weighted by the weight coefficients and then input to the fully connected layer 222c. Also, the output from the second output layer 23b is input to the fully connected layer 222c. From the third output layer 23c, a value between 0 and 1 for the "possibility of skin peeling" is set to be output.

[0057] Between each layer of the neural network models 20a and 20b, synapses that connect the neurons between the layers are formed, and weights are associated with each synapse of the convolutional layers 220a and 220b and the fully connected layers 222a and 222b of the intermediate layers 22a and 22b.

[0058] The machine learning unit 401 inputs the learning data into the neural network model 20 and causes the neural network model 20 to perform machine learning on the correlation between the input data (image data Gn cut out from the learning image 30) and the output data. Specifically, the machine learning unit 401 inputs the image data Gn constituting the learning data as the input data into the input layer 21 of the neural network model 20. Note that the machine learning unit 401 may perform predetermined image adjustment (for example, image format, image size, image filter, image mask, etc.) on the image data Gn as preprocessing when inputting the learning image 30 into the input layer 21.

[0059] The machine learning unit 401 uses an error function that compares the output data output as the inference result from the output layer 23 with the output data (correct label) constituting the learning data, and repeats adjusting the weights associated with each synapse (backpropagation) so that the evaluation value of the error function becomes smaller.

[0060] The machine learning unit 401 repeatedly executes the above series of processes a predetermined number of times, and when it determines that predetermined learning end conditions such as the evaluation value of the error function becoming smaller than the allowable value are satisfied, it ends the machine learning, and the neural network model 20 at that time (a weight parameter group consisting of all the weights associated with each synapse) is stored in the learned model storage unit 43 as the learned model 2. Note that when the machine learning unit 401 performs machine learning on the learning model 2, as a method for adjusting the weights, for example, online learning, batch learning, mini-batch learning, etc. may be adopted, or as a method for evaluating the learning model 2 by dividing a plurality of sets of learning data into training data and test data, for example, the hold-out method, cross-validation, etc. may be adopted, or as a predetermined learning end condition, it may be determined that the misjudgment rate is the minimum. (Hair loss prediction device 5) FIG. 10 is a block diagram showing an example of the hair loss prediction device 5 according to an embodiment of the present invention. The hair loss prediction device 5 includes a control unit 50, a communication unit 51, and a learned model storage unit 52.

[0061] The control unit 50 functions as a determination data acquisition unit 500, an inference unit 501, and an output processing unit 502. The communication unit 51 is connected to an external device (for example, the imaging device 3, the machine learning device 4, the terminal device 6, etc.) via the network 7 and functions as a communication interface for transmitting and receiving various data.

[0062] The determination data acquisition unit 500 is connected via the communication unit 51 and the network 7 to an external device, and acquires determination data including a prediction image 31 in which the tunnel face 10 is imaged. The prediction image 31 is cut out as image data consisting of four grids, similarly to during learning, and is to be used for peeling prediction. The image data cut out from the prediction image 31 will be referred to as Hn (n is a natural number). From the peeling prediction device 5, a value between 0 and 1 for the "peeling possibility" is output for the image data Hn cut out from the prediction image 31. The prediction image 31 corresponds to the learning image 30 when the learning model 2 is machine-learned by the machine learning device 4. Also, for each Hn, a weight coefficient is calculated and input to the learning model 2, similarly to during learning. Note that the determination data includes at least the prediction image 31, but may also include other data useful for peeling prediction.

[0063] The inference unit 501 performs an inference process of inferring the "peeling possibility" of the tunnel face 10 by inputting the image data Hn acquired by the determination data acquisition unit 500 into the learning model 2.

[0064] The learned model storage unit 52 is a database that stores the learned learning model 2 used in the inference process of the inference unit 501. Note that the number of learning models 2 stored in the learned model storage unit 52 is not limited to one. For example, a plurality of learned models with different machine learning methods may be stored and selectively available. Also, the learned model storage unit 52 may be substituted by the storage unit of an external computer (for example, a server-type computer or a cloud-type computer). In that case, the inference unit 501 may perform the above inference process by accessing the external computer.

[0065] The output processing unit 502 performs output processing for outputting the prediction result of the "possibility of skin peeling" for the image data Hn of the tunnel face 10 inferred by the inference unit 501. Various means can be adopted as specific output means for outputting the prediction result. For example, the output processing unit 502 may transmit the determination result to the terminal device 6, further display it on the screen, or store the determination result in the storage unit of the skin peeling prediction device 5.

[0066] Here, the learning model 2 is obtained by machine learning in the machine learning device 4 using a plurality of sets of learning data, the relationship between the image data Gn cut out from the learning image 30, the skin peeling prediction pointed location, and the skin peeling factor pattern pointed location, and it is a learning model 2 with the weight parameter group adjusted (learned). Therefore, the inference unit 501 can infer the "possibility of skin peeling" in the image data Hn of the prediction image 31 by inputting the image data Hn cut out from the prediction image 31 and the weight coefficient into the learning model 2.

[0067] Note that the prediction result of skin peeling for the image data Hn included in the prediction image 31 is preferably stored in the learned model storage unit 52 or another storage device (not shown). The past prediction results may be used as learning data for online learning or re-learning, for example, to further improve the inference accuracy of the learned learning model 2. (Configuration of the computer 900) FIG. 11 is a hardware configuration diagram showing an example of the computer 900 constituting each device 3 to 6 of the skin peeling prediction system 1.

[0068] Each device 3 to 6 of the muscle loss prediction system 1 is composed of a general-purpose or dedicated computer 900. The main components of the computer 900 include a bus 910, a processor 912, a memory 914, an input device 916, an output device 917, a display device 918, a storage device 920, a communication I / F (interface) unit 922, an external device I / F unit 924, an I / O (input / output) device I / F unit 926, and a media input / output unit 928. Note that the above components may be appropriately omitted according to the application for which the computer 900 is used.

[0069] The processor 912 is composed of one or more arithmetic processing units (such as a CPU (Central Processing Unit), MPU (Micro-processing unit), DSP (digital signal processor), GPU (Graphics Processing Unit), etc.) and operates as a control unit that oversees the entire computer 900. The memory 914 stores various data and programs 930 and is composed of, for example, a volatile memory (such as DRAM, SRAM, etc.) that functions as a main memory and a non-volatile memory (such as ROM, flash memory, etc.).

[0070] The input device 916 is composed of, for example, a keyboard, a mouse, a numeric keypad, an electronic pen, etc. and functions as an input unit. The output device 917 is composed of, for example, a sound (voice) output device, a vibration device, etc. and functions as an output unit. The display device 918 is composed of, for example, a liquid crystal display, an organic EL display, an electronic paper, a projector, etc. and functions as an output unit. The input device 916 and the display device 918 may be integrally configured, such as a touch panel display. The storage device 920 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc. and functions as a storage unit. The storage device 920 stores various data necessary for the execution of the operating system and the program 930.

[0071] The communication I / F unit 922 is connected to a network 940 such as the Internet or an intranet (which may be the same as the network 7 in FIG. 1) either wired or wirelessly, and functions as a communication unit that transmits and receives data to and from other computers according to a predetermined communication standard. The external device I / F unit 924 is connected to an external device 950 such as a camera, printer, scanner, or reader / writer either wired or wirelessly, and functions as a communication unit that transmits and receives data to and from the external device 950 according to a predetermined communication standard. The I / O device I / F unit 926 is connected to an I / O device 960 such as various sensors and actuators, and functions as a communication unit that transmits and receives various signals and data such as detection signals from sensors and control signals to actuators to and from the I / O device 960. The media input / output unit 928 is composed of, for example, drive devices such as a DVD (Digital Versatile Disc) drive and a CD (Compact Disc) drive, and reads and writes data to a media 970 such as a DVD or a CD (non-volatile storage medium).

[0072] In the computer 900 having the above configuration, the processor 912 calls and executes the program 930 stored in the storage device 920 in the memory 914, and controls each part of the computer 900 via the bus 910. Note that the program 930 may be stored in the memory 914 instead of the storage device 920. The program 930 may be recorded in the media 970 in an installable file format or an executable file format, and provided to the computer 900 via the media input / output unit 928. The program 930 may be provided to the computer 900 by being downloaded via the network 940 through the communication I / F unit 922. Also, the computer 900 may implement various functions realized when the processor 912 executes the program 930 in hardware such as an FPGA (field-programmable gate array) or an ASIC (application specific integrated circuit).

[0073] The computer 900 is composed of, for example, a stationary computer or a portable computer, and is an electronic device in any form. The computer 900 may be composed of a client-type computer or an edge-type computer according to the usage purpose of the computer 900, or may be composed of a server-type computer or a cloud-type computer. (Machine learning method) FIG. 12 is a flowchart showing an example of a machine learning method by the machine learning apparatus 4 according to an embodiment of the present invention.

[0074] First, in step S100, the learning data acquisition unit 400 prepares a desired number of learning data as preliminary preparation for starting machine learning, and stores the prepared learning data in the learning data storage unit 42. The number of learning data to be prepared here may be set in consideration of the inference accuracy required for the finally obtained learning model 2.

[0075] Several methods can be adopted for preparing the learning data. For example, the imaging device 3 is used to image the learning image 30 of the tunnel face 10, and the terminal device 6 is used for a skilled technician to record the peeling prediction points and the peeling factor pattern points in the learning image 30. By repeating such an operation every time a new tunnel face 10 appears, it is possible to prepare a plurality of sets of learning data.

[0076] Next, in step S110, the machine learning unit 401 prepares a pre-learning learning model 2 in order to start machine learning. The pre-learning learning model 2 prepared here adopts the convolutional neural network illustrated in FIG. 8, and the weight of each synapse is set to an initial value. Each pixel of the image data Gn cut out from the learning image 30 as input data constituting the learning data is associated with each neuron in the input layer 21. To each neuron in the first output layer 23a, the presence or absence of a predicted skin loss location as output data constituting the learning data is associated. To each neuron in the second output layer 23b, the ratio of each classification based on the predicted skin loss factor pattern location as output data constituting the learning data is associated. To each neuron in the third output layer 23c, the skin loss possibility as output data constituting the learning data is associated.

[0077] Next, in step S120, the machine learning unit 401 acquires, for example, one set of learning data randomly from a plurality of sets of learning data stored in the learning data storage unit 42.

[0078] Next, in step S130, the machine learning unit 401 inputs the input data included in one set of learning data to the input layer of the pre-prepared (or during learning) learning model 2. As a result, output data is output as an inference result from the output layer of the learning model 2, and the output data is generated by the pre-prepared (or during learning) learning model 2. Therefore, in the pre-prepared (or during learning) state, the output data output as an inference result indicates information different from the output data (correct label) included in the learning data.

[0079] Next, in step S140, the machine learning unit 401 compares the output data (correct label) included in the one set of learning data acquired in step S120 with the output data output as an inference result from the output layer in step S130, and performs machine learning by adjusting the weights of each synapse. Thereby, the machine learning unit 401 causes the learning model 2 to learn the correlation between the input data and the output data.

[0080] Next, in step S150, the machine learning unit 401 determines whether a predetermined learning end condition is satisfied, for example, based on the evaluation value of the error function based on the inference result and the output data (correct label) included in the learning data, or based on the remaining number of unlearned learning data stored in the learning data storage unit 42.

[0081] In step S150, when the machine learning unit 401 determines that the learning end condition is not satisfied and machine learning is to be continued (No in step S150), it returns to step S120, and the processes of steps S120 to S140 are performed multiple times on the learning model 2 being learned using unlearned learning data. On the other hand, in step S150, when the machine learning unit 401 determines that the learning end condition is satisfied and machine learning is to end (Yes in step S150), it proceeds to step S160.

[0082] Then, in step S160, the machine learning unit 401 stores the learned learning model 2 (adjusted weight parameter group) that has been machine learned by adjusting the weights associated with each synapse in the learned model storage unit 43, and ends the series of machine learning methods shown in FIG. 8. In the machine learning method, step S100 corresponds to the learning data storage step, steps S110 to S150 correspond to the machine learning step, and step S160 corresponds to the learned model storage step. Note that in the series of machine learning methods shown in FIG. 12, the case where online learning is adopted as the method for adjusting the weights has been described, but batch learning or mini-batch learning (for example, in units of 100 sets of learning data) may be adopted. Further, when a plurality of sets of learning data are divided into training data and test data, the series of machine learning methods shown in FIG. 12 may be executed using the training data. Furthermore, it may be determined based on the error rate whether or not a predetermined learning end condition is satisfied.

[0083] As described above, according to the machine learning apparatus 4 and the machine learning method according to the present embodiment, it is possible to provide a learning model 2 that can accurately infer (determine, predict) the skin peeling of the tunnel face 10 from the learning image 30 in which at least a part of the tunnel face 10 is imaged by the imaging apparatus 3. (Skin peeling prediction method) FIG. 13 is a flowchart showing an example of a skin peeling prediction method by the skin peeling prediction device 5 according to an embodiment of the present invention. Note that the series of skin peeling prediction methods shown in FIG. 13 is repeatedly executed by the skin peeling prediction device 5 at a predetermined timing. The predetermined timing may be any timing. For example, it may be when the imaging device 3 newly images the prediction image 31, or when a predetermined event occurs (when the terminal device 6 receives a determination operation by an administrator or an operator, when the skin peeling prediction system 1 receives a determination command, etc.).

[0084] In step S200, the tunnel face 10 is imaged by the imaging device 3, and the prediction image 31 is transmitted to the skin peeling prediction device 5, whereby the determination data acquisition unit 500 acquires determination data including the prediction image 31.

[0085] Subsequently, a process of cutting out a plurality of image data Hn based on the grid as described above is executed from the prediction image in step S210. Then, in step S220, the weight coefficient of each image data Hn is calculated in the manner described above.

[0086] Next, in step S230, the inference unit 501 inputs the image data Hn and its weight coefficient to the input layer of the learned learning model 2, and obtains the "skin peeling possibility" for the image data Hn as an inference result from the output layer of the learning model 2 as a value between 0 and 1.

[0087] In step S240, it is determined whether an inference result has been obtained for all the image data Hn. If the result of the determination is No, in step S280, the next target image data is selected and the process proceeds to step S230 again. On the other hand, if the result of the determination is Yes, the process proceeds to step S250.

[0088] In step S250, the inference result based on the image data Hn is restored to the one corresponding to the prediction image. As an inference result, a value of "possibility of skin peeling" is obtained for each image data Hn. However, since there are overlapping grids in the image data Hn, different values of "possibility of skin peeling" may be obtained for a certain grid. In such a case, in order to err on the safe side, it is better to associate a higher value as the value of "possibility of skin peeling" with that grid.

[0089] Also, in step S250, as described above, in the prediction image in which the value of "possibility of skin peeling" is associated with each grid, a label is generated so that the locations where the value of the possibility of skin peeling is equal to or greater than a predetermined value can be displayed as "skin peeling prediction locations".

[0090] Subsequently, in step S260, when there are locations in the prediction image where the value of the possibility of skin peeling is equal to or greater than a predetermined value, a process of visualizing the basis for prediction is executed. As a result, it is possible to obtain knowledge about from what viewpoints the neural network selected the skin peeling prediction locations. In this sense, it is also possible to prevent the black-boxing of skin peeling prediction. Note that, for the visualization of the basis for prediction, a well-known algorithm such as Grad-CAM (Gradient-weightend Class Actvation Mapping) can be used.

[0091] In step S270, the output processing unit 502 executes output such as display. An example of the output display based on such step S270 is shown in FIG. 14. FIG. 14(A) shows an example of the prediction image labeled as the skin peeling prediction location in step S250. In the example of FIG. 14(A), the skin peeling prediction locations are surrounded by labels such as frames so that they can be recognized. Also, FIG. 14(B) shows an example of the prediction image processed for visualizing the basis for prediction in step S260. In the example of FIG. 14(B), it is possible to recognize on what basis the learning model 2 made the prediction by shading (or another color difference, etc.).

[0092] As described above, according to the skin peeling prediction device 5 of the present invention, in addition to the skin peeling prediction pointed-out location, the skin peeling prediction location is inferred based on the skin peeling factor pattern pointed-out location. Therefore, it is possible to more accurately predict the location where there is a risk of skin peeling. In addition, since the inference is made based on the skin peeling factor pattern pointed-out location, it is possible to prevent the skin peeling prediction factor from being black-boxed.

[0093] Further, according to the machine learning device 4 of the present invention, it is possible to generate a learning model used for the skin peeling prediction device 5 that can more accurately predict the location where there is a risk of skin peeling.

[0094] According to the skin peeling prediction method of the present invention, in addition to the skin peeling prediction pointed-out location, the skin peeling prediction location is inferred based on the skin peeling factor pattern pointed-out location. Therefore, it is possible to more accurately predict the location where there is a risk of skin peeling. In addition, since the inference is made based on the skin peeling factor pattern pointed-out location, it is possible to prevent the skin peeling prediction factor from being black-boxed.

[0095] Further, according to the machine learning method of the present invention, it is possible to generate a learning model used for the skin peeling prediction method that can more accurately predict the location where there is a risk of skin peeling.

[0096] In addition, according to the skin peeling prediction device 5 and the skin peeling prediction method of the present invention, even a technician or worker with little experience can obtain knowledge based on artificial intelligence similar to the skills of a geology expert, and thus can recognize in advance the locations where skin peeling may occur. As a result, the technician or worker can accumulate experience, and the safety of the work can be ensured.

[0097] In addition, according to the skin peeling prediction device 5 and the skin peeling prediction method of the present invention, just by taking a photo with an imaging device, it is possible to know the locations where skin peeling may occur. Therefore, even without skilled techniques, it is possible to observe and monitor the face, and thus the labor saving of skilled technicians can be achieved.

[0098] In addition, according to the skin peeling prediction device 5 and the skin peeling prediction method according to the present invention, it is possible to confirm whether kosoku (a pumice dropping operation for dropping rocks, etc.) is sufficient, so that the working time of kosoku can be shortened.

[0099] In addition, according to the skin peeling prediction device 5 and the skin peeling prediction method according to the present invention, kosoku is thoroughly performed and pumice is removed, so the quality is improved.

[0100] In addition, according to the skin peeling prediction device 5 and the skin peeling prediction method according to the present invention, an appropriate spraying thickness can be proposed, so that reasonable mirror spraying can be performed.

[0101] In addition, according to the skin peeling prediction device 5 and the skin peeling prediction method according to the present invention, skin peeling prevention equipment becomes unnecessary, which is economical and improves the face work efficiency.

[0102] Situations similar to the tunnel face can also be seen in large-scale rock cutting slopes during rock mass excavation, quarry remaining wall slopes, natural slopes, etc. The skin peeling prediction device 5 and the skin peeling prediction method according to the present invention are not limited to the face of mountain tunnels, but can also be applied to these slopes and inclined surfaces, and become one of the monitoring methods for preventing skin peeling and rockfall disasters.

[0103] In the embodiments described so far, as a learning method of machine learning, the case where the learning model 2 is configured by a neural network model (including deep learning) 20 that treats the prediction and determination of skin peeling on the tunnel face 10 as a classification problem has been described. However, the learning model may be configured by a neural network model that treats it as a regression problem. Furthermore, as long as it is something that learns the correlation between the input data and the output data in the above embodiments from the learning data, it is not limited to the above examples, and other learning methods may be adopted. For example, the learning model may use a recurrent neural network (RNN) or ensemble learning. Also, when treating it as a regression problem, the learning model may adopt a statistical learning method. For example, it may use a statistical model such as an autoregressive integrated moving average (ARIMA) model or Bayesian estimation.

[0104] Further, each part included in the control unit 40 of the machine learning device 4 or the control unit 50 of the hair loss prediction device 5 in the above embodiment may be realized by causing the processor 912 of the computer 900 shown in FIG. 11 to execute it by a program. It can be done.

[0105] Further, instead of the aspect of the hair loss prediction device 5 according to the above embodiment, it may be provided in the aspect of an inference device (inference method or inference program) used for predicting and determining the hair loss of the tunnel face 10. In this case, the inference device includes a memory and a processor, and among them, the processor executes a series of processes.

[0106] The series of processes includes a data acquisition process of acquiring determination data including the prediction image 31, and when the determination data is acquired in the data acquisition process, the prediction image 31 is input to the learning model, and an inference process of inferring the hair loss prediction location of the tunnel face imaged in the prediction image 31.

[0107] By providing in the aspect of the above inference device (inference method or inference program), it becomes possible to easily apply to various devices as compared with the case of implementing the hair loss prediction device 5. It should be understood by those skilled in the art that when the inference device (inference method or inference program) predicts and determines the hair loss of the tunnel face 10, the inference method implemented by the inference unit 501 of the hair loss prediction device 5 may be applied using the learned learning model 2 generated by the machine learning device 4 according to the above embodiment.

Explanation of Signs

[0108] 1 ··· Hair loss prediction system, 2, 2a, 2b ··· Learning models, 3 ··· Imaging device, 4 ··· Machine learning device, 5 ··· Hair loss prediction device, 6 ··· Terminal device, 7 ··· Network, 8A ··· Environment observation device, 8B ··· Environmental information providing device, 10 ··· Tunnel face, 20 ··· Neural network model, 21 ··· Input layer, 22a ··· First intermediate layer, 22b ··· Second intermediate layer, 23a ··· First output layer, 23b ··· Second output layer, 23c ··· Third output layer 30 ··· Learning image, 31 ··· Prediction image, 40 ··· Control unit, 41 ··· Communication unit, 42 ··· Learning data storage unit, 43 ··· Trained model storage unit, 50 ··· Control unit, 51 ··· Communication unit, 52 ··· Trained model storage unit, 80 ··· Learning environmental information, 81 ··· Judgment environmental information, 220a, 220b ··· Convolution layer, 221a, 221b ··· Pooling layer, 222a, 222b, 222c ··· Fully connected layer, 400, 400a, 400b ··· Learning data acquisition unit, 401, 401a, 401b ··· Machine learning unit, 500, 500a ··· Judgment data acquisition unit, 501, 501a, 501b ··· Inference unit, 502 ··· Output processing unit, 900 ··· Computer 910 ··· Bus, 912 ··· Processor, 914 ··· Memory, 916 ··· Input device, 917 ··· Output device, 918 ··· Display device, 920 ··· Storage device, 922 ··· Communication I / F (Interface) unit, 924 ··· External device I / F section, 926 ··· I / O (Input / Output) device I / F section, 928 ··· Media input / output section

Claims

1. A peeling prediction device for predicting peeling in a tunnel face, comprising: a determination data acquisition unit that acquires determination data including a prediction image in which the tunnel face is imaged; a learned model storage unit that stores a learned model obtained by machine learning the correlation between input data including a learning image in which the tunnel face is imaged and output data including data related to a peeling prediction pointed-out portion and a peeling factor pattern pointed-out portion included in the learning image; an inference unit that inputs the determination data acquired by the determination data acquisition unit into the learned model and infers a peeling prediction portion of the tunnel face imaged in the prediction image, wherein the prediction image is cut out and used in a plurality of image data, each of the plurality of image data has an overlapping portion with other image data, each of the plurality of image data has a weight coefficient, and the weight coefficient of the image data is input into the learned model together with the image data, the overlapping portion includes a plurality of images divided in a grid pattern, and the weight coefficient is higher at the peripheral portion than at the central portion of the tunnel face. A peeling prediction device characterized by this.

2. The peeling prediction device according to claim 1, wherein the peeling prediction portion is marked in the prediction image.

3. The peeling prediction device according to claim 1 or claim 2, wherein the basis for predicting the peeling prediction portion is visualized in the prediction image.

4. A machine learning device for generating a learned model used in a peeling prediction device for predicting peeling in a tunnel face, comprising: a learning data storage unit that stores a plurality of sets of learning data composed of input data including a learning image in which the tunnel face is imaged and output data including data related to a peeling prediction pointed-out portion and a peeling factor pattern pointed-out portion included in the learning image; a machine learning unit that inputs a plurality of sets of the learning data into the learned model to machine-learn the correlation between the input data and the output data in the learned model; and a learned model storage unit that stores the learned model machine-learned by the machine learning unit, wherein the learning image is cut out and used in a plurality of image data, each of the plurality of image data has an overlapping portion with other image data, each of the plurality of image data has a weight coefficient, and the weight coefficient of the image data is input into the learned model together with the image data. The overlapping part includes a plurality of images divided in a grid pattern, A machine learning device characterized in that the weight coefficient is higher at the peripheral part than at the central part of the tunnel face.

5. A peeling prediction method for predicting peeling at a tunnel face, comprising: A determination data acquisition step of acquiring determination data including a prediction image in which the tunnel face is imaged; Inputting the determination data acquired in the determination data acquisition step into a learning model obtained by machine learning the correlation between the input data including a learning image in which the tunnel face is imaged and output data including data related to a peeling prediction point and a peeling factor pattern point included in the learning image, and inferring a peeling prediction point of the tunnel face imaged in the prediction image; The prediction image is cut out and used in a plurality of image data, Each of the plurality of image data has an overlapping part with other image data, Each of the plurality of image data has a weight coefficient, and the weight coefficient of the image data is input into the learning model together with the image data, The overlapping part includes a plurality of images divided in a grid pattern, A peeling prediction method characterized in that the weight coefficient is higher at the peripheral part than at the central part of the tunnel face.

6. A machine learning method for generating a learning model used in a peeling prediction method for predicting peeling at a tunnel face, comprising: A learning data storage step of storing a plurality of sets of learning data composed of input data including a learning image in which the tunnel face is imaged and output data including data related to a peeling prediction point and a peeling factor pattern point included in the learning image in a learning data storage unit; A machine learning step of machine learning the correlation between the input data and the output data in the learning model by inputting a plurality of sets of the learning data into the learning model; A learned model storage step of storing the learning model machine-learned in the machine learning step in a learned model storage unit; The learning image is cut out and used in a plurality of image data, Each of the plurality of image data has an overlapping part with other image data, Each of the plurality of image data has a weight coefficient, and the weight coefficient of the image data is input into the learning model together with the image data, The overlapping part includes a plurality of images divided in a grid pattern, A machine learning method characterized in that the weight coefficient is higher at the peripheral part than at the central part of the tunnel face.

Images