Soil type estimation device, soil type estimation method, and soil type estimation program
The soil type estimation device addresses the inefficiency of machine learning in tunnel construction by using individually set annotation frames to generate accurate soil type predictions, enhancing estimation accuracy and adaptability with reduced training data.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- OHBAYASHI GUMI LTD
- Filing Date
- 2022-06-10
- Publication Date
- 2026-07-29
AI Technical Summary
Existing machine learning algorithms for determining soil fluidity in tunnel construction require extensive learning based on large datasets, which is time-consuming and labor-intensive due to varying tunnel site environments, necessitating individual learning.
A soil type estimation device with an imaging device, learning result storage, and control unit that uses a prediction model generated by machine learning to determine soil type from transported soil images, employing individually set annotation frames for each soil type.
Enables efficient and accurate estimation of soil type, improving accuracy, recall, precision, and harmonic mean through individually set annotation frames, allowing for rapid predictive model generation with minimal training data and adaptive soil type determination.
Smart Images
Figure 0007896360000002 
Figure 0007896360000003 
Figure 0007896360000004
Abstract
Description
Technical Field
[0002] , , , , , , ,
[0004] , , , , , , , , , , ,
[0003] , , , ,
[0001] The present invention relates to a soil property estimation device, a soil property estimation method, and a soil property estimation program for estimating the soil property of transported soil.
Background Art
[0002] In order to construct a tunnel, a shield method using a shield tunneling machine may be adopted. In this shield method, the earth and sand excavated by the cutter head are taken into the chamber and filled. Then, the stability of the face is achieved by the earth pressure in the chamber, and the soil is discharged through a screw conveyor. Here, a technique for determining the fluidity of the excavated earth and sand has been studied (see Patent Document 1). In the technique described in this document, a teacher dataset is obtained, which consists of an earth and sand image obtained by imaging a predetermined region in the conveying direction of a belt conveyor that conveys the earth and sand discharged from the shield tunneling machine, and a measurement result of the plastic fluidity of the earth and sand corresponding to the earth and sand image. Then, the fluidity of the earth and sand is determined by estimating the plastic fluidity of the target earth and sand, which is the earth and sand corresponding to a new earth and sand image, using a machine learning algorithm learned based on the teacher dataset.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
Means for Solving the Problems
[0005] A soil type estimation device for solving the above problems includes: an imaging device that photographs transported soil being transported from a first transport area to a second transport area; a learning result storage unit that records a prediction model generated by machine learning using training information with annotation frames set for each soil type in the images of the transported soil captured by the imaging device; and a control unit that uses the prediction model recorded in the learning result storage unit to determine the soil type of the transported soil from the images of the transported soil being transported from the first transport area to the second transport area. [Effects of the Invention]
[0006] According to this disclosure, the soil type of transported soil can be estimated efficiently and accurately. [Brief explanation of the drawing]
[0007] [Figure 1] This is a schematic diagram of the soil type estimation device according to the embodiment. [Figure 2] This is an explanatory diagram of the hardware configuration of the embodiment. [Figure 3] These are explanatory diagrams of annotation frames for the embodiments, where (a) is an explanatory diagram for "Belt", (b) is for "Hard", (c) is for "Normal", (d) is for "Soft", and (e) is for "Verysoft". [Figure 4] This is an explanatory diagram of the processing procedure of the embodiment. [Figure 5] This is an explanatory diagram of the processing procedure of the embodiment. [Figure 6] This is an explanatory diagram of the annotation frame for the comparative example. [Figure 7] This is an explanatory diagram of the effects of the embodiment. [Modes for carrying out the invention]
[0008] An embodiment of a soil type estimation device, soil type estimation method, and soil type estimation program will be described below with reference to Figures 1 to 7. In this embodiment, the soil type of transported soil excavated by the shield tunneling method is continuously measured. As shown in Figure 1, the shield tunneling machine 10 excavates the ground while sequentially assembling segments SG1 with the erector 10a to form a lining T1 that covers the inner surface of the excavation.
[0009] The shield tunneling machine 10 is equipped with a skin plate 11, a bulkhead 12, a cutter motor 13, a cutter head 14, an additive injection pipe 15, a screw conveyor 16, a belt conveyor 17, and the like.
[0010] The skin plate 11 is a cylindrical steel member that forms the outer shell of the shield tunneling machine 10. The partition wall 12 demarcates the chamber SP1 in front of the skin plate 11 in the direction of excavation. The cutter head 14 excavates the ground by rotation in front of the skin plate 11. The cutter motor 13 is the drive source that rotates the cutter head 14 via a support arm.
[0011] The additive injection pipe 15 is a tubular member for supplying an additive that fluidizes (turns into mud) the excavated soil. This additive is obtained by foaming a liquid foaming agent in a foaming device (not shown). The additive discharged from the additive injection pipe 15 is then injected onto the surface of the cutter head 14. Alternatively, the additive may be injected into the chamber SP1.
[0012] The excavated soil from the cutter head 14 is mixed with additives by the rotation of the cutter head 14, resulting in a highly fluid mixed soil that flows into the chamber SP1.
[0013] The screw conveyor 16 takes the mixed soil that has flowed into chamber SP1 into the working space behind the partition wall 12 in the direction of excavation. The pressure of the mixed soil that has flowed into chamber SP1 is measured by an earth pressure sensor. The thrust of the shield tunneling machine 10 and the discharge rate of excavated soil by the screw conveyor 16 are adjusted according to the pressure of the mixed soil measured by this earth pressure sensor. The belt conveyor 17 transports the mixed soil discharged from the screw conveyor 16 further to the shaft side.
[0014] At the exit of the screw conveyor 16 (the first conveying area), in the vicinity of the entrance of the belt conveyor 17 (the second conveying area), the imaging device 20a of the management device 20 is arranged to capture the earth and sand (conveyed soil). The imaging device 20a is fixed with the dumping area where the transfer is made from the screw conveyor 16 to the belt conveyor 17 as the field of view. This imaging device 20a is connected to the management device 20 and transmits the captured image data to the management device 20. In this embodiment, the soil quality estimation system is called including the imaging device 20a and the management device 20.
[0015] (Hardware configuration example) FIG. 2 is a hardware configuration example of the information processing device H10 that functions as the management device 20 and the like.
[0016] The information processing device H10 has a communication device H11, an input device H12, a display device H13, a storage device H14, and a processor H15. Note that this hardware configuration is an example, and it may have other hardware.
[0017] The communication device H11 is an interface that establishes a communication path with other devices and executes data transmission and reception, such as a network interface or a wireless interface.
[0018] The input device H12 is a device that receives inputs from an administrator or the like, such as a mouse or a keyboard. The display device H13 is a display or a touch panel that displays various information.
[0019] The storage device H14 is a device that stores data and various programs for executing various functions of the management device 20. Examples of the storage device H14 include a ROM, a RAM, and a hard disk.
[0020] The processor H15 uses programs and data stored in the memory device H14 to control each process in the management device 20 (for example, the processes in the control unit 21 described later). An example of the processor H15 is a CPU or MPU. This processor H15 loads programs stored in ROM, etc., into RAM and executes various processes corresponding to various operations. For example, when the application program of the management device 20 is started, the processor H15 operates the processes that execute each of the operations described later.
[0021] The processor H15 is not limited to performing all of its operations through software processing. For example, the processor H15 may include dedicated hardware circuits (e.g., application-specific integrated circuits: ASICs) that perform hardware processing for at least some of the operations it performs. In other words, the processor H15 may be configured as follows:
[0022] (1) One or more processors that operate according to a computer program (software) (2) One or more dedicated hardware circuits that perform at least some of the various processes, (3) Circuits that include combinations of those. The H15 processor includes the CPU and memory such as RAM and ROM, where memory stores program code or instructions configured to cause the CPU to execute processes. Memory, or computer-readable media, includes any media that can be accessed and used by a general-purpose or dedicated computer.
[0023] (System Configuration) As shown in Figure 1, the management device 20 includes a control unit 21, an annotation information storage unit 22, a teacher information storage unit 23, a learning result storage unit 24, and a measurement information storage unit 25.
[0024] The control unit 21 functions as an acquisition unit 211, a learning unit 212, and a management unit 213 by executing a soil type estimation program. The acquisition unit 211 performs the process of acquiring captured images and the process of generating training information. The learning unit 212 uses the annotation frames set on the captured image to generate a soil type prediction model. The management unit 213 uses a predictive model to perform a process that predicts the soil type of the transported soil.
[0025] The annotation information storage unit 22 stores annotation management information for setting annotation frames in captured images. This annotation management information is recorded when the learning process is executed. This annotation management information includes information about labels and frame placement.
[0026] The labels are classifications that indicate the image content, such as soil type, shown in the captured image. In this embodiment, five types of labels are used: "Belt (no transported soil)", "Hard (hard)", "Normal (normal)", "Soft (soft)", and "Verysoft (very soft)".
[0027] The frame placement refers to information about the position where the annotation frame is placed in the captured image to which this label is assigned. In this embodiment, a rectangular annotation frame is used, and the coordinates of each of the four corners of the rectangle are recorded relative to the local coordinates on the captured image.
[0028] Figure 3 shows the placement of annotation frames for each label. These annotation frames are placed in locations where the characteristics of each label are likely to be expressed in the captured image of the same field of view. For example, for the label "Belt," annotation frame F1 shown in Figure 3(a) is used. Annotation frame F1 is set in the area where the belt of the belt conveyor 17 is visible. For the labels "Hard," "Normal," and "Soft," annotation frames F2, F3, and F4 shown in Figures 3(b), (c), and (d) are used, respectively. Annotation frames F2, F3, and F4 are set in the area where the discharge port of the screw conveyor 16 is visible. For the label "Verysoft," annotation frame F5 shown in Figure 3(e) is used. Annotation frame F5 is set in the area from the screw conveyor 16 to the belt conveyor 17.
[0029] The training information storage unit 23 shown in Figure 1 stores training information used for machine learning. This training information is recorded with annotation frames set during the training process described later. For each captured image, labels and annotation frames are set in this training information.
[0030] The image shows the area where the transported soil falls from the screw conveyor 16 onto the belt conveyor 17. The label indicates the soil type classification designated by the administrator for this photograph. An annotation frame is an annotation frame set in a captured image using annotation management information.
[0031] The learning result storage unit 24 stores a prediction model for predicting soil type. This prediction model is recorded when the learning process is executed.
[0032] This prediction model is a learning model composed of a network (hidden layer) with an input layer containing captured images with annotation frames and an output layer containing labels. This prediction model can be generated, for example, by deep learning. However, the learning method is not limited to deep learning.
[0033] The measurement information storage unit 25 records estimated soil type measurement information for the transported soil. This measurement information is recorded when estimation processing is performed. This measurement information includes the estimated date and time and a label.
[0034] The estimated date and time is the date and time when the soil properties of the transported soil were estimated using images of the transported soil being evaluated. The labels represent the estimated soil type classification based on images of the transported soil being evaluated.
[0035] (Processing during learning) Next, we will explain the processing steps during the learning phase using Figure 4. First, the control unit 21 of the management device 20 performs the annotation frame setting process (step S11). Specifically, the acquisition unit 211 of the control unit 21 outputs the annotation setting screen to the display device H13. This annotation setting screen displays the captured image (initial image) taken using the imaging device 20a. In this case, the administrator sets the annotation frame for each tag. When the acquisition unit 211 receives input indicating that the annotation frame setting for each tag is complete, it records the annotation management information in the annotation information storage unit 22.
[0036] Next, the control unit 21 of the management device 20 performs the image registration process (step S12). Specifically, the acquisition unit 211 of the control unit 21 acquires images (training images) from the imaging device 20a. These training images are taken at the same position as the initial images. These images include a variety of images containing soil types such as "Belt," "Hard," "Normal," "Soft," and "Verysoft." In this embodiment, about 100 images are used. The acquisition unit 211 then records the acquired images in the training information storage unit 23.
[0037] Next, the control unit 21 of the management device 20 repeats the following process for each captured image. Here, the control unit 21 of the management device 20 performs image identification processing (step S13). Specifically, the acquisition unit 211 of the control unit 21 outputs the captured image recorded in the teacher information storage unit 23 to the display device H13. In this case, the administrator inputs a label indicating the soil type in the captured image into the input device H12. The acquisition unit 211 associates the input label with the outputted captured image and records it in memory.
[0038] Next, the control unit 21 of the management device 20 performs the annotation frame setting process (step S14). Specifically, the acquisition unit 211 of the control unit 21 sets annotation frames on the captured image according to the labels assigned based on the frame arrangement recorded in the annotation information storage unit 22.
[0039] Repeat the above process for all captured images until it is complete. Next, the control unit 21 of the management device 20 performs machine learning processing (step S15). Specifically, the learning unit 212 of the control unit 21 uses the training information stored in the training information storage unit 23 to generate a predictive model that estimates labels from captured images. This machine learning utilizes annotation frames set on the captured images. The learning unit 212 then records the generated predictive model in the learning result storage unit 24.
[0040] (Estimated processing) Next, the processing procedure for the estimation process will be explained using Figure 5. This process is performed continuously on the soil being transported by the screw conveyor 16 and the belt conveyor 17.
[0041] First, the control unit 21 of the management device 20 performs image acquisition processing (step S21). Specifically, the acquisition unit 211 of the control unit 21 acquires captured images (evaluation target images) from the imaging device 20a at predetermined time intervals (for example, several images / second). These evaluation target images are also captured at the same position as the initial image.
[0042] Next, the control unit 21 of the management device 20 performs soil evaluation processing (step S22). Specifically, the management unit 213 of the control unit 21 inputs the captured image into the prediction model recorded in the learning result storage unit 24 and calculates the probability for each label. The management unit 213 then identifies the label with the highest probability and temporarily stores it in memory. If there are no labels with a probability equal to or higher than the standard value, it assigns "no judgment".
[0043] Next, the control unit 21 of the control device 20 performs a determination process to determine whether the reference time has elapsed (step S23). Specifically, the management unit 213 of the control unit 21 calculates the elapsed time from the start of processing (for example, 1 second).
[0044] If it is determined that the reference time has not elapsed (i.e., "NO" in step S23), the control unit 21 of the management device 20 repeats the processing from the image acquisition process (step S21) onward.
[0045] If it is determined that the reference time has elapsed (if the answer is "YES" in step S23), the control unit 21 of the management device 20 performs the frequency of occurrence calculation process (step S24). Specifically, the management unit 213 of the control unit 21 calculates the frequency of occurrence of each label temporarily stored in memory. The management unit 213 then identifies the label with the highest frequency of occurrence and records it in the measurement information storage unit 25 in association with the current time. The management unit 213 then resets the memory.
[0046] Next, the control unit 21 of the management device 20 performs a determination process to determine whether it is a work stop (step S25). Specifically, the management unit 213 of the control unit 21 determines a work stop according to the excavation status of the shield tunneling method. For example, it determines a work stop when the excavation work for one ring is completed or based on the elapsed time since the previous work stop.
[0047] If it is determined that it is not a work section (i.e., "NO" in step S25), the control unit 21 of the management device 20 repeats the processing from the image acquisition process (step S21) onward. On the other hand, if it is determined that a work section has been completed (if the answer is "YES" in step S25), the control unit 21 of the management device 20 executes the output processing of the soil change history (step S26). Specifically, the management unit 213 of the control unit 21 outputs the history of measurement information recorded in the measurement information storage unit 25 to the display device H13. In this case, the manager checks the measurement information and adjusts the excavation conditions, the amount of additive injected, etc.
[0048] Next, the control unit 21 of the management device 20 performs a determination process to determine whether the process is complete (step S27). Specifically, the management unit 213 of the control unit 21 detects input from the administrator indicating the completion of the work via the input device H12 and determines that the process is complete ("YES" in step S27). If it is determined that the process is not complete (i.e., "NO" in step S27), the control unit 21 of the management device 20 repeats the processing from the image acquisition process (step S21) onward.
[0049] (action) When the field of view during image capture is fixed, the properties of the transported soil tend to manifest in specific areas of the captured image. For example, if the transported soil is relatively hard, its properties (morphology) tend to manifest when it is transported by a screw conveyor. Also, if the transported soil becomes soft, it spreads not only across the screw conveyor but also across the entire belt conveyor, and its properties (morphology) tend to manifest there as well. By individually setting annotation frames according to the labels, it is possible to generate a predictive model that takes into account the areas where the properties manifest.
[0050] According to this embodiment, the following effects can be obtained. (1) In this embodiment, annotation frames are individually set in the captured image, which is the training information, according to the label. This makes it possible to perform machine learning using regions in which properties corresponding to the soil type of the transported soil are likely to be expressed.
[0051] The evaluation of this prediction model was performed using the following process. The following mixing matrix was created for each label's soil type R1.
[0052] [Table 1]
[0053] Then, the accuracy, recall, precision, and harmonic mean for the estimation results are calculated. ·Correct answer rate ((TP+TN) / (TP+FP+TN+FN)) The accuracy rate is the percentage of predictions that were correct out of all predictions, and represents the percentage of predictions that matched visual observations.
[0054] ·Recall rate (TP / (TP+TN)) Recall is the proportion of predictions that are positive among those that are actually positive. If the visual observation is R1, then the proportion of predictions that are also R1 is also R1.
[0055] ·Precision rate (TP / (TP+FP)) Precision is the proportion of actual positive results among those that are actually positive. It is the proportion of R1 results for both predictions and visual observations.
[0056] ·Harmonic mean (2*precision*recall / (precision+recall)) The harmonic mean is the harmonic mean of precision and recall, which have contrasting characteristics. In this case, the value will be small if there are many FP and FN.
[0057] Figure 6 shows the annotation frame F0 that is set in common for each label. The estimation results using the prediction model trained with this annotation frame F0 are compared with the estimation results of this embodiment as a comparative example.
[0058] Figure 7 shows a comparison of the estimation results. Compared to the comparative example, by individually setting annotation frames F1 to F5 as in this embodiment, the accuracy, recall, precision, and harmonic mean can be improved.
[0059] (2) In this embodiment, the control unit 21 of the management device 20 performs image identification processing (step S13) and annotation frame setting processing (step S14). This makes it possible to efficiently generate training information with annotation frames set. The condition of the transported soil and the shooting conditions differ for each construction site, but it is possible to efficiently create a predictive model for each site using a small amount of training information.
[0060] (3) In this embodiment, if it is determined that the reference time has elapsed (if the answer is "YES" in step S23), the control unit 21 of the management device 20 performs the process of calculating the frequency of occurrence (step S24). This makes it possible to statistically determine the soil type of the transported soil even if there are fluctuations in the estimation results.
[0061] (4) In this embodiment, if it is determined that a work section has been completed (if the answer is "YES" in step S25), the control unit 21 of the management device 20 performs output processing of the soil change history (step S26). This makes it possible to adjust the amount of additive injected, the excavation conditions, etc., according to the condition of the transported soil.
[0062] Furthermore, this embodiment can be implemented with the following modifications. Note that this embodiment and the following modifications can be combined with each other to the extent that they do not contradict each other technically.
[0063] In the above embodiment, the soil type of the excavated soil from the shield tunneling machine 10 is estimated, but the application of the present invention is not limited to this. In the above embodiment, annotation frames F1 to F5 are set individually as shown in Figure 3. The arrangement of the annotation frames is not limited to this. For example, the annotation frames may be gradually widened according to the degree of softness. Alternatively, in the captured images to which each level has been assigned, regions with large feature quantities may be identified using machine learning. For example, a predictive model that predicts labels can be created using machine learning that can be explained by training information with a common annotation frame. Then, characteristic regions are identified in the training information to which each label has been assigned, an annotation frame is set for each label, and retraining is performed.
[0064] In the above embodiment, five types of labels are used. However, the number of label types is not limited to five. In the above embodiment, images of the soil discharge area from the screw conveyor 16 (first conveying area) to the belt conveyor 17 (second conveying area) are used. The imaging area is not limited to the discharge port. Specifically, an area in which the properties of the conveyed soil can be altered is desirable. For example, a step may be provided in the belt conveyor 17, and the falling conditions from the step and the conditions immediately after falling may be photographed and evaluated using the imaging device 20a.
[0065] Furthermore, it is possible to place a molding device above the belt conveyor 17, against which the top of the conveyed soil will strike, and to use images of the shape of the conveyed soil that has been blocked and accumulated by the molding device. In this case as well, according to the soil type classification, annotation frames are set in the areas where the properties are likely to be expressed.
[0066] In the above embodiment, the control unit 21 of the management device 20 performs a determination process to determine whether the reference time has elapsed (step S23). Here, the reference time may be set according to the transport speed of the screw conveyor 16 or the belt conveyor 17. [Explanation of Symbols]
[0067] 10...Shield tunneling machine, 11...Skin plate, 12...Bulkhead, 13...Cutter motor, 14...Cutter, 15...Additive injection pipe, 16...Screw conveyor, 17...Belt conveyor, 20...Management device, 20a...Photography device, 21...Control unit, 211...Acquisition unit, 212...Learning unit, 213...Management unit, 22...Annotation information storage unit, 23...Teacher information storage unit, 24...Learning result storage unit, 25...Measurement result storage unit.
Claims
1. A camera for photographing the transported soil being transported from the first transport area to the second transport area, An annotation information storage unit stores local coordinates of annotation frames set in regions where characteristics are likely to be expressed according to the soil type in the image of the field of view captured by the aforementioned imaging device, A learning result storage unit that uses the annotation information storage unit to set annotation frames in local coordinates corresponding to specified soil types in images of transported soil captured by the imaging device, and records a predictive model generated by machine learning using training information consisting of images with the annotation frames set. A soil type estimation device comprising: an imaging device that acquires images of the transported soil being transported from the first transport area to the second transport area from the imaging device; a control unit that uses a prediction model recorded in the learning result storage unit to calculate the likelihood for each soil type from the images of the transported soil being transported from the first transport area to the second transport area; and a control unit that determines the soil type of the transported soil according to the level of certainty.
2. The soil type estimation device according to claim 1, characterized in that the annotation frame is set to expand in accordance with the softness of the soil.
3. The control unit acquires the captured image taken by the imaging device, Using the aforementioned captured images, the soil type identified by the administrator is obtained. The soil type estimation device according to claim 1 or 2, characterized in that it generates training information in which annotation frames corresponding to the soil type are set in the captured image.
4. The soil type estimation device according to claim 1, characterized in that the first conveying area and the second conveying area are conveying areas on different types of conveyors, and the area where characteristics are likely to manifest according to the soil type is the area where the conveyed soil falls between the different types of conveyors.
5. The soil type estimation device according to claim 1, characterized in that the control unit estimates the soil type by calculating the frequency of soil types determined within a reference time.
6. The soil type estimation device according to claim 5, characterized in that the control unit outputs the history of the determined soil type for each work section.
7. The imaging device captures images of the soil being transported from the first transport area to the second transport area. In the image of the field of view captured by the aforementioned imaging device, an annotation information storage unit stores the local coordinates of annotation frames set in areas where characteristics are likely to be expressed according to the soil type for each soil type. In the image of the transported soil captured by the aforementioned imaging device, a prediction model generated by machine learning using training information that sets the local coordinates of annotation frames set in areas where characteristics are likely to be expressed according to the soil type for each soil type is recorded in the learning result storage unit. A soil type estimation method characterized in that the soil type estimation device acquires images of the transported soil being transported from the first transport area to the second transport area from the imaging device, calculates the probability for each soil type from the images of the transported soil being transported from the first transport area to the second transport area using the prediction model recorded in the learning result storage unit, and determines the soil type of the transported soil according to the level of the probability.
8. An annotation information storage unit that stores local coordinates of annotation frames set in regions where characteristics are likely to appear according to the soil type in an image of the field of view captured by a shooting device, A training information storage unit records images of the transported soil being transported from the first transport area to the second transport area, A soil type estimation program for determining the soil type of transported soil using a soil type estimation device comprising a control unit connected to the aforementioned teacher information storage unit, The control unit, In the captured images recorded in the aforementioned training information storage unit, annotation frames are set at local coordinates corresponding to the specified soil type, and a prediction model is generated using the training information consisting of the captured images with the annotation frames set, and recorded in the learning result storage unit. A soil type estimation program characterized by acquiring images of the transported soil being transported from the first transport area to the second transport area from the aforementioned imaging device, calculating the likelihood for each soil type from the images of the transported soil being transported from the first transport area to the second transport area using the prediction model recorded in the learning result storage unit, and determining the soil type of the transported soil according to the level of certainty.