Estimation device, estimation method and program
The estimation device uses machine learning to analyze sensor data and images from construction machines, addressing the inefficiencies of manual Schmidt hammer methods by providing rapid and consistent soil quality assessment in geological surveys.
Patent Information
- Application Number
- JP2023074997
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2043-04-28
AI Technical Summary
Current geological survey methods using a Schmidt hammer are time-consuming and subjective, requiring manual measurements and visual judgments, leading to variability in survey results and operator-dependent accuracy.
An estimation device and method that utilizes time-series data from construction machine sensors and machine learning models to automatically estimate soil quality by analyzing vibrations and images during excavation, employing classifiers to identify the work type and soil characteristics.
Enables rapid and consistent soil quality estimation with reduced variability, independent of the operator, by leveraging machine learning to analyze sensor data and images for accurate geological surveying.
Smart Images

Figure 2024159168000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an estimation device, an estimation method, and a program. [Background technology]
[0002] When carrying out construction work on the ground to build a new road, dam, tunnel, etc., a geological survey of the ground must be carried out before construction work begins. In a geological survey, the soil quality of the ground is investigated, for example, to determine whether the ground is made up of hard rock, medium-hard rock, soft rock, or soil and sand.
[0003] Currently, geological surveys are carried out by striking multiple points on the ground being surveyed with a Schmidt hammer (e.g., Patent Document 1), and making a comprehensive judgment based on the measurements of the load received by the Schmidt hammer at each point, as well as on the characteristics of the rock mass confirmed visually. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2023-13488 A Summary of the Invention [Problem to be solved by the invention]
[0005] In this method, workers manually measure multiple points on the rock mass using a Schmidt hammer, which takes time, and because workers must visually check and judge the characteristics of the rock mass, their judgments may vary depending on the worker's level of skill. It also takes time for workers to be trained. Therefore, a system that can perform geological surveys quickly and with a certain level of accuracy is desired. Here, a certain level of accuracy means that when geological surveys are repeated on the same ground, the fluctuation range of the survey results falls within a specified range.
[0006] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide a soil quality estimation device, an estimation method, and a program that enable geological surveys to be conducted with a certain degree of accuracy in a short construction period without relying on workers. [Means for solving the problem]
[0007] The estimation device according to a first aspect of the present invention includes a time series acquisition unit that acquires time series data detected by one or more sensors possessed by a construction machine, a division unit that divides the acquired time series data into waveform data of a predetermined time length, a first estimation unit that estimates the work performed by the construction machine at the time the divided waveform data was detected using a first classifier, an excavation extraction unit that extracts waveform data indicating that the estimated work is excavation from the divided waveform data, and a second estimation unit that estimates, using a second classifier, the soil type of the soil excavated at the time the waveform data extracted by the excavation extraction unit was detected.
[0008] Furthermore, the estimation method according to a second aspect of the present invention is characterized in that a computer acquires time series data detected by one or more sensors possessed by a construction machine, divides the acquired time series data into waveform data of a predetermined time length, estimates the work performed by the construction machine at the time when the divided waveform data was detected using a first classifier, extracts waveform data for which the estimated work is excavation from the divided waveform data, and estimates the soil quality of the soil excavated at the time when the waveform data extracted by the excavation extraction unit was detected using a second classifier.
[0009] Furthermore, a program according to a third aspect of the present invention is characterized in that it causes a computer to function as a time series acquisition unit that acquires time series data detected by one or more sensors possessed by a construction machine, a division unit that divides the acquired time series data into waveform data of a predetermined time length, a first estimation unit that estimates using a first classifier the operation of the construction machine at the time the divided waveform data was detected, an excavation extraction unit that extracts waveform data from the divided waveform data where the estimated operation is excavation, and a second estimation unit that estimates using a second classifier the soil type of the soil excavated at the time the waveform data extracted by the excavation extraction unit was detected. Effect of the Invention
[0010] According to the present invention, the soil quality can be estimated quickly and with a consistent quality regardless of the operator. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a diagram for explaining a configuration of a hydraulic excavator which is a construction machine according to an embodiment of the present invention. [Diagram 2] 1A is a diagram for explaining a vehicle body coordinate system, a global coordinate system, and a pitch angle, and FIG. 1B is a diagram for explaining a vehicle body coordinate system, a global coordinate system, and a roll angle. [Diagram 3] FIG. 2 is a diagram for explaining a hardware configuration of the estimation device according to the present embodiment. [Figure 4] FIG. 2 is a diagram for explaining the configurations and relationship between an estimation device and a construction machine according to the present embodiment. [Diagram 5] 5 is a flowchart showing an estimation process according to the present embodiment. [Figure 6] 1A is a diagram showing an example of time-series data, and FIG. 1B is a diagram showing an example of waveform data. [Figure 7] 10 is a flowchart showing one aspect of a determination process according to the present embodiment. [Figure 8] 10 is a flowchart showing another aspect of the determination process according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Hereinafter, an estimation device according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present embodiment is for illustrative purposes only and does not limit the scope of the present invention. Therefore, a person skilled in the art can adopt an embodiment in which each or all of the elements are replaced with an equivalent, and such an embodiment is also included in the scope of the present invention.
[0013] In this embodiment, work is performed on the ground that is the target of the geological survey using a construction machine such as a hydraulic excavator, and the soil quality of the soil is estimated based on data and images related to vibrations during work. An inference model trained by machine learning is used to estimate the soil quality of the soil. This inference model is prepared by learning using training data in which data or images related to vibrations during work, which are acquired in advance prior to the geological survey, are used as input data, and the soil quality of the soil during the work is used as correct answer data.
[0014] In this embodiment, a hydraulic excavator is used as a construction machine to excavate the ground to be surveyed, and the soil quality is estimated based on data related to vibrations during excavation and images of the discharged soil. As shown in Fig. 1, a hydraulic excavator, which is a construction machine 200 according to this embodiment, includes a vehicle body including a running body 10 and a rotating body 20 supported by the running body 10, a work machine 30 supported by the rotating body 20, and a blade 40 supported by the running body 10.
[0015] The running body 10 has tracks on the left and right sides, and travels by rotating the tracks. The rotating body 20 is supported on the running body 10 so as to be rotatable about a rotation axis.
[0016] The work machine 30 includes a boom 31, an arm 32, and a bucket 33. The boom 31 is rotatably connected to the rotating body 20, and rotates in the direction of an angle α when a boom cylinder 34 is hydraulically extended and retracted. The boom 31 rises when work oil flows into a port on the head side of the boom cylinder 34, and lowers when work oil flows into a port on the bottom side. In addition, the boom cylinder 34 is provided with a hydraulic sensor (not shown) that can measure the boom head pressure, which is the hydraulic pressure on the head side, and the boom bottom pressure, which is the hydraulic pressure on the bottom side.
[0017] The arm 32 is rotatably connected to the boom 31, and rotates in the direction of an angle β as the arm cylinder 35 expands and contracts due to hydraulic pressure. The arm cylinder 35 is also provided with a hydraulic sensor (not shown) so that the hydraulic pressure can be measured in the same manner as the boom cylinder 34.
[0018] The bucket 33 is rotatably connected to the arm 32, and rotates in the direction of angle γ as the bucket cylinder 36 expands and contracts hydraulically. The bucket 33 has a cutting edge for cutting an object to be excavated and a storage section for storing the cut object. The bucket cylinder 36 is also equipped with a hydraulic sensor (not shown) so that the hydraulic pressure can be measured in the same manner as the boom cylinder 34.
[0019] In this embodiment, data from a hydraulic sensor provided in the boom cylinder 34 is utilized, however, instead of or in addition to this, data from a hydraulic sensor provided in the arm cylinder 35 or bucket cylinder 36 can also be utilized as appropriate.
[0020] The blade 40 rotates up and down as indicated by the arrow by the extension and contraction of a lift cylinder (not shown). The lift cylinder is also provided with a hydraulic sensor (not shown) so that the hydraulic pressure can be measured in the same manner as the boom cylinder 34.
[0021] The worker sits in a driver's seat 221 arranged on the vehicle body of the construction machine 200 and operates the work implement 30 and the blade 40 by operating an operating device 220 such as an operating lever 222 .
[0022] A worker operates the work machine 30 to excavate the ground to be surveyed with the cutting edge of the bucket 33, and after storing the excavated soil in the bucket 33, turns the work machine 30 to discharge the soil. In order to obtain data and images during this series of operations, the operation device 220 of the construction machine 200 is provided with a vehicle angle sensor 232 and a camera 233.
[0023] The vehicle body angle sensor 232 detects the attitude of the construction machine 200. As shown in FIG. 2, a local coordinate system (x, y, z) is set with the intersection of the plane passing through the center of the revolving body 20 and the revolving axis as the origin. The vehicle body angle sensor 232 has an inclination sensor and a geomagnetic sensor. The inclination sensor is, for example, an IMU (Inertial Measurement Unit), and measures the acceleration and angular velocity of the vehicle body using a three-axis gyro and a three-directional accelerometer. The inclination sensor detects the inclination of the vehicle body based on the measurement results, that is, measures a roll angle θ2 representing the rotation about the X axis of the global coordinate system (X, Y, Z) as shown in FIG. 2(a) and a pitch angle θ1 representing the rotation about the Y axis of the global coordinate system as shown in FIG. 2(b). The pitch angle θ1 is the direction in which the vehicle body is lifted as the excavation work is performed by the work machine.
[0024] The geomagnetic sensor detects north and south, and measures the angular displacement of the yaw angle θ3 (not shown) which represents the rotation of the vehicle body relative to the Z axis of the global coordinate system.
[0025] The camera 233 is equipped with an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) whose light receiving surface including an imaging surface is composed of a plurality of light receiving elements arranged two-dimensionally. The camera 233 captures a video or image of the construction machine 200 during work. The video or image includes an image of the soil removed during work by the construction machine 200.
[0026] Next, a configuration of the estimation device 100 that acquires various data during work by the above-mentioned construction machine 200 and images of the discharged soil, and estimates the soil quality of the soil, will be described.
[0027] 3 is a diagram showing a hardware configuration of the estimation device 100. The estimation device 100 includes an input unit 710, a display unit 720, a control unit 730, a main memory unit 740, an auxiliary memory unit 750, and a communication unit 760, which are connected to each other via a bus 770.
[0028] The input unit 710 includes a mouse, a keyboard, etc. The display unit 720 includes a liquid crystal display, an organic EL (Electro Luminescence) display, etc.
[0029] The control unit 730 includes, for example, a central processing unit (CPU) that is an integrated circuit. The control unit 730 executes a program stored in the auxiliary storage unit 750 to realize various functions of the estimation device 100 and execute an estimation process described later.
[0030] The main memory unit 740 includes a RAM (Random Access Memory). Programs are loaded into the main memory unit 740 from the auxiliary memory unit 750. The main memory unit 740 is used as a working area for the control unit 730.
[0031] The auxiliary storage unit 750 includes a non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory), an SSD (Solid State Drive), and an HDD (Hard Disk Drive). In addition to programs, the auxiliary storage unit 750 stores various data used in the processing of the control unit 730. The auxiliary storage unit 750 supplies the data used by the control unit 730 to the control unit 730 in accordance with instructions from the control unit 730, and stores the data supplied from the control unit 730.
[0032] The communication unit 760 includes a network interface circuit for transmitting and receiving Ethernet frames from an external device. The communication unit 760 receives a signal from the outside and outputs data indicated by this signal to the control unit 730. The communication unit 760 also transmits a signal indicating the data output from the control unit 730 to the external device.
[0033] The input unit 710 and the display unit 720 may be configured as separate units from the estimation device 100. For example, the input unit 710 and the display unit 720 may be connected to the estimation device 100 from a control box of the estimation device 100 via an extension cable or the like.
[0034] Furthermore, the functions of the estimation device 100 can be realized by dedicated hardware or by a general computer system.
[0035] Furthermore, the program executed by the control unit 730 may be stored in a non-transitory computer-readable recording medium, distributed, and the program may be installed in a computer to configure the estimation device 100 according to this embodiment. Examples of such recording media include flexible disks, CD-ROMs (Compact Disc Read-Only Memory), DVDs (Digital Versatile Discs), and MOs (Magnetro-Optical Discs).
[0036] Also, the program may be stored in a disk device of a server device on a communication network such as the Internet, and may be downloaded to a computer, for example, by being superimposed on a carrier wave.
[0037] The estimation device 100 according to this embodiment can also be achieved by starting and executing a program while transferring it via a communication network.
[0038] Furthermore, the estimation device 100 according to this embodiment can also be achieved by executing all or part of the program on a server device and executing the program while a computer sends and receives information related to the processing via a communication network.
[0039] In addition, when the functions of the estimation device 100 are shared and realized by an OS (Operating System) or by cooperation between the OS and an application, only the parts other than the OS may be stored on a medium and distributed, or may be downloaded to a computer.
[0040] Furthermore, the means for realizing the functions of the estimation device 100 is not limited to software, and some or all of the functions may be realized by dedicated hardware including circuits.
[0041] 4 is a diagram showing the configuration of the estimation device 100 and the construction machine 200. First, the estimation device 100 includes a time series acquisition unit 110, an image acquisition unit 120, a division unit 130, a first estimation unit 140, an excavation extraction unit 150, a second estimation unit 160, a discharged soil extraction unit 170, a third estimation unit 180, and a determination unit 190.
[0042] The time series acquisition unit 110 acquires time series data detected by one or more sensors of the construction machine 200. The time series acquisition unit 110 receives time series data from the construction machine 200 via a wired or wireless network. Time series data related to a series of operations, such as the payload calculation value, boom head pressure, boom bottom pressure, rotation angular velocity, pitch angle θ1, roll angle θ2, yaw angle θ3, etc., are acquired. For example, when detecting the above values, the time series data acquired here is a 7-dimensional vector arranged in chronological order.
[0043] The image acquiring unit 120 acquires images captured by the camera 233 of the construction machine 200 from the construction machine 200 via a wired or wireless network. The dividing unit 130 divides the time series data acquired by the time series acquiring unit 110 into waveform data of a predetermined time length.
[0044] The first estimation unit 140 estimates the work performed by the construction machine 200 at the time when the waveform data divided by the division unit 130 is detected, using a first classifier. The first classifier is an inference model trained by machine learning, and is trained in advance using training data in which previously acquired waveform data is input data and labeling of the work performed by the construction machine 200 at the time the waveform data was obtained is output data. For example, a Long Short Term Memory (LSTM), a Recurrent Neural Network (RNN), or a decision tree (Classification and Regression Trees (CART)) can be applied to the first classifier, and various types of neural networks may also be adopted.
[0045] The first estimation unit 140 estimates whether the work indicated by the waveform data is excavation, turning, or earth removal by providing the waveform data divided by the division unit 130 as an input to the first classifier. The first estimation unit 140 also outputs a first confidence level C1 indicating how reliable the estimation by the first classifier is. Various types of machine learning technology, deep learning technology, neural network technology, etc. can be applied to the first classifier.
[0046] The excavation extraction unit 150 extracts, from the waveform data divided by the division unit 130, waveform data for which the work estimated by the first estimating unit 140 is excavation.
[0047] The second estimation unit 160 estimates, by a second classifier, the soil quality of the soil excavated at the time when the waveform data extracted by the excavation and extraction unit 150 was detected. The second classifier is an inference model trained by machine learning, and has been trained in advance using training data in which previously acquired waveform data is used as input data and the soil quality of the soil at the time the waveform data was obtained is labeled as output data.
[0048] For example, a Long Short Term Memory (LSTM), a Recurrent Neural Network (RNN), or Classification and Regression Trees (CART) can be applied to the second classifier, and various other neural networks may also be adopted.
[0049] The second estimation unit 160 estimates whether the soil quality of the soil indicated by the waveform data is, for example, soft rock or soil and sand by providing the waveform data extracted by the excavation and extraction unit 150 as input to the second classifier. The second estimation unit 160 also outputs a second confidence level C2 indicating how reliable the estimation by the second classifier is.
[0050] The earth discharge extraction unit 170 extracts, from the waveform data divided by the division unit 130, waveform data for which the work estimated by the first estimating unit 140 is earth discharge.
[0051] The third estimation unit 180 estimates the soil quality of the soil captured in the image acquired by the image acquisition unit 120 using a third classifier. The third classifier is an inference model trained by machine learning, and is trained in advance using training data in which a previously acquired image of dumped soil is used as input data and the image of dumped soil is labeled as output data. For example, a convolutional neural network (CNN) or a decision tree can be applied to the third classifier, and various types of neural networks may also be used. The techniques applicable to the first classifier, the second classifier, and the third classifier may be changed as appropriate. Furthermore, the techniques described above for each classifier may be applied to other classifiers.
[0052] The third estimation unit 180 estimates whether the soil quality of the soil included in the image is, for example, soft rock or soil and sand by providing the image at the time when the waveform data extracted by the soil discharge extraction unit 170 is detected as an input to the third classifier. The third estimation unit 180 also outputs a third confidence level C3 indicating how reliable the estimation by the third classifier is.
[0053] The determination unit 190 determines the soil quality of the soil in which work was done during the time period when the time series data was detected, based on the soil quality estimated by the second estimation unit 160 and the soil quality estimated by the third estimation unit 180.
[0054] Next, the construction machine 200 includes a construction machine control device 210, an operation device 220, and various sensors such as a hydraulic sensor 231, a vehicle body angle sensor 232, and a camera 233. The construction machine control device 210 includes a controller, a predetermined storage device, and the like, and reads out and executes programs and data stored in the storage device to control the construction machine 200 according to instructions from the operation device 220 and acquire data from the various sensors.
[0055] The construction machine control device 210 includes a payload calculation unit 211. The payload calculation unit 211 calculates the weight of the contents stored in the bucket 33 based on time series data from each hydraulic sensor 231 and time series data of the position of the bucket 33 calculated from the vehicle body angle sensor 232. In addition, the construction machine control device 210 calculates the rotation angular velocity during the rotation operation based on the drive of the rotation shaft that rotates the rotating body 20.
[0056] Furthermore, the construction machine control device 210 acquires, for example, time series data of the boom head pressure and boom bottom pressure from the hydraulic sensor 231, acquires time series data of the pitch angle θ1, roll angle θ2, and yaw angle θ3 from the vehicle body angle sensor 232, and acquires video or images from the camera. Then, in response to a request from the estimation device 100, the construction machine control device 210 transmits time series data of the payload calculation value, turning angular velocity, boom head pressure, boom bottom pressure, pitch angle θ1, roll angle θ2, yaw angle θ3, etc. to the estimation device 100 via a wired or wireless network.
[0057] Fig. 5 is a flowchart explaining the estimation process executed by the estimation device 100. In the estimation process, various time-series data and images of work on the ground are acquired from the construction machine 200, and the soil quality of the soil contained in the ground is estimated. The estimation process will be explained below with reference to Fig. 5.
[0058] The estimation process starts, for example, when an application or program for the estimation process is launched. When the estimation process starts, first, the time series acquisition unit 110 requests and acquires time series data for a series of works from the construction machine 200 (step S501). The time series data is a collection of the above-mentioned various types of time series data for a series of works. The time series data may be data obtained in real time, or may be data obtained collectively after a series of works is completed.
[0059] Next, the division unit 130 divides the time series data into waveform data of a predetermined time length (step S502). Here, it is assumed that the division unit 130 divides the time series data D shown in the example of FIG. 6(a) into waveform data d1 to d12 shown in the example of FIG. 6(b) at 5-second intervals. In FIGS. 6(a) and 6(b), the time series data is illustrated as a time series of seven numerical values detected by seven sensors for ease of understanding. In general, as described above, the time series data is data in which multidimensional vectors are arranged in chronological order. If the time series data is a seven-dimensional vector detected at 0.1-second intervals and is divided at 5-second intervals, the waveform data will be a set of 50 seven-dimensional vectors arranged.
[0060] Returning to Fig. 5, steps S503 to S505 are loop processes. The first estimation unit 140 provides the divided waveform data d1 as an input to the first classifier, and estimates the task in the waveform data d1 (step S504). Here, it is assumed that the first estimation unit 140 estimates that the task in the waveform data d1 is turning, as shown in Fig. 6(b).
[0061] The loop process is repeated until the operations in all the waveform data d1 to d12 are estimated (step S505). It is assumed that the first estimation unit 140 estimates that the operation in the waveform data d1 and d2 is turning, the operation in the waveform data d3 to d6 is excavation, the operation in the waveform data d7 and d8 is turning, the operation in the waveform data d9 to d11 is earth removal, and the operation in the waveform data d12 is stop.
[0062] 5, when the first estimating unit 140 estimates the work in all of the waveform data d1 to d12, the excavation extracting unit 150 extracts waveform data in which the estimated work is excavation (step S506). In the example shown in FIG 6, waveform data d3 to d6 are extracted.
[0063] Returning to Fig. 5, steps S507 to S509 are loop processes. The second estimation unit 160 first inputs the waveform data d3 of the extracted waveform data d3 to d6 to the second classifier, and estimates the soil quality indicated by the waveform data d3 (step S508). The loop process is repeated until the soil quality indicated by all of the extracted waveform data d3 to d6 has been estimated (step S509).
[0064] After estimating the soil properties indicated by all the extracted waveform data d3 to d6, the second estimation unit 160 determines the estimation results of the soil properties in the series of works (step S510). For example, the second estimation unit 160 determines the most frequently estimated result among the estimation results of the four waveform data d3 to d6 as the soil property.
[0065] Thereafter, the earth discharge extraction unit 170 extracts waveform data for which the work estimated by the first estimation unit 140 is earth discharge (step S511). In the example shown in Fig. 6, waveform data d9 to d11 are extracted.
[0066] 5, steps S512 to S515 are loop processes. The image acquisition unit 120 specifies the time when the waveform data d9 was detected among the waveform data d9 to d11 extracted by the soil discharge extraction unit 170, and requests and acquires an image at the specified time from the construction machine 200 (step S513).
[0067] The third estimation unit 180 provides the image acquired by the image acquisition unit 120 as an input to the third classifier, and estimates the soil quality of the soil contained in the image (step S514). The loop process is repeated until the time when all the waveform data d9 to d11 extracted by the soil discharge extraction unit 170 are detected is specified, and the soil quality of the soil contained in the image at each specified time is estimated (step S515).
[0068] After estimating the soil type for all images of the discharged soil, the third estimation unit 180 determines the estimation result (step S516). For example, the third estimation unit 180 determines the most frequently estimated result of the images related to the three waveform data d9 to d11 as the soil type.
[0069] Thereafter, the determination unit 190 executes a determination process described below, and finally determines the soil quality of the soil in the series of works (step S600) based on the estimation result by the second estimation unit 160 (step S510) and the estimation result by the third estimation unit 180 (step S516). When the determination process by the determination unit 190 ends, the estimation process ends.
[0070] 7 is a flowchart illustrating one aspect of the determination process by the determination unit 190. When the determination process starts, the determination unit 190 first determines whether the soil quality estimated by the second estimating unit 160 is the same as the soil quality estimated by the third estimating unit 180 (step S601).
[0071] If the judgment unit 190 determines that the soil quality estimated by the second estimation unit 160 and the soil quality estimated by the third estimation unit 180 are the same (Yes in step S601), it adopts that soil quality and judges that it is the soil quality of the soil worked on during the time period when the time series data D was detected (step S602).
[0072] On the other hand, if determining unit 190 determines that the soil quality estimated by second estimating unit 160 and the soil quality estimated by third estimating unit 180 are different (No in step S601), determining unit 190 obtains second confidence factor C2 and third confidence factor C3 from second estimating unit 160 and third estimating unit 180 (step S603). Then, determining unit 190 compares second confidence factor C2 and third confidence factor C3 (step S604).
[0073] If the determination unit 190 determines that the second confidence level C2 is greater than the third confidence level C3 (Yes in step S604), it adopts the soil quality estimated by the second estimation unit 160 and determines that the soil quality is that of the soil worked on in the time period when the time series data D was detected (step S605).
[0074] On the other hand, if the judgment unit 190 determines that the second confidence level C2 is smaller than the third confidence level C3 (No in step S604), it adopts the soil quality estimated by the third estimation unit 180 and determines that the soil quality is the soil on which work was performed during the time period when the time series data D was detected (step S606).
[0075] When the determining unit 190 has determined the final soil quality, the process proceeds to "A", where the estimation process described in FIG. 5 ends.
[0076] An inference model learned by machine learning may be used in the determination process by the determination unit 190. FIG.
[0077] The inference model used in the determination process described in Fig. 8 is prepared by learning from training data acquired in advance prior to the determination process. This inference model may be prepared in advance by determination unit 190. That is, the inference model is prepared by learning from training data in which the estimation result by second estimator 160 and second confidence factor C2 and the estimation result by third estimator 180 and third confidence factor C3 are input, and the soil type worked on in the time period in which time series data D was detected is output.
[0078] When the determination process starts, determination unit 190 provides the estimation result and second confidence factor C2 by second estimator 160 and the estimation result and third confidence factor C3 by third estimator 180 as inputs to a prepared inference model (step S621). Then, determination unit 190 determines the result output from the inference model as the soil quality of the soil worked in the time period when time-series data D was detected (step S622).
[0079] When the determining unit 190 has determined the final soil quality, the process proceeds to "A", where the estimation process described in FIG. 5 ends.
[0080] As described above, according to this embodiment, time-series data detected by a plurality of sensors of the construction machine during a series of operations is acquired, data during excavation operations is extracted, and then the soil quality of the soil is estimated using an inference model learned by machine learning. As a result, when a worker works on the ground to be surveyed using the construction machine, the soil quality of the soil of the ground can be estimated, thereby shortening the time required for geological survey. In addition, since an inference model learned by machine learning is used, a geological survey with a certain quality can be performed regardless of the worker.
[0081] Furthermore, according to this embodiment, in addition to the above-mentioned estimation, images of the construction machine during the earth removal work are acquired, and the soil quality is estimated using another inference model learned by machine learning, and used for the final judgment. This makes it possible to estimate the soil quality with higher accuracy.
[0082] Various embodiments and modifications of the present disclosure are possible without departing from the broad spirit and scope of the present disclosure. The above-described embodiments are for explaining the present disclosure and do not limit the scope of the present disclosure. That is, the scope of the present disclosure is indicated by the claims, not the embodiments. Various modifications made within the scope of the claims and the scope of the disclosure equivalent thereto are considered to be within the scope of the present disclosure. [Industrial Applicability]
[0083] The present invention can be widely applied to estimation devices for geological surveys. [Explanation of symbols]
[0084] 10 Running body 20 Rotating body 30 Work Machine 31 Boom 32 Arm 33 Bucket 34 Boom cylinder 35 Arm Cylinder 36 Bucket Cylinder 40 Blades 100 Estimator 110 Time series acquisition unit 120 Image acquisition unit 130 Split section 140 1st estimation part 150 Excavation and Extraction Section 160 Second estimation part 170 Soil extraction section 180 Third Estimation Department 190 Judgment section 200 Construction Machinery 210 Construction machinery control device 211 Payload Calculation Unit 220 Operating device 221 Driver's seat 222 Operating lever 231 Oil Pressure Sensor 232 Body angle sensor 233 Camera 710 Input section 720 Display section 730 Control Unit 740 Main memory 750 Auxiliary storage 760 Communications Department 770 Bus d1~d12 Waveform data D. Time series data α, β, γ angles θ1 Pitch angle θ2 roll angle
Claims
1. a time series acquisition unit that acquires time series data detected by one or more sensors of the construction machine; A division unit that divides the acquired time series data into waveform data of a predetermined time length; a first estimation unit that estimates an operation performed by the construction machine at a time when the divided waveform data is detected using a first classifier; an excavation extraction unit that extracts waveform data in which the estimated work is excavation from the divided waveform data; A second estimation unit that estimates the soil type of the excavated soil at the time when the waveform data extracted by the excavation extraction unit was detected using a second classifier; An estimation device comprising:
2. an earth discharge extraction unit that extracts waveform data in which the estimated work is earth discharge from the divided waveform data; An image acquisition unit that acquires an image captured by a camera of the construction machine at a time when the waveform data extracted by the soil discharge extraction unit is detected; and a third estimation unit that estimates the soil type of the soil captured in the acquired image by a third classifier.
2. The estimation device according to claim 1 .
3. The soil quality estimation unit further includes a determination unit for determining the soil quality of the soil in which work was performed during the time period in which the time-series data was detected, based on the estimated soil quality.
3. The estimation device according to claim 1 or 2.
4. The second estimation unit further outputs a confidence level for the soil type estimated by the second estimation unit, The third estimation unit further outputs a confidence level for the soil type estimated by the third estimation unit, The determination unit is configured to determine whether the soil quality estimated by the second estimation unit and the soil quality estimated by the third estimation unit are If they are the same, the soil quality is determined to be the soil quality of the soil worked during the time period when the time series data was detected; If the soil types are different, the soil type estimated by the estimation unit with the higher degree of certainty is determined to be the soil type of the soil worked on at the time when the time series data was detected.
4. The estimation device according to claim 3.
5. The second estimation unit further outputs a confidence level for the soil type estimated by the second estimation unit, The third estimation unit further outputs a confidence level for the soil type estimated by the third estimation unit, The determination unit is an inference model is trained on training data in which the estimation result by the second estimation unit and its certainty and the estimation result by the third estimation unit and its certainty are input, and the soil type worked on during the time period in which the time series data was detected is output; a new estimation result by the second estimation unit and its certainty factor, and a new estimation result by the third estimation unit and its certainty factor are provided as inputs to the trained inference model, thereby determining the soil quality of the soil worked during the time period when the time-series data was newly detected; 4. The estimation device according to claim 3.
6. The second classifier is an RNN, an LSTM, or a decision tree.
3. The estimation device according to claim 1 or 2.
7. The third classifier is a CNN or a decision tree.
3. The estimation device according to claim 2 .
8. The computer Acquire time series data detected by one or more sensors of the construction machine, Dividing the acquired time series data into waveform data of a predetermined time length; A first classifier estimates an operation performed by the construction machine at a time when the divided waveform data is detected; Extracting waveform data for which the estimated work is excavation from the divided waveform data; A second classifier estimates the soil quality of the excavated soil at the time when the waveform data extracted by the excavation extraction unit was detected. The estimation method according to claim 1,
9. Computer, a time series acquisition unit that acquires time series data detected by one or more sensors of the construction machine; A division unit that divides the acquired time series data into waveform data of a predetermined time length; a first estimation unit that estimates an operation performed by the construction machine at a time when the divided waveform data was detected using a first classifier; an excavation extraction unit that extracts waveform data in which the estimated work is excavation from the divided waveform data; a second estimation unit that estimates, by a second classifier, the soil quality of the soil excavated at the time when the waveform data extracted by the excavation extraction unit was detected; A program characterized by causing the program to function as a
Citation Information
Patent Citations
Management device for work machine and method for determining abnormality in work machine
JP2015083731A
Shovel support device
JP2016003462A
Work machine
JP2020002751A
Work machine and control method thereof
JP2021088815A
Motion Recognition Device
JP7234398B2
