Detection of work equipment ground contact state using vibration signals

By employing vibration sensors and machine learning to predict Implement-On-Ground periods, the system addresses inaccuracies in ground surface mapping and material tracking, ensuring precise interaction detection and efficient construction operations.

JP2024540834AActive Publication Date: 2024-11-06CATERPILLAR TRIMBLE CONTROL TECHNOLOGIES LLC
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2024520069
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-10-04
Filing Date
2022-09-29
Publication Date
2024-11-06
Estimated Expiration
2042-09-29

AI Technical Summary

Technical Problem

Existing construction equipment systems struggle to accurately predict the period during which a work implement interacts with the ground surface, leading to inaccuracies in ground surface mapping and material tracking.

Method used

Utilizing vibration sensors to capture movement signals, which are analyzed by a machine learning model to determine Implement-On-Ground (IOG) and Implement-In-Air (IIA) periods, enabling precise prediction of IOG start and end times through features like signal amplitude and frequency.

Benefits of technology

Enables accurate real-time adjustment of ground surface maps and enhances material tracking by precisely identifying when construction equipment interacts with the ground, improving operational efficiency and material management.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024540834000001_ABST
    Figure 2024540834000001_ABST
Patent Text Reader

Abstract

Systems, methods, and other techniques for determining a period of time during which an implement of a construction machine is interacting with a ground surface are described herein. A vibration signal indicative of implement movement is captured. One or more features are extracted from the vibration signal. The one or more features are provided to a machine learning model to generate a model output. From the model output, an implement on ground (IOG) start time and an IOG end time are predicted. The IOG start time and the IOG end time constitute a period of time.
Need to check novelty before this filing date? Find Prior Art

Description

[Background technology]

[0001]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. patent application Ser. No. 17 / 493,793, filed on October 4, 2021, entitled "IMPLEMENT-ON-GROUND DETECTION USING VIBRATION SIGNALS," the entire contents of which are incorporated herein by reference for all purposes.

[0002] Modern mobile machines, including construction and agricultural machines, have dramatically increased the efficiency of performing various work-related tasks. For example, earthmoving machines using automatic grade control systems can grade a project area using fewer passes than previously possible by hand. As another example, modern asphalt pavers and other road construction companies can replace old roads and build new roads in hours or days instead of weeks or months as was previously the case. Various aspects of automation allow construction and agricultural projects to be performed by smaller groups of workers than previously required. Many of the technological advances in mobile machines are due in part to the availability of precise sensors that allow real-time monitoring of the condition and location of the machine's components and / or the environment surrounding the machine. Summary of the Invention

[0003]

[0003] A summary of various embodiments of the invention is presented below as a list of examples. As used below, any reference to a series of examples should be understood disjunctively as a reference to each of those examples (e.g., "Examples 1-4" should be understood as "Examples 1, 2, 3 or 4").

[0004]

[0004] Example 1 is a method for determining a period during which a work implement of a construction machine is interacting with a ground surface, the method including capturing a vibration signal indicative of work implement movement, extracting one or more features from the vibration signal, providing the one or more features to a machine learning model to generate model output, and predicting an implement-on-ground (IOG) start time and an IOG end time based on the model output, where the IOG start time and IOG end time constitute a period during which the work implement is interacting with the ground surface.

[0005] Example 2 is the method of example 1, wherein the vibration signal is captured using a vibration sensor attached to the construction machine.

[0006] Example 3 is the method of example 2, wherein the vibration sensor includes an accelerometer and the vibration signal includes an acceleration signal.

[0007] Example 4 is the method of example 2, wherein the vibration sensor includes a gyroscope and the vibration signal includes a rotation signal.

[0008] Example 5 is the method of example 2, wherein the vibration sensor is attached to the work machine.

[0009] Example 6 is the method of examples 1-5, wherein the model output includes a set of IOG candidates corresponding to times when the implement is interacting with the ground surface.

[0010] Example 7 is the method of examples 1-6, wherein the model output includes a set of candidate implement-in-air states (IIA) corresponding to times when the implement is not interacting with the ground surface.

[0011] Example 8 is the method of examples 1-7, wherein the one or more features include at least one of a signal amplitude feature or a signal frequency feature.

[0012] Example 9 is the method of Examples 1-8, wherein the machine learning model is a pre-trained support vector machine.

[0013] Example 10 is the method of examples 1-9, further comprising adjusting the terrain surface map based on a period during which the implement interacts with the terrain surface and a path of the implement.

[0014]

[0014] Example 11 is a system that includes one or more processors and a computer-readable medium containing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including capturing a vibration signal indicative of movement of a work implement of a construction machine; extracting one or more features from the vibration signal; providing the one or more features to a machine learning model to generate model output; and predicting an implement on ground condition (IOG) start time and an IOG end time based on the model output, where the IOG start time and the IOG end time form a period during which the work implement is interacting with the ground surface.

[0015] Example 12 is the system of Example 11, where the vibration signal is captured using a vibration sensor attached to the construction machine.

[0016] Example 13 is the system of example 12, wherein the vibration sensor includes an accelerometer and the vibration signal includes an acceleration signal.

[0017] Example 14 is the system of example 12, wherein the vibration sensor is attached to the work machine.

[0018] Example 15 is the system of Examples 11-14, where the model output includes a set of IOG candidates corresponding to times when the implement is interacting with the ground surface.

[0019] Example 16 is the system of Examples 11-15, wherein the model output includes a set of candidate implement airborne states (IIA) corresponding to times when the implement is not interacting with the ground surface.

[0020] Example 17 is the system of examples 11-16, wherein the one or more features include at least one of a signal amplitude feature or a signal frequency feature.

[0021]

[0021] Example 18 is the system of Examples 11-17, in which the machine learning model is a pre-trained support vector machine.

[0022] Example 19 is the system of Examples 11-18, wherein the IOG start time and the IOG end time are predicted further based on a set of implement positions.

[0023]

[0023] Example 20 is a method for determining a period during which a work machine of a construction machine interacts with a ground surface, the method including capturing a vibration signal indicative of work machine movement, determining a set of work machine on ground condition (IOG) candidates corresponding to a time when the work machine is interacting with the ground surface, determining a set of work machine airborne condition (IIA) candidates corresponding to a time when the work machine is not interacting with the ground surface, and predicting an IOG start time and an IOG end time based on the set of IOG candidates and the set of IIA candidates, wherein the IOG start time and the IOG end time form a period during which the work machine is interacting with the ground surface.

[0024]

[0024] Example 21 is a non-transitory computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods of Examples 1-10 or Example 20.

[0025]

[0025] The accompanying drawings, which are included to provide a further understanding of the present disclosure, are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure, and together with the detailed description, serve to explain the principles of the present disclosure. No attempt is made to show structural details of the present disclosure in more detail than may be necessary for a fundamental understanding of the present disclosure and the various aspects in which the present disclosure can be implemented. [Brief description of the drawings]

[0026] [Figure 1A] 4 illustrates an example modification of a terrain surface map based on an implement path. [Figure 1B] 4 illustrates an example modification of a terrain surface map based on an implement path. [Figure 1C] 4 illustrates an example modification of a terrain surface map based on an implement path. [Diagram 2] 1 shows an example of prediction of IOG duration based on vibration signals. [Diagram 3] 1 illustrates an example embodiment of one or more techniques of the present disclosure within a built environment. [Figure 4] FIG. 1 illustrates an example machine control system. [Diagram 5] FIG. 1 illustrates an exemplary IOG detector. [Figure 6A] 1 shows an example of a vibration signal. [Figure 6B] 4 shows an example of extracted frequency features corresponding to a vibration signal. [Figure 7A] 4 shows examples of work machine operation characteristics. [Figure 7B] 4 shows examples of work machine operation characteristics. [Figure 8] 1 illustrates an example of prediction of IOG duration based on a vibration signal and its corresponding amplitude features, according to some embodiments of the present disclosure. [Figure 9] 13 shows an example of prediction of IOG end time based on vibration signal and implement distance. [Figure 10] An exemplary method is shown. [Figure 11] 1 illustrates an exemplary computer system comprising various hardware elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027]

[0037] In the accompanying drawings, similar components and / or features may have the same reference numbers. Furthermore, various components of the same type may be distinguished by appending a letter after the reference label, or by appending a dash after the reference label followed by a second numerical reference label that distinguishes between the similar components and / or features. When only a first numerical reference label is used herein, the description is applicable to any one of the similar components and / or features having the same first numerical reference label, regardless of the subscript.

[0028]

[0038] In some cases, a construction machine, such as an earthmoving machine, may keep a map of the surrounding ground surface. This map, which may alternatively be referred to as a "ground surface map," may include the current elevation or height of each of a number of locations surrounding the machine. As the construction machine performs a construction project by lowering the implement, cutting into the ground surface, and digging and lifting dirt, the ground surface map may be updated based on the path of the implement and the period of time the implement has been interacting with the ground surface. For example, if the implement has been cutting into the ground surface for three seconds, the new ground surface may be estimated to be approximately equal to the implement's position (e.g., the position of the tip of the excavator's blade) during those three seconds. The ground surface map may then be adjusted to reflect the new ground surface.

[0029]

[0039] Accurate adjustment of the ground surface map depends on two factors: the first is knowledge of the implement position and path, and the second is a determination of the period during which the implement is actually interacting with the ground (e.g., digging), which may alternatively be referred to as an "interaction period," an "implementation on ground (IOG) period," or an "IOG instance." If either of these factors cannot be accurately determined, the ground surface map may be unreliable and may contain error spikes, especially where the implement path intersects with the original ground surface. For example, if the IOG period is predicted to start before the actual digging start time, this may cause the ground level map to contain an upward spike because the implement is above the actual ground surface. Similarly, if the IOG period is predicted to end after the actual digging end time, this may also cause the ground level map to contain an upward spike because the implement has finished digging and is above the actual ground surface.

[0030]

[0040] The disclosed embodiments provide techniques for accurately predicting the duration of time that a work implement of a construction machine is interacting with the ground. The embodiments may include predicting an IOG start time when the work implement begins interacting with the ground and an IOG end time when the work implement ends interacting with the ground, which may correspond to the start and end of the IOG period, respectively. The disclosed techniques may rely on vibration signals captured using a vibration sensor mounted on the construction machine. The vibration signal may be indicative of the movement of the work implement during an excavation event. In various embodiments, the vibration sensor may be mounted directly on the work implement or elsewhere on the construction machine.

[0031]

[0041] Some embodiments may utilize a trained machine learning model, such as a neural network, that receives vibration signal features as input to generate IOG periods. In some cases, the machine learning model may generate a set of IOG candidates corresponding to times when the implement is predicted to be interacting with the ground and a set of implement airborne (IIA) candidates corresponding to times when the implement is predicted not to be interacting with the ground. The set of IOG candidates and the set of IIA candidates may be analyzed to identify clusters to predict IOG periods, where the IOG periods typically include as many IOG candidates as possible and exclude as many IIA candidates as possible.

[0032]

[0042] Several advantages are obtained by the present disclosure. For example, as described above, by predicting precise IOG periods, embodiments of the present disclosure allow for precise adjustments to the ground surface map. Such adjustments can be made in real-time as construction equipment operates within the construction environment. As another example, embodiments provide the ability to track material movement (e.g., where material comes from and where it is going). In certain applications, such as mining, it may be desirable to know whether material is valuable ore or low-value waste. This determination can be derived from precise knowledge of when and where the equipment contacts the ground surface.

[0033]

[0043] In the following description, various examples are described. For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the examples. However, it will be apparent to those skilled in the art that the examples may be practiced without the specific details. Furthermore, well-known features may be omitted or simplified in order not to obscure the embodiments being described.

[0034]

[0044] 1A-1C illustrate example modifications of a ground surface map 124 based on a work implement path 114, according to some embodiments of the present disclosure. In FIG. 1A, a construction machine 150 begins moving a work implement 121 along a work implement path 114. In various embodiments, the work implement path 114 may be set by an operator of the construction machine and / or may be automatically generated based on the details of a construction project or other work related task. In FIG. 1B, the work implement 121 follows the work implement path 114 and cuts into the ground surface 112, thereby modifying the ground surface 112 by removing dirt above the work implement path 114 while leaving dirt below the work implement path 114.

[0035]

[0045] 1C, the ground surface map 124 is adjusted based on a ground surface map adjustment 125 that is determined based on the implement path 114 and an IOG period that corresponds to the time frame of the excavation operation. In the illustrated example, the previous ground surface map 124-1 is adjusted to generate a new ground surface map 124-2 by replacing the previous ground surface map 124-1 with the ground surface map adjustment 125 for locations where the ground surface map adjustment 125 contains data (e.g., altitude or height). For locations outside of where the ground surface map adjustment 125 contains data, the new ground surface map 124-2 is set the same as the previous ground surface map 124-1.

[0036]

[0046] 2 illustrates an example of predicting IOG duration 240 based on a vibration signal 242, according to some embodiments of the present disclosure. In the illustrated example, the vibration signal 242 is captured using a vibration sensor while the work implement 221 follows a work implement path 214 through a ground surface 212. Although not illustrated, the vibration sensor may be mounted directly on the work implement 221 of the construction machine or elsewhere on the construction machine. The illustrated vibration signal is an acceleration signal along a particular axis (or any direction in space) that includes both positive and negative acceleration values.

[0037]

[0047] Based on the characteristics of the vibration signal 242, an IOG start time 236 (between 1.3 minutes and 1.31 minutes) when the implement 221 begins digging along the ground surface 212 and an IOG end time 238 (between 1.46 minutes and 1.47 minutes) when the implement 221 ends digging along the ground surface 212 are predicted. In the illustrated example, the vibration signal 242 is shown to have a higher amplitude during the IOG period 240 compared to outside of the IOG period 240. Also shown in FIG. 2 is an actual IOG period (or IOG truth 239) and a truth reference showing the corresponding actual IOG start and end times, illustrating the accuracy of the described technique of using vibration signatures to predict the IOG period 240.

[0038]

[0048] FIG. 3 illustrates an exemplary embodiment of one or more techniques of the present disclosure within a construction environment. Specifically, FIG. 3 illustrates a construction machine 350 deployed at a construction site 310 that is controlled at least in part by a control unit 360 that, in various embodiments, may be communicatively connected to a position sensor 358 and a vibration sensor 365 via wired and / or wireless connections. Although the construction site 310 is generally described herein as corresponding to an earthwork site, the present disclosure may be applicable to a wide variety of construction, maintenance, or agricultural projects in which heavy or mobile machinery is used. Similarly, while the construction machine 350 is generally described herein as corresponding to an earthwork machine, the various techniques described herein may be applicable to a wide variety of construction or heavy machinery such as graders, excavators, bulldozers, backhoes, paving machines (e.g., for concrete, asphalt, slipform, vibration, etc.), compactors, scrapers, loaders, material handlers, combines, spreaders, etc.

[0039]

[0049] In some embodiments, the construction machine 350 may include a tractor having wheels, axles, and a gasoline, diesel, electric, or steam powered engine to provide power and traction to the construction machine 350 to drive it along a desired path, often at a constant speed. The construction machine 350 may be a tracked vehicle incorporating a continuous track of treads or track plates driven by the wheels of the vehicle. An operator of the construction machine 350 may use a variety of input devices, such as levers, switches, buttons, pedals, steering wheels, and touch screens, to provide input to the control unit 360, which in turn causes various actuators to move the construction machine 350.

[0040]

[0050] In some cases, the construction machine 350 may include an implement 321, which may be the primary component of the construction machine 350 that is controlled to interact with the elements of the construction site 310. For example, at an earthmoving site, the implement 321 may be the component of the construction machine 350 that interacts with the dirt (e.g., pushes, scoops, cuts, etc.), such as a bulldozer blade, an excavator bucket, or a compactor drum. As another example, at an agricultural site, the implement 321 may be a combine header or a spreader boom. As another example, at a road construction site, the implement 321 may be an asphalt paving screed.

[0041]

[0051] It should be understood that in some embodiments, the work implement 321 and the construction machine 350 may be considered separate main parts with a semi-rigid joint between them. The joint is semi-rigid in that the main parts can move relative to each other, but can be fixed in a given orientation. Some examples of semi-rigid joints used on construction or earthmoving equipment include c-frames, angle c-frames, push arms, L-shaped push arms, etc.

[0042]

[0052] In some embodiments, the control unit 360 can determine a geospatial position 322 of the construction machine 350 based on sensor data captured by one or more sensors (e.g., position sensor 358) attached to the construction machine 350. For example, the position sensor 358 can be a GNSS receiver that receives radio signals from one or more Global Navigation Satellite Systems (GNSS) satellites 302. By processing the received radio signals, the position of the GNSS receiver can be calculated. During operation of the construction machine 350, the geospatial position 322 is determined and used to query a site design that can result in a translation between a two-dimensional (2D) position within the construction site 110 and a desired ground surface or excavation depth.

[0043]

[0053] As described herein, in some embodiments, a vibration sensor 365 may be used to capture a vibration signal 342 indicative of movement of the work implement 321. The vibration signal 342 may be provided to an IOG detector 334, which may extract features from the vibration signal 342 and provide the extracted features to a machine learning model used to predict one or more IOG period(s) 340. Each of the IOG period(s) 340 occurs during a larger excavation event (or excavation period), and each may correspond to a period during which the work implement 321 is interacting with the ground at the construction site 310. The IOG period(s) 340 may be associated with an IOG start time(s) 336 and an IOG end time(s) 338.

[0044]

[0054] 4 illustrates an example machine control system 400 according to some embodiments of the disclosure. The machine control system 400 may include various input devices 452, sensors 454, actuators 456, and computing devices to enable one or more operators of the construction machine to complete work related tasks. The components of the machine control system 400 may be attached to or integrated with components of the construction machine such that the construction machine may be considered to include the machine control system 400. The components of the machine control system 400 may be communicatively connected to one another via one or more wired and / or wireless connections.

[0045]

[0055] The machine control system 400 may include a control unit 460 that receives data from various sensors and inputs and generates commands that are sent to various actuators and output devices. In the illustrated example, the control unit 460 receives input data 453 from input device(s) 452, receives sensor data 455 from sensor(s) 454, and generates control signal(s) 457 that are sent to actuator(s) 456. The control unit 460 may include one or more processors and associated memory. In some embodiments, the control unit 460 may be communicatively connected to an external computing system 462 located outside the machine control system 400 and the construction machine. The external computing system 462 may send instructions to the control unit 460 for details of work-related tasks. The external computing system 462 may also send alerts and other general information, such as traffic conditions, weather conditions, and the location and status of material handling vehicles, to the control unit 460.

[0046]

[0056] In some embodiments, input device(s) 452 may receive input data 453 indicative of a desired movement of the vehicle, a desired movement of the implement, a desired height of the implement, actuation of one or more mechanisms on the implement (e.g., rotation of the bucket of an excavator), etc. Input device(s) 452 may include a keyboard, a touch screen, a touch pad, a switch, a lever, a button, a steering wheel, an accelerator pedal, a brake pedal, etc. In some embodiments, input device(s) 452 may be attached to any physical portion of the vehicle, such as within the cab of the vehicle.

[0047]

[0057] In some embodiments, the sensor(s) 454 may include one or more position sensors 458 and / or vibration sensor(s) 465. The position sensor(s) 458 may be a combination of a GNSS receiver that determines position using radio signals received from satellites and a total station that determines position by combining measurements of distance, vertical angle, and horizontal angle. The vibration sensor(s) 465 may include one or more sensors that detect movement of components of the construction machine to which they are securely attached. In some cases, the vibration sensor(s) 465 may include one or more microphones or other acoustic sensors. In some cases, the vibration sensor(s) 465 may include one or more inertial measurement unit (IMU) sensors and / or elements thereof. For example, the vibration sensor(s) 465 may include, among other possibilities, one or more gyroscopes for detecting angular acceleration, angular velocity, and / or angular position (or other rotational signals or data), one or more accelerometers for detecting linear acceleration, linear velocity, and / or linear position, and / or one or more magnetometers for detecting the above types of data.

[0048]

[0058] In some embodiments (e.g., when implemented as a gyroscope), the vibration sensor(s) 465 can directly detect angular rate and integrate to obtain angular position, or the vibration sensor can directly measure angular position and determine the change in angular position (e.g., calculate the derivative) to obtain angular rate. In many cases, the vibration sensor(s) 465 may be used to determine the yaw angle (angle of rotation about a vertical axis), pitch angle (angle of rotation about a lateral axis), and / or roll angle (angle of rotation about a longitudinal axis) of the construction machine.

[0049]

[0059] The control unit 460 may include various controllers and modules to assist in generating the control signal(s) 457 and computing other data. Each of the controllers and modules may include dedicated hardware and / or may execute using the main processor and / or memory of the control unit 460. In some embodiments, the control unit 460 may include a machine component position estimator 428 that estimates the position of the machine components based on the sensor data 455. For example, the machine component position estimator 428 may use the sensor data 455 to estimate the position of the work machine during the IOG process. Optionally, the machine component position estimator 428 may generate a model of the entire construction machine as it is currently located within the construction site. In such an embodiment, the model may be generated based on the sensor data 455 but also using feedback from the actuator(s) 456 and / or by tracking the control signal(s) 457.

[0050]

[0060] In some embodiments, the control unit 460 may include an IOG detector 434 that uses the vibration signal 442 along with other sensor data 455 to predict each IOG period that will occur during the excavation period. Each IOG period may be provided to a ground surface map creator 425 that maintains a ground surface map. The ground surface map creator 425 may adjust the ground surface map based on each IOG period provided by the IOG detector 434 along with the implement position provided by the machine component position estimator 428.

[0051]

[0061] The control signal(s) 457 may include direct current (DC) or alternating current (AC) voltage signals, DC or AC current signals, and / or information-containing signals. An example of an information-containing signal may be a controller area network (CAN) message that may be transmitted along a CAN bus or other communication medium. In some cases, the control signal(s) 457 include pneumatic or hydraulic. Upon receiving the control signal(s) 457, the actuator(s) 456 may operate in a specified manner, such as by extending, retracting, rotating, raising, or lowering a specified amount. The actuator(s) 456 may use various forms of power to operate components of the construction machine. For example, the actuator(s) 456 may be electric, hydraulic, pneumatic, mechanical, or thermal, among other possibilities.

[0052]

[0062] 5 illustrates an example IOG detector 534, according to some embodiments of the present disclosure. As inputs, the IOG detector 534 may be provided with a vibration signal 542 and a set of implement position(s) 529, which may be estimated by a machine component position estimator 528 based on sensor data 555 (and optionally based on the vibration signal 542). The IOG detector 435 may include a feature extractor 546 that extracts one or more features 547 (and optionally the implement position(s) 529) from the vibration signal 542.

[0053]

[0063] In some embodiments, features 547 may include signal amplitude features 547-1 related to the amplitude of vibration signal 542. For example, signal amplitude features 547-1 may include a time-varying amplitude of vibration signal 542, a magnitude of the amplitude of vibration signal 542, an average of the amplitude of vibration signal 542, a standard deviation of the amplitude of vibration signal 542, a short-time average amplitude to a long-term average amplitude (STA / LTA) of vibration signal 542, a long-term vibration change of vibration signal 542 (e.g., when vibration signal 542 is captured by a gyro), among other possibilities.

[0054]

[0064] In some embodiments, features 547 may include signal frequency features 547-2 related to the amplitude of vibration signal 542. For example, signal frequency features 547-2 may include the time-varying frequency of vibration signal 542 (e.g., over a contiguous time window), high frequency energy of vibration signal 542, low frequency energy of vibration signal 542, power spectral density of vibration signal 542, among other possibilities.

[0055]

[0065] In some embodiments, features 547 may include work machine motion features 547-3 determined based on work machine position(s) 529 (and, optionally, based on vibration signal 542 if vibration signal 542 includes an IMU signal). For example, work machine motion features 547-3 may include a curling motion or rotation of the work machine (e.g., curling motion of a bucket in an excavator), a distance between the work machine and a reference point on the construction machine (e.g., the distance between the work machine and the cab), a direction of travel of the work machine, a turning of the construction machine, among other possibilities.

[0056]

[0066] The features 547 may be provided to the machine learning model 532, which generates an output based on the features 547. For example, the features 547 may be used as an input to the machine learning model 532 and may pass through layers or nodes of the machine learning model 532. The output generated by the machine learning model 532 may be referred to as a model output. The model output may include a set of IOG candidates 544 and / or a set of IIA candidates 545. In various embodiments, the IOG candidates 544 and the IIA candidates 545 may be output directly by the machine learning model 532 or may be obtained by performing one or more post-processing steps on the model output.

[0057]

[0067] In some embodiments, the IOG detector 534 may include a post-processing step 551 for analyzing the model output (optionally including the IOG candidates 544 and / or the IIA candidates 545) and predicting the IOG period 540 and / or the IOG start time 536 and the IOG end time 538 based on this analysis. For example, the post-processing step 551 may include a step for identifying clusters of the IOG candidates 544 and / or the IIA candidates 545 and predicting the IOG period 540 and / or the IOG start time 536 and the IOG end time 538 based on the identified clusters. In one example, a time window encompassing a cluster of the IOG candidates 544 may be used for predicting the IOG period 540. In another example, a time window between two clusters of the IIA candidates 545 excluding each other may be used for predicting the IOG period 540. In another example, the transition between a cluster of IIA candidates 545 and a cluster of IOG candidates 544 may be used to predict IOG start time 536, and the transition between a cluster of IOG candidates 544 and a cluster of IIA candidates 545 may be used to predict IOG end time 538. In some embodiments, the machine learning model 532 may directly output the IOG duration 540, and / or the IOG start time 536 and IOG end time 538 without using a post-processing step 551.

[0058]

[0068] In some embodiments, the post-processing step 551 can implement implement motion characteristics 547-3 to predict the IOG period 540 and / or the IOG start time 536 and IOG end time 538. For example, the movement of the implement toward the cab of the construction machine can be indicative of an IOG process. In one example, the distance between the implement and the cab can be analyzed to determine a minimum distance value and its associated time, which can be a strong indicator of when the IOG end time 538 may occur. In another example, the maximum distance between the implement and the cab can be determined, which can be a strong indicator of when the IOG start time 536 may occur.

[0059]

[0069] The machine learning model 532 may be a support vector machine, a neural network, a convolutional neural network, a Bayesian network, among other possibilities. The machine learning model 532 may be trained through a training process in which multiple training vibration signals are provided to a feature extractor 546 to generate training features that are provided to the machine learning model 532. For each iteration during the training process, the IOG periods 540 generated using the training vibration signals may be compared to a truth reference to determine any difference (also referred to as "loss") between the training results and the truth reference. The loss may be used to modify the machine learning model 532 (e.g., by modifying weights or parameters of the machine learning model 532) to reduce loss during subsequent inference using the machine learning model 532. It should be noted that while the machine learning model 532 is shown as a single machine learning model, multiple machine learning models may be used to process different types of features 547 or different types of model outputs.

[0060]

[0070] 6A and 6B show examples of a vibration signal 642 and its corresponding extracted frequency features 647, respectively, according to some embodiments of the present disclosure. In FIG. 6A, an acceleration signal captured by an accelerometer is shown along with a truth reference indicating when IOG periods occur. In FIG. 6B, a power spectral density of the acceleration signal is shown along with a truth reference indicating when IOG periods occur. In some cases, the illustrated vibration signal and frequency features can be used to train a machine learning model by using the machine learning model to generate predicted IOG periods, comparing the predicted IOG periods to the truth reference, and modifying the machine learning model based on the comparison.

[0061]

[0071] 7A and 7B show examples of implement motion characteristics according to some embodiments of the present disclosure. In FIG. 7A, the implement distance (or bucket distance) between the implement and the cab is shown along with a truth reference indicating when the IOG period occurs. In FIG. 7B, the implement rotation (or bucket rotation, i.e., bucket angle) is shown along with a truth reference indicating when the IOG period occurs. Both FIG. 7A and FIG. 7B show the original data along with the smoothed data.

[0062]

[0072] 7A and 7B demonstrate a strong correlation between implement operation characteristics and IOG duration. For example, FIG. 7A shows that IOG duration is strongly correlated with decreasing bucket distance. In some embodiments, maximum and minimum bucket distance values ​​(i.e., peaks and valleys, respectively) can be identified, and the IOG duration (including IOG start and end times) can be constrained to be between the times associated with the maximum and minimum values. As another example, FIG. 7B shows that IOG duration is strongly correlated with decreasing bucket rotation. In some embodiments, maximum and minimum bucket rotation values ​​can be identified, and the IOG duration (including IOG start and end times) can be constrained to be between the times associated with the maximum and minimum values.

[0063]

[0073] FIG. 8 illustrates an example of a prediction of an IOG period based on a vibration signal and its corresponding amplitude feature 847, according to some embodiments of the present disclosure. Also shown are predictions of an IOG start time 836 and an IOG end time 838, as well as a truth reference indicating when an IOG period occurs. In the illustrated example, the amplitude feature 847 represents a change in amplitude of the vibration / acceleration signal. Each value of the amplitude feature 847 is calculated by summing the magnitude of a first window containing eight values ​​and dividing by the sum of the magnitude of a second window containing 64 values. The two windows are aligned to the last sample of the window.

[0064]

[0074] The amplitude features of the illustrated vibration signal are provided to a machine learning model that generates IOG candidates 844 and IIA candidates 845, which are shown in FIG. 8 with associated times. During a post-processing step, a cluster of IOG candidates 844 and a cluster of IIA candidates 845 can be identified. The transition from the cluster of IIA candidates 845 to the cluster of IOG candidates 844 can be used to predict IOG start time 836 based, for example, on the time of the first IOG candidate in each IOG cluster. The transition from the cluster of IOG candidates 844 to the cluster of IIA candidates 845 can be used to predict IOG end time 838 based, for example, on the time of the first IIA candidate in each IIA cluster and / or the time of the last IOG candidate in each IOG cluster.

[0065]

[0075] FIG. 9 illustrates an example of prediction of IOG end time 938 based on vibration signal 942 and implement distance (bucket distance) according to some embodiments of the present disclosure. A truth reference is also shown indicating when the IOG period occurs. In the illustrated example, several IOG candidates are generated by a machine learning model based on the vibration signal features. During a post-processing step, a cluster of IOG candidates is identified and IOG start and end times are predicted. In some embodiments, the IOG start and end times are constrained to be between the maximum implement distance (max bucket distance) and the minimum implement distance (min bucket distance).

[0066]

[0076] In some embodiments, an outlier IOG candidate (approximately 2.4 minutes) may be used to determine whether to predict IOG end time 838 or whether to ignore it. By examining the time difference between the IOG candidate at 2.4 minutes and the time of minimum bucket distance (also approximately 2.4 minutes), it may be determined that the IOG candidate at 2.4 minutes should be used to predict IOG end time 838 and should not be ignored. In some embodiments, the time difference between a particular IOG candidate and the time of minimum bucket distance may be compared to a threshold to determine whether the particular IOG candidate should be considered for determining IOG end time, with a time difference exceeding the threshold causing the particular IOG candidate to be ignored.

[0067]

[0077] FIG. 10 illustrates an exemplary method 1000 according to some embodiments of the present disclosure. In various embodiments, the method 1000 may be considered to be a method for determining a period during which an implement of a construction machine interacts with a ground surface, a method for determining a start time and / or an end time during which an implement of a construction machine interacts with a ground surface, a method for adjusting a ground surface map, a method for generating a ground surface map, among other possibilities. During execution of the method 1000, one or more steps in the method 1000 may be omitted, and the steps of the method 1000 may be executed in any order and / or in parallel. One or more steps of the method 1000 may be executed by one or more processors, such as those included in a control unit (e.g., control unit 360, 460). The method 1000 may be implemented as a computer-readable medium or computer program product including instructions that, when executed by one or more computers, cause one or more computers to execute the steps of the method 1000.

[0068]

[0078] In step 1002, a vibration signal (e.g., vibration signal 242, 342, 442, 542, 642, 942) is captured. The vibration signal may be captured using a vibration sensor (e.g., vibration sensor 365, 465). The vibration sensor may be attached to a work implement (e.g., work implement 121, 221, 321) of a construction machine (e.g., construction machine 150, 350). The vibration signal may be indicative of movement of the work implement. In some embodiments, the vibration sensor may include an accelerometer, and the vibration signal may include an acceleration signal. In some embodiments, the vibration sensor may include a gyroscope, and the vibration signal may include a rotation signal. In some embodiments, the vibration sensor may include an IMU sensor (e.g., an accelerometer or a gyroscope), and the vibration signal may include an IMU signal (e.g., an acceleration signal or a rotation signal). In some embodiments, the vibration sensor may include an acoustic sensor (e.g., a microphone).

[0069]

[0079] In step 1004, one or more features (e.g., features 547, 647, 847) are extracted from the vibration signal. The one or more features may include signal amplitude features (e.g., signal amplitude feature 547-1), signal frequency features (e.g., signal frequency feature 547-2), implement motion features (e.g., implement motion features 547-3), among other possibilities. The one or more features may be extracted by a feature extractor (e.g., feature extractor 546). The feature extractor may extract one or more features from a set of implement positions (e.g., implement positions 529) in addition to the vibration signal.

[0070]

[0080] In step 1006, the one or more features are provided to a machine learning model (e.g., machine learning model 532). Upon providing the one or more features to the machine learning model, the machine learning model can generate a model output based on the one or more features. The model output may include a set of IOG candidates (e.g., IOG candidates 544, 844) and / or a set of IIA candidates (e.g., IIA candidates 545, 845). The set of IOG candidates may correspond to times when the implement is interacting with the ground surface. The set of IIA candidates may correspond to times when the implement is not interacting with the ground surface (i.e., the implement is above the ground surface). The machine learning model may be a pre-trained support vector machine, a neural network, among other possibilities.

[0071]

[0081] In step 1008, an IOG start time (e.g., IOG start time 336, 536, 836) and / or an IOG end time (e.g., IOG end time 338, 538, 838, 938) is predicted based on the model output. The IOG start time and the IOG end time may form an IOG period (e.g., IOG period 340, 540) during which the implement is interacting with the ground surface (e.g., ground surface 112, 212) of the construction site (e.g., construction site 310). The implement may interact with the ground surface when the implement is cutting or excavating into the ground surface. The IOG start time and / or the IOG end time may be predicted based on a set of IOG candidates and / or a set of IIA candidates. In some embodiments, the IOG start time and / or the IOG end time may be predicted based on the implement's operational characteristics and / or a set of implement positions.

[0072]

[0082] In step 1010, the ground surface map (e.g., ground surface map 124) is adjusted based on the period of time that the implement is interacting with the ground surface (and / or the IOG start time and IOG end time). The ground surface map may be further adjusted based on the path of the implement during the period of time. Adjusting the ground surface map may include modifying / changing / updating the ground surface map to reflect the path of the implement during the period of time. In some embodiments, the ground surface map may be set equal to the path of the implement during that period of time (e.g., the position of the bucket edge).

[0073]

[0083] It should be noted that method 1000 may be modified to predict various periods associated with the operation of a construction machine in addition to or instead of predicting IOG periods. For example, in various embodiments, method 1000 may be modified to predict periods when the construction machine is dumping, turning, tramming, idling, reversing, accelerating, and decelerating, among other possibilities.

[0074]

[0084] FIG. 11 illustrates an exemplary computer system 1100 with various hardware elements according to some embodiments of the present disclosure. The computer system 1100 may be incorporated or integrated into the apparatus described herein and / or may be configured to execute some or all of the steps of the methods provided by various embodiments. For example, in various embodiments, the computer system 1100 may be incorporated into the machine control system 400 and / or may be configured to execute the method 1000. It should be noted that FIG. 11 is only meant to present a generalized description of various components, any or all of which may be appropriately utilized. Thus, FIG. 11 broadly illustrates how individual system elements may be implemented in a relatively separated manner or in a relatively more integrated manner.

[0075]

[0085] In the illustrated example, computer system 1100 includes a communication medium 1102, one or more processor(s) 1104, one or more input device(s) 1106, one or more output device(s) 1108, a communication subsystem 1110, and one or more memory device(s) 1112. Computer system 1100 may be implemented using a variety of hardware implementations and embedded systems technologies. For example, one or more elements of computer system 1100 may be implemented as a field programmable gate array (FPGA), such as those commercially available from XILINX®, INTEL®, or LATTICE SEMICONDUCTOR®, a system on a chip (SoC), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a microcontroller, and / or a hybrid device such as a SoC FPGA, among other possibilities.

[0076]

[0086] The various hardware elements of the computer system 1100 may be communicatively coupled via a communication medium 1102. While the communication medium 1102 is shown as a single connection for clarity, it should be understood that the communication medium 1102 may include various numbers and types of communication media for transferring data between the hardware elements. For example, the communication medium 1102 may include one or more wires (e.g., conductive traces, paths, or leads on a printed circuit board (PCB) or integrated circuit (IC), microstrip, stripline, coaxial cable), one or more optical waveguides (e.g., optical fiber, strip waveguide), and / or one or more wireless connections or links (e.g., infrared wireless communication, wireless communication, microwave wireless communication), among other possibilities.

[0077]

[0087] In some embodiments, the communication medium 1102 may include one or more buses connecting pins of hardware elements of the computer system 1100. For example, the communication medium 1102 may include a bus connecting the processor(s) 1104 and the main memory 1114, referred to as a system bus, and a bus connecting the main memory 1114 and the input device(s) 1106 or the output device(s) 1108, referred to as an expansion bus. The system bus itself may be composed of several buses including an address bus, a data bus, and a control bus. The address bus may transmit memory addresses from the processor(s) 1104 to an address bus circuit associated with the main memory 1114 so that the data bus may access data contained in the memory addresses and return them to the processor(s) 1104. The control bus may transmit commands from the processor(s) 1104 and return status signals from the main memory 1114. Each bus may include multiple wires for transmitting multiple bits of information, and each bus may support serial or parallel transmission of data.

[0078]

[0088] The processor(s) 1104 may include one or more central processing units (CPUs), graphics processing units (GPUs), neural network processors or accelerators, digital signal processors (DSPs), and / or other general-purpose or special-purpose processors capable of executing instructions. The CPU may take the form of a microprocessor that may be fabricated on a single IC chip of metal-oxide-semiconductor field-effect transistor (MOSFET) construction. The processor(s) 1104 may include one or more multi-core processors, where each core may read and execute program instructions simultaneously with the other cores, enhancing the speed of programs that support multi-threading.

[0079]

[0089] The input device(s) 1106 may include one or more of various user input devices, such as a mouse, keyboard, microphone, and various sensor input devices, such as an image capture device, a pressure sensor (e.g., barometer, tactile sensor), a temperature sensor (e.g., thermometer, thermocouple, thermistor), a motion sensor (e.g., accelerometer, gyroscope, tilt sensor), a light sensor (e.g., photodiode, photodetector, charge-coupled device), and / or the like. The input device(s) 1106 may also include a device for reading and / or receiving a removable storage device or other removable media. Such removable media may include optical disks (e.g., Blu-ray® disks, DVDs, CDs), memory cards (e.g., Compact Flash® cards, Secure Digital (SD) cards, memory sticks), floppy disks, Universal Serial Bus (USB) flash drives, external hard disk drives (HDD) or solid state drives (SSD), and / or the like.

[0080]

[0090] The output device(s) 1108 may include one or more of a variety of devices that convert information into a human readable form, such as, but not limited to, a display device, a speaker, a printer, a haptic or tactile device, and / or the like. The output device(s) 1108 may also include a device for writing to a removable storage device or other removable medium, such as those described with reference to the input device(s) 1106. The output device(s) 1108 may also include various actuators for causing physical movement of one or more components. Such actuators may be hydraulic, pneumatic, electric, and may be controlled using control signals generated by the computer system 1100.

[0081]

[0091] The communications subsystem 1110 may include hardware components for connecting the computer system 1100 to systems or devices located outside the computer system 1100, such as via a computer network. In various embodiments, the communications subsystem 1110 may include a wired communications device connected to one or more input / output ports (e.g., a universal asynchronous receiver / transmitter (UART)), an optical communications device (e.g., an optical modem), an infrared communications device, a wireless communications device (e.g., a wireless network interface controller, a BLUETOOTH device, an IEEE 802.11 device, a Wi-Fi device, a Wi-Max device, a cellular device), among other possibilities.

[0082]

[0092] The memory device(s) 1112 may include various data storage devices of the computer system 1100. For example, the memory device(s) 1112 may include various types of computer memory having various response times and capacities, ranging from fast response times and low capacity memory, such as processor registers and caches (e.g., L0, L1, L2), to medium response times and medium capacity memory, such as random access memory (RAM), to slow response times and low capacity memory, such as solid state drives and hard disk disks. Although the processor(s) 1104 and the memory device(s) 1112 are shown as separate elements, it should be understood that the processor(s) 1104 may include various levels of on-processor memory, such as processor registers and caches, that may be utilized by a single processor or shared among multiple processors.

[0083]

[0093] The memory device(s) 1112 may include a main memory 1114 that may be directly accessible by the processor(s) 1104 via a memory bus of the communication medium 1102. For example, the processor(s) 1104 may continuously read and execute instructions stored in the main memory 1114. Thus, as shown in FIG. 11, various software elements may be loaded into the main memory 1114 and read and executed by the processor(s) 1104. Typically, the main memory 1114 is a volatile memory, which loses all data when the power is turned off and therefore requires power to save the stored data. The main memory 1114 may further include a small portion of non-volatile memory that includes software (e.g., firmware such as a BIOS) used to read other software stored in the memory device(s) 1112 into the main memory 1114. In some embodiments, the volatile memory of main memory 1114 is implemented as RAM, such as dynamic random access memory (DRAM), and the non-volatile memory of main memory 1114 is implemented as read-only memory (ROM), such as flash memory, erasable programmable read-only memory (EPROM), or electrically erasable programmable read-only memory (EEPROM).

[0084]

[0094] Computer system 1100 may include software elements shown as currently residing in main memory 1114, which may include an operating system, device driver(s), firmware, compilers, and / or other code, such as one or more application programs, which may include computer programs provided by various embodiments of the present disclosure. By way of example only, one or more steps described with respect to any of the methods described above may be implemented as instructions 1116 executable by computer system 1100. In one example, such instructions 1116 may be received by computer system 1100 using communications subsystem 1110 (e.g., via a wireless or wired signal carrying the instructions 1116), transmitted over communications medium 1102 to memory device(s) 1112, stored within memory device(s) 1112, read into main memory 1114, and executed by processor(s) 1104 to perform one or more steps of the described methods. In another example, the instructions 1116 may be received by the computer system 1100 using the input device(s) 1106 (e.g., via a removable media reader), transmitted over the communication medium 1102 to the memory device(s) 1112, stored in the memory device(s) 1112, read into the main memory 1114, and executed by the processor(s) 1104 to perform one or more steps of the described method.

[0085]

[0095] In some embodiments of the present disclosure, the instructions 1116 are stored in a computer-readable storage medium (or simply a computer-readable medium). Such a computer-readable medium may be non-transitory and therefore may be referred to as a non-transitory computer-readable medium. In some cases, the non-transitory computer-readable medium may be incorporated within the computer system 1100. For example, the non-transitory computer-readable medium may be one of the memory device(s) 1112 (shown in FIG. 11). In some cases, the non-transitory computer-readable medium may be separate from the computer system 1100. In one example, the non-transitory computer-readable medium may be a removable medium provided on the input device(s) 1106 (shown in FIG. 11), such as those described with reference to the input device(s) 1106, and the instructions 1116 are read into the computer system 1100 by the input device(s) 1106. In another example, the non-transitory computer-readable medium may be a component of a remote electronic device, such as a mobile phone, that can wirelessly transmit data signals carrying the instructions 1116 to the computer system 1100 and be received by the communications subsystem 1110 (shown in FIG. 11 ).

[0086]

[0096] The instructions 1116 may take any suitable form that can be read and / or executed by the computer system 1100. For example, the instructions 1116 may be source code (written in a human readable programming language such as Java, C, C++, C#, Python, etc.), object code, assembly language, machine code, microcode, executable code, etc. In one example, the instructions 1116 may be provided to the computer system 1100 in the form of source code, a compiler may be used to convert the instructions 1116 from source code to machine code, and then loaded into the main memory 1114 for execution by the processor(s) 1104. As another example, the instructions 1116 are provided to the computer system 1100 in the form of an executable file having machine code that can be immediately loaded into the main memory 1114 for execution by the processor(s) 1104. In various examples, the instructions 1116 may be provided to the computer system 1100 in encrypted or unencrypted, compressed or uncompressed format, among other possibilities, as an installation package or initial setup for a broader software deployment.

[0087]

[0097] In one aspect of the disclosure, a system (e.g., computer system 1100) is provided for performing methods according to various embodiments of the disclosure. For example, some embodiments may include a system comprising one or more processors (e.g., processor(s) 1104) communicatively coupled to a non-transitory computer-readable medium (e.g., memory device(s) 1112 or main memory 1114). The non-transitory computer-readable medium may store instructions (e.g., instructions 1116) that, when executed by the one or more processors, cause the one or more processors to perform the methods described in the various embodiments.

[0088]

[0098] In another aspect of the disclosure, a computer program product is provided that includes instructions (e.g., instructions 1116) for performing methods according to various embodiments of the disclosure. The computer program product may be tangibly embodied in a non-transitory computer-readable medium (e.g., memory device(s) 1112 or main memory 1114). The instructions may be configured to cause one or more processors (e.g., processor(s) 1104) to perform the methods described in the various embodiments.

[0089]

[0099] In another aspect of the disclosure, a non-transitory computer-readable medium (e.g., memory device(s) 1112 or main memory 1114) is provided. The non-transitory computer-readable medium can store instructions (e.g., instructions 1116) that, when executed by one or more processors (e.g., processor(s) 1104), cause the one or more processors to perform the methods described in the various embodiments.

[0090]

[0100] The methods, systems, and devices described above are examples. Various configurations may omit, substitute, or add various procedures or components, as appropriate. For example, in alternative configurations, the methods may be performed in an order different from that described, and / or various steps may be added, omitted, and / or combined. Also, features described with respect to particular configurations may be combined in various other configurations. Different aspects and elements of the configurations may be combined in a similar manner. Also, technology evolves, and thus many of the elements are examples and are not limiting of the disclosure or the claims.

[0091]

[0101] Specific details are given in the above description to provide a thorough understanding of example configurations including embodiments. However, configurations can be implemented without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques are shown without unnecessary detail to avoid obscuring the configurations. This description provides only example configurations and does not limit the scope, applicability, or configuration of the claims. Rather, the above description of the configurations provides those skilled in the art with an enabling description for implementing the described technology. Various changes can be made in the function and arrangement of elements without departing from the spirit or scope of the present disclosure.

[0092]

[0102] Although several example configurations have been described, various modifications, alternative configurations, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may be components of a larger system, in which case other rules may take precedence over or otherwise modify the application of the technology. Also, some steps may be taken before, during, or after the above elements are considered. Thus, the above description is not intended to be binding on the scope of the claims.

[0093]

[0103] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Thus, for example, a reference to "a user" includes a reference to one or more such users, a reference to "a processor" includes a reference to one or more processors and equivalents thereof known to those skilled in the art, and so forth.

[0094]

[0104] Additionally, the terms "comprise," "comprising," "contains," "containing," "include," "including," and "includes," when used in this specification and the claims that follow, are intended to specify the presence of stated features, integers, components, or steps, but do not exclude the presence or addition of one or more other features, integers, components, steps, acts, or groups.

[0095]

[0105] It is also understood that the examples and embodiments described herein are for illustrative purposes only, and various modifications or changes in light thereof will be suggested to those skilled in the art and are to be included within the spirit and scope of this application and the scope of the appended claims.

Claims

1. 1. A method for determining a period of time during which a work implement of a construction machine interacts with a ground surface, comprising: capturing a vibration signal indicative of movement of the work machine; Extracting one or more features from the vibration signal; and providing the one or more features to a machine learning model to generate a model output; predicting an implement on ground (IOG) start time and an IOG end time based on the model output, the IOG start time and the IOG end time forming the period during which the implement is interacting with the ground surface; A method comprising:

2. The method of claim 1 , wherein the vibration signal is captured using a vibration sensor attached to the construction machine.

3. The method of claim 2 , wherein the vibration sensor comprises an accelerometer and the vibration signal comprises an acceleration signal.

4. The method of claim 2 , wherein the vibration sensor comprises a gyroscope and the vibration signal comprises a rotational signal.

5. The method of claim 2 , wherein the vibration sensor is attached to the work machine.

6. The method of claim 1 , wherein the model output includes a set of IOG candidates corresponding to times when the implement is interacting with the ground surface.

7. The method of claim 1 , wherein the model output includes a set of candidate implement airborne states (IIA) corresponding to times when the implement is not interacting with the ground surface.

8. The method of claim 1 , wherein the one or more features include at least one of a signal amplitude feature or a signal frequency feature.

9. The method of claim 1 , wherein the machine learning model is a pre-trained support vector machine.

10. adjusting a terrain surface map based on the period that the implement is interacting with the terrain surface and a path of the implement. The method of claim 1.

11. one or more processors; When executed by the one or more processors, the one or more processors are Acquiring a vibration signal indicative of a movement of a working implement of a construction machine; Extracting one or more features from the vibration signal; and providing the one or more features to a machine learning model to generate a model output; predicting an implement on ground (IOG) start time and an IOG end time based on the model output, the IOG start time and the IOG end time defining a period during which the implement is interacting with a ground surface; A computer readable medium including instructions for performing operations including: A system comprising:

12. The system of claim 11 , wherein the vibration signal is captured using a vibration sensor attached to the construction machine.

13. The system of claim 12 , wherein the vibration sensor comprises an accelerometer and the vibration signal comprises an acceleration signal.

14. The system of claim 12 , wherein the vibration sensor is attached to the work machine.

15. The system of claim 11 , wherein the model output includes a set of IOG candidates corresponding to a time when the implement is interacting with the ground surface.

16. The system of claim 11 , wherein the model output includes a set of candidate implement airborne states (IIA) corresponding to times when the implement is not interacting with the ground surface.

17. The system of claim 11 , wherein the one or more features include at least one of a signal amplitude feature or a signal frequency feature.

18. The system of claim 11 , wherein the machine learning model is a pre-trained support vector machine.

19. The system of claim 11 , wherein the IOG start time and the IOG end time are predicted further based on a set of implement positions.

20. 1. A method for determining a period of time during which a work implement of a construction machine interacts with a ground surface, comprising: capturing a vibration signal indicative of movement of the work machine; determining a set of candidate implement on ground conditions (IOG) corresponding to times when the implement is interacting with the ground surface; determining a set of candidate implement airborne states (IIA) corresponding to times when the implement is not interacting with the ground surface; predicting an IOG start time and an IOG end time based on the set of IOG candidates and the set of IIA candidates, the IOG start time and the IOG end time forming the period during which the implement is interacting with the ground surface; A method comprising:

Citation Information

Patent Citations

  • Real-time soil body category identification method and system and excavator

    CN113065388A

  • Operation state recording device of construction machinery

    JP2016099744A

  • Attachment monitoring system

    JP2017201119A

  • Work analyzing apparatus and work analyzing method

    JP2019163626A

  • Index value specifying device and index value specifying method

    JP2020020145A