System and computer-implemented method for determining the wear level of a ground engaging tool of a work machine indicating tool change conditions

Abstract

The exemplary wear detection system (110) receives first image data from one or more sensors associated with at least one ground engaging tool (GET) of the work machine at a first time point in a dig-dump cycle of the work machine (100). The wear detection system processes the first image data to determine a first wear measurement and a first wear level of the at least one GET. The wear detection system determines whether the first wear level is indicative of a GET replacement condition. The wear detection system generates an alert if the first wear level is indicative of a GET replacement condition. The wear detection system receives second image data associated with the at least one GET at a second time point, different from the first time point, and determines a second wear measurement and a second wear level for the at least one GET if the first wear level does not indicate a GET replacement condition. The wear detection system generates an alert indicative of the first wear level and the second wear level based on determining that the first wear level and the second wear level are indicative of a GET replacement condition.

Description

Technical Field

[0001] The present disclosure relates to a system and method for detecting wear of an object over time, and more particularly, to a system and method for detecting wear or loss of one or more ground engaging tools (GETs) over time using imaging technology.

Background Art

[0002] Machines can be used to perform various tasks at a work site. For example, a machine can be used to excavate, move, shape, contour, and / or remove materials present at the work site, such as gravel, concrete, asphalt, soil, and / or other materials. These machines can include buckets used to collect such materials, and the buckets can include a set of GETs, such as teeth, to loosen the materials. The GETs can also include shrouds attached to the inter-tooth buckets to protect the edges of the buckets. Over time, the GETs wear down and become smaller in size, their effectiveness decreases, and it becomes more difficult for the buckets to collect materials at the work site. The GETs can break off from the buckets. If GET breakage is not detected, the GETs can mix with the materials at the work site and damage downstream processing equipment, such as crushers and grinders. Work machines can utilize wear detection systems to identify worn or broken GETs before damage occurs to downstream equipment.

[0003] An attempt to provide a wear detection system is described in International Publication No. 2020 / 237324 (the "324 Publication"), published on December 2, 2020. The 324 Publication describes a system that includes one or more sensors attached to a work device and directed at a GET, and monitors the state of the GET. This system receives data from the one or more sensors, generates a three-dimensional (3D) representation of at least a portion of the GET, and compares the currently generated 3D representation of the GET with a previously generated 3D representation of the GET. Then, based on this comparison, the system determines wear or loss of the GET. An operator can set an "adaptive threshold" that can be tuned to adjust the sensitivity of the detection.

[0004] The system described in the 324 Publication has several drawbacks. For example, by relying solely on the adaptive threshold and comparing 3D representations over time, acute GET wear (e.g., loss of the GET) may not be detected if the adaptive threshold is set high such that the system recognizes it as a statistical outlier or an incorrect measurement. Conversely, if the adaptive threshold is too low, the system described in Publication No. 324 may generate overly frequent warnings or alerts, creating a high frequency of false positives that can discomfort the operator. This discomfort can lead the operator to ignore the warnings or alerts, adjust the adaptive threshold to a too-high value, or turn off the wear detection system completely, failing to achieve its purpose. The systems and methods described herein are intended to solve one or more of these problems. SUMMARY OF THE INVENTION

[0005] A method implemented by a computer for GET wear detection according to a first aspect includes receiving, from one or more sensors associated with a work machine, first image data related to at least one GET of the work machine at a first time point in a digging-dumping cycle of the work machine. The method also includes determining a first wear measurement value of the at least one GET based on the first image data, and determining a first wear level of the at least one GET corresponding to the first time point based on the first wear measurement value. The method implemented by the computer includes determining whether the first wear level indicates a GET replacement condition. When it is determined that the first wear level indicates the GET replacement condition, a warning indicating the first wear level is generated. When it is determined that the first wear level does not indicate the GET replacement condition, second image data related to the at least one GET of the work machine is received from the one or more sensors at a second time point different from the first time point in the digging-dumping cycle of the work machine. The method implemented by the computer determines a second wear measurement value of the at least one GET based on the second image data, and determines a second wear level of the at least one GET based on the second wear measurement value. The method implemented by the computer determines whether the first wear level and the second wear level indicate the GET replacement condition, and when it is determined that the first wear level and the second wear level indicate the GET replacement condition, a warning indicating the first wear level and the second wear level is generated.

[0006] A GET wear detection system according to a further aspect comprises one or more processors and one or more sensors associated with a work machine. The GET wear detection system also comprises a non-transitory computer-readable medium storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including receiving, from the one or more sensors associated with the work machine, first image data related to at least one GET of the work machine at a first point in time in a digging-dumping cycle of the work machine. The one or more processors determine a first wear measurement value of the at least one GET based on the first image data, and determine a first wear level of the at least one GET based on the first wear measurement value. The one or more processors also determine whether the first wear level indicates a GET replacement condition. When it is determined that the first wear level indicates the GET replacement condition, a warning indicating the first wear level is generated. When it is determined that the first wear level does not indicate the GET replacement condition, second image data related to the at least one GET of the work machine is received from the one or more sensors at a second point in time different from the first point in time in the digging-dumping cycle of the work machine. The one or more processors determine a second wear measurement value of the at least one GET based on the second image data, and determine a second wear level of the at least one GET based on the second wear measurement value. The one or more processors determine whether the first wear level and the second wear level indicate the GET replacement condition, and when the first wear level and the second wear level indicate the GET replacement condition, a warning indicating the first wear level and the second wear level is generated.

[0007] A work machine according to another aspect includes a bucket including at least one GET, a plurality of sensors including at least a left image sensor and a right image sensor, a display, one or more processors, and a non-transitory computer-readable medium storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform an operation including the following. The operation includes receiving, from the plurality of sensors, first image data related to the at least one GET at a first point in time in the excavation-dumping cycle of the work machine that is closer to the start of the excavation-dumping cycle than to the end of the excavation-dumping cycle. The one or more processors determine a first wear measurement value of the at least one GET based on the first image data, and determine a first wear level of the at least one GET based on the first wear measurement value. The one or more processors determine whether the first wear level indicates a GET replacement condition. When it is determined that the first wear level indicates the GET replacement condition, a warning indicating the first wear level is generated and the warning is rendered on the display. When it is determined that the first wear level does not indicate the GET replacement condition, second image data corresponding to the at least one GET is received from the plurality of sensors at a second point in time in the excavation-dumping cycle of the work machine. The second point in time is after the first point in time. The one or more processors determine a second wear measurement value of the at least one GET based on the second image data, and determine the second wear level of the at least one GET based on the second wear measurement value. The one or more processors determine whether the first wear level and the second wear level indicate the GET replacement condition, and when the first wear level and the second wear level indicate the GET replacement condition, a warning indicating the first wear level and the second wear level is generated and the warning is rendered on the display.

Brief Description of the Drawings

[0008] Specific embodiments will be described with reference to the accompanying drawings. In each figure, the leftmost digit of the reference number identifies the figure in which that reference number first appears. The same reference numerals in different drawings indicate similar or identical items.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

DETAILED DESCRIPTION OF THE INVENTION

[0009] The present disclosure generally relates to systems and methods for detecting wear of components of a work machine in an environment such as a work site using one or more sensors. The one or more sensors can include an imaging sensor (which can be part of a stereo camera or “stereo camera”) capable of imaging imaging data related to the component, a LiDAR sensor, an infrared (IR) sensor, a sonar sensor, a temperature sensor, or a radar sensor. The imaging data can include, but is not limited to, video, images, LiDAR imaging data, IR imaging data, audio-based imaging data, or radar data. The imaging data is analyzed by a wear detection computer system associated with the work machine, and in this aspect, can be located on the work machine, within the stereo camera, within one or more sensors of the work machine, or external to them or external to the work machine to detect wear of the component. For example, the component may be one or more GETs of a bucket of the work machine. The one or more sensors of the work machine may each have a different field of view and may generate slightly different imaging data for the component. The different fields of view can reduce errors associated with bad lighting conditions, shadows, or debris that can negatively affect imaging of the component. The wear detection system can determine image points associated with marker points (e.g., edges, corners, or visual indicators on the component) of the GET from the captured imaging data and use those image points to determine measurements of the component. The wear detection system can determine the wear level or loss of the component based on the determined measurements and / or past or baseline measurements. The wear detection system can receive imaging data from one or more sensors at various times during the excavation-dump cycle of the work machine and determine whether to generate a warning or capture additional imaging data based on the location of the time point within the excavation-dump cycle.

[0010] FIG. 1 is a block diagram schematically showing an exemplary work machine 100 including an exemplary wear detection computer system 110. In FIG. 1, the work machine 100 is depicted as a hydraulic excavator, but in other examples, the work machine 100 can include any machine that moves, sculpts, excavates, or removes materials such as soil, rock, or minerals. As shown in FIG. 1, the work machine 100 can include a bucket 120 attached to an arm 122. The bucket 120 can include one or more ground engaging tools (GETs) 125 such as teeth that help the work machine 100 loosen the material. In the examples provided by this disclosure, the GET 125 is generally referred to as a tooth, but other types of GETs are also considered to be within the scope of the embodiments provided by this disclosure. For example, the GET can include a lip shroud, an edge guard, an adapter, a ripper protector, a cutting edge, a side bar protector, a tip, or other tools associated with a work machine that wear over time due to friction with the materials at the work site.

[0011] The work machine 100 also includes one or more sensors having respective fields of view, such as a sensor 126 having a field of view 127 and a stereo camera 128 having a field of view 129. Both the field of view 127 and the field of view 129 are directed at the bucket 120 and the GET 125. As shown in FIG. 1, the field of view 127 and the field of view 129 overlap but are different. The sensor 126 can include, by way of just a few examples, an image sensor, a LiDAR sensor, an IR sensor, a sonar sensor, or a radar sensor.

[0012] As used in this disclosure, the term "imaging data" refers to data generated by sensor 126 or stereo camera 128 and received by wear detection computer system 110 that can be decoded or processed to reflect the size, shape, or appearance of GET 125. Although this disclosure refers to a single sensor 126, in some embodiments, work machine 100 includes, in addition to stereo camera 128, typically two or more sensors 126 each having its own field of view 127. For example, work machine 100 may include a camera 128 having a field of view 129, a LiDAR sensor, additional imaging sensors, and IR sensors, all of which may generate imaging data processed by a wear detection computer system according to embodiments of this disclosure.

[0013] In some embodiments, sensor 126 includes an adaptive scan LiDAR sensor, i.e., a LiDAR sensor whose resolution and field of view can be commanded, controlled, and configured. For example, sensor 126 can include the AEYE 4Sight M (trademark). In some embodiments, the field of view 127 may start from a 60-degree by 30-degree baseline (representing a "low" resolution range scan) and then be adjusted in 0.1-degree increments up to a high-definition region of interest spanning 0.025 degrees, although in other embodiments, other fields of view and angular resolutions may exist. Sensor 126 can be configured to collect up to 1600 points per square degree at a frequency of 100 Hz. The accuracy of sensor 126 is a function of the angular resolution of field of view 127 and the distance between sensor 126 and GET 125. For example, if GET 125 is approximately 6 meters away from sensor 126 and field of view 127 is configured as 60 degrees by 30 degrees, a scan of 1600 points per square degree generates LiDAR hits within an imaging rectangle of approximately 7.2 meters by 3.2 meters. By readjusting the focus of the field of view, the LiDAR hits can record 2.6 millimeters horizontally and vertically. Although an example of sensor 126 has been described above, different LiDAR sensors capable of adaptive scanning can be used in various embodiments.

[0014] Sensor 126 can also include an infrared sensor or a sensor capable of detecting the heat signature of GET 125. For example, sensor 126 can include a long-wave FLIR® infrared camera with a resolution of 640×512, a refresh rate of 9 Hz, and a field of view of 75 degrees. Infrared sensor 126 can complement camera 128 in an environment with little light or an environment where debris may adhere to GET 125 during operation. As another example of sensor 126, a sonar or radar sensor can be included.

[0015] Stereo camera 128 includes a left image sensor and a right image sensor arranged at intervals to capture stereo images of objects within the field of view 129, such as bucket 120 and GET 125. In some embodiments, the left image sensor and the right image sensor capture monochrome images. Stereo camera 128 can further include a color image sensor for capturing color images of objects within the field of view 129. In some embodiments, camera 128 outputs a digital image, or work machine 100 can include an analog-to-digital converter arranged between camera 128 and wear detection computer system 110 to convert the analog image to a digital image before wear detection computer system 110 receives it. Although the present disclosure refers to stereo camera 128 having a single field of view 129 for ease of discussion, those skilled in the art will understand that each image sensor of camera 128 (e.g., left, right, color) has its own field of view for generating a stereo image from which GET 125 can be measured in accordance with embodiments of the present disclosure.

[0016] In some embodiments, one or more sensors of the work machine 100, such as the sensor 126 and the camera 128, can include a lens cleaning device for removing debris, fog, or other obstacles from the surface of the lens (or screen) of the one or more sensors. The lens cleaning device can include, for example, a nozzle that sprays compressed air, a cleaning solvent, or a cleaning antifreeze. The lens cleaning device can further include a movable wiper configured to wipe in contact with the surface of the lens and push debris or other obstacles out of the lens surface. In some embodiments, the cover of the lens of the one or more sensors can include an actuator that rotates the lens screen (in the case of a cylindrical lens screen) or slides the lens screen (in the case of a flat lens screen) into contact with one or more wipers to remove debris from the screen.

[0017] When the work machine 100 operates within the work site, the arm 122 can be moved to position the bucket 120 to move or excavate materials within the work site as part of the excavation-dumping cycle. The work machine 100 can move the bucket 120 in and out of the fields of view 127 and 129 when positioning the bucket 120 by the excavation-dumping cycle. The sensor 126 and the camera 128 may be arranged so as to be visible without obstructing the GET 125 during the excavation-dumping cycle. For example, the sensor 126 and the camera 128 may be arranged on the work machine 100 so that the bucket 120 and the GET 125 are visible at the moment when the bucket 120 empties the material within the excavation-dumping cycle. As another example, the sensor 126 and the camera 128 may be arranged so that the bucket 120 enters the field of view when the arm 122 is fully extended or retracted during the excavation-dumping cycle. As will be described below with reference to FIGS. 2-4, the positions of the sensor 126 and the camera 128 (and their respective fields of view 127 and 129) may vary depending on the type of the work machine 100 and the details regarding the work site.

[0018] In some embodiments, the fields of view 127 and 129 may capture image data of the bucket 120 and the GET 125 at different points in the excavation-dump cycle. For example, the sensor 126 may capture image data of the GET 125 at an early stage of the excavation-dump cycle (e.g., closer to the start of the cycle than to the end of the cycle), and the camera 128 may capture image data of the GET 125 at a late stage of the excavation-dump cycle (e.g., closer to the end of the cycle than to the start of the cycle). In some embodiments, the sensor 126 and / or the camera 128 may adjust their respective fields of view 127, 129 to collect image data of the GET 125 at different points in the excavation-dump cycle. For example, in some embodiments, both the sensor 126 and the camera 128 may capture image data of the GET 125 at an early stage of the excavation-dump cycle and then adjust the fields of view 127, 129 to capture image data of the GET 125 at a late stage of the excavation-dump cycle.

[0019] According to some embodiments, the work machine 100 includes an operator control panel 130. The operator control panel 130 can include a display 133 that generates an output for an operator of the work machine 100, whereby the operator can receive a status or an alert regarding the wear detection computer system 110. The display 133 can include a liquid crystal display (LCD), a light emitting diode display (LED), a cathode ray tube (CRT) display, or other types of displays known in the art. In some examples, the display 133 includes an audio output such as a speaker or headphones or a port for peripheral speakers. The display 133 can also include an audio input device such as a microphone or a port for peripheral microphones. In some embodiments, the display 133 includes a touch-sensitive display screen that also functions as an input device.

[0020] The display 133 can display information regarding the wear level or loss of the GET 125 rendered by the wear computer detection system 110. For example, the display 133 may display the calculated measurement value of the GET 125. In some embodiments, the calculated measurement value may be color-coded to reflect the health state of the GET 125. For example, the calculated measurement value may be displayed with a green background if the GET 125 is considered to have an acceptable wear level, may be displayed with a yellow background if the replacement of the GET 125 is approaching, and may be displayed in red if the GET 125 has reached the point where it is damaged or worn and needs to be replaced. The display 133 can also display an image of the bucket 120, an image of the GET 125, or an image of the region of interest within the fields of view 127, 129 related to the GET 125 rendered by the wear computer detection system 110.

[0021] In some embodiments, the operator control panel 130 also includes a keyboard 137. The keyboard 137 provides input capabilities for the wear detection computer system 110. The keyboard 137 includes a plurality of keys that enable the operator of the work machine 100 to provide input to the wear detection computer system 110. For example, according to the examples of the present disclosure, the operator can press the keys of the keyboard 137 to select or input the type of the work machine 100, the bucket 120, and / or the GET 125. The keyboard 137 may be non-virtual (e.g., including those that can physically press keys), or the keyboard 137 may be a virtual keyboard displayed on a touch-sensitive embodiment of the display 133.

[0022] As shown in FIG. 1, the wear detection computer system 110 includes one or more processors 140. The processor 140 can include a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), one or more of a combination of a CPU, a GPU, or an FPGA, or any other type of processing unit. The processor 140 can have a plurality of arithmetic logic units (ALUs) that perform arithmetic and logical operations, and one or more control units (CUs) that extract instructions and stored content from the processor cache memory and then call the ALU as needed during program execution to execute the instructions. The processor 140 may also be responsible for executing drivers and other computer-executable instructions of applications, routines, or processes stored in the memory 143, which may be associated with a general type of volatile (RAM) and / or non-volatile (ROM) memory.

[0023] The wear detection computer system 110 further includes a memory 143. The memory 143 can include a system memory that can be volatile (e.g., RAM), non-volatile (e.g., ROM, flash memory, etc.), or a combination of both. The memory 143 can also include non-transitory computer-readable media such as volatile and non-volatile, removable and non-removable media, realized by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. System memory, removable storage, and non-removable storage are examples of non-transitory computer-readable media. Examples of non-transitory computer-readable media include RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, cassette tapes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or other non-transitory media used to store the necessary information and accessible by the wear detection computer system 110, but are not limited thereto.

[0024] Memory 143 stores data including computer-executable instructions for the wear detection computer system 110 as described herein. For example, memory 143 can store one or more components of wear detection computer system 110 such as physical parameter library 145, image analyzer 150, wear analyzer 153, warning manager 155, and GET wear level storage 157. Memory 143 can also store additional components, modules, or other code executable by processor 140 to enable the operation of wear detection computer system 110. For example, memory 143 can include code related to input / output functions, software drivers, operating systems, or other components.

[0025] According to some embodiments, various aspects of wear detection computer system 110 may be disposed within camera 128. For example, camera 128 may include one or more of processor 140 and / or memory 143. Similarly, various aspects of wear detection computer system 110 may be disposed within sensor 126. Additionally, or alternatively, various aspects of wear detection computer system 110 may be disposed on work machine 100 and outside of sensor 126 or camera 128.

[0026] Physical parameter library 145 includes a set of physical parameters related to work machine 100, bucket 120, GET 125, sensor 126, and / or camera 128. For example, physical parameter library 145 can include, as some examples, measurement data related to the size of bucket 120, the shape of bucket 120, the size of GET 125, the shape of GET 125, and the spatial relationship between GET 125 and bucket 120, and / or the spatial relationship between sensor 126 and camera 128. Physical parameter library 145 can further include parameters related to the size and shape of GET 125 in a new or unworn state, and parameters related to the size and shape of GET 125 when maximum wear has been reached.

[0027] In some embodiments, the physical parameter library 145 can include geometric parameters related to the marker points of the bucket 120 or the GET 125. The marker points are related to aspects or reference points of the bucket 120 and the GET 125 for measuring wear or loss of the GET 125 that can be used by the wear analyzer 153. For example, the marker points can include the corners of the GET 125, the edges of the bucket 120 where the GET 125 engages with the bucket 120, the corners between the edges of the bucket 120 and the GET 125, or physical markers applied to the GET 125 such as welds, paints, grooves, reflective tapes, or barcodes. The physical parameter library 145 can include a set of physical parameters having information regarding the relative position of the marker points with respect to the GET 125 or the bucket 120. For example, the set of physical parameters can include the angles of the corners of the GET 125, the relative positions of the physical markers on the GET 125 with respect to the edges of the GET 125 or the edges of the bucket 120. Embodiments of the wear detection computer system 110 may include sets of physical parameters beyond the specific examples described herein, and those skilled in the art will understand that other methods of detecting marker points on the bucket 120 or the GET 125 can be used, and that the physical parameter library 145 can include sets of physical parameters that can be used by the wear analyzer 153 to identify marker points in the image data collected by the sensor 126 and / or the camera 128.

[0028] The physical parameter library 145 can also include templates or reference images (e.g., bucket tool templates) related to the combination of the bucket 120 and the GET 125. For example, in the case of the work machine 100, one of the templates stored in the physical parameter library 145 can include an image of the bucket 120 with the GET 125, since it is expected that the bucket 120 will be located within the field of view 127 and / or the field of view 129. The bucket tool template can represent an unworn GET 125 (e.g., an unworn or expected edge), or a GET 125 that has reached maximum wear (e.g., a threshold edge). The physical parameter library 145 can also include other information regarding the wear of the GET 125 to assist the wear analyzer 153 in determining when the GET needs to be replaced due to wear. The wear data regarding the GET 125 can be, by way of several examples, in a form of actual measured dimensions (e.g., metric or inch sizes) or in a form of pixel values.

[0029] As another example, the physical parameter library 145 can include a CAD-based model of the GET 125. The CAD-based model can use a reference GET 125 model developed using a computer-aided design program such as AutoCAD®, Autodesk®, SolidWorks®, or other well-known CAD programs. The wear detection computer system 110 can determine the wear or loss of the GET 125 by comparing the observed size and shape of the GET 125 with that of the same type of model, a standard, or an unworn GET, using the CAD-based model as a reference point. In some embodiments, the CAD-based model can include the position, orientation, and / or relative positioning of points on the GET 125.

[0030] The physical parameter library 145 can include a plurality of physical parameter sets, and each physical parameter set corresponds to a work machine, a bucket, a GET, or a combination thereof. During operation, the operator can use the operator control panel 130 to select a physical parameter set from the physical parameter library 145 that is compatible with the bucket 120 and GET 125, or the work machine 100. For example, if the work machine 100 is a hydraulic excavator with a model number "6015B", the operator can use the operator control panel 130 to enter the model number "6015B", and the wear detection computer system 110 can load the physical parameter set corresponding to the hydraulic excavator of model "6015B" from the physical parameter library 145 into the memory 143. In some examples, when the wear detection computer system 110 is powered on or reset, a list of templates available in the physical parameter library 145 can be displayed on the display 133, and the operator can select and operate one physical parameter set from the list according to the model number of the work machine 100, the type of the bucket 120, or the type of the GET 125.

[0031] In some embodiments, at the start of a work shift, the operator can place the bucket 120 and GET 125 within the field of view 129 of the camera 128 and use the input on the operator control panel 130 to enable the wear detection computer system 110 to capture images of the bucket 120 and GET 125. The wear detection computer system 110 then performs an image matching process that matches the bucket 120 and GET 125 with the physical parameter sets and can configure itself for the wear detection and image processing processes disclosed herein based on the matching physical parameter sets. In some embodiments, the wear detection computer system 110 can use the sensor 126 and the field of view 127 in the configuration process instead of the camera 128 and the field of view 129.

[0032] The image analyzer 150 can be configured to analyze the imaging data captured by either the sensor 126 or the camera 128 to identify the GET 125 within the fields of view 127 and 129, and measure the wear of the GET 125 based on the processing of the imaging data. For example, the image analyzer 150 can receive a stereoscopic image from the camera 128 in the form of a left corrected image (captured by the left image sensor of the camera 128) and a right corrected image (captured by the right image sensor of the camera 128). The image analyzer 150 can perform various computer vision techniques on the left corrected image and the right corrected image to identify or determine the region of interest corresponding to the GET 125. As another example, the image analyzer 150 can receive the imaging data captured by the sensor 126 that can be used to identify the region of interest corresponding to the GET 125. In the disclosed embodiments, the image analyzer 150 receives data from the sensor 126 and determines the wear or loss of the GET 125, as will be described in more detail below.

[0033] In some embodiments, when wear or loss of the GET 125 is detected, the image analyzer 150 processes two sets of imaging data. The first set of imaging data is captured to identify the region of interest within the field of view 127 or 129. The region of interest corresponds to the relative position of the GET 125 within the field of view 127 or 129. The first set of imaging data for detecting the region of interest is an imaging of wide and low-resolution imaging data designed to identify the position of the general region of interest of the GET 125, and may be referred to as a "coarse scan". In some embodiments, the first set of imaging data may be captured using the camera 128, and the image analyzer 150 determines the region of interest using computer vision or machine learning techniques. In other embodiments, the first set of imaging data may be captured using the sensor 126 at a relatively wide first low resolution (e.g., 60 degrees × 30 degrees in the LiDAR embodiment of the sensor 126). In some embodiments, the image analyzer 150 receives the first set of imaging data from the sensor 126 and the camera 128.

[0034] When the image analyzer 150 identifies the region of interest corresponding to the GET 125, it controls the sensor 126 to focus on the region of interest and may perform a higher-resolution scan or, in some embodiments, a "fine scan". For example, the image analyzer 150 can communicate with the application programming interface (API) of the sensor 126 to change the sensor 126 to narrow the field of view 127 by focusing on the identified region of interest. Next, the sensor 126 scans the GET 125 again to collect a second set of imaging data. The second set of imaging data captured by the sensor 126 with the narrow field of view 127 has a higher resolution than the first imaging data captured by either the sensor 126 (when set to a wide field of view) or the camera 128.

[0035] In some embodiments, the coarse scan can be performed by either the sensor 126 or the camera 128, and the fine scan can be performed by the other of the camera 128 or the sensor 126. Alternatively, both the sensor 126 and the camera 128 may perform both the coarse scan and the fine scan.

[0036] In one embodiment, the image analyzer 150 creates a high-density stereo disparity map based on the left corrected image and the right corrected image received from the camera 128. The image analyzer can divide the high-density stereo disparity map to identify the region of interest. In addition, the image analyzer 150 can also create a three-dimensional point cloud based on the high-density stereo disparity map, and can also divide the three-dimensional point cloud to identify the region of interest.

[0037] In addition to, or instead of using, computer vision technology, the image analyzer 150 can identify regions of interest in the left corrected image and / or the right corrected image captured by the camera 128 using deep learning or machine learning techniques. For example, the image analyzer 150 can use a deep learning GET detection algorithm that uses a neural network trained to identify regions of interest based on an image corpus in which individual GETs, GET groups, or combinations of GETs and buckets are labeled. The image analyzer 150 can also use a deep learning GET position algorithm that employs a neural network trained to identify the positions of GETs in the image. The GET position algorithm can be trained using a corpus of images in which individual GETs are labeled. When the GET position algorithm identifies an individual GET in the image, it outputs the corresponding position of the GET. For example, the GET position algorithm can output a pixel position or a bounding box output regarding the GET position.

[0038] In some embodiments, the deep learning GET detection algorithm includes a neural network trained to identify regions of interest based on a disparity map. For example, the corpus of training data for the deep learning GET detection algorithm can include disparity images between the left corrected and right corrected images in which individual GETs, GET groups, or combinations of GETs and buckets are labeled.

[0039] As described above, when the image analyzer 150 identifies the region of interest including the GET 125, the image analyzer 150 may command and control the sensor 126 to focus on the field of view 127, or may command and control the camera 128 to focus on the field of view 129 in the region of interest. In some embodiments, the image analyzer 150 uses the spatial relationship data between the sensor 126 and the camera 128 to command the sensor 126 to change the field of view 127 over the region of interest. For example, in a LiDAR embodiment, when the sensor 126 receives a command to change its field of view, it can change the configuration of its MEMS (Micro-Electro-Mechanical System) mirror to a narrow field of view 127 to capture higher-resolution imaging data regarding the GET 125.

[0040] The image analyzer 150 can create a three-dimensional point cloud corresponding to the GET 125 from the captured higher-resolution imaging data. Each point in the three-dimensional point cloud corresponds to a "hit" or detected point imaged by the sensor 126, such as a LiDAR hit or an infrared thermal signature. In some embodiments, the actual distance between points may be as small as about 1 mm. In embodiments with a sufficiently high resolution (i.e., when the actual distance between points is less than about 2.5 mm), the image analyzer 150 sends the three-dimensional point cloud data to the wear analyzer 153 for wear detection analysis. In other embodiments, the image analyzer 150 may perform additional processing to further refine the three-dimensional point cloud data for wear analysis.

[0041] For example, in some embodiments, the image analyzer 150 can convert a three-dimensional point cloud into a dense mesh surface. The image analyzer 150 can further convert the dense mesh surface into a sparse mesh surface before communicating the GET imaging data to the wear analyzer 153. When comparing the imaging data captured by the sensor 126 with the CAD-based GET model, the conversion from the three-dimensional point cloud to a dense mesh surface and then to a sparse mesh surface may be desirable to reduce computational cost. The conversion from the three-dimensional point cloud to a dense mesh surface and then to a sparse mesh surface can also remove noise that may be present in the imaging data due to oversampling.

[0042] In some embodiments, the wear analyzer 153 fuses the lower-resolution first received imaging data from the camera 128 with the higher-resolution second received imaging data from the sensor 126 to obtain reliability for the observed measurements of the GET 125. In such embodiments, the image analyzer 150 performs additional processing on the left and right images captured by the camera 128. For example, when the image analyzer 150 identifies regions of interest, it can further process them to create a left edge digital image corresponding to the left corrected image and a right edge digital image corresponding to the right corrected image. The image analyzer 150 may employ gradient magnitude search-based edge detection, but in other embodiments, it may employ other edge detection techniques employed in the field of computer vision (e.g., zero-crossing-based edge detection techniques) to create the left edge digital image and the right edge digital image.

[0043] In some examples, the image analyzer 150 can identify individual GETs 125 by refining the edge estimation of the GETs 125 and / or using the expected positions of the GETs 125 within the captured image. For example, the image analyzer 150 can know the expected positions of the GETs 125 relative to the bucket 120 based on a set of physical parameters stored in the physical parameter library 145 corresponding to the type of the bucket 120 in use and the GETs 125. Using this information, the image analyzer 150 can reach the expected positions within the selected image and capture the pixel region closest to the tooth. Next, the tooth can be further identified using the pixel region based on computer vision techniques such as applying a convolutional filter, segmentation analysis, edge detection, or pixel intensity / darkness analysis within the pixel region. In some embodiments, the image analyzer 150 can apply individual tooth templates to the pixel region and further refine the position of the tooth using computer vision techniques. The image analyzer 150 may further refine the edges using dynamic programming techniques. The dynamic programming techniques can include smoothing based on the intensity of the edges, whether the edges are close to holes or uncertain regions within a high-density stereo disparity map, or other edge detection optimization methods. The image analyzer 150 can also obtain reliability in determining the position of the GET using the output of the GET position algorithm and further refine the edge estimation based on the output of the GET position algorithm.

[0044] The image analyzer 150 can also create a sparse stereo disparity that can be used with the higher resolution imaging data captured by the sensor 126 for the wear analyzer 153, which is a wear analyzer 153 for determining wear or loss in the GET 125. In some embodiments, the image analyzer 150 creates a sparse stereo disparity between a left edge digital image (associated with the left corrected image) and a right edge digital image (associated with the right corrected image), and this disparity is used by the wear analyzer 153. Alternatively, the image analyzer 150 may calculate a sparse stereo disparity from a first region of interest image (associated with the left corrected image) and a second region of interest image (associated with the right corrected image), and detect edges from the sparse stereo disparity image.

[0045] The wear analyzer 153 can be configured to analyze the sparse stereo disparity generated by the image analyzer 150 for wear. For example, the set of physical parameters associated with the bucket 120 and the GET 125 can include expected data related to a set of non-worn GET 125s calibrated based on the expected image acquisition of the non-worn GET 125 or the camera 128. As some examples, the expected data can be in the form of pixels, measured values, a CAD-based model of the GET 125, or an edge image for a non-worn GET. When receiving the sparse stereo disparity, the wear analyzer 153 can fuse and correlate the sparse stereo disparity with the 3D point cloud of the higher resolution imaging data captured by the sensor 126 (or, in some embodiments, a dense mesh surface or a sparse mesh surface determined based on a 3D point cloud) to determine measurement data regarding the GET 125. Next, to determine the wear level or loss of the GET 125, the determined measurement data can be compared with the expected data corresponding to the non-worn version of the GET 125.

[0046] In some embodiments, the image analyzer 150 identifies marker points in the image data collected from the sensor 126 and / or the camera 128. As described above, the marker points can refer to the corners of the GET 125, the edges of the bucket 120 that fits the GET 125, or physical or visual markers on the bucket 120 or the GET 125. For example, if the image analyzer 150 identifies the corners of the GET 125 as marker points, the image analyzer 150 may perform corner detection techniques known in the field of computer vision, such as, by way of example, Moravec, Harris & Stephens, Shi - Tomasi, Forstner, multi - scale Harris, Laplacian of Gaussian, Wang and Brady, SUSAN (Smallest Univalue Segment Assimilating Nucleus), Trajkovic, Hedley, etc., and / or Hessian techniques. In addition to performing segmentation, pattern matching, or template matching analysis, other corner detection methods may be used to identify marker points based on the image data collected by the sensor 126 and / or the camera 128.

[0047] In some embodiments, the image analyzer 150 may identify marker points within the region of interest using deep learning or machine learning techniques in addition to, or as an alternative to, computer vision techniques. For example, the image analyzer 150 may deploy a marker point detection algorithm trained using a data corpus in which marker points (e.g., corners, edges, markers) of the GET 125 are labeled. The marker point detection algorithm may be trained using, as some examples, a normalized monochrome image (e.g., similar to the imaging data provided by the left and right image sensors of the camera 128), a color image (e.g., similar to the imaging data provided by the color image sensor of the camera 128), a disparity map (e.g., similar to the disparity generated based on the imaging data provided by the left and right image sensors of the camera 128), LiDAR point cloud imaging data, and / or infrared imaging data. The training data for the marker point detection algorithm may vary and correspond to the embodiments of the sensors 126 and the camera 128 disposed in the embodiment of the work machine 100.

[0048] The image analyzer 150 also maps image points within the imaging data across the image data sources. For example, the image analyzer 150 may determine image points associated with the end tip of the GET 125 associated with the imaging data captured by the sensor 126 and the imaging data captured by the 128. The image analyzer 150 may do this to determine errors in the imaging data that may be caused by debris or poor lighting conditions. For example, if the image point data from a set of imaging data captured by the sensor 126 appears to be an outlier when compared to the image point data of another set of imaging data captured by the imaging data of the camera 128, the image analyzer 150 may ignore this, and vice versa. In some embodiments, the image analyzer 150 may compare the image point data determined from the most recently captured imaging data with the past image point data and ignore the image point data that does not match the past image point data.

[0049] In some embodiments, the wear or loss of the GET can be measured using the number of pixels related to sparse stereo disparity. The number of pixels includes, but is merely illustrative in several examples, the area (e.g., the total pixels of the GET), the height of the GET in pixel units, the width of the GET in pixel units, and the sum of the height and width of the GET. The method for determining the number of pixels can vary depending on the shape and style of the GET. For example, for a GET whose length is much larger than its width, the number of height pixels can be used, and for a GET whose width is much larger than its length, the number of width pixels can be used. Without departing from the spirit and scope of the present disclosure, various methods for determining the number of pixels can be used.

[0050] In some embodiments, the wear analyzer 153 can calculate a similarity score between the determined measurement data and the predicted data corresponding to the unworn GET 125. The similarity score can reflect a measure of how well the determined measurement data of the GET 125 matches the predicted data of the physical parameter set. For example, the similarity score can include using the union intersection or Jaccard index method for detecting similarity. In some embodiments, the similarity score may be determined using the Dice coefficient or F1 score method for detecting similarity. The similarity score can also include a value that reflects the proportion of the pixels of sparse stereo disparity that overlap with the edge image where they are predicted. In some embodiments, the similarity score may be scaled or normalized from 0 to 100.

[0051] The similarity score can provide an indication of the wear of the GET 125. For example, a low score (e.g., in the range of 0 to 20) may indicate that one of the GETs 125 is broken or shows a defect indicating tooth loss. A high score (e.g., in the range of 80 - 100) may indicate that the tooth health condition is good and there may be no need for replacement. The score between the low score and the high score can indicate the tooth wear level, and a higher score can indicate that the lead time for tooth replacement is longer than that of a lower score.

[0052] In some embodiments, the wear analyzer 153 can collect measurement data regarding the GET 125 over time and use the collected measurement data to determine the wear level of the GET 125 and the wear trend of the GET 125. The wear analyzer 153 may store the collected measurement data in the GET wear level storage 157. For example, the work machine 100 can continue to operate at the work site for several days for a job. When the work machine 100 moves materials during the job, the camera 128 provides the stereo image bucket 120 and the GET 125 to the wear detection computer system 110, and the image analyzer 150 creates a sparse stereo disparity for the GET 125. The wear analyzer 153 can map measurement data (e.g., number of pixels, metric measurement, inch measurement) related to the GET 125 at several points in time over the duration of the job. As the bucket 120 and the GET 125 engage the material at the work site, it is expected that the size of the GET 125 will decrease due to wear. Accordingly, the measurement data related to the GET 125 also decreases over time, and the number of pixels over time reflects the wear trend. The wear analyzer 153 can determine the wear level of the GET 125 at a specific point in time using the wear trend at that specific point in time. The wear level of the GET 125 may indicate that the GET 125 needs to be replaced or may indicate one or more losses of the GET 125. In some embodiments, the measurement data related to the GET 125 can be stored in the memory 143, can be applied to multiple jobs and multiple work sites, and the wear trend can be applied to the life of the GET 125. In such embodiments, when the bucket 120 or the GET 125 is replaced, the number of pixels related to the GET 125 imaged by the wear analyzer 153 can be reset, and the wear analyzer 153 can resume collecting the number of pixels of the GET 125 from the zero point.

[0053] Since the wear analyzer 153 determines the wear tendency based on the measurement data measured over time by the GET 125, the wear analyzer 153 can also create a prediction when the GET 125 may need to be replaced. For example, if the measurement data related to the GET 125 indicates that the GET 125 loses 1% of its life every 10 working hours (since the measurement data decreases by 1% every 10 working hours), and it is determined that the GET 125 has been used for 800 working hours, the wear analyzer 153 can determine that the GET 125 needs to be replaced within 200 hours.

[0054] In some embodiments, the wear detection computer system 110 can include a warning manager 155. The warning manager 155 can communicate with the wear analyzer 153 and monitor the wear tendency and wear level identified by the wear analyzer 153. The warning manager 155 can provide a message warning to the operator control panel 130 based on the information identified by the wear analyzer 153. For example, when the wear level reaches the wear threshold, the warning manager 155 can generate a warning shown on the display 133 of the operator control panel 130. The threshold can correspond to a value indicating extreme GET wear or, in some cases, complete GET loss. The warning can indicate to the operator of the work machine 100 that one or more GET 125s need to be replaced. The wear threshold can vary depending on the embodiment and can depend on the type of GET 125 and the material of the work site with which the GET 125 engages.

[0055] The warning manager 155 can also provide a warning that the GET 125 may need to be replaced at a future point in time, such as within two weeks. The replacement warning can include information regarding the wear tendency prediction of the GET 125. For example, the replacement warning can include quantification of the wear tendency (e.g., the GET 125 is wearing 2% per day), the time the teeth have been used, or the date or time when the GET 125 is expected to reach the wear threshold based on usage data.

[0056] In some embodiments, the warning manager 155 can monitor the wear trend identified by the wear analyzer 153 and provide a wear level value to the display 133 to notify the operator of the work machine 100 of the current wear level. For example, if the wear trend indicates that the GET 125 is 60% worn, the warning manager 155 can provide a display indicating that 40% of the life of the GET 125 remains before replacement is required, based on the wear trend. The display 133 can also inform the operator of a broken tooth or indicate tooth loss (e.g., when one or more lives of the GET 125 are less than 20%).

[0057] In some embodiments, the warning manager 155 can generate an instruction to render the wear level on the display 133 showing the wear level or measured value of the GET 125. For example, if the wear analyzer 153 determines, based on the processed imaging data, that one of the GETs 125 is currently 325 mm, the warning manager 155 may generate an instruction that, when executed by the processor, causes the display 133 to show that that one GET is currently 325 mm.

[0058] In some embodiments, the wear detection computer system 110 can communicate with a remote monitoring computer system 160. The remote monitoring computer system 160 can include a remote display 163, one or more remote processors 165, and a remote memory 170. The remote monitoring computer system 160 can be located at a job site where one or more work machines 100 operate and may communicate with the wear detection computer system 110 at relevant times at the job site. The remote monitoring computer system 160 can be configured to display GET wear levels for a plurality of work machines 100 and store GET wear information to facilitate monitoring the health of GETs across the entire job site. For example, the remote display 163 may be configured to display the wear levels of each GET 125 in one location and display a user interface corresponding to one or more work machines 100 to facilitate monitoring. The remote monitoring computer system 160 may be implemented as a laptop computer, a desktop computer system, or a mobile device.

[0059] The remote monitoring computer system 160 includes one or more remote processors 165. The remote processor 165 can include one or more of a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), a combination of a CPU, GPU, or FPGA, or any other type of processing unit. The remote processor 165 can have a plurality of arithmetic logic units (ALUs) that perform arithmetic and logical operations, and one or more control units (CUs) that extract instructions and stored content from the processor cache memory and call the ALUs as needed during program execution to execute the instructions. The remote processor 165 may also be responsible for executing drivers and other computer-executable instructions of applications, routines, or processes stored in the remote memory 170, which may be associated with a common type of volatile (RAM) and / or non-volatile (ROM) memory.

[0060] The remote monitoring computer system 160 also includes a remote memory 170. The remote memory 170 can include a system memory that can be volatile (e.g., RAM), non-volatile (e.g., ROM, flash memory, etc.), or any combination thereof. The remote memory 170 can include non-transitory computer-readable media such as volatile and non-volatile, removable and non-removable media, realized by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. System memory, removable storage, and non-removable storage are examples of non-transitory computer-readable media. Examples of non-transitory computer-readable media include RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, cassette tapes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or other non-transitory media used to store the necessary information and accessible by the remote monitoring computer system, but are not limited thereto.

[0061] The remote memory 170 stores data including computer-executable instructions for instructing and controlling the remote display 163 and implementing the remote log manager 175 in the remote monitoring computer system 160. The remote memory 170 can also store additional components, modules, or other code executable by the remote processor 165 to enable the operation of the remote monitoring computer system 160. For example, the remote memory 170 can include code related to input / output functions, software drivers, operating systems, or other components.

[0062] The remote monitoring computer system 160 includes a remote display 163 that generates monitoring output for a manager at the work site related to the health status of the GET, and receives warnings and alarms related to the GET wear level of one or more work machines 100 at the work site. The remote display 163 can include a liquid crystal display (LCD), a light emitting diode display (LED), a cathode ray tube (CRT) display, or other types of displays known in the art. In some examples, the remote display 163 includes audio output such as a speaker or headphones or a port for peripheral speakers. The remote display 133 can also include an audio input device such as a microphone or a port for peripheral microphones. In some embodiments, the remote display 133 includes a touch-sensitive display that also functions as an input device.

[0063] Similar to the display 133 (on the work machine 100), the remote display 163 of the remote monitoring computer system 160 can display information regarding the wear level or loss of the GET 125 rendered by the wear computer detection system 110 at the work site. For example, the remote display 163 may display the calculated measurements of the GET 125 of one or more work machines 100. The calculated measurements may be color-coded in some embodiments to reflect the health status of the GET 125. For example, the calculated measurements may be displayed with a green background if the GET 125 is considered to have an acceptable wear level, with a yellow background if the replacement of the GET 125 is approaching, and with red if the GET 125 has reached the point where it is damaged or worn and needs to be replaced. The display 133 can also show images of the regions of interest within the buckets 120, GET 125, or the fields of view 127, 129 related to the GET 125, rendered by each wear computer detection system 110 of the work machines 100 at the work site.

[0064] The wear detection computer system 110 enables the operator of the work machine 100 to be notified when the GET 125 needs to be replaced due to extensive wear or is damaged. The processes employed by the wear detection computer system 110 and described in more detail below provide high-precision and accurate measurement of GET wear on a scale of less than 5 mm, enabling the operator to stop the operation of the work machine 100 in the event of extreme GET wear or GET loss. The processes and techniques deployed by the wear detection computer system 110 can be used in various work machines.

[0065] For example, FIG. 2 is a schematic side view of an exemplary environment 200 in which a wheel loader work machine 201 operates. The wheel loader work machine 201 can include a bucket 220 and one or more buckets 225. As shown in FIG. 2, the sensors 226 and the camera 228 are arranged such that the GET 225 and the bucket 220 are within the field of view 227 (of the sensor 226) and the field of view 229 (of the camera 228) during the end of the dump in the excavation-dump cycle. As a result, in such an embodiment, the LiDAR sensor 226 and the camera 228 can be configured to capture imaging data when the bucket 220 is stationary at the end of the dump in the excavation-dump cycle.

[0066] As another example, FIG. 3 is a schematic side view of an exemplary environment 300 in which a hydraulic excavator work machine 301 is operating. The hydraulic excavator work machine 301 can include a bucket 320 and one or more buckets 325. In contrast to the positions of the sensors 226 and the camera 228 of the wheel loader work machine 201, the sensors 326 and the camera 328 are arranged such that the GET 325 is within the field of view 327 (of the sensor 326) and the field of view 329 (of the camera 328) during the end of the excavation in the excavation-dump cycle. In such an embodiment, the sensors 326 and the camera 328 may be configured to capture imaging data while the bucket 320 is stationary at the end of the excavation in the excavation-dump cycle.

[0067] In yet another example, FIG. 4 is a schematic side view of an example of an environment 400 in which an electric rope shovel work machine 401 is operating. The electric rope shovel work machine 401 can include a bucket 420, one or more GETs 425, a sensor 426, and a camera 428. As shown in FIG. 4, the GET 425 can be located within the field of view 427 (of the sensor 426) and the field of view 429 (of the camera 428) at an intermediate point of the excavation-dump cycle, but is located when the bucket 420 is relatively close to the sensor 426 and the camera 428. In such an embodiment, the sensor 426 and the camera 428 can be configured to capture imaging data when the bucket 420 enters a position range related to the field of view 427 and the field of view 429.

[0068] Although FIGS. 2-4 show only examples of the positions of specific work machines and sensors / cameras, it should be noted that the sensors 226, 326, 426 and the cameras 228, 328, 428 can be arranged such that their respective fields of view 227, 327, 427, 229, 329, 429 capture image data at any point in the excavation-dump cycle of the work machines 201, 301, 401. Further, the described positionings of the sensors 226, 326, 426 and the cameras 228, 328, 428 can be combined in some embodiments. For example, the present disclosure contemplates embodiments of wheel loader work machines 201, hydraulic excavator work machines 301, and electric rope shovel work machines 401 having sensors and cameras with fields of view directed towards the start, middle, and / or end of the excavation-dump cycle.

[0069] FIG. 5 shows an image data flow diagram 500 that illustrates an example of the flow of imaging data for a region of interest detection process using computer vision technology. The image data flow diagram 500 includes an image received, processed, and generated by an image analyzer 150 when a region of interest in the imaging data captured by a camera 128 with respect to GET125 is detected. The image data flow diagram 500 includes a left image 510 and a right image 520 captured by the camera 128. The left image 510 may be a corrected image captured by the left image sensor of the camera 128. The right image 520 may be a corrected image captured by the right image sensor of the camera 128. Both the left image 510 and the right image 520 include images of the bucket 120 and GET125.

[0070] The image analyzer 150 can process the left image 510 and the right image 520 to create a disparity map 530. The disparity map 530 may be a high-density stereo disparity map that shows the disparity between each pixel of the left image 510 and each pixel of the right image 520. Using the disparity map 530 and a set of physical parameters 535 obtained from a physical parameter library 145 and associated with the bucket 120, GET125, and / or the work machine 100, the image analyzer 150 can construct a 3D point cloud 540. The 3D point cloud 540 shows the disparity between the left image 510 and the right image 520 in three dimensions. Next, the image analyzer 150 can perform segmentation analysis on the 3D point cloud 540 to identify a region of interest 550 that includes GET125 in the left image 510, the right image 520, or both. In some embodiments, the image analyzer 150 can command and control the sensor 126 to capture higher-resolution imaging data for GET125 using the region of interest 550.

[0071] FIG. 6 shows an image data flow diagram 600 illustrating an example of the flow of imaging data for region of interest detection processing using deep learning techniques. Similar to the above-described image data flow diagram 500, the output of the region of interest detection processing is a region of interest 550 corresponding to GET125 that is used by the image analyzer 150 to further analyze the image data. However, unlike the image data flow diagram 500, the image analyzer 150 utilizes deep learning techniques to detect the region of interest 550 in the image data flow diagram 600.

[0072] The image data flow diagram 600 includes an image 610 captured by the camera 128. The image 610 can be a corrected image captured by either the left image sensor or the right image sensor of the camera 128, or can be image data captured from the sensor 126. The image analyzer 150 can apply a deep learning GET detection algorithm to the image 610. The deep learning GET detection algorithm can employ a neural network trained on an image data corpus in which GETs are individually identified, labeled, and / or GET groups are individually identified and labeled. When the image analyzer 150 applies the deep learning GET detection algorithm to the image 610, it can identify a plurality of individual GET bounding boxes 620 of the image including the individual GETs 125. In some embodiments, the image analyzer 150 can also identify a GET group bounding box 630 including the individual GET bounding boxes 620. When the image analyzer 150 identifies the GET group bounding box 630, the image analyzer 150 can extract the pixels therein as the region of interest 550. FIG. 6 shows the detection of the region of interest 550 from the image 610, which may represent image data captured by the left image sensor of the camera 128, the right image sensor of the camera 128, the color image sensor of the camera 128, or the sensor 126, and in some embodiments, the image data flow diagram 600 may be applied to two or more of these at a particular time. For example, the image analyzer 150 may detect regions of interest of the captured data from two or more of the left image sensor of the camera 128, the right image sensor of the camera 128, the color image sensor of the camera 128, and / or the image data captured by the sensor 126 at a particular time.

[0073] FIG. 7 shows an image data flow diagram 700 illustrating an example of the flow of imaging data for region of interest detection processing using deep learning techniques. Similar to the above-described image data flow diagram 600, the output of the region of interest detection processing is a region of interest 550 corresponding to the GET 125 that the image analyzer 150 uses to further analyze the image data. However, unlike the image data flow diagram 500, the deep learning GET detection algorithm described with respect to FIG. 7 is trained using a corpus of data including a disparity map, and the region of interest is detected using a disparity map generated from two or more image sensors (e.g., the left and right image sensors of the camera 128).

[0074] The image data flow diagram 700 includes a first image 710 and a second image 720 captured by the camera 128 or the sensor 126. As a mere example, the first image 710 may be a corrected image captured by the left image sensor of the camera 128, and the second image 720 may be a corrected image captured by the right image sensor of the camera 128, but the first image 710 and the second image 720 are not necessarily received from the camera 128. As another example, either one of the first image 710 or the second image 720 may be infrared image data captured by the sensor 126. As yet another example, the first image 710 may be captured by the sensor 126, while the second image 720 may be captured by the camera 128, or vice versa. Both the first image 710 and the second image 720 include the images of the bucket 120 and the GET 125. The image analyzer 150 may process the first image 710 and the second image 720 to create a disparity map 730. The disparity map 730 may be a high-density stereo disparity map showing the disparity between each pixel of the first image 710 and each pixel of the second image 720.

[0075] The image analyzer 150 can apply a deep learning GET detection algorithm to the disparity map 730. The deep learning GET detection algorithm may employ a neural network trained with a corpus of image data in which GETs are individually identified and labeled, and / or a GET group individually identified and labeled in a disparity map corresponding to the types of buckets 120 and GETs 125 imaged within the first image 710 and the second image 720. When the image analyzer 150 applies the deep learning GET detection algorithm to the disparity map 730, it can identify a plurality of individual GET bounding boxes 740 of the image including the individual GETs 125. In some embodiments, the image analyzer 150 can also identify a GET group bounding box 750 that includes the individual GET bounding boxes 740. When the image analyzer 150 identifies the GET group bounding box 750, the image analyzer 150 can extract the pixels therein as the region of interest 550.

[0076] FIG. 8 shows an image data flow diagram 800 illustrating an example of the flow of imaging data for region of interest detection processing of imaging data captured by the sensor 126. The following description of FIG. 8 uses the LiDAR embodiment of the sensor 126 as an example, but other examples are also conceivable. The imaging data captured by the sensor 126 according to the image data flow diagram 800 substantially corresponds to the field of view shown in the image 810. As shown, the field of view includes the bucket 120 and the GET 125.

[0077] When sensor 126 detects an object surface, such as the surface corresponding to either bucket 120 or GET 125 for example, sensor 126 performs LiDAR data imaging including a plurality of LiDAR "hits". The LiDAR hits can be represented as a three-dimensional point cloud 820, and each point in the three-dimensional point cloud 820 corresponds to a LiDAR hit. Image analyzer 150 determines region of interest 510 based on the three-dimensional point cloud 820 by performing segmentation analysis or other object recognition analysis techniques. In some embodiments, image analyzer 150 can identify region of interest 550 using physical parameter set 535. For example, image analyzer 150 can identify region of interest 550 within the three-dimensional point cloud 820 using a bucket tooth template, a CAD-based model of GET 125, or pattern matching techniques.

[0078] FIG. 9 shows an image data flow diagram 900 illustrating an example of the flow of imaging data for a wear detection process using marker point identification. As described above, the features of GET 125 can be regarded as marker points, and wear detection computer system 110 may determine measurements of GET 125 based on the identification of marker points in the image data or regions of interest in the image data corresponding to GET 125. The marker points can include, for example, the edges, corners, or specific angles of GET 125. Although data flow diagram 900 shows the flow of data for the identification of marker points using one region of interest (e.g., region of interest 910), in operation, wear detection computer system 110 may process the imaging data according to data flow diagram 900 for each time point for a plurality of sets of image data received from sensor 126 and / or camera 128.

[0079] When the image analyzer 150 identifies the region of interest 910, in order to assist in identifying the marker points 920 in the region of interest 910, it refers to the physical parameter set 535 and identifies the marker points 920 within the region of interest 910 corresponding to the GET 125. For example, the physical parameter set 535 can include the size or shape of the angles of each corner of the GET 125 that the image analyzer can use when performing corner detection analysis on the region of interest 910. As another example, the physical parameter set 535 can include a template that matches the marker points (e.g., images of corners, marks, welds, or other identifiers) within the GET 125 that the image analyzer 150 applies in the segmentation analysis to the region of interest 910. In an embodiment where the region of interest 910 corresponds to image data captured by an IR camera, the physical parameter set 535 can include the predicted temperature value of the GET 125 and temperature information related to the working environment in which the GET 125 is used.

[0080] When the image analyzer 150 determines the marker points 920, the GET end 930 can be identified and / or determined in other ways. The GET end 930 corresponds to the length of the GET 125, with one end corresponding to the bucket side edge of the GET 125 (e.g., where the GET 125 contacts the bucket 120), and the other end corresponding to the front edge or engagement edge of the GET 125 (e.g., where the GET 125 engages with the ground). Based on the determination of the GET end 930, the wear analyzer 153 can determine the GET measurement value 940 for the GET 125 within the region of interest 910. For example, the wear analyzer 153 can determine the number of pixels between the GET ends 930 for a particular GET 125 and then convert the number of pixels into a distance measurement (e.g., a real-world distance measurement using the metric or yard-pound method).

[0081] FIG. 10 shows a flowchart representing an exemplary wear detection process 1000 for detecting wear of the GET 125. In some embodiments, process 1000 may be performed by the image analyzer 150 and the wear analyzer 153. Process 1000 generally follows the image data flow of FIGS. 5-9 and should be interpreted to be consistent with the description of these figures and the description of the image analyzer 150 and the wear analyzer 153 described above with respect to FIG. 1. In the following discussion, aspects of process 1000 performed by the image analyzer 150 or the wear analyzer 153 will be described, but other components of the wear detection computer system 110 may perform one or more blocks of process 1000 without departing from the spirit and scope of the present disclosure.

[0082] Process 1000 begins at step 1010, where the image analyzer 150 receives imaging data from a plurality of sensors associated with the work machine 100. The plurality of sensors can include, for example, the left monochrome, right monochrome, or color image sensors of the sensor 126 and the camera 128. In some embodiments, each of the plurality of sensors has a different field of view. For example, the left monochrome image sensor of the camera 128 and the right monochrome image sensor of the camera 128 may have slightly different fields of view to create the parallax necessary for stereo imaging. As another example, the sensor 126 and the camera 128 may have different orientations on the work machine 100 to image the GET 125 related image data from different angles. The image analyzer 150 receives imaging data from each of the plurality of sensors during the excavation-dump cycle of the work machine, and the image analyzer 150 may correlate the imaging data of each of the plurality of sensors for analysis purposes. For example, for one iteration of process 1000, the image analyzer 150 receives first image data from one of the plurality of sensors, receives second image data from another one of the plurality of sensors, and processes them together to determine the GET wear measurement value for one excavation-dump cycle.

[0083] In step 1020, the image analyzer 150 identifies each region of interest in the image data received from the plurality of sensors in step 1010. For example, the image analyzer 150 may use computer vision techniques (e.g., see FIG. 5) to identify a first region of interest in the first image data received from a first sensor among the plurality of sensors and a second region of interest in the second image data received from a second sensor among the plurality of sensors. The image analyzer 150 may use deep learning techniques (e.g., see FIGS. 6 and 7) to identify a first region of interest in the first image data received from a first sensor among the plurality of sensors and a second region of interest in the second image data received from a second sensor among the plurality of sensors. If one of the plurality of sensors is a LiDAR sensor, the image analyzer 150 may use a point cloud analysis method (e.g., see FIG. 8) to determine the region of interest in the image data.

[0084] In step 1030, the image analyzer 150 may also identify one or more image points within the region of interest (e.g., see FIG. 9). In some examples, the image points identified by the image analyzer 150 in step 1030 are associated with the edges of the GET125, as described above with respect to FIGS. 1 and 9. In some embodiments, the image analyzer 150 determines the image points using geometric parameters that describe the GET125. The geometric parameters may describe the corners of the GET125 (e.g., the relative lengths of the side edges of the front edge, the angles of the corners, the shapes of the corners), or other physical aspects such as the overall size, shape, or thickness of the GET125.

[0085] In step 1040, the wear analyzer 153 determines the GET measurement value based on the image points identified in step 1030. The wear analyzer 153 may determine the GET measurement value by correlating the number of pixels between the image points and the distance measurement value (for example, see FIG. 9). In some embodiments, the wear analyzer 153 may track and record the GET measurement value at the pixel. Based on the GET measurement value determined in step 1040, the wear analyzer 153 determines, in step 1050, the wear level or loss of the GET. The wear level or loss can be quantified in real-world measurements (for example, millimeters), pixel units, or percentage units of the expected size (for example, a CAD-based model based on GET125). As described above, the wear analyzer 153 can use a CAD-based model of GET125 in an unworn state and compare it with the observed GET125 measurement value to determine the GET wear level or loss. The wear analyzer 153 can also use past measurement data of the GET to determine the wear level over time or to determine the wear level trend to predict when the GET125 needs to be replaced. In some embodiments, the wear analyzer 153 may be configured to determine the loss when the wear exceeds a threshold. For example, when its size has decreased by 50% or more, or when a certain measurement quantity (for example, a length of 5 cm) has decreased, the wear analyzer can determine the loss of the GET. The wear analyzer 153 can issue a warning when the wear of the GET reaches or exceeds the threshold.

[0086] In some embodiments, the image analyzer 150 and the wear analyzer 153 execute the process 1000 several times within one excavation-dump cycle. In such embodiments, the wear analyzer 153 may compare the GET measurement values determined in the current excavation-dump cycle (step 1040) with one or more GET measurement values previously (or later) determined within the same excavation-dump cycle, or historical measurement values, and determine whether the currently determined GET measurement values match. If the measurement values do not match (e.g., are different from other GET measurement values within the same excavation-dump cycle by some thresholds), the wear analyzer 153 can discard the current measurement as noise or incorrect data. The thresholds can be configured based on the environment, the work machine, or the type of GET. For example, for a work machine that excavates soft materials with long GETs, the threshold may be set to a low value (e.g., less than 10%), and for a work machine that excavates hard materials with short GETs, the threshold may be set to a higher value (e.g., more than 20%). Also, in some embodiments, the image analyzer 150 and the wear analyzer 153 may also consider the point in time within the excavation-dump cycle when executing the process 1000 to determine whether the determined GET measurement values are noise.

[0087] FIG. 11 shows a flowchart representing an exemplary wear detection process 1100 for detecting wear of the GET 125. In some embodiments, the process 1100 may be executed by the image analyzer 150 and the wear analyzer 153. The process 1100 generally follows the image data flow of FIGS. 5-9 and should be interpreted to be consistent with the descriptions of these figures, the descriptions of the image analyzer 150 and the wear analyzer 153 described above with respect to FIG. 1, and the description of the wear detection process 1000. In the following discussion, aspects of the process 1100 executed by the image analyzer 150 or the wear analyzer 153 will be described, but other components of the wear detection computer system 110 can execute one or more blocks of the process 1100 without departing from the spirit and scope of the present disclosure.

[0088] Process 1100 can be executed within one excavation-dump cycle of the work machine 100 or over multiple excavation-dump cycles of the work machine 100. In some embodiments, the image analyzer 150 and the wear analyzer 153 execute process 1100 several times within a single excavation-dump cycle.

[0089] After the start of the excavation-dump cycle (block 1105), at step 1110, the image analyzer 150 receives first image data from the sensor 126 or the camera 128. After processing the captured first image data (using process 1000 as just one non-limiting example), the wear analyzer 153 determines a first wear measurement value for the GET 125 at block 1115 and then determines a first wear level for the GET 125. The wear analyzer 153 may use the techniques described above with respect to FIGS. 1, 9, and 10 as just some examples to determine the first wear measurement value and the first wear level.

[0090] After the wear analyzer 153 determines the first wear level of the GET125, the wear analyzer 153 determines whether the first wear level indicates a GET replacement condition, for example, whether the wear analyzer 153 should generate a warning notifying the operator of the work machine 100 that GET replacement is necessary. The determination of whether the first wear level indicates a GET replacement condition can include two determinations in some embodiments. One of the determinations is whether the wear level indicates a need for a warning (step 1127). If the wear level does not indicate a need for a warning (step 1127: NO), the process 1100 ends and may be repeated at another time within the same excavation-dump cycle consistent with the embodiments of the present disclosure (block 1195). However, if the wear level indicates a need for a warning (step 1127: YES), the process proceeds to another determination (step 1129) to determine whether the first image data was captured in the second half of the excavation-dump cycle. In some embodiments, the other determination may be based on a "delay point" within the excavation-dump cycle. If the first image data is received after the delay point, the wear analyzer 153 may determine that the excavation-dump cycle is delayed (step 1129: YES), and the process 1100 proceeds to step 1135 where the wear analyzer 153 generates a warning. If the first image data is received before the delay point, the wear analyzer 153 may determine that the excavation-dump cycle is not late (step 1129: NO), and the process 1100 proceeds to step 1140. In some embodiments, the delay point may be set by the operator of the work machine 100 and may have a default setting. For example, the default setting is the midpoint or 50% point of the excavation-dump cycle. Image data captured near the end of the excavation-dump cycle is captured in the second half of the cycle, while image data captured near the start of the excavation-dump cycle is captured in the first half of the excavation-dump cycle (not late).

[0091] If wear analyzer 153 determines that the first GET wear level indicates a need for warning and that the first set of image data was not received in the second half of the excavation-dump cycle, a second set of image data of GET 125 is collected to confirm the need for warning. In step 1140, image analyzer 150 receives the second set of image data from sensor 126 or camera 128. After processing the second set of image data (using process 1000 as merely one non-limiting example), wear analyzer 153 determines a second wear measurement for GET 125 at block 1145, and then determines a second wear level for GET 125 in the same manner as the first wear level for GET 125 was determined above (step 1150). Wear analyzer 153 then determines whether the GET replacement condition is satisfied. If both the first wear level and the second wear level indicate a need for warning (step 1155: YES), the wear analyzer generates a warning at step 1135. However, if the second wear level does not indicate a need for warning (step 1155: NO), the GET replacement condition is not satisfied and process 1100 ends for the current excavation-dump cycle (block 1195).

[0092] In some embodiments, wear analyzer 153 may record the first GET wear level or the second GET wear level (if calculated) for each iteration of process 1100. Logging may include storing GET wear measurements, storing GET wear levels, and / or storing imaging image data. Wear analyzer 153 can record this information in GET wear level storage 157 in wear detection computer system 110 or provide it to a remote monitoring computer system for storage in remote log manager 175.

[0093] FIG. 12 shows a wear detection user interface 1200 as an example that can be rendered on the display 133 or the remote display 163. The user interface 1200 can include captured images 1210 of the bucket 120 and the GET 125. The captured images 1210 can be still images or real-time video images of the bucket 120 and the GET 125 in some embodiments. The user interface 1200 can also include a state user interface element 1220 of the GET 125 that displays the current state of the GET 125. The state user interface element 1220 can display, for example, the current measured value of the GET 125 (as shown in FIG. 12), the wear rate of the GET 125, or both. In some embodiments, the state user interface can include an indicator of GET wear, such as color coding, to provide information to the operator of the work machine 100. For example, when the GET 125 is in good condition without significant wear, the state user interface element 1220 can be rendered using the health indicator 1230. The health indicator 1230 can be color-coded (e.g., green) to indicate that no action is required. As another example, when the GET 125 is partially worn and approaching the need for replacement, the state user interface element 1220 can be rendered using the partial wear indicator 1240. The partial wear indicator 1240 may be color-coded (e.g., yellow), have a special font, or be highlighted within the user interface 1200. As another example, when the GET 125 is worn to the point of needing replacement or is damaged, the state user interface element 1220 can be rendered with the complete wear indicator 1250. The complete wear indicator 1250 can be color-coded (e.g., red) in a special font and highlighted within the user interface 1200 or blink consistently to warn the operator.

[0094] The user interface 1200 can also include a close-up view 1260 of the GET 125. The close-up view 1260 can correspond to, for example, a specified region of interest within the image data. In some embodiments, in addition to the full wear indicator 1250, the user interface 1200 can include a caution notice 1270 for warning the operator of the GET replacement condition. A voice cue or alarm can also accompany the partial wear indicator 1240, the full wear indicator 1250, and / or the caution notice 1270.

[0095] Throughout the above description, some components of the wear detection computer system 110 have been described as performing some operations. However, in some embodiments of the wear detection computer system 110, components other than those described above can perform these operations. Further, the wear detection computer system 110 can include additional components or fewer components than those described above in the example embodiments. Those skilled in the art will understand that the wear detection computer system 110 need not be limited to the specific embodiments disclosed above.

Industrial Applicability

[0096] The systems and methods of the present disclosure can be used in connection with the operation of work machines at work sites for excavating, moving, shaping, contouring, and / or removing materials such as soil, rock, minerals, etc. These work machines may be equipped with buckets used to scoop, dig, and discard materials at the work site. The bucket may be provided with one or more buckets to help loosen the material during operation. The work machine can also include a system having a processor and memory configured to perform a wear detection method according to the examples described herein. This system and method can detect wear or loss of components of a work machine such as a GET, so that an operator of such a work machine can take corrective measures before a failure that could damage downstream processing equipment occurs.

[0097] In some examples, the system and method image imaging data related to the GET from one or more sensors of the work machine and process the imaging data to determine wear or loss of the GET. The one or more sensors can include an image sensor, a stereo camera, a LiDAR sensor, an infrared sensor, a temperature sensor, a sonar sensor, and / or a radar.

[0098] In some examples, the one or more sensors collect imaging data twice during the excavation-dump cycle of the work machine. The first collection of imaging data can be imaged, for example, closer to the start of the cycle than the end of the cycle, such as during an early part of the excavation-dump cycle. If the processing of the first collection of imaging data at the first time indicates GET wear detection conditions that suggest loss of the GET, the system creates a second collection of imaging data of the GET for analysis and confirmation of acute GET wear or loss. If the processing of the second collection of imaging data indicates GET wear detection conditions that suggest loss of the GET, the system can generate a warning to the operator of the work machine so that the operator of the work machine can take corrective measures (such as stopping the operation of the work machine to remove the broken GET from the work site).

[0099] The first collection of imaging data can also be imaged, for example, in the second half of the excavation-dump cycle, closer to the end of the cycle than the start of the cycle. In such a situation, if the processing of the first collection of imaging data indicates GET wear detection conditions that suggest loss of the GET, the system warns the operator without obtaining a second collection of imaging data. The system may do this if there is insufficient time to accurately image the second collection of imaging data to confirm the GET wear detection conditions.

[0100] The wear detection system described in this disclosure provides visual information regarding the wear of the GET to the operator of the work machine while the work machine is operating. For example, the wear level of the GET can be rendered on the operator's display. When a GET wear detection condition indicating loss or severe wear is detected, the system may render an image of the GET for which the wear detection condition was detected on the operator's display. The operator can then use the rendered image to assist in determining whether corrective action is necessary.

[0101] By the process described in this disclosure, it is possible to detect the wear state of the GET in a timely manner while reducing false positives and operator discomfort.

[0102] Aspects of the present disclosure have been particularly shown and described with reference to the above examples, but it will be understood by those skilled in the art that various additional embodiments can be contemplated by modifying the devices, systems, and methods of the present disclosure without departing from the spirit and scope of the present disclosure. Such embodiments are to be understood as being within the scope of the present disclosure as determined based on the claims and any equivalents thereof.

Claims

Claim 1 Receiving, from one or more sensors (126, 128) associated with a work machine (100), first image data related to at least one ground engaging tool (GET) of the work machine at a first point in the excavation-dump cycle of the work machine; Determining a first wear measurement value in the at least one GET based on the first image data; Determining a first wear level of the at least one GET corresponding to the first point in time based on the first wear measurement value; Determining whether the first wear level indicates a GET replacement condition; A method implemented by a computer, comprising: If it is determined that the first wear level indicates the GET replacement condition, Generating a warning indicating the first wear level, or If it is determined that the first wear level does not indicate the GET replacement condition, Receiving, from the one or more sensors, second image data related to the at least one GET of the work machine at a second point in time different from the first point in time in the excavation-dump cycle of the work machine; Determining a second wear measurement value of the at least one GET based on the second image data; Determining a second wear level of the at least one GET corresponding to the second point in time based on the second wear measurement value; Determining whether the second wear level indicates the GET replacement condition, and if it is determined that the second wear level indicates the GET replacement condition, generating a warning indicating the second wear level. Method. Claim 2 The method implemented by a computer according to claim 1, wherein determining that the first wear level indicates the GET replacement condition includes determining that the first point in time is closer to the end of the excavation-dump cycle than to the start of the excavation-dump cycle. Claim 3 The method implemented by a computer according to claim 1, wherein generating the warning indicating the first wear level includes rendering a graphic indicator (1220) of the first wear level on a display (133) associated with the work machine. Claim 4 Determining the first wear measurement value of the at least one GET includes generating a disparity map (530, 730) based on the first image data, the method implemented by a computer according to claim 1.

5. Determining the first wear measurement value of the at least one GET further includes applying a deep learning GET detection algorithm to the disparity map, the method implemented by a computer according to claim 4.

6. One or more sensors (126, 128) associated with a work machine (100), One or more processors (140), A non-transitory computer-readable medium (143) storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including the following: A system comprising: The operations are: Receiving, from the one or more sensors associated with the work machine, first image data related to at least one ground engaging tool (GET) of the work machine at a first point in time in the excavation-dump cycle of the work machine; Determining a first wear measurement value of the at least one GET based on the first image data; Determining a first wear level of the at least one GET corresponding to the first point in time based on the first wear measurement value; Determining whether the first wear level indicates a GET replacement condition; Including, If it is determined that the first wear level indicates the GET replacement condition, Generating a warning indicating the first wear level, or If it is determined that the first wear level does not indicate the GET replacement condition, Receiving, from the one or more sensors, second image data related to the at least one GET of the work machine at a second point in time different from the first point in time in the excavation-dump cycle of the work machine; Determining a second wear measurement value of the at least one GET based on the second image data; Determining a second wear level of the at least one GET corresponding to the second point in time based on the second wear measurement value; Determining whether the second wear level indicates the GET replacement condition, and if it is determined that the second wear level indicates the GET replacement condition, generating a warning indicating the second wear level. System.

7. Determining that the first wear level indicates the GET replacement condition includes determining that the first point in time is closer to the end of the excavation-dump cycle than to the start of the excavation-dump cycle, the system of claim 6. **Claim 8** Generating the warning indicating the first wear level includes generating a graphic indicator (1220) of the first wear level on a display (133) associated with the work machine, the system of claim 6. **Claim 9** Determining the first wear measurement of the at least one GET includes generating a disparity map (530, 730) based on the first image data, the system of claim 6. **Claim 10** Determining the first wear measurement of the at least one GET further includes applying a deep learning GET detection algorithm to the disparity map, the system of claim 9.

Images