Ground engaging tool wear and loss detection system and method

Abstract

An example wear detection system (110) receives a left image (510) and a right image (520) of a bucket (120) of a work machine (100) having at least one ground engaging tool (GET) (125). An example system identifies a first region of interest (550) corresponding to the GET from the left image and a second region of interest (560) corresponding to the GET from the right image. The example system also generates a left edge digital image (740) corresponding to the first region of interest and a right edge digital image (750) corresponding to the second region of interest. Further, the example system determines a sparse stereo disparity (760) between the left edge digital image and the right edge digital image, determines measurement data related to the GET, and determines a wear level or loss of the at least one GET also based on the measurement data.

Description

Technical Field

[0001] This disclosure relates to a system and method for detecting wear and tear on an object over time, and more specifically to a system and method for detecting wear or damage to a ground bonding tool (GET) over time using computer vision techniques. Background Technology

[0002] Machines can be used to perform a variety of tasks at a work site. For example, machines can be used to excavate, move, shape, profile, and / or remove materials present at the work site, such as gravel, concrete, asphalt, soil, and / or other materials. These machines may include a bucket for collecting such materials, and the bucket may include a set of ground engagement tools (GETs), such as teeth, to loosen the material. The GET may also include a guard attached between the teeth to the bucket to protect the edges of the bucket. Over time, the GET wears and shrinks in size, reducing its effectiveness and making it more difficult for the bucket to collect material from the work site. GETs may also break off from the bucket. When GET breakage is not detected, the GET can mix with the work site material and may damage downstream processing equipment such as crushers or pulverizers. The machine can utilize a wear detection system to identify worn or broken GETs before they cause damage to downstream equipment.

[0003] An attempt to provide a wear detection system is described in WIPO PCT Publication WO2019 / 227194A1 (“'194 Publication”), published on December 5, 2019. Specifically, '194 Publication describes a method and system for monitoring the condition of an operating tool of heavy equipment, such as the teeth of a machine bucket. The method and system receive an image of the operating tool and process the image using a first neural network trained to identify regions of interest within the image. Each region of interest has an associated name called a critical region or a non-critical region. An embedded processor adjacent to the heavy equipment further processes the critical regions using a second neural network to identify “wear marks” on the operating tool. The system and method then compare the wear marks with a reference image to determine the wear on the operating tool.

[0004] The '194 disclosure's reliance on neural networks and machine learning to identify wear levels can be problematic because it is difficult to measure GET wear at a scale that allows for rapid detection of wear conditions and accurate scheduling of preventative maintenance. For example, using neural networks and machine learning specifically may only provide measurement accuracy within a few centimeters, but accuracy within a range of less than 5 millimeters may be desirable. Furthermore, machine learning techniques, such as those described in the '194 disclosure, can be prone to a large number of "false alarms" regarding GET wear or depletion, rendering them ineffective. The systems and methods described herein address one or more of these concerns. Summary of the Invention

[0005] According to a first aspect, a method for detecting wear or damage to a ground engagement tool (GET) includes receiving a left image and a right image of the bucket of a working machine from a stereo camera associated with the machine. The bucket has at least one GET. The method further includes: identifying a first region of interest (ROI) corresponding to the at least one GET from the left image; and identifying a second ROI corresponding to the at least one GET from the right image. The method further includes generating a left edge digital image corresponding to the first ROI; and generating a right edge digital image corresponding to the second ROI. The method further includes determining a sparse stereo parallax between the left edge digital image and the right edge digital image. Based on the sparse stereo parallax, the method determines the wear level or damage of the at least one GET.

[0006] According to another aspect, a GET wear detection system includes a stereo camera, one or more processors, and a non-transitory computer-readable medium storing executable instructions. When executed by the processor, the executable instructions cause the processor to perform operations including receiving left and right images of a bucket of a working machine from the stereo camera. The bucket has at least one GET. The operations also include identifying a first region of interest (ROI) corresponding to the at least one GET from the left image and a second ROI corresponding to the at least one GET from the right image. The operations further include generating a left edge digital image corresponding to the first ROI and generating a right edge digital image corresponding to the second ROI. The operations also include determining a sparse stereo parallax between the left edge digital image and the right edge digital image. Based on the sparse stereo parallax, the processor determines the wear level or wear of the at least one GET.

[0007] According to another aspect, the working machine includes a left monochrome image sensor, a right monochrome image sensor, and a color image sensor. The working machine also includes a bucket having at least one ground engagement tool (GET), one or more processors; and a non-transitory computer-readable medium storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations may include receiving a left image of the bucket captured by the left monochrome image sensor, a right image of the bucket captured by the left monochrome image sensor, and a color image of the bucket captured by the color image sensor from a stereo camera. The operations also include identifying a first region of interest from the left image and a second region of interest from the right image by applying a deep learning GET detection algorithm to the left and right images and generating dense stereo disparity maps for the left and right images. The operations further include generating a left edge digital image corresponding to the left region of interest and a right edge digital image corresponding to the right region of interest using gradient magnitude edge detection technology. The operations also include determining a sparse stereo disparity between the left edge digital image and the right edge digital image, and determining a wear level or depletion of the at least one GET based on the sparse stereo disparity. Attached Figure Description

[0008] The accompanying drawings are described in detail. In the drawings, the leftmost numeral of the reference numeral indicates the figure in which the reference numeral first appears. The same reference numerals in different figures indicate similar or identical items.

[0009] Figure 1 This is a block diagram depicting a sample machine, which includes an example system for detecting wear in GET.

[0010] Figure 2 This is a schematic side view of an example environment with an example machine, including an example system for detecting wear in GET.

[0011] Figure 3 This is a schematic side view of another example environment with an example machine, which includes an example system for detecting wear in GET.

[0012] Figure 4 This is a schematic side view of another example environment with an example machine, which includes an example system for detecting wear in GET.

[0013] Figure 5 This is an image data flow diagram illustrating an example flow of image data for the region of interest detection process using computer vision techniques.

[0014] Figure 6 This is an image data flow diagram illustrating an example flow of image data for the region of interest detection process using deep learning techniques.

[0015] Figure 7 This is an image data flow diagram illustrating an example flow of image data in a wear detection process using computer vision technology.

[0016] Figure 8 This is an image data flow diagram illustrating an example flow of image data for a wear detection process using deep learning techniques.

[0017] Figure 9 This is an example process for detecting wear and tear in an example environment using computer vision technology.

[0018] Figure 10 This is an example process for detecting wear and tear in a sample environment using deep learning techniques.

[0019] Figure 11 This is an example process that uses a combination of computer vision and deep learning techniques to detect wear and tear in a sample environment. Detailed Implementation

[0020] This disclosure generally relates to systems and methods for detecting wear on components of a working machine in an environment such as a work site using computer vision techniques. In some instances, a stereoscopic camera (or "stereo camera") associated with the working machine captures video of the machine's components. The video is analyzed by a wear detection computer system associated with the working machine, which may be located inside or outside the stereoscopic camera, to detect wear on the components. As an example, the component may be one or more ground engagement tools (GETs) of the bucket of the working machine. The stereoscopic camera captures a left and a right image including the GET, and the wear detection computer system processes the images using stereo computer vision techniques to identify regions of interest (ROIs) corresponding to the GETs in both the left and right images. Alternatively or additionally, a deep learning GET detection algorithm may be employed, trained to identify ROIs in both the left and right images. Once the wear detection computer system has identified the ROIs in both the left and right images, it further processes the ROIs to generate edges associated with the GETs. The wear detection computer system uses a left-edge digital image and a right-edge digital image to determine sparse stereo parallax. Wear or damage to the GET is determined based on the number of pixels contained within the sparse stereo parallax, either by comparing this sparse stereo parallax with a previously determined sparse stereo parallax of the same GET, or by comparing this sparse stereo parallax with a reference image. By using the sparse stereo parallax between the left-edge and right-edge digital images, the system and method disclosed herein can accurately measure wear or damage to the GET at an accuracy level of less than 5 mm.

[0021] Figure 1 This is a block diagram depicting an example work machine 100, including an example wear detection computer system 110. Although Figure 1 The working machine 100 is depicted as a hydraulic excavator, but in other instances, the working machine 100 may include any machine that moves, sculpts, excavates, or removes materials such as soil, rock, or minerals. Figure 1 As shown, the working machine 100 may include a bucket 120 attached to the arm 122. The bucket 120 may include one or more ground engagement tools (GET) 125, such as teeth, to assist the working machine 100 in loosening material. While the examples provided in this disclosure generally refer to the GET 125 as teeth, other types of GETs are contemplated within the scope of the embodiments provided by this disclosure. For example, a GET may include a lip guard, edge protector, adapter, ripper protector, cutting edge, side bar protector, tip, or any other tool associated with the working machine that wears down over time due to friction with materials at the work site.

[0022] The working machine 100 may also include a stereo camera 128. The camera 128 has a field of view 129 pointing towards the bucket 120 and the GET 125. The stereo camera 128 includes a left image sensor and a right image sensor spaced apart to capture stereo images of objects (e.g., the bucket 120 and the GET 125) within the field of view 129. In some embodiments, the left and right image sensors capture monochrome images. The stereo camera 128 may also include a color image sensor to capture color images of objects within the field of view 129. In some embodiments, the camera 128 outputs digital images, or the working machine 100 may include an analog-to-digital converter disposed between the camera 128 and the wear detection computer system 110 to convert analog images into digital images before the wear detection computer system 110 receives analog images.

[0023] When machine 100 operates within the work site, it can move its boom 122 to position the bucket 120 to move or excavate material within the work site as part of a dig-and-dump cycle. As machine 100 positions the bucket 120 through the dig-and-dump cycle, the bucket 120 can move in and out of the field of view 129 of camera 128. Camera 128 can be positioned such that it has an unobstructed view GET 125 during the dig-and-dump cycle. For example, camera 128 can be positioned on machine 100 such that the bucket 120 and GET 125 are visible at the moment the bucket 120 is emptied of material during the dig-and-dump cycle. As another example, camera 128 can be positioned such that the bucket 120 enters the camera's field of view when boom 122 is fully extended or fully retracted during the dig-and-dump cycle. (See below for more details.) Figure 2-4 As explained, the position of camera 128 can vary depending on the type of machine 100 and details related to its work site.

[0024] According to some embodiments, the machine tool 100 may include an operator control panel 130. The operator control panel 130 may include a display 133 that generates output for the operator of the machine tool 100, allowing the operator to receive status or alarms related to the wear detection computer system 110. The display 133 may include a liquid crystal display (LCD), a light-emitting diode (LED) display, a cathode ray tube (CRT) display, or any other type of display known in the art. In some instances, the display 133 may include audio output, such as a speaker or a port for headphones or peripheral speakers. The display 133 may also include an audio input device, such as a microphone or a port for a peripheral microphone. In some embodiments, the display 133 may include a touch-sensitive display screen, which may also function as an input device.

[0025] In some embodiments, the operator control panel 130 may further include a keyboard 137. The keyboard 137 may provide input capability to the wear detection computer system 110. The keyboard 137 may include a plurality of keys that allow the operator of the working machine 100 to provide input to the wear detection computer system 110. For example, according to an embodiment of this disclosure, the operator may press keys on the keyboard 137 to select or input the type of the working machine 100, bucket 120, and / or GET 125. The keyboard 127 may be non-virtual (e.g., containing physically pressable keys) or the keyboard 127 may be a virtual keyboard displayed on a touch-sensitive embodiment of the display 133.

[0026] like Figure 1 As shown, the wear detection computer system 110 may include one or more processors 140. Processor 140 may include one or more of a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), some combination of CPU, GPU, or FPGA, or any other type of processing unit. Processor 140 may have a plurality of arithmetic logic units (ALUs) that perform arithmetic and logical operations, and one or more control units (CUs) that fetch instructions and stored contents from processor cache memory and then execute the instructions by invoking the ALUs as necessary during program execution. Processor 140 may also be responsible for executing drivers and other computer-executable instructions for application programs, routines, or processes stored in memory 150, which may be associated with common types of volatile (RAM) and / or non-volatile (ROM) memory.

[0027] In some embodiments, the wear detection computer system 110 may include memory 150. Memory 150 may include system memory, which may be volatile (e.g., RAM), non-volatile (e.g., ROM, flash memory, etc.), or some combination of both. Memory 150 may also include non-transitory computer-readable media, such as volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. System memory, removable storage devices, and non-removable storage devices are all examples of non-transitory computer-readable media. Examples of non-transitory computer-readable media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other non-transitory media that can be used to store desired information and is accessible by the wear detection computer system 110.

[0028] Memory 150 may store data including computer-executable instructions for use with the wear detection computer system 110 as described herein. For example, memory 150 may store one or more components of the wear detection computer system 110, such as a physical parameter library 160, an image analyzer 170, a wear analyzer 175, and an alarm manager 180. Memory 150 may also store additional components, modules, or other code executable by processor 140 to enable the operation of the wear detection computer system 110. For example, memory 150 may include code related to input / output functions, software drivers, operating systems, or other components.

[0029] According to some embodiments, aspects of the wear detection computer system 110 may be located within the camera 128. For example, the camera 128 may include one or more of a processor 140 and / or a memory 150. Alternatively, aspects of the wear detection computer system 110 may be located on the machine tool 100 and outside the camera 128. For example, both the machine tool 100 and the camera 128 may include one or more processors 140 or memory 150, or one or more processors 140 or memory 150 may be located entirely outside the camera 128 but within the machine tool 100.

[0030] The physical parameter library 160 may include a set of physical parameters relating to the machine 100, bucket 120, and / or GET 125. For example, the physical parameter library 160 may include measurement data relating to the dimensions and shape of the bucket 120, the dimensions and shape of the GET 125, and the spatial relationship between the GET 125 and the bucket 120 (to name just a few examples). The physical parameter library 160 may also include parameters relating to the dimensions and shape of the GET 125 in a new or unworn condition, as well as parameters relating to the dimensions and shape of the GET 125 when it has reached maximum wear.

[0031] The physical parameter library 160 may also include templates or reference images (e.g., bucket-tool templates) relating to the combination of bucket 120 and GET 125. For example, for machine 100, one of the templates stored in the physical parameter library 160 may include an image of bucket 120 and GET 125 when bucket 120 is intended to be positioned within the field of view of camera 128. The bucket-tool template may be for GET 125 when it is unworn (e.g., unworn or expected edge) or when it has reached maximum wear (e.g., threshold edge). The physical parameter library 160 may also include additional information relating to wear of GET 125 to assist wear analyzer 175 in determining when GET has worn to the point of needing replacement. Wear data relating to GET 125 may be in the form of actual measurements (e.g., metric or imperial dimensions) or in the form of pixel values.

[0032] The physical parameter library 160 may include multiple physical parameter sets, each corresponding to a working machine, bucket, GET, or a combination thereof. During operation, the operator can use the operator control panel 130 to select a physical parameter set from the physical parameter library 160 that matches the bucket 120 and GET 125 or the working machine 100. For example, if the working machine 100 is a hydraulic excavator with model number "6015B", the operator can enter model number "6015B" using the operator control panel 130, and the wear detection computer system 110 can load the physical parameter set corresponding to the model 6015B hydraulic excavator from the physical parameter library 160 into the memory 150. In some instances, after the wear detection computer system 110 is powered on or reset, a list of templates available in the physical parameter library 160 can be displayed on the display 133, and the operator can select one of the physical parameter sets from the list for operation based on the model of the working machine 100, the bucket type of the bucket 120, or the type of the GET 125.

[0033] In some embodiments, an operator may use input on an operator control panel 130 to position bucket 120 and GET 125 within the field of view 129 of camera 128 at the start of a work shift, and cause wear detection computer system 110 to capture images of bucket 120 and GET 125. Wear detection computer system 110 may then perform an image matching process to match bucket 120 and GET 125 with a set of physical parameters, and configure itself based on the matched set of physical parameters for use in the wear detection and image processing processes disclosed herein.

[0034] Image analyzer 170 can be configured to analyze images captured by camera 128 to identify GET 125 within the images and to measure wear on GET 125 based on the processed form of these images. For example, image analyzer 170 can receive stereo images from camera 128 in the form of a left-corrected image (captured by the left image sensor of camera 128) and a right-corrected image (captured by the right image sensor of camera 128). Image analyzer 170 can perform various computer vision techniques on the left-corrected and right-corrected images to identify or determine regions of interest within them corresponding to GET 125.

[0035] In one embodiment, the image analyzer 170 can create a dense stereo disparity map based on a left-corrected image and a right-corrected image. The image analyzer can segment the dense stereo disparity map to identify regions of interest. Additionally, the image analyzer 170 can also create a 3D point cloud based on the dense stereo disparity map, and can segment the 3D point cloud to identify regions of interest.

[0036] Once the image analyzer 170 identifies the region of interest, it can further process the region of interest to create a left-edge digital image corresponding to the left-corrected image and a right-edge digital image corresponding to the right-corrected image. The image analyzer 170 may employ edge detection based on gradient magnitude search, but in other embodiments, it may employ other edge detection techniques used in the field of computer vision (e.g., zero-crossing edge detection techniques) to create the left-edge and right-edge digital images.

[0037] In addition to, or as an alternative to, computer vision techniques, image analyzer 170 may employ deep learning or machine learning techniques to identify regions of interest within the left-corrected and right-corrected images captured by camera 128. For example, image analyzer 170 may use a deep learning GET detection algorithm that employs a neural network trained on an image corpus in which individual GETs, groups of GETs, or combinations of GETs and buckets are labeled to identify regions of interest. Image analyzer 170 may also use a deep learning GET-localization algorithm that employs a neural network trained to localize GETs within an image. The GET-localization algorithm has been trained using an image corpus in which the individual GET is labeled. Once the GET-localization algorithm identifies an individual GET within the image, it outputs the corresponding location of the GET. For example, the GET-localization algorithm may output a pixel location or bounding box output related to the location of the GET.

[0038] In some instances, image analyzer 170 can refine edge estimation of the GET and / or identify individual GET 125s by using the expected location of the GET 125 within the captured image. For example, image analyzer 170 can know the expected location of the GET 125 relative to the bucket 120 based on a set of physical parameters stored in physical parameter library 160 corresponding to the types of the bucket 120 and GET 125 in use. Using this information, image analyzer 170 can navigate to the expected location in a selected image and capture a pixel region adjacent to the tooth. The pixel region can then be used to further identify the tooth based on computer vision techniques, such as applying convolutional filters, segmentation analysis, edge detection, or pixel intensity / darkness analysis within the pixel region. In some embodiments, image analyzer 170 can use computer vision techniques to apply individual tooth templates to the pixel region to further refine the tooth location. Image analyzer 170 can further refine the edges using dynamic programming techniques. Dynamic programming techniques may include smoothing or other edge detection optimization techniques based on edge intensity (regardless of whether the edge is close to a hole or an uncertain region in a dense stereo parallax map). Image analyzer 170 can also use the output of the GET-localization algorithm to obtain confidence in determining the location of the GET, and further refine edge estimation based on the output of the GET-localization algorithm.

[0039] Image analyzer 170 can also generate sparse stereo disparity, which is provided to wear analyzer 175 so that wear analyzer 175 can determine wear in GET 125. In some embodiments, image analyzer 170 generates sparse stereo disparity between a left edge digital image (associated with a left-corrected image) and a right edge digital image (associated with a right-corrected image), and this disparity is used by wear analyzer 175. Alternatively, sparse stereo disparity can be calculated from a first region of interest image (associated with a left-corrected image) and a second region of interest image (associated with a right-corrected image), and image analyzer 170 can detect edges from the sparse stereo disparity image.

[0040] Wear analyzer 175 can be configured to analyze sparse stereo parallax generated by image analyzer 170 for wear analysis. For example, the set of physical parameters associated with bucket 120 and GET 125 may include expected data related to an unworn GET 125 or a set of unworn GET 125 that has been calibrated based on expected image capture from camera 128. The expected data may be in the form of pixels, actual measurements, or edge images related to the unworn GET. Once wear analyzer 175 receives the sparse stereo parallax, it can determine measurement data related to the GET 125 used by machine 100. The wear analyzer can then compare the determined measurement data with the expected data corresponding to the unworn form of the GET 125 to determine the wear level or wear of the GET 125.

[0041] In some embodiments, pixel counts associated with sparse stereo parallax can be used to measure wear or damage to the GET. As just some examples, pixel counts may include area (e.g., total pixels of the GET), the height of the GET (pixels), the width of the GET (pixels), or the sum of the height and width of the GET. The method for determining pixel counts can vary depending on the shape and style of the GET. For example, for a GET whose length is much greater than its width, a height pixel count may be used, while for a GET whose width is much greater than its length, a width pixel count may be used. Various methods for determining pixel counts may be used without departing from the spirit and scope of this disclosure.

[0042] In some embodiments, the wear analyzer 175 may calculate a similarity score between determined measurement data extracted from sparse stereo parallax and expected data corresponding to an unworn GET 125. The similarity score may be a measure of how well the determined measurement data of the GET 125 matches the expected data of the physical parameter set. For example, the similarity score may include a cross-union ratio or Jaccard index method for detecting similarity. In some embodiments, a Dice coefficient or F1 score method for detecting similarity may be used to determine the similarity score. The similarity score may also include a value reflecting the percentage of pixels that overlap between the sparse stereo parallax and the expected edge image. In some embodiments, the similarity score may be scaled or normalized from zero to one hundred.

[0043] Similarity ratings can provide an indication of GET 125 wear. For example, a low rating (e.g., a range of 0 to 20) may indicate that one of the GET 125 teeth is broken or missing, indicating tooth wear. A high rating (e.g., a range of 80-100) may indicate that the tooth is in good health and does not require replacement. Ratings between low and high ratings can provide the level of tooth wear, where a higher rating indicates a longer lead time for tooth replacement compared to a lower rating.

[0044] In some embodiments, wear analyzer 175 may collect measurement data associated with GET 125 over time and use the collected measurement data to determine the wear level and wear trend of GET 125. For example, machine 100 may operate at its work site for several days to perform work. As machine 100 moves material during work, camera 128 provides stereo images of bucket 120 and GET 125 to wear detection computer system 110, and image analyzer 170 creates sparse stereo parallax for GET 125. Wear analyzer 175 may plot measurement data associated with GET 125 at several points within the time period of work (e.g., pixel counts, metric measurements, imperial measurements). As bucket 120 and GET 125 engage with material at the work site, GET 125 is expected to shrink in size due to wear. Therefore, the measurement data associated with GET 125 will also decrease over time, and the pixel count over time will reflect the wear trend. Wear analyzer 172 can determine the wear level of GET 125 at a specific time point using wear trends at that specific time point. The wear level of GET 125 can indicate that GET 125 needs to be replaced, or it can indicate the wear of one or more of GET 125. In some embodiments, measurement data associated with GET 125 can be stored in memory 150 and applied to multiple jobs and multiple work sites, and the wear trend can be applied to the service life of GET 125. In such embodiments, the pixel counts captured by wear analyzer 175 associated with GET 125 can be reset when bucket 120 or GET 125 is replaced, and wear analyzer 175 can restart collecting pixel counts of GET 125 from a zero time point.

[0045] Because the wear analyzer 175 determines wear trends based on measurement data of the GET 125 taken over time, it can also predict when the GET 125 may need to be replaced. For example, if the wear analyzer 175 determines that the measurement data associated with the GET 125 shows that the GET 125 loses 1% of its lifespan every ten working hours (because the measurement data decreases by 1% every ten working hours), and the GET 125 has been used for eight hundred working hours, the wear analyzer 175 may determine that the GET 125 needs to be replaced within 200 hours.

[0046] In some embodiments, the wear detection computer system 110 may include an alarm manager 180. The alarm manager 180 may communicate with a wear analyzer 175 and may monitor wear trends and wear levels determined by the wear analyzer 175. The alarm manager 180 may provide message alerts to an operator control panel 130 based on the information determined by the wear analyzer 175. For example, when the wear level reaches a wear threshold, the alarm manager 180 may generate an alarm displayed on a display 133 of the operator control panel 130. The threshold may correspond to a value indicating extreme GET wear, or in some cases, a value corresponding to complete GET wear. The alarm may provide an indication to the operator of the machine 100 that one or more GETs 125 need to be replaced. The wear threshold may vary depending on the embodiment and may depend on the type of GET 125 and the material at the work site where the GET 125 is engaged.

[0047] The alarm manager 180 can also provide alarms indicating that GET 125 may need to be replaced at some point in the future, such as within two weeks. Replacement alarms may include information related to wear trend predictions for GET 125. For example, replacement alarms may include quantifications of wear trends (e.g., GET 125 wears 2% per workday), the amount of time the teeth have been used, or the expected date or time when GET 125 will reach a wear threshold based on usage data.

[0048] In some embodiments, the alarm manager 180 may monitor wear trends determined by the wear analyzer 175 and provide wear level values ​​to the display 133 to notify the operator of the machine 100 of the current wear level. For example, if the wear trend indicates that GET 125 is 60% worn, the alarm manager 180 may provide an indication that GET 125 has 40% of its life remaining before replacement is required, based on the wear trend. The display 133 may also notify the operator that a tooth has broken, indicating tooth wear (e.g., when one or more of GET 125 have less than 20% of their life remaining).

[0049] The wear detection computer system 110 allows the operator of machine 100 to be notified when the GET 125 needs replacement or has broken due to excessive wear. The process employed by the wear detection computer system 110—described in more detail below—provides accurate and precise measurements of GET wear on a scale of less than 5 mm, allowing the operator to stop machine 100 in the event of extreme GET wear or damage. The processes and techniques deployed by the wear detection computer system 110 can be used with a variety of machines.

[0050] For example, Figure 2This is a schematic side view depicting an example environment 200 in which a wheel loader working machine 201 is operating. The wheel loader working machine 201 may include a bucket 220 and one or more GET 225. Figure 2 As shown, camera 228 is positioned such that GET 225 and bucket 220 are within the field of view 229 of camera 228 during the end of the digging-dumping cycle. Therefore, in such embodiments, camera 228 can be configured to capture images when bucket 220 is stationary at the end of the digging-dumping cycle.

[0051] As another example, Figure 3 This is a schematic side view depicting an example environment 300 in which a hydraulic excavator shovel machine 301 is operating. The hydraulic excavator shovel machine 301 may include a bucket 320 and one or more GETs 325. The camera 328 is positioned relative to the position of a camera 228 for a wheel loader machine 201 such that the GET 325 is within the field of view 329 of the camera 328 during the end of an excavation cycle. In such embodiments, the camera 328 may be configured to capture images when the bucket 320 is stationary at the end of an excavation cycle.

[0052] In yet another example, Figure 4 This is a schematic side view depicting an example environment 400 in which an electric rope shovel working machine 401 is operating. The electric rope shovel working machine 401 may include a bucket 420, one or more GETs 425, and a camera 428. Figure 4 As shown, GET 425 may be at the midpoint of the digging-dumping cycle, but within the field of view 429 of camera 428 when bucket 420 is relatively close to camera 428. In such embodiments, camera 428 may be configured to capture images when bucket 420 enters a positional range associated with the field of view 429 of camera 428.

[0053] Figure 5 Image data flow diagram 500 is depicted, illustrating an example flow of image data for a region of interest (ROI) detection process using computer vision techniques. Image data flow diagram 500 includes images received, processed, and generated by image analyzer 170 when detecting ROIs in an image captured by camera 128 related to GET 125. Image data flow diagram 500 includes a left image 510 and a right image 520 captured by camera 128. Left image 510 may be a corrected image captured by the left image sensor of camera 128. Right image 520 may be a corrected image captured by the right image sensor of camera 128. Both left image 510 and right image 520 include images of bucket 120 and GET 125.

[0054] Image analyzer 170 can process left image 510 and right image 520 to generate disparity map 530. Disparity map 530 can be a dense stereo disparity map, which shows the disparity between each pixel of left image 510 and each pixel of right image 520. Using disparity map 530 and a set of physical parameters 535 obtained from physical parameter library 160 and associated with bucket 120, GET 125 and / or working machine 100, image analyzer 170 can construct 3D point cloud 540. 3D point cloud 540 shows the disparity between left image 510 and right image 520 in three dimensions. Image analyzer 170 can then perform conventional segmentation analysis on 3D point cloud 540 to identify a first region of interest 550 corresponding to left image 510 and a second region of interest 560 corresponding to right image 520.

[0055] Figure 6 Image data flow diagram 600 is depicted, illustrating an example flow of image data for a region of interest (ROI) detection process using deep learning techniques. Similar to the image data flow diagram 500 above, the output of the ROI detection process will correspond to a first ROI 550 and a second ROI 560 for GET 125. However, unlike image data flow diagram 500, image analyzer 170 utilizes deep learning techniques to detect the first ROI 550 and the second ROI 560.

[0056] Image data flow diagram 600 includes a left image 510 and a right image 520 captured by camera 128. Left image 510 may be a corrected image captured by the left image sensor of camera 128. Right image 520 may be a corrected image captured by the right image sensor of camera 128. Both left image 510 and right image 520 include images of bucket 120 and GET 125.

[0057] Image analyzer 170 can apply a deep learning GET detection algorithm to the left image 510. The deep learning GET detection algorithm can employ a neural network trained on an image data corpus in which GETs have been individually identified and labeled, and / or GET groups have been individually identified and labeled. When image analyzer 170 applies the deep learning GET detection algorithm to the left image 510, it can identify multiple individual GET bounding boxes 610 that contain a single GET 125. In some embodiments, image analyzer 170 can also identify GET group bounding boxes 615 that cover individual GET bounding boxes 610. Similarly, when image analyzer 170 applies the deep learning GET detection algorithm to the right image 520, it can identify multiple individual GET bounding boxes 620 that contain a single GET 125. In some embodiments, image analyzer 170 can also identify GET group bounding boxes 625 that cover individual GET bounding boxes 620 in the right image 520. Once the image analyzer 170 identifies the GET group bounding box 615, it can extract the pixels of the GET group bounding box as a first region of interest 550. The image analyzer 170 can also extract the pixels of the GET group bounding box 625 as a second region of interest 560.

[0058] Figure 7 Image data flow diagram 700 is depicted, illustrating an example flow of image data in a wear detection process using computer vision techniques. Image data flow diagram 700 includes an image analyzer 170 receiving, processing, and generating images as it produces output, which is ultimately provided to a wear analyzer 175 to detect wear or damage to the GET 125. In some embodiments, the generated output will be in the form of a sparse stereoscopic image.

[0059] Image data flow diagram 700 includes a first region of interest 550 and a second region of interest 560. Both the first region of interest 550 and the second region of interest 560 may have already been generated by the image analyzer 170, as described above. Figure 5 or Figure 6 Specifically, the image analyzer 170 may have generated a first region of interest 550 and a second region of interest 560 using computer vision or deep learning techniques. In some embodiments, the first region of interest 550 and the second region of interest 560 may have been generated using a combination of computer vision and deep learning techniques.

[0060] Image data flow diagram 700 also includes an unthinned left-edge digital image 710. Image analyzer 170 can generate the unthinned left-edge digital image 710 by applying computer vision edge detection techniques to a first region of interest 550. Computer vision edge detection techniques may include search-based edge detection techniques, such as gradient magnitude edge detection techniques. Computer vision edge detection techniques may also include zero-crossing based techniques. Image analyzer 170 may also perform preprocessing steps, such as Gaussian smoothing, before generating the unthinned left-edge digital image 710. In some embodiments, image analyzer 170 may employ the Canny edge detector or other well-known edge detectors in the field of computer vision. Similarly, image analyzer 170 may use similar computer vision edge techniques to generate an unthinned right-edge digital image 720.

[0061] In some embodiments, the image analyzer 170 can reduce errors in both the unthinned left-edge digital image 710 and the unthinned right-edge digital image 720 by performing dynamic programming on both. Dynamic programming may include applying a physical parameter set 535 to a series of optimization routines to reduce errors. Optimization routines may include evaluating the intensity of detected edges, analyzing whether the detected edges are close to holes or uncertain regions in the disparity map 530 (or the location of edges that can be predicted based on the physical parameter set 535) or included in said holes or uncertain regions (or the location of edges that can be predicted based on the physical parameter set). The image analyzer 170 can then output a thinned left-edge digital image 740 and a thinned right-edge digital image 750 based on the dynamic programming. The image analyzer 170 can then generate a sparse stereo disparity 760 by calculating the sparse stereo disparity between the thinned left-edge digital image 740 and the thinned right-edge digital image 750. Image analyzer 170 can then provide sparse stereo parallax 760 to wear analyzer 175 to detect wear loss of GET 125 according to the disclosed embodiments.

[0062] Figure 8 Image data flow diagram 800 is depicted, illustrating an example flow of image data in a wear detection process using deep learning techniques. Image data flow diagram 800 includes an image analyzer 170 receiving, processing, and generating images during output generation, which will ultimately be provided to a wear analyzer 175 to detect wear or damage to GET 125. The generated output may be in the form of a sparse stereo image, or it may be in the form of location information corresponding to GET 125. Image analyzer 170 can use the output generated in either form to build a model from... Figure 8 The confidence level of the edges determined by the process described in the text.

[0063] Image data flow diagram 800 includes a first region of interest 550 and a second region of interest 560. Both the first region of interest 550 and the second region of interest 560 may have already been generated by the image analyzer 170, as described above. Figure 5 or Figure 6 Specifically, the image analyzer 170 may have generated a first region of interest 550 and a second region of interest using computer vision or deep learning techniques. In some embodiments, the first region of interest 550 and the second region of interest 560 may have been generated using a combination of computer vision and deep learning techniques.

[0064] Image data flow diagram 800 also includes a left edge digital image 810 and a right edge digital image 820. Image analyzer 170 can generate the left edge digital image 810 and the right edge digital image 820 using deep learning techniques. For example, image analyzer 170 can generate the left edge digital image 810 and the right edge digital image 820 using a deep learning GET-localization algorithm. The deep learning GET-localization algorithm can employ a neural network that has been trained using an image corpus that identifies where edges of GET 125 groups have been labeled and identified for training purposes. In some embodiments, image analyzer 170 determines the location of edges corresponding to GET 125. Alternatively or additionally, image analyzer 170 determines the location of individual GET 125s.

[0065] In some embodiments, after the image analyzer 170 generates the left edge digital image 810 and the right edge digital image 820, the image analyzer can use the position relative to either or both edges of the region of interest to establish confidence in the edges created using computer vision techniques (such as those described above with respect to the image data flow diagram). In some instances, the image analyzer 170 may further refine its edge-image created using computer vision techniques with the output of a deep learning GET-localization algorithm, or, in cases where the output of the GET-localization algorithm differs significantly from the edge-image generated by conventional computer vision techniques, the image used for wear detection processing may be completely ignored.

[0066] In some embodiments, image analyzer 170 creates sparse stereo disparity 760 by calculating sparse stereo disparity between the left edge digital image 810 and the right edge digital image 820. Image analyzer 170 may then provide sparse stereo disparity 760 to wear analyzer 175 to detect wear and tear on GET 125 according to the disclosed embodiments.

[0067] Figure 9A flowchart illustrating an example computer vision wear detection process 900 using computer vision technology to detect wear on a GET 125 is shown. In some embodiments, process 900 may be performed by an image analyzer 170 and a wear analyzer 175. Process 900 generally follows... Figure 5 and 7 The image data flow is described below and should be interpreted in accordance with the description in these figures. Although the following discussion describes aspects of process 900 performed by image analyzer 170 or wear analyzer 175, other components of the wear detection computer system 110 may perform one or more blocks of process 900 without departing from the spirit and scope of this disclosure.

[0068] Process 900 begins at block 910, where image analyzer 170 receives left and right images of bucket 120 and GET 125. The left image may be captured by the left image sensor of camera 128, and the right image may be captured by the right image sensor of camera 128. Both the left and right images can be corrected images. At block 920, the image analyzer can generate a dense stereo disparity map based on the disparity between pixels in the left and right images. At block 925, using the generated dense stereo disparity map and a set of physical parameters associated with bucket 120 and GET 125, image analyzer 170 can generate a 3D point cloud representing the 3D images of bucket 120 and GET 125. At block 940, from the 3D point cloud, image analyzer 170 can identify a first region of interest associated with the left image and a second region of interest associated with the right image. As described above, this can be achieved by performing segmentation analysis on the 3D point cloud using information from the set of physical parameters associated with bucket 120 and GET 125. Both the first and second regions of interest include images of GET 125.

[0069] At block 950, image analyzer 170 can generate a left edge digital image associated with a first region of interest and a right edge digital image associated with a second region of interest. The left and right edge digital images can be generated using computer vision edge detection techniques according to the disclosed embodiments. At block 955, image analyzer 170 can refine the left and right edge digital images using dynamic programming techniques and a set of physical parameters associated with bucket 120 and GET 125. Then, at block 970, image analyzer 170 can determine sparse stereo parallax based on the left and right edge digital images. At block 980, wear analyzer 175 can use the techniques described above regarding... Figure 1 The described technique determines the wear level or depletion of one or more GET 125s.

[0070] Figure 10A flowchart illustrating an example deep learning wear detection process 1000 for detecting wear on a GET 125 using deep learning or machine learning techniques is shown. In some embodiments, process 1000 may be performed by an image analyzer 170 and a wear analyzer 175. Process 1000 generally follows the above description... Figure 6 and Figure 8 The process described herein should be interpreted in accordance with the description of these figures. Although the following discussion describes aspects of process 1000 performed by image analyzer 170 or wear analyzer 175, other components of the wear detection computer system 110 may perform one or more blocks of process 1000 without departing from the spirit and scope of this disclosure.

[0071] At block 1010, image analyzer 170 receives stereo images of bucket 120 and GET 125. This stereo image may include a left image captured by the left image sensor of camera 128 and a right image captured by the right image sensor of camera 128. The stereo image may also include a color image captured by the color image sensor of camera 128. At block 1030, image analyzer 170 may use a deep learning GET detection algorithm on the left image, right image, and color image to identify a first region of interest corresponding to the left image and a second region of interest corresponding to the right image. In some embodiments, the use of a color image may improve the performance of the deep learning GET detection algorithm.

[0072] At block 1060, image analyzer 170 uses a deep learning GET-localization algorithm to locate a GET within a first region of interest (ROI) or a second ROI. The deep learning GET-localization algorithm may include a neural network trained using an image corpus in which GETs have been labeled within the images. Labels may include labels for a single GET or a group of GETs. In some embodiments, the deep learning GET-localization algorithm analyzes the first ROI (corresponding to the left image) or the second ROI (corresponding to the right image) by applying a digitized ROI as input to the neural network, and the output of the neural network may be the pixel location of the GET within the ROI or a bounding box relating the GET's location within the ROI. In some embodiments, the deep learning GET-localization algorithm may detect the location of the GET within both the first and second ROIs, while in other embodiments, it may use either the first or second ROI. Whether the deep learning GET-localization algorithm detects the location of the GET within the first ROI, the second ROI, or both may vary depending on the implementation and may vary depending on the type of GET, the type of bucket, or the type of machine being worked.

[0073] In some embodiments, at block 1060, the deep learning GET-localization algorithm outputs edge images corresponding to the GET. The edge images (e.g., left edge digit image 810, right edge digit image 820) may correspond to the contours of the GET within a first region of interest (ROI) or a second region of interest (ROI). In such embodiments, the deep learning GET-localization algorithm may include a neural network trained using an image corpus comprising the GET or a GET attached to a bucket, wherein the edges of the GET have been labeled. The deep learning GET-localization algorithm analyzes the first ROI (corresponding to the left image) or the second ROI (corresponding to the right image) by applying the digitized ROI as input to the neural network, and the output of the neural network will include a left edge digit image corresponding to the first ROI and a right edge digit image corresponding to the second ROI.

[0074] At block 1070, image analyzer 170 can determine sparse stereo disparity. In an embodiment where the deep learning GET-localization algorithm outputs left-edge and right-edge digitized images at block 1060, image analyzer 170 can determine sparse stereo disparity between the left-edge and right-edge digitized images. In an embodiment where the deep learning GET-localization algorithm outputs the location of the GET within a first region of interest, a second region of interest, or both, image analyzer 170 can use computer vision techniques (e.g., as described above with respect to block 970 of process 900) to determine sparse stereo disparity, and can use the output of the deep learning GET-localization algorithm to establish confidence or verify the accuracy of the left-edge and right-edge digitized images generated by computer vision techniques.

[0075] At block 1080, wear analyzer 175 can use the above-mentioned... Figure 1 The described technique determines the wear level or depletion of one or more GET 125s.

[0076] Figure 11 A flowchart illustrating an example hybrid wear detection process 1100 employing a combination of deep learning or machine learning techniques to detect wear on a GET 125 is shown. In some embodiments, process 1100 may be performed by an image analyzer 170 and a wear analyzer 175. Although the following discussion describes aspects of process 1100 performed by image analyzer 170 and wear analyzer 175, other components of the wear detection computer system 110 may perform one or more blocks of process 1100 without departing from the spirit and scope of this disclosure.

[0077] According to some embodiments, certain blocks of process 1100 are executed in parallel and in combination to improve performance in identifying regions of interest and determining sparse stereo parallax. Many blocks shown in process 1100 are similar to those in processes 900 and 1000 described above, and blocks performing similar operations have the same last two digits. For example, block 1110 of process 1100 is functionally similar to block 910 of process 900 and block 1010 of process 1000.

[0078] At block 1110, image analyzer 170 receives stereo images of bucket 120 and GET 125. Image analyzer 170 then performs a set of operations (blocks 1120, 1125) using computer vision techniques in parallel with a second set of operations (block 1130) using deep learning techniques to identify a first region of interest associated with the left image of the stereo image and a second region of interest associated with the right image of the stereo image. Therefore, at block 1120, image analyzer 170 generates a dense stereo disparity map based on the stereo image, similar to block 920 of process 900. At block 1125, image analyzer 170 generates a 3D point cloud, similar to block 925 of process 900. At block 1130, image analyzer 170 applies a deep learning GET detection algorithm based on the stereo image, similar to block 1030 of process 1000.

[0079] Process 1100 then continues to block 1135, where image analyzer 170 fuses the results from blocks 1120 and 1125 with the results from block 1130. Image analyzer 170 can use dynamic programming techniques to generate an optimized first region of interest corresponding to the left image and an optimized second region of interest corresponding to the right image. For example, image analyzer 170 can utilize a set of physical parameters describing the spatial relationship between bucket 120 and GET 125, combined with 3D point clouds and information output by a deep learning GET detection algorithm, to determine a more accurate extraction of the region of interest. At block 1140, using the fused results, image analyzer 170 can identify the first and second regions of interest.

[0080] Process 1100 then performs sparse stereo disparity generation using both computer vision and deep learning techniques. Therefore, at block 1150, image analyzer 170 generates left-edge and right-edge digital images using computer vision edge detection techniques, similar to what is described above regarding block 950 of process 900. At block 1155, the left-edge and right-edge digital images are refined as described above regarding block 955 of process 900. At block 1160, image analyzer 170 locates the GET within a first or second region of interest using a deep learning GET-localization algorithm, similar to what is described above regarding block 1060 of process 1000.

[0081] At block 1170, image analyzer 170 uses the refined left-edge digital image and refined right-edge digital image determined at block 1155, along with the output of block 1160, to determine sparse stereo parallax 1170 corresponding to the shape and size of GET 125. In some embodiments, when operating at block 1170, image analyzer 170 can use a set of physical parameters associated with bucket 120 and GET 125 to fuse information between computer vision techniques (blocks 1150 and 1155) and deep learning techniques (block 1160). For example, image analyzer 170 can utilize the expected size and shape of the GET, the relative positioning of GET 125 relative to the side of the bucket 120 engaging GET 125, the smoothness or continuity of the refined left-edge digital image and the refined right-edge digital image (from block 1155), the positioning of the GET in the first or second region of interest (from block 1160), and / or the smoothness or continuity of the left-edge digital image and the right-edge digital image (from block 1160) to perform optimization analysis and determine sparse stereo parallax. At block 1180, wear analyzer 175 can use the above-mentioned... Figure 1 The described technique determines the wear level or depletion of one or more GET 125s.

[0082] In some embodiments, process 1100 may flow to various combinations including computer vision techniques or deep learning techniques. For example, process 1100 may follow the process flow of blocks 1110, 1120, 1125, 1140, 1160, 1170, and 1180 (i.e., excluding blocks 1130, 1135, 1150, and 1155). As another example, process 1100 may follow the process flow of blocks 1110, 1130, 1140, 1150, 1155, 1170, and 1180 (i.e., excluding blocks 1120, 1125, 1135, and 1160). Additionally, process 1100 may use computer vision techniques to identify regions of interest (blocks 1120 and 1125) and deep learning techniques to identify left-edge and right-edge digitized images (block 1160). Conversely, process 1100 can use deep learning techniques to identify regions of interest (block 1130) and computer vision techniques (blocks 1150 and 1155) to identify left-edge and right-edge digital images.

[0083] Throughout the foregoing description, certain components of the wear detection computer system 110 are described as performing certain operations. However, in some embodiments of the wear detection computer system 110, other components may perform these operations in addition to those described above. Furthermore, the wear detection computer system 110 may include additional or fewer components than those presented in the example embodiments above. Those skilled in the art will understand that the wear detection computer system 110 is not necessarily limited to the specific embodiments disclosed above.

[0084] Industrial applicability

[0085] The systems and methods described herein can be used in conjunction with the operation of working machines that are excavating, moving, shaping, contouring, and / or removing materials (e.g., soil, rock, minerals, etc.) at a work site. These working machines may be equipped with a bucket for shoveling, digging, or dumping materials at the work site. The bucket may be equipped with a series of ground engagement tools (GET) to assist in loosening materials during operation. The working machine may also include a system with a processor and memory configured to perform wear detection methods according to the examples described herein. The systems and methods can detect wear or damage to working machine components, such as GETs, so that operators of such working machines can take corrective action before potential failures that could damage downstream processing equipment.

[0086] In some instances, the system and method may capture stereoscopic images of machine parts from a stereo camera associated with the machine for wear detection processing. The stereoscopic images may be captured as a left-side image and a right-side image.

[0087] In some instances, the system and method may use computer vision techniques and the creation of a dense stereo disparity map to process the left and right images. The dense stereo disparity map can be segmented to identify regions of interest within the image associated with the GET. The system and method may further use computer vision techniques—such as gradient magnitude edge detection techniques—to process the regions of interest to identify left and right edge digital images of the shape of the contoured GET. The system and method can then use the left and right edge digital images to determine sparse stereo disparity, from which the GET captured in the left and right images can be measured. GET wear or deterioration can be determined based on these measurements.

[0088] Using sparse stereo parallax to determine the size of the GET (Getting Gear) can improve the accuracy of GET wear measurement because it allows wear to be measured with an accuracy of less than 5 mm. Therefore, systems and methods using the above technique have advantages over systems that do not use sparse stereo parallax, which can only detect wear at a particle size on the order of centimeters or tens of centimeters. More accurate wear detection can reduce the possibility of catastrophic GET damage or wear that could cause damage to downstream processing equipment.

[0089] In some instances, the system and method can use deep learning techniques to process the left and right images. The system and method can employ a deep learning GET detection algorithm, which uses a neural network trained to identify regions of interest (ROIs) corresponding to GETs within the left and right images. Once the ROI is detected, sparse stereo parallax can be determined between the left edge digital image corresponding to the left image and the right edge digital image corresponding to the right image. As described above, using sparse stereo parallax improves the accuracy of wear measurement.

[0090] In other instances, the system and method may use both computer vision and deep learning techniques to process the left and right images, and in some cases, apply these techniques in parallel. In such instances, the system and method can improve GET measurement accuracy by fusing the outputs of each technique to find the optimal measurement for GET. By using the fused outputs, GET measurement accuracy can be improved compared to conventional systems that do not employ sparse stereo parallax or do not use both computer vision and machine learning techniques.

[0091] Although various aspects of this disclosure have been specifically shown and described with reference to the foregoing examples, those skilled in the art will understand that various additional embodiments can be contemplated through modifications to the disclosed apparatus, systems, and methods without departing from the spirit and scope of the disclosure. Such embodiments should be understood to fall within the scope of this disclosure as defined by the claims and any equivalents.

Claims

1. A computer-implemented method, comprising: A left image (510) and a right image (520) of the bucket (120) of the working machine (100) are received from a stereo camera (128) associated with the working machine (100), the bucket having at least one ground engagement tool (GET) (125), wherein the left image and the right image are digital; A first region of interest (550) is identified from the left image, the first region of interest corresponding to the at least one GET; A second region of interest (560) is identified from the right image, the second region of interest corresponding to the at least one GET; Computer vision edge detection is used to generate a digital image of the left edge corresponding to the first region of interest (740). Computer vision edge detection is used to generate a digital image of the right edge corresponding to the second region of interest (750); The accuracy of the following items was verified using the deep learning GET localization algorithm: A digital image of the left edge generated using computer vision edge detection; A digital image of the right edge generated using computer vision edge detection; Determine the sparse stereo parallax between the left edge digital image and the right edge digital image (760); The sparse stereo parallax is used to determine measurement data related to the GET; as well as, The determined measurement data is compared with the expected data corresponding to the unworn form of the GET, thereby determining the wear level or loss of the at least one GET based on the sparse stereo parallax.

2. The method of claim 1, wherein identifying the first region of interest from the left image and identifying the second region of interest from the right image comprises applying a deep learning GET detection algorithm to the left image and the right image.

3. The method of claim 1, wherein identifying the first region of interest from the left image and identifying the second region of interest from the right image comprises generating a dense stereo disparity map for the left image and the right image (530).

4. The method of claim 3, wherein identifying the first region of interest from the left image and identifying the second region of interest from the right image further comprises generating a 3D point cloud (540) at least in part based on the dense stereo disparity map.

5. A wear detection system, comprising: Stereo cameras (128); One or more processors (140); as well as A non-transitory computer-readable medium (150) storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including: The stereo camera receives a left image (510) and a right image (530) of the bucket (120) of the working machine (100), the bucket having at least one ground engagement tool (GET) (125), wherein the left image and the right image are digital; A first region of interest (550) is identified from the left image, the first region of interest corresponding to the at least one GET; A second region of interest (560) is identified from the right image, the second region of interest corresponding to the at least one GET; Computer vision edge detection is used to generate a digital image of the left edge corresponding to the first region of interest (740). Computer vision edge detection is used to generate a digital image of the right edge corresponding to the second region of interest (750); The accuracy of the following items was verified using the deep learning GET localization algorithm: A digital image of the left edge generated using computer vision edge detection; A digital image of the right edge generated using computer vision edge detection; Determine the sparse stereo parallax between the left edge digital image and the right edge digital image (530); The sparse stereo parallax is used to determine measurement data related to the GET; as well as, The determined measurement data is compared with the expected data corresponding to the unworn form of the GET, thereby determining the wear level or loss of the at least one GET based on the sparse stereo parallax.

6. The system of claim 5, wherein identifying the first region of interest from the left image and identifying the second region of interest from the right image comprises applying a deep learning GET detection algorithm to the left image and the right image.

7. The system of claim 5, wherein identifying the first region of interest from the left image and identifying the second region of interest from the right image comprises generating a dense stereo disparity map for the left image and the right image (530).

8. The system of claim 7, wherein identifying the first region of interest from the left image and identifying the second region of interest from the right image further comprises generating a 3D point cloud (540) at least in part based on the dense stereo disparity map.

Images