Vehicle dispatch system and method of use
AI-driven video analytics integrated with multi-sensor systems improve truck dispatch accuracy by detecting and classifying truck IDs, addressing FMS errors and enhancing material handling efficiency in mining operations.
Patent Information
- Application Number
- PCT/IB2026/000230
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-05
- Filing Date
- 2026-05-05
- Publication Date
- 2026-11-12
AI Technical Summary
Current truck batching methods in mining operations are prone to errors due to inaccuracies in Fleet Management Systems (FMS), leading to incorrect bucket-to-truck batching, inefficient routing, and reduced profitability.
Integrating AI-driven video analytics with multi-sensor ore sorting systems to enhance truck material grade estimation accuracy by using cameras mounted on mining shovels to detect and classify truck IDs, and integrating this data with FMS for verification and correction.
Reduces misclassified trucks by 10% and improves truck ore grade assignment accuracy by 4.7%, ensuring accurate material handling and optimizing routing decisions.
Smart Images

Figure 00000025_0000 
Figure 00000026_0000 
Figure 00000027_0000
Abstract
Description
[0001] VEHICLE DISPATCH SYSTEM AND METHOD OF USE
[0002] BACKGROUND
[0003] In mining operations, precise truck dispatching is highly desirable for controlling ore processing and waste management. Current truck batching methods rely on data from Fleet Management Systems (FMS), which often can be prone to errors. These errors lead to incorrect bucket-to-truck batching resulting in inaccurate truck level composition estimates, inefficient routing, and a reduced profitability. The applicant recognizes the need for further improvement in this area and the present application addresses these issues by integrating Al-driven video analytics with multi-sensor ore sorting systems, enhancing truck material grade estimation accuracy and overall system resilience. More specifically, the present application is a novel approach to truck identification in mining, addressing the limitations of traditional FMS-reliant systems.
[0004] 1
[0005] 200284777vlSUMMARY
[0006] In one form the present application contemplates a method for matching a mining truck to shovel loads of ore material from a mining shovel to ensure the material composition data of the shovel loads is assigned to the truck, the method comprising: sensing the ore material in the shovel loads to gather composition data about the ore material; detecting a truck identifier on the truck as the mining shovel loads the shovel load of ore material into the truck; capturing the truck identifier during the loading of the shovel load of ore material into the truck; and, matching the sensed composition data with the captured truck identifier.
[0007] Another form of the present application contemplates a method for matching a mining truck to at least one shovel load of ore material to ensure the material composition data of the shovel load is assigned to the identified mining truck, the method comprising: operating a mining shovel to obtain the at least one shovel load of ore material; sensing the ore material in the shovel load to gather composition data about the ore material; video recording the mining shovel loading the ore material in the mining truck; detecting a truck identifier data set of the mining truck on at least one frame in the video recording; processing the truck identifier data set with at least one of a CNN based digit classification model or a machine vision model to classify or interpret the detected data; and, matching the sensed composition data with the captured truck identifier data set.
[0008] 2
[0009] 200284777vlAnother form of the present application contemplates a system for matching a mining truck to at least one shovel load of ore material to ensure the material composition data of the shovel load is assigned to the identified mining truck, comprising: a mining shovel having a bucket for holding ore; at least one compositional sensor proximate the bucket for gathering compositional data about the ore; a mining truck configured to receive a load of ore form the mining shovel, the mining shovel including a first region; a truck ID located at the first region and being used to identify a discrete load of ore; a camera focused on the first region to detect and record images of the truck ID when the load of ore passes to the mining truck; and a data system to integrate the compositional ore data and the truck ID data.
[0010] Advantages and benefits of the present application should be apparent for the present application.
[0011] 3
[0012] 200284777vlBRIEF DESCRIPTION OF THE FIGURES FIG. 1 depicts a schematic view of an truck tracking system of the present application.
[0013] FIG. 2 depicts an illustrative view of a truck tracking system of the present application. .
[0014] FIG. 3 depicts an embodiment of a computer vision of a truck tracking system of the present application. .
[0015] 200284777vlDETAILED DESCRIPTION OF THE ILLUSTRATIVE EMBODIMENTS For the purposes of promoting an understanding of the principles of the invention, reference will now be made to the embodiments illustrated in the drawings and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended. Any alterations and further modifications in the described embodiments, and any further applications of the principles of the invention as described herein are contemplated as would normally occur to one skilled in the art to which the invention relates.
[0016] With reference to the figures, there is disclosed embodiments of a truck tracker systemWO designed to improve truck dispatch accuracy in mining operations. While the system 100 and methods of use are discussed relative to mining trucks 50 other embodiments contemplate a variety of vehicles. It enhances ore sorting and material handling efficiency by integrating camera data with a multi-sensor ore sorting system. In one embodiment the system detects truck IDs 51 using video footage from one or more cameras 52 mounted on mining shovels 70 or disposed proximate and applicable viewing area 53 of the trucks, ensuring proper bucket-to-truck batching and minimizing dispatch errors. The mining shovel 70 includes a moveable bucket 71 for grasping loads of ore from the mine environment, Through high-precision object detection, digit recognition, and modeling recognizing and interprets each digit. The present application significantly improves operational efficiency and reduces ore misclassification. In one study of a particular embodiment of the present
[0017] 5
[0018] 200284777vlapplication of the truck tracker system it was found to reduce unclassified trucks by 10% and improves the accuracy of truck ore grade assignments for 4.7% of total hauls.
[0019] In one aspect of the present application the system 100 provides more accurate assignment of real-time data collected regarding the relationship of ore in mining shovels 70 to mining trucks 50, reducing mis-aggregations and optimizing routing decisions. In one embodiment the present application discloses embodiments that integrate computer vision systems to identify faults in fleet management tracking, enhances the accuracy of actions based on sensor-based ore classification technologies. As a result of more accurate batching real-time material grade composition data to the appropriate mining trucks 50 , the system 100 addresses a need to ensure that material is properly handled, thereby improving overall operational efficiency.
[0020] In one embodiment of the present application, the system 100 consists of one or more cameras 52 mounted on mining shovel 70 (the cameras may be mounted at other locations in other embodiments), working in conjunction with material composition sensors 80 located in the bucket 71. In one preferred embodiment the present application uses the types of sensors disclosed (ShovelSense sensors) in MineSenses’s intellectual property publications that provide sensors to obtain material composition data. United States Patent No.
[0021] 8,958,905and 10,036,142 are incorporated herein by reference to provide information on MineSense’s product called ShovelSense. These cameras 52 detect truck IDs 51 so the system 100 can match individual mining trucks 50 to
[0022] 6
[0023] 200284777vltheir corresponding shovel loads 81 to ensure that the correct material composition data is assigned to specific mining trucks 50. In one form includes Al-powered computer vision models, trained for truck detection, ID recognition, and digit classification, process this information and are applicable to crossvalidate it against fleet management system (FMS) data. This integrated approach ensures that mining trucks 50 are accurately tracked with the appropriate ore, mitigating errors caused by FMS misreporting and improving overall material handling efficiency.
[0024] In one embodiment the system 100 operates through a structured Al pipeline that includes object detection, digit localization, classification, and postprocessing. The model assembly phase combines convolutional neural networks (CNNs) and OCR techniques to extract and validate truck numbers from video footage. By continuously comparing detected truck IDs with FMS records, the system 100 can identify inconsistencies and provide corrections.
[0025] The present application represents a significant advancement in mining operations, offering a robust and scalable solution for improving ore tracking in specific mining trucks 50, reducing misclassified material movements, and increasing operational efficiency through the integration of Al-driven computer vision with multi-sensor sorting 80 technologies. An exemplary diagram of overall workflow of a system 100 is set forth in FIG. 2.
[0026] One form of the Camera 52 for truck ID data capture utilizes a standard RGB camera, The camera or plurality of cameras are mounted on the mining shovel 70. The at least one camera is mounted to provide a field of view of one
[0027] 7
[0028] 200284777vlor more mining trucks 50as the shovel is dumping with the ability to capture realtime video footage. The camera is oriented to place the field of view so that the truck IDs are disposed within the field of view while the mining shovel 70 is in operation. The preferred placement of the camera allows the camera to capture the truck ID 51 when the shovel bucket 71 turns to dump ore material 81 into the mining truck 50. The present application has found that even if the truck ID 51 is visible for only a brief period, the 100 system processes multiple framesof the camera 52 video to extract reliable information.
[0029] In one embodiment the system 100 includes an Al pipeline that consists of multiple deep learning models to detect, classify, and extract relevant information. The Al pipeline includes an Object Detection (Truck and ID Localization) portion that in its first layer of object detection identifies the mining truck 51 itself within the video frame. Once the mining truck 51 is detected, the system 100 then searches for the mining truck ID region 53 on the detected mining truck 50. This embodiment performs object detection in both steps using a YOLOv8-based Object Detection Model. This YOLOv8 based Object Detection Model is just one example of an applicable mode, but the present application contemplates any appropriate alternative object detection model. Aspects of the present embodiment contemplate that the model is fine-tuned on a custom-labeled dataset specific to the mine operation, and the model is capable of detecting mining trucks 50 of different types and identifying the area 53 where the truck ID 51 is located. The present application contemplates alternative
[0030] 8
[0031] 200284777vlembodiments that directly detect the ID’s 51 on the mining truck 50 without having an intermediate step for detecting a truck.
[0032] An embodiment of the system 100 then utilizes a Digit Localization operation and the mining truck 50 detected by the previous act is used as the input in this act. An algorithm detects the location of the tracking ID 51 or elements of the ID on the mining truck 50. One embodiment of the present application uses an object detection model to detect individual digits within the truck ID. This embodiment utilizes YOLOv8 model, but any alternative object detection model with sufficient performance characteristics can be used instead. The present embodiment is designed to be flexible and is not constrained by a fixed number of digits, allowing it to detect IDs of varying lengths. The present application contemplates other embodiments that can detect non-numeric embodiments. For example, QR codes, alphanumeric codes, or any distinct symbols can be used to identify mining trucks 50. Alternative embodiments, such as QR codes, do not necessarily require a machine learning model.
[0033] After detecting the location of each digit within the ID region 53, a Convolution Neurol Network (CNN) -based digit classification model recognizes and interprets each digit. The present application contemplates the application of other machine vision models to classify or interpret digits. The recognized digits are ordered based on their positions to reconstruct the full truck ID 51. The CNN-based digit classifier is specifically trained to handle challenging conditions, such as dust, poor lighting, and motion blur. The present application
[0034] 9
[0035] 200284777vlcontemplates alternative embodiments that include other algorithms that can be used to interpret unique identifiers51 for a mining truck 50. These algorithms are not limited to machine learning algorithms. For example, QR or barcode detection algorithms can be applied when such truck ID’s 51 are present on a mining truck 50.
[0036] In form of the present application the system 100 includes a timestamp extraction feature. A CNN-based timestamp detection model extracts date and time information from the video frames. This timestamp extraction feature can be used to compare the start time to the finish time and compare to the FMS data to ascertain where the truck is located. In an alternate embodiment, if the system the NVR has reliable timestamps, the system 100 can integrate real-time date-time metadata instead of extracting timestamps from the video frames.
[0037] The present application contemplates embodiments having additional features and / or techniques. One embodiment contemplates the post-processing & data Aggregation for enhanced accuracy. The detected truck IDs 51 are aggregated over multiple video frames to improve accuracy. The system 100 applies filters and data validation techniques to remove false detections and improve reliability. The extracted truck IDs 51 are then linked to shovel operation data (the data from the sensors 80 in the bucket 71 for one example ) at the run level, ensuring consistency between truck ID 51 readings and material loading events from the bucket 71 to the mine truck 50..
[0038] 10
[0039] 200284777vlAnother form of the present application contemplates embodiments that provide for integration with a fleet management system (FMS). These alternate embodiments can integrate FMS-reported truck IDs with detected IDs for verification and correction via an API. If discrepancies between the FMS truck ID and the computer vision-detected ID exist, the system can flag potential errors for review; a correction or change to dispatch. In an alternate form the system shares information with end-users via a web portal, through push notifications, or any other method of directly contacting a client.
[0040] The present application contemplates embodiments that provide for MultiSensor Data Fusion. The detected truck IDs can be integrated with any real-time data related to material collected by a mining shovel or mining shovel operation, including but not limited to: ore-sorting results; particle size analysis; truck profile detection; operational data; geological data generated via assays of blastholes; geostatistical block model data. The embodiments including multi-sensor data fusion provide for a more accurate assignment of ore material to the correct mining truck 50, reducing misclassification errors in material handling.
[0041] A baseline system 100 of the present application includes: a single camera 52 mounted on a mining shovel 70; a YOLOv8-based (or any object detection model) truck and ID detection model running on a processing device; a CNNbased digit classifier (or any object or digit detection model) for recognizing truck IDs; a basic data aggregation for improving ID detection accuracy across multiple video frames; and an output of detected truck ID stream with timestamps.
[0042] 11
[0043] 200284777vlAdditional features that are included with the baseline system includes: FMS truck ID validation to compare detected truck IDs with dispatch-FMS reported truck IDs; an automated correction mechanisms to adjust FMS errors; shovel run-level data aggregation for better assignment of truck IDs 51 to specific loads.
[0044] A preferred embodiment includes some or all of the above baseline system plus the additional features plus the below additions to define a Multi-Sensor Integration Embodiment This preferred embodiment includes: multi-camera setups for increased ID detection reliability; integration with other mining data, such as ore-sorting information and particle size data, to enhance material handling; and Al-driven anomaly detection, which flags inconsistencies in truck ID 51 reporting, improving operational efficiency.
[0045] The camera system 52 component of the present application is responsible for identifying trucks ID’s. The preferred choice is an RGB industrial camera, specifically designed to withstand harsh conditions, including dust, vibration, and extreme lighting variations. In a preferred form the camera has a self-cleaning system suitable for mining environments. However, alternative cameras, such as 3D depth cameras, or any other imaging cameras, could be used depending on specific environmental challenges.
[0046] The cameras 52 are typically mounted on mining shovels 70, positioned to capture truck ID 51 (or plates) during the loading process. Alternative mounting locations include fixed infrastructure points, on different shovel points, or fixed locations on the truck road, such as poles at truck loading road, which could 12
[0047] 200284777vlprovide a more stable imaging setup. Construction-wise, the cameras 52 must be shock-resistant, waterproof, and capable of operating in extreme temperatures. Advanced features such as auto-focus, motion compensation, and Al-powered image stabilization could enhance ID detection accuracy, reducing the impact of external disturbances. Ultimately, selecting the right camera system influences the quality of image input, which directly affects the overall detection and classification accuracy of the system.
[0048] The Al-based processing pipeline forms a portion of the present application, enabling automated truck ID recognition. The pipeline integrates multiple deep learning models, including object detection models, a digit identification algorithm, and in some embodiments a timestamp detection model to synchronize images with truck loading events. In one embodiment the object detection is performed using YOLOv8 models and digit identification is performed using CNN-based classification models. Alternative Al architectures can be considered in alternate embodiments for all steps in the machine learning pipeline. For object detection, Faster R-CNN, SSD (Single Shot Multibox Detector), or even transformer-based models like DETR are contemplated. The OCR component, which is used for digit classification, could use alternatives such as PaddleOCR, Google Vision API, or self-supervised learning techniques that require fewer labeled samples for adaptation to new environments. If alternative ID’s, such as QR codes, are used by a mine then standards algorithms appropriate for those ID’s can be used instead.
[0049] 13
[0050] 200284777vlAnother embodiment contemplates the use of ensemble learning, where multiple Al models vote on the final prediction to increase robustness. The impact of the Al pipeline depends not only on model accuracy but also on efficient data handling, real-time synchronization with the Fleet Management System (FMS), and the ability to adapt to dynamic site conditions, such as dust accumulation on truck ID plates or varying truck angles relative to the camera.
[0051] In low-light conditions, external lighting or special camera imaging could be integrated, allowing better truck detection. Given that the system 100 generally operates in a mining environment, the detection model must be robust to environmental variations, including fluctuating lighting, moving backgrounds, and dirt accumulation on truck surfaces. Motion tracking algorithms could be incorporated to track mining trucks 50 across multiple video frames, reducing false detections. In terms of hardware, the object detection system requires GPU acceleration for training but can run efficiently on CPU once deployed.
[0052] Improvements in detection reliability will directly enhance downstream digit recognition, ensuring that truck IDs are accurately extracted and processed in subsequent stages.
[0053] After identifying the truck ID location 53, the digit localization and recognition process extracts and deciphers the numerical sequence. One embodiment employs YOLOv8 for digit localization followed by a CNN-based classifier (or PaddleOCR) to recognize individual digits. However, variations and enhancements in alternate embodiments are contemplated to address
[0054] 14
[0055] 200284777vlrobustness under different conditions. Instead of a two-stage pipeline, an end-to-end OCR-based approach like PaddleOCR could directly extract and recognize truck IDs 51 in a single step, reducing computational overhead. Attention-based sequence models, such as Transformer OCR networks, could further enhance accuracy by focusing on the IDs while ignoring distortions or occlusions. Hybrid CNN-RNN architectures are another alternative, particularly useful for recognizing sequences of digits when individual digit separation is challenging. To increase robustness, an ensemble approach combining multiple OCR models (including CNN) could be implemented, ensuring better recognition rates by leveraging different perspectives on the same input. One of the challenges in digit recognition is dealing with partially visible, blurry, or dirt-covered digits. Implementing self-supervised learning techniques could enable the model to adapt overtime, improving performance even with limited labeled training data. Additionally, real-time confidence scoring and error detection mechanisms could be introduced, flagging uncertain predictions for manual review or reprocessing. The overall performance of the digit recognition module directly impacts the final truck ID reporting accuracy, making it a critical component of the system 100.
[0056] A later act in the TruckTracker pipeline is integrating the detected truck IDs 51 with the Fleet Management System (FMS), which logs mining truck 50 activities and dispatch information. The system 100 must ensure that detected truck IDs 51 match FMS-reported truck numbers, enabling real-time validation of operational records. An embodiment having a Al-driven anomaly detection, which would flag discrepancies between detected IDs and FMS records, alerting
[0057] 15
[0058] 200284777vloperators to potential dispatching errors or data entry mistakes. In addition to the integration with FMS, incorporating data from a MultiSensor System 80 can offer several advantages, further enhancing the overall functionality of the system. The present application contemplates multiple ways to add value in the integration including: automated Dispatching and Routing; Al-driven Anomaly Detection; real-time Operational Insights; and predictive Maintenance.
[0059] Aspects of the present application include a system that is specifically trained for mining conditions, ensuring higher accuracy despite challenging visual environments. The system provides an independent verification layer for FMS data, reducing reliance on potentially faulty tracking systems. Further, the present application integrates seamlessly with existing ore-sorting and sensor technologies, providing a unified and automated solution for truck batching and material classification. The present application is generally hardware-agnostic, requiring only a robust camera installation, making it more cost-effective than RFID or GPS-based solutions. In the embodiment where the TruckTracker is focused on the shovel, it can link individual scoops to specific trucks with only one hardware component with high accuracy and can identify when such a match has been missed.
[0060] While the invention has been illustrated and described in detail in the drawings and foregoing description, the same is to be considered as illustrative and not restrictive in character, it being understood that only the preferred embodiments have been shown and described and that all changes and modifications that come within the spirit of the inventions are desired to be
[0061] 16
[0062] 200284777vlprotected. It should be understood that while the use of words such as preferable, preferably, preferred or more preferred utilized in the description above indicate that the feature so described may be more desirable, it nonetheless may not be necessary and embodiments lacking the same may be contemplated as within the scope of the invention, the scope being defined by the claims that follow. In reading the claims, it is intended that when words such as “a,” “an,” “at least one,” or “at least one portion” are used there is no intention to limit the claim to only one item unless specifically stated to the contrary in the claim. When the language “at least a portion” and / or “a portion” is used the item can include a portion and / or the entire item unless specifically stated to the contrary.
[0063] 17
[0064] 200284777vl
Claims
CLAIMSWHAT IS CLAIMED IS:
1. A method for matching a mining truck to shovel loads of ore material from a mining shovel to ensure the material composition data of the shovel loads is assigned to the truck, comprising:sensing the ore material in the shovel loads to gather composition data about the ore material;detecting a truck identifier on the truck as the mining shovel loads the shovel load of ore material into the truck;capturing the truck identifier during the loading of the shovel load of ore material into the truck; andmatching the sensed composition data with the captured truck identifier.
2. The method of claim 1 , which further comprises validating the matched sensed composition data and captured truck identifier with a fleet management system data.
3. The method of claim 2, wherein said validating continuously compares the captured truck identifier with the fleet management system data.18200284777vl4. The method of claim 2, which further comprises flagging discrepancies between the matched sensed composition data and captured truck identifier and the fleet management system data.
5. The method of claim 1 , wherein said matching integrates the sensed composition data with the captured truck identifier; andfurther comprises batching real time data to the mining truck of the sensed composition data with the captured truck identifier.
6. The method of claim 1 , which further comprises taking video footage from a camera of the truck;wherein said detecting involves searching for a truck identifier region within the truck.
7. The method of claim 6, wherein the detecting includes identifying the truck within video footage and identifying the area where truck identifier region of the truck is located.
8. The method of claim 6, which further comprises detecting individual digits, QR codes, alphanumeric codes or any distinct symbols for a truck identifier; and processing the detected data with one of a CNN based digit classification model or a machine vision model to classify or interpret the detected data.19200284777vl9. The method of claim 8, wherein the detected individual digits are ordered based upon their positions to provide the captured truck identifier.
10. The method of claim 8, wherein the CNN based digit classification model or a machine based vision model have been trained to address challenging conditions.
11. The method of claim 10, wherein the challenging conditions include at least one of dust, poor lighting or motion blur.
12. The method of claim 8, wherein the model extracts time information from the video frames within the video footage.
13. The method of claim 1 , wherein said sensed composition data includes at least one of particle size detection and ore sorting information.
14. The method of claim 1 , which further comprises taking video footage from a camera of the truck;wherein said matching comprises automated truck identifier recognition; and20200284777vlthe automated truck identifier recognition integrates a plurality of deep learning models including an object detection model and a symbol identification model.
15. The method of claim 14, wherein said automated truck identifier recognition includes a timestamp detection model to synchronize video images with loading events of the ore material.
16. The method of claim 1 , which further comprises taking video footage from a camera of the truck;wherein said detecting involves searching for a truck identifier region within the truck;which further comprises detecting individual digits, QR codes, alphanumeric codes or any distinct symbols for a truck identifier; and processing the detected data with one of a CNN based digit classification model or a machine vision model to classify or interpret the detected data;, wherein the CNN based digit classification model or a machine based vision model have been trained to address challenging conditions; and wherein the challenging conditions include at least one of dust, poor lighting or motion blur.21200284777vl17. A method for matching a mining truck to at least one shovel load of ore material to ensure the material composition data of the shovel load is assigned to the identified mining truck, comprising:operating a mining shovel to obtain the at least one shovel load of ore material;sensing the ore material in the shovel load to gather composition data about the ore material;video recording the mining shovel loading the ore material in the mining truck;detecting a truck identifier data set of the mining truck on at least one frame in the video recording;processing the truck identifier data set with at least one of a CNN based digit classification model or a machine vision model to classify or interpret the detected data;matching the sensed composition data with the captured truck identifier data set.
18. The method of claim 17, which further comprises validating the matched sensed composition data and captured truck identifier data set with a fleet management system data.
19. The method of claim 18, wherein said validating continuously compares the captured truck identifier data set with the fleet management system data.22200284777vl20. The method of claim of claim 18, which further comprises flagging discrepancies between the matched sensed composition data and captured truck identifier data set and the fleet management system data.
21. The method of claim 17, wherein said matching integrates the sensed composition data with the captured truck identifier; andfurther comprises batching real time data to the mining truck of the sensed composition data with the captured truck identifier data set.
22. The method of claim 17, wherein said sensed composition data includes at least one of particle size detection and ore sorting information.
23. The method of claim 17, wherein said processing includes a plurality of deep learning models including an object detection model and a symbol identification model.
24. A system for matching a mining truck to at least one shovel load of ore material to ensure the material composition data of the shovel load is assigned to the identified mining truck, comprising:a mining shovel having a bucket for holding ore;at least one compositional sensor proximate the bucket for gathering compositional data about the ore;23200284777vla mining truck configured to receive a load of ore form the mining shovel, the mining shovel including a first region;a truck ID located at the first region and being used to identify a discrete load of ore;a camera focused on the first region to detect and record images of the truck ID when the load of ore passes to the mining truck; anda data system to integrate the compositional ore data and the truck ID data.24200284777vl
