System and method for real-time material stripback deduction in job cycles

Abstract

This invention relates to a system and method for real-time material repatriation deduction in loading and unloading operation cycles. A system and method are provided for real-time deduction of material repatriated from a loading container of a transport vehicle, wherein material is loaded into the loading container by operating machinery at a first station and unloaded from the loading container by the transport vehicle at a second station. In a first operation state (e.g., loading), a first sensor (e.g., a camera associated with the operating machinery) provides first data corresponding to the volume of material loaded in the loading container. In a second operation state (e.g., unloading), a second sensor (e.g., a camera or payload measurement unit associated with the transport vehicle) provides second data corresponding to the volume of material loaded in the loading container. The generated output signal corresponds to a calculated total volume of material associated with the operation cycle, said total volume being based at least on the provided first data and the provided second data.

Description

Technical Field

[0001] This disclosure generally relates to work cycles including work machinery for loading materials and transport vehicles for carrying and unloading loaded materials, and more specifically to systems and methods for real-time deduction of loaded materials. Background Technology

[0002] The working machinery discussed in this article may specifically refer to excavators for illustrative purposes, but may also include, for example, various other self-propelled or other machinery and equipment that alter the terrain or equivalent working environment in some way and are further responsible for loading materials from adjacent terrain into transport vehicles for delivery to individual dumping sites. Tracked or wheeled ground engagement units support a chassis from the ground, and the chassis may typically further support one or more working attachments (or alternatively, working implements) for digging or otherwise extracting materials from the terrain and selectively discharging the materials into loading areas associated with transport vehicles, such as the containers of articulated dump trucks.

[0003] As used herein, the term "bring-back material" refers to material that is undesirably retained in the loading container of the transport vehicle after the unloading process. This can occur for a number of reasons, including, for example, wet conditions, the inherent properties of the transported material, the construction of the loading container, the slope of the unloading site, etc. The presence of material bring-back is undesirable, at least because it increases inefficiency in the work cycle and because it increases the uncertainty in estimating the volume of material loaded and transported during the work cycle. While payload weight can be measured more accurately for each load, volume is more important to the end user, and conventional methods for estimating volume from measured payload weight are relatively inaccurate. Therefore, it is often necessary to wait until all the material has been dispersed at the destination before measuring with a target.

[0004] Therefore, it is desirable to provide an easy and efficient way to measure and accordingly subtract the amount of material brought back in a given load. Unfortunately, when a loading container is filled with material, cameras or equivalent imaging devices on transport vehicles such as articulated dump trucks cannot scan or otherwise measure the contents of the loading container. Furthermore, in many applications, cameras or equivalent imaging devices mounted on operating machinery such as excavators will not be able to scan or otherwise measure the material at the bottom of the loading container. Summary of the Invention

[0005] This disclosure provides an enhancement to conventional systems, at least in part, by introducing a novel method for utilizing multiple image data sources mounted on multiple machines or vehicles in a work cycle and coordinating data processing for return deduction.

[0006] In one embodiment, a method for real-time deduction of material brought back from a loading container of a transport vehicle, as disclosed herein, is provided, wherein the material is loaded into the loading container by a working machine at a first station and dumped from the loading container by the transport vehicle at a second station. A first sensor is associated with either the working machine or the transport vehicle and provides first data corresponding to the volume of material loaded in the loading container in a first operating state. A second sensor is associated with the other of the working machine or the transport vehicle and provides second data corresponding to the volume of material loaded in the loading container in a second operating state. An output signal corresponding to a calculated total volume of material associated with the operating cycle may be further generated, wherein the total volume is calculated at least in part based on the provided first data and the provided second data.

[0007] In one exemplary aspect of the above embodiments, the generated output signal is provided to fill the data structure with the calculated total volume of material associated with at least one of the first and second stations.

[0008] In another exemplary aspect of the above embodiments, the first volume may be determined at least in part based on the provided first data and information about one or more dimensions of the loading container, the second volume may be determined at least in part based on the provided second data and information about one or more dimensions of the loading container, and the total volume of material associated with the work cycle may be calculated based on the difference between the determined first volume and the determined second volume.

[0009] In another exemplary aspect of the above embodiments, information about one or more dimensions of the loading container may be stored in association with a transport vehicle, wherein a first sensor is associated with the operating machinery, first data is transmitted from the operating machinery to the transport vehicle, and a first volume is calculated based on the transmitted first data and the stored information about one or more dimensions of the loading container.

[0010] In another exemplary aspect of the above embodiments, the first sensor may include a first image data source configured to generate a signal of a first contour of material loaded in a loading container in a first operating state, and the second sensor may include a second image data source configured to generate a signal corresponding to a second contour of material loaded in a loading container in a second operating state, and the total volume is calculated at least in part based on the determined first contour and the determined second contour of the loaded material.

[0011] In another exemplary aspect of the above embodiments, information about one or more dimensions of the loading container is obtained via scanned images from a first image data source and / or a second image data source.

[0012] In another exemplary aspect of the above embodiments, information about one or more dimensions of the loading container is retrieved from a data storage device based on a scanned image including an identifier associated with the transport vehicle.

[0013] In another exemplary aspect of the above embodiments, the information regarding one or more dimensions of the loading container is retrieved from a data storage device based on communication between the operating machinery and the transport vehicle, including an identifier associated with the transport vehicle.

[0014] In another exemplary aspect of the above embodiments, the first sensor may include an image data source associated with the operating machinery and configured to generate a signal corresponding to the outline of material loaded in a loading container in a first operating state, and the second sensor may include a payload measurement unit associated with a transport vehicle.

[0015] In another exemplary aspect of the above embodiments, the material density of the material loaded in the transport vehicle can be determined based on inputs from a first sensor and inputs from a payload measurement unit in the first operating state. The volume of the remaining material in the transport vehicle in the second operating state can be determined based on inputs from the payload measurement unit in the second operating state and further considering the determined material density. Information regarding one or more dimensions of the loading container can be obtained via scanned images from an image data source. Information regarding one or more dimensions of the loading container can be retrieved from a data storage device based on scanned images including identifiers associated with the transport vehicle and / or based on communication between the operating machinery and the transport vehicle, including identifiers associated with the transport vehicle.

[0016] In another embodiment, this document discloses a system for real-time deduction of material brought back from a loading container of a transport vehicle, wherein the material is loaded into the loading container by a working machine at a first station and dumped from the loading container by the transport vehicle at a second station. The system includes: a first sensor associated with one of the working machine or the transport vehicle and configured to provide first data corresponding to the volume of material loaded in the loading container in a first operating state; and a second sensor associated with the other of the working machine or the transport vehicle and configured to provide second data corresponding to the volume of material loaded in the loading container in a second operating state. The computing device includes a computer-readable medium residing on one of the working machine or the transport vehicle and having program instructions residing on the computer-readable medium, the program instructions being executable by a processor to direct the execution of steps in the method according to the above embodiments and any exemplary aspects optionally associated therewith.

[0017] Many objects, features, and advantages of the embodiments set forth herein will be apparent to those skilled in the art when the following disclosure is read in conjunction with the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a side view showing an exemplary operating machine according to the present disclosure.

[0019] Figure 2 This is a side view showing an exemplary transport vehicle according to this disclosure.

[0020] Figure 3 It shows the location Figure 1 A three-dimensional image of an image data source on a machine in operation, the image data source scanning the machine in the first operating state. Figure 2 The outline of the materials on the transport vehicle.

[0021] Figure 4 It shows the location Figure 2 A stereoscopic view of the image data source on the transport vehicle, which scans the outline of the remaining material in the loading container during the second operating state.

[0022] Figure 5 This is a block diagram illustrating a control system for a work machine according to an embodiment of the present disclosure.

[0023] Figure 6 This is a block diagram illustrating a control system for a transport vehicle according to an embodiment of the present disclosure.

[0024] Figure 7 This is a flowchart illustrating an exemplary method according to an embodiment of the present disclosure. Detailed Implementation

[0025] Now for reference Figures 1 to 7 Various embodiments of the systems and methods of the present invention can now be described.

[0026] Figure 1 A representative work machine 20 is shown, which, in a particular embodiment disclosed herein, is, for example, in the form of a tracked excavator. The work machine 20 includes a chassis 22 having first and second ground engagement units 24 driven by first and second travel motors (not shown), respectively.

[0027] The main frame 32 is supported from the chassis 22 by a slewing bearing 34, allowing the main frame 32 to pivot relative to the chassis 22 about a pivot axis 36. The pivot axis 36 is substantially vertical when the ground 38, engaged by the ground engagement unit 24, is substantially horizontal. A slewing motor (not shown) is configured to pivot the main frame 32 on the slewing bearing 34 relative to the chassis 22 about the pivot axis 36.

[0028] In the context of working machinery 20, working implement 42 includes a boom assembly having: a boom 44; a boom 46 pivotally connected to the boom 44; and a tool 48. The term "implementation" may be used herein to collectively describe the boom assembly (or its equivalents) or to describe individual elements of the boom assembly or its equivalents. The boom 44 is pivotally connected to the main frame 32 to pivot relative to the main frame 32 about a generally horizontal axis. In this embodiment, the tool is an excavator shovel (or bucket) 48 pivotally connected to the boom 46. The boom assembly extends from the main frame 32 along the working direction of the boom assembly. The working direction may also be described as the working direction of the boom 44. As described herein, control of the working implement 42 may involve control of any one or more associated components (e.g., boom 44, boom 46, tool 48).

[0029] Within the scope of this disclosure, the working machinery 20 can take various alternative forms and further utilize alternative working implements 42 to modify the adjacent terrain.

[0030] exist Figure 1In this embodiment, the first and second ground engagement units 24 are tracked ground engagement units, although various alternative embodiments of the work machinery 20 are contemplated, wherein the ground engagement unit 24 may be a wheeled ground engagement unit. Each tracked ground engagement unit 24 includes a guide wheel 52, a drive sprocket 54, and a track chain 56 extending around the guide wheel 52 and the drive sprocket 54. A travel motor of each tracked ground engagement unit 24 drives its respective drive sprocket 54. Each tracked ground engagement unit 24 is shown having a forward travel direction 58 defined from the drive sprocket 54 toward the guide wheel 52. The forward travel direction 58 of the tracked ground engagement unit 24 also defines a forward travel direction 58 of the chassis 22, and thus defines a forward travel direction 58 of the work machinery 20. In some applications (including uphill driving, which will be discussed further below), the orientation of chassis 22 can be reversed, such that the direction of travel of work machinery 20 is limited from guide wheel 52 toward its corresponding drive sprocket 54, while work implement 42 remains in front of chassis 22 in the direction of travel.

[0031] Although the excavator as working machinery 20 is self-propelled according to the above-mentioned components, other forms of working machinery 20 that are not self-propelled can be conceived within the scope of this disclosure, unless otherwise specifically indicated.

[0032] The cab 60 may be located on the main frame 32. Both the cab 60 and the work implement 42 (e.g., boom assembly) may be mounted on the main frame 32 such that the cab 60 faces the working direction 58 of the boom assembly. A control console (not shown) may be located in the cab 60. The control console may include or be associated with a user interface further described below. As used herein, the orientation of the work implement 20 can be referenced from the perspective of the driver seated in the cab 60; the left side of the work implement is the driver's left side, the right side of the work implement is the driver's right side, the front end (or front end) of the work implement is the direction the driver is facing, the rear end (or rear end) of the work implement is behind the driver, the top of the work implement is above the driver, and the bottom of the work implement is below the driver.

[0033] An engine 64, which powers the work machinery 20, is also mounted on the main frame 32. The engine 64 may be a diesel internal combustion engine, but is not limited to this, and within the scope of this disclosure, the work machinery 20 may alternatively be driven by a non-combustion power source (not shown). The engine 64 may drive a hydraulic pump to provide hydraulic power to various operating systems of the work machinery 20.

[0034] Shown as Figure 2The articulated dump truck 10 may include multiple wheels and associated axles, and a frame 12 supporting a loading container 14 (e.g., a cargo box), the loading container having a loading surface at the bottom of an interior area surrounded by sidewalls and a top edge, at least a portion of which may generally be parallel to the ground. A hydraulic piston-cylinder unit 16 may be coupled between the frame 12 and the loading container 14 and configured to selectively extend and raise / pivot the loading container 14 rearward to a tilting position and retract and lower / pivot the loading container forward from the tilting position to a driving and loading position (as shown). The cab 18 of the truck 10 may be located on the frame 12, wherein the orientation of the truck 10 can be referenced from the perspective of a driver seated in the cab 18; the left side of the truck is the driver's left side, the right side of the truck is the driver's right side, the front portion (or front end) of the truck is the direction the driver is facing, the rear portion (or rear end) of the truck is behind the driver, the top of the truck is above the driver, and the bottom of the truck is below the driver.

[0035] The controller 212 for the transport vehicle 10 may, in some embodiments, include a driver interface in the cab 18 or otherwise be associated with a driver interface, as further described below.

[0036] Next reference Figure 3 According to this disclosure, image data source 104 ( Figure 1 (Not shown) can be mounted on the operating machinery 20. The position of the image data source 104 can be selected such that during at least a portion of the material loading operation as a first operating state, the field of view 106 surrounds the loading container 14 of the transport vehicle 10, in which the surface of the loading container retracts to a substantially horizontal orientation as shown, and the position of the image data source 104 can preferably be selected such that the field of view 106 surrounds all four top edges of the loading container 14. Figure 3 As shown, the operating machinery 20 is at the same level relative to the transport vehicle 10; however, it will be understood that in various loading applications, the operating machinery 20 may be in an elevated position relative to the transport vehicle 10 and / or in various corresponding orientations relative to each other. In some embodiments, multiple image data sources 104 or image data sources 104 that are movable or reconfigurable in terms of position may be provided to address potential differences in relative height, position, and orientation relative to the transport vehicle during loading.

[0037] Reference Figure 4 According to this disclosure, another image data source 204 ( Figure 2(Not shown) can be installed on the transport vehicle 10. The position of the image data source 204 can be selected such that when the material dumping operation, which is the second working state, is at least completed, the field of view 206 surrounds the loading container 14 of the transport vehicle 10. In the second working state, the surface of the loading container is pivoted to an angled orientation as shown in the figure, and the position of the image data source 204 can preferably be selected such that the field of view 206 completely surrounds the bottom surface of the loading container 14.

[0038] like Figure 5 As shown, the operating machinery 20 includes a control system with a controller 112. The controller 112 may be part of the mechanical control system of the operating machinery 20, or it may be a separate control module.

[0039] As described above, controller 112 is configured to receive input signals from some or all of a variety of image data sources 104, such as cameras, which collectively define the imaging system. Image data sources 104 may include cameras configured to record raw image streams and send corresponding data to controller 112. Alternatively or additionally, image data sources 104 may include one or more of infrared cameras, stereo cameras, PMD cameras, etc. Those skilled in the art will understand that, within the scope of this disclosure, high-resolution light detection and ranging (LiDAR) scanners, radar detectors, laser scanners, etc., can be implemented as image data sources. The number and orientation of the image data sources 104 may vary depending on the type of work machinery 20 and the associated application, but they may be provided at least in an area relative to the direction of travel of the work machinery 20 and are configured to capture image data associated with the loading area of ​​the adjacent work machinery 20 (e.g., corresponding to loading container 14).

[0040] The location and size of the image region recorded by the corresponding camera, which serves as image data source 104, may depend on the arrangement and orientation of the camera and camera lens system (particularly the focal length of the camera lenses), but can be desirously configured to capture substantially the entire loading container 14 throughout the loading operation. Those skilled in the art will also understand that, if a given image data source is properly configured, image data processing functions can be performed separately at that image data source, and these image data processing functions can typically also include, or otherwise comprise, at least some image data processing performed by a controller or other downstream data processor. For example, by using image data processing tools known in the art in conjunction with the disclosed objectives, image data from any one or more image data sources can be provided for 3D point cloud generation, image segmentation, object depiction, and classification, etc.

[0041] The controller 112 of the operating machinery 20 can be configured to generate output (as further described below) to a user interface 114 associated with a display unit 118 for display to the operator. The controller 112 can be configured to receive input from the user interface 114, such as user input provided via the user interface 114. Figure 5 Not specifically shown herein, in some embodiments, the controller 112 of the work machinery 20 may also receive input from a remote device associated with a user via a corresponding user interface (e.g., a display unit with a touchscreen interface) and generate output to the remote device. Data transmission between, for example, the vehicle control user interface and the remote user interface may take the form of a wireless communication system and related components, as is well known in the art. In some embodiments, the remote user interface and the vehicle control system for the corresponding work machinery 20 may further coordinate or otherwise interact with a remote server or other computing device to perform operations as disclosed herein.

[0042] In various embodiments, controller 112 may be configured to generate control signals for controlling the operation of a corresponding actuator, or for indirect control via an intermediate control unit associated with mechanical steering control system 126, machine tool control system 128, and engine speed control system 130. Control systems 126, 128, and 130 may be independent or otherwise integrated together, or as part of a machine control unit in various ways known in the art. Controller 112 may, for example, generate control signals for controlling the operation of various actuators such as hydraulic motors or hydraulic piston-cylinder units (not shown), and the electronic control signals from controller 112 may actually be received by an electro-hydraulic control valve associated with the actuator, such that the electro-hydraulic control valve controls the flow of hydraulic fluid to and from the corresponding hydraulic actuator in response to the control signals from controller 112, thereby controlling the actuation of the corresponding hydraulic actuator.

[0043] Alternatively, a reading device 132 (e.g., an RFID device, a barcode scanner, etc.) conventionally known in the art can be provided, and the reading device 132 can be communicatively connected to the controller 112 to obtain readable information associated with a particular transport vehicle 10.

[0044] The attitude sensor unit 134 may also be connected to the controller 112 for capturing and processing data associated with, for example, the current or predicted attitude of the loading container 14 of the transport vehicle 10. In some embodiments, the attitude sensor unit 134 may be integrated with or otherwise associated with an image data source 104, for example, where images of the loading container are captured and processed to associate the current attitude with a corresponding operational state. In some embodiments, the attitude sensor unit 134 may be integrated with or otherwise associated with a reading device 132, for example, where data is captured or otherwise received from the transport vehicle and associated with the current attitude of the loading container and further with a corresponding operational state.

[0045] Controller 112 includes or may be associated with processor 150, computer-readable medium 152, communication unit 154, and data storage 156 (e.g., a database network). It should be understood that controller 112 described herein may be a single controller having some or all of the described functions, or it may include multiple controllers, wherein some or all of the described functions are distributed among the multiple controllers.

[0046] The various operations, steps, or algorithms described in conjunction with controller 112 can be embodied directly in hardware, in a computer program product (such as a software module executed by processor 150), or in a combination of both. The computer program product may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, or any other form of computer-readable medium 152 known in the art. An exemplary computer-readable medium 152 may be coupled to processor 150 such that processor 150 can read information from and write information to memory / storage medium 152. Alternatively, medium 152 may be integrated with processor 150. Processor 150 and medium 152 may reside in an application-specific integrated circuit (ASIC). The ASIC may reside in a user terminal. Alternatively, processor 150 and medium 152 may reside as discrete components in a user terminal.

[0047] As used herein, the term "processor" 150 may refer to at least general-purpose or special-purpose processing devices and / or logic that are understood by those skilled in the art, including but not limited to microprocessors, microcontrollers, state machines, etc. Processor 150 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0048] Communication unit 154 may support or provide communication between controller 112 and external communication units, systems, or devices, and / or support or provide communication interfaces regarding internal components of the working machinery 20. The communication unit may include wireless communication system components (e.g., via a cellular modem, WiFi, Bluetooth, etc.) and / or may include one or more wired communication terminals, such as universal serial bus ports.

[0049] Unless otherwise stated, the data storage 156 further described below may generally include hardware (such as volatile or non-volatile storage devices, drives, electronic memory, and optical or other storage media) and, in some embodiments, one or more databases residing on the hardware.

[0050] like Figure 6 As shown, in embodiments of the system disclosed herein, each of the multiple transport vehicles 10 may include its own control system, which includes a controller 212. The controller 212 may be part of the vehicle control system of the transport vehicle 10, or it may be a separate control module.

[0051] The controller 212 of the corresponding transport vehicle 10 can be configured to receive input signals from a payload measurement unit 222, as is conventionally known in the art for certain articulated dump trucks. The controller 212 can also be integrated with or otherwise communicate with a tipping control system 224 to selectively guide the operation of the hydraulic piston-cylinder unit 16 to articulate the loading container 14 between the loading and tipping positions. The transport vehicle 10 may also include a barcode 332 or otherwise generate another form of machine-readable identifier 232 (e.g., an RFID signal) to transmit readable information to the operating machinery 20, etc., via a transceiver.

[0052] The attitude sensor unit 234 may also be connected to the controller 212 for capturing and processing data associated with, for example, the current or predicted attitude of the loading container 14 of the transport vehicle 10. In some embodiments, the attitude sensor unit 234 may be integrated with or otherwise associated with the image data source 204, for example, where images of the loading container are captured and processed to associate the current attitude with a corresponding operational state. In some embodiments, the attitude sensor unit 234 may be integrated with or otherwise associated with the machine control system or the user interface 214, for example, to receive input or output signals corresponding to a command position of the loading container for controlling the attitude of the loading container via the tilting control system 224.

[0053] In some embodiments, as described above, the controller 212 may also be integrated with or otherwise communicate with an image data source 204, such as an in-vehicle camera.

[0054] The controller 212 of the corresponding transport vehicle 10 can be configured to generate output to a user interface 214 associated with the display unit 218 for display to the driver, as further described below. The controller 212 can be configured to receive input from the user interface 214, such as user input provided via the user interface 214.

[0055] The controller 212 of the corresponding transport vehicle 10 may also include or be associated with a processor 250, a computer-readable medium 252, a communication unit 254, and a data storage 256 (e.g., a database network). It should be understood that the controller 212 described herein may be a single controller having some or all of the described functions, or it may include multiple controllers, wherein some or all of the described functions are distributed among the multiple controllers.

[0056] Next reference Figure 7 And further reference Figures 1 to 6 Embodiments of method 300 are now described. Method 300 is exemplary but does not limit the scope of this disclosure unless specifically indicated otherwise. Those skilled in the art will understand that alternative embodiments may include fewer or additional steps, and some disclosed steps may be performed, for example, in a different temporal order or simultaneously. Unless specifically indicated otherwise, the operations, steps, functions, processes, etc., disclosed in association with method 300 may be performed or directed by a single computing device, or via multiple computing devices operatively communicating via a communication network. Exemplary such computing devices may include vehicle controllers or mechanical control systems, remote servers, mobile user devices, etc.

[0057] The first step 310 may include detecting a first orientation of the loading area / loading container 14 of the transport vehicle 10, such as an orientation associated with a first operating state (e.g., a loading phase). In the case where the excavator, acting as the work machinery 20, loads material into the loading container of the articulated dump truck, acting as the transport vehicle 10, the first operating state may be associated with a loading container orientation in which the loading container is fully retracted to receive material, and optionally further, in which the transport vehicle is properly positioned relative to the frame 12 of the work machinery 20. The image data source 104 may be mounted on the work machinery, for example, on the underside of the work implement 42, but is not explicitly limited thereto, such that images can be sufficiently captured to process the upper edge of the loading container and determine, for example, the orientation of the loading container relative to the ground.

[0058] When it is determined that the loading process is complete (i.e., in response to the query in step 320 being "yes"), for example via manual user input or other forms of input signals or processed image data, the method may continue in step 330: scanning the outline of the material loaded in the loading container via an image data source 104 mounted on the operating machinery 20. As previously mentioned, the image data source 204 mounted on the transport vehicle 10 typically cannot capture the outline of the material because the material is loaded high enough relative to the shape of the loading container to obstruct the corresponding field of view 206.

[0059] As previously described, using image data processing tools known in the art, contours can be scanned and further analyzed using techniques such as 3D point cloud generation, image segmentation, object delineation, and classification. Optionally, a reference contour corresponding to a predetermined shape of the loading container 14 can be considered to estimate, for example, the volume of material loaded in the container based on the scanned contour. Certain dimensions (such as information typically corresponding to the relevant shape of the loading container 14) can be dynamically determined using an image data source, or a reference contour can be predetermined and retrieved from a data storage when identifying a specific transport vehicle 10 or transport vehicle type, or a reference contour can be directly input from a user interface. For example, information about one or more dimensions of the loading container 14 can be stored in association with the transport vehicle 10, wherein the detected first contour is associated with and transmitted from the operating machinery 20 to the transport vehicle 10. And a first volume is calculated based on the transmitted detected first contour and the stored information about one or more dimensions of the loading container 14.

[0060] In one embodiment, raw data corresponding to the scanned contour can be captured by image data source 104 on the operating machinery 20 and then transmitted to transport vehicle 10, which does not need to obtain reference contour data corresponding to the dimensions of the loading container 14 (e.g., dimensions related to the shape of the loading container 14) from another location and therefore can perform volume estimation without such a sub-step. In some embodiments, a model can be implemented that estimates the volume of material directly from the scanned contour without supplementing the reference shape or other parameters of the loading container 14. Such a model can be developed and implemented in the form of a lookup table based on the definition level of the material relative to one or more identified shapes / edges of the loading container 14, or it can be progressively refined over time using machine learning techniques, etc.

[0061] The next step 340 may include detecting a second orientation of the loading area / loading container 14 of the transport vehicle 10, such as an orientation associated with a second operating state (e.g., a dumping phase). In the above context of the transport vehicle 10 being an articulated dump truck, the second operating state may be associated with a loading container orientation in which the piston-cylinder unit 16 coupled to the loading container 14 is substantially fully extended to dump material out of the loading container 14. In one embodiment, a signal may be generated to indicate that the loading container 14 pivots to the second orientation; this signal may come, for example, from a sensor associated with the piston-cylinder unit 16 or from manual input from a user interface. The second orientation may be determined based on processing of images from the image data source 204.

[0062] When it is determined that the dumping process is complete (i.e., in response to the query "yes" in step 350), for example via manual user input or other forms of input signals or processed image data, the method may continue in step 360: scanning the outline of any remaining material in the loading container 14 after dumping. The scanned outline may be generated by an image data source 204, which may, for example, be mounted above a plane corresponding to the upper edge of the loading container 14 and configured such that an image can be sufficiently captured to process at least the bottom surface of the loading container 14. As previously mentioned, the image data source 104 mounted on the operating machinery 20 may not reliably capture the outline of the remaining material in the loading container 14 (where the loading container is empty except for the remaining material) because the corresponding field of view 106 typically does not extend to the bottom surface of the loading container 14.

[0063] The outline of the remaining (brought back) material in the second working state can be scanned and further analyzed, for example, in a manner similar to that of the material loaded in the first working state; however, in some embodiments, the processing techniques are not necessarily the same and can be performed in different locations.

[0064] Figure 7 The next step 370 shown may include: transferring the scanned contour and / or processed volume data from the initial position to the downstream position for additional processing and to support further functions such as reporting, display, control, etc.

[0065] For example, as initially described above, in one embodiment, raw data from an initial scan of material loaded in loading container 14, captured by an image data source mounted on the work machinery 20, can be transmitted to transport vehicle 10. The scan data can be provided as an input data string along with identifiers for the work machinery, identifiers for loading locations, etc., so that after loading container 14 has been substantially emptied, such first scan data can be supplemented with additional scan data, and calculations can be performed for each scan to determine the total volume dumped in the current iteration of the work cycle. Generally, if the second scan does not produce any material brought back in loading container 14, then all material loaded in loading container 14 during the first scan, or in other words, all material dumped on the ground before the second scan, can be considered a positive code number. Alternatively, if the second scan produces a profile corresponding to a certain amount of return material in loading container 14, the volume of material dumped on the ground before the second scan can be calculated / estimated based on a comparison of the corresponding scans, further optionally taking into account supporting parameters (such as the shape of loading container 14).

[0066] In one embodiment, the material density can be determined at least based on a first input (e.g., from the payload measurement unit) acquired before the material is loaded into the transport vehicle 10, a second input (e.g., from image data source 104 on the operating machinery 20) acquired after the material is loaded into the transport vehicle 10, and a third input (e.g., from the payload measurement unit) that is parallel to the second input but compared with the first and second inputs to determine the actual loaded volume and weight of the material added since the first input. After the material corresponding to the third input has been discharged from the loading container, the volume of the remaining material (e.g., the material brought back) in the transport vehicle 10 can be determined based on a fourth input (e.g., from the payload measurement unit) and further taking into account the previously determined material density.

[0067] In various embodiments, the different data processing steps disclosed herein can be performed in any of a variety of possible stations. For example, the controller 212 for the transport vehicle 10 can be configured to receive data from the image data source 104 of the operating machinery 20 and perform data processing and volume estimation steps based on supplementary data from the image data source 204 of the transport vehicle 10. As another example, the respective controllers 112, 212 can individually perform image data processing steps to calculate / estimate the volume of loaded and returned material, respectively, wherein the volume data can be transmitted to a destination including the controllers 112, 212, an associated user interface, or a third-party device / system.

[0068] In one embodiment, a data structure (e.g., a custom database) may be developed and stored in association with, for example, a third-party system configured to receive scan data and / or volume data from either or both of the work machinery 20 and the transport vehicle 10 (step 380). The data structure may be populated with such data and / or calculations based on that data and / or derivations thereof to provide the total volume of material removed from a location associated with the work machinery 20, the total volume of material removed from a location associated with the dumping state of the transport vehicle 10, or other aggregated values, for example, associated with multiple work machinery 20s and / or transport vehicles 10 at a collective work site. Data received at a third-party system (such as a central server, mobile user device, etc.) may also include identifiers associated with the corresponding point, the transported work machinery 20 or transport vehicle 10, the user, time / date, etc., as part of a data string, used to appropriately populate the data structure or feed it into a program engine to generate data entering the data structure.

[0069] Considering the preceding steps and their variations, method 300 may further include step 390 for displaying relevant information to the user and / or selectively executing control functions based on this. For example, the driver of transport vehicle 10 may be notified in real time of the volume of material remaining in loading container 14 after dumping, or the volume of material currently loaded in loading container 14. An automatic control function may generate an alarm when the volume of material in loading container 14, determined during a loading operation state, exceeds a threshold, or when the volume of remaining material in loading container 14 exceeds a threshold at the end of a dumping operation state. In one embodiment, if the volume of remaining material in loading container 14 exceeds a threshold at the end of a dumping operation state, the automatic control function may repeat one or more aspects of the dumping operation state.

[0070] In view of the above embodiments and their equivalents, as those skilled in the art will understand, more accurate and timely estimation of the volume of material at each stage of operation leads to more accurate calculations of productivity and efficiency.

[0071] As used in this article, the phrase "one or more" when used with a list of items means that different combinations of one or more items may be used, and it may be necessary to have only one of each item in the list. For example, "one or more" in items A, B, and C could include, for example, but not limited to (item A) or (items A and B). The example could also include (items A, B, and C) or (items B and C).

[0072] Those skilled in the art will understand that when an element is referred to herein as being “connected” to another element, it may be directly connected to the other element, or there may be an intermediate element present.

[0073] Therefore, it can be seen that the apparatus and methods of this disclosure readily achieve the mentioned and inherent purposes and advantages. Although certain preferred embodiments of this disclosure have been shown and described for this purpose, many changes can be made by those skilled in the art to the arrangement and construction of the components and steps, which are included within the scope and spirit of this disclosure as defined by the appended claims. Each feature or embodiment of the disclosure may be combined with any other feature or embodiment of the disclosure.

Claims

1. A method (300) for real-time deduction of material brought back from the loading container (14) of a transport vehicle (10), wherein, The material is loaded into the loading container by the operating machinery (20) at a first station and unloaded from the loading container by the transport vehicle at a second station, the method comprising: In a first operating state, first data corresponding to the volume of material loaded in the loading container is provided via a first sensor (104) associated with one of the operating machinery or the transport vehicle; In the second operating state, second data corresponding to the volume of material loaded in the loading container is provided via a second sensor (204) associated with the other of the operating machinery or the transport vehicle; An output signal is generated corresponding to the calculated total volume of materials associated with the work cycle, the total volume being at least partially based on the provided first data and the provided second data.

2. The method according to claim 1, wherein, The generated output signal is provided to fill the data structure with the calculated total volume of the material associated with at least one of the first and second sites.

3. The method according to claim 1 or 2, wherein the method comprises: The first volume is determined at least in part based on the first data provided and information about one or more dimensions of the loading container; The second volume is determined at least in part based on the provided second data and the information regarding one or more dimensions of the loading container; as well as The total volume of material associated with the work cycle is calculated based on the difference between the determined first volume and the determined second volume.

4. The method according to claim 3, wherein: The information regarding one or more dimensions of the loading container is stored in association with the transport vehicle; The first sensor is associated with the operating machinery, and the first data is transmitted from the operating machinery to the transport vehicle; as well as The first volume is calculated based on the transmitted first data and the stored information about one or more dimensions of the loading container.

5. The method according to claim 3, wherein: The first sensor includes a first image data source, which is configured to generate a signal corresponding to a first profile of material loaded in the loading container in the first operating state; as well as The second sensor includes a second image data source configured to generate a signal corresponding to a second profile of the material loaded in the loading container during the second operating state.

6. The method according to claim 5, wherein: The total volume is calculated at least in part based on the determined first profile and the determined second profile of the loaded material.

7. The method according to claim 5 or 6, wherein, The information about one or more dimensions of the loading container is obtained via scanned images from the first image data source and / or the second image data source.

8. The method according to claim 5 or 6, wherein, Based on scanned images including identifiers associated with the transport vehicle, information about one or more dimensions of the loading container is retrieved from a data storage device.

9. The method according to claim 5 or 6, wherein, Based on communication between the operating machinery and the transport vehicle, including an identifier associated with the transport vehicle, the information regarding one or more dimensions of the loading container is retrieved from a data storage device.

10. The method according to claim 3, wherein: The first sensor includes an image data source associated with the operating machinery and configured to generate a signal corresponding to the outline of the material loaded in the loading container in the first operating state; as well as The second sensor includes a payload measurement unit associated with the transport vehicle.

11. The method according to claim 10, further comprising: The material density of the material loaded in the transport vehicle is determined based on the input from the first sensor and the input from the payload measurement unit during the first operating state.

12. The method according to claim 11, further comprising: Based on the input from the payload measurement unit in the second operating state and further considering the determined material density, the volume of the remaining material in the transport vehicle in the second operating state is determined.

13. The method according to any one of claims 10 to 12, wherein, The information about one or more dimensions of the loading container is obtained via scanned images from the image data source.

14. The method according to any one of claims 10 to 12, wherein, Information about one or more dimensions of the loading container is retrieved from a data storage device based on scanned images including identifiers associated with the transport vehicle and / or based on communications between the operating machinery and the transport vehicle including identifiers associated with the transport vehicle.

15. A system for real-time deduction of material brought back from the loading container (14) of a transport vehicle (10), wherein, The material is loaded into the loading container by the operating machinery (20) at a first station and unloaded from the loading container by the transport vehicle at a second station. The system includes: A first sensor (104) is associated with one of the operating machinery or the transport vehicle and is configured to provide first data corresponding to the volume of material loaded in the loading container in a first operating state; A second sensor (204) is associated with another of the operating machinery or the transport vehicle and is configured to provide second data corresponding to the volume of material loaded in the loading container in a second operating state; A computing device (112, 212) comprising a computer-readable medium (152, 252) residing on one of the operating machinery or the transport vehicle and having program instructions residing on the computer-readable medium, the program instructions being executable by a processor (150, 250) to direct the execution of steps in the method according to any one of claims 1 to 14.

Images