Spatial positioning imaging system

Abstract

A spatially positioned imaging system for a machine, comprising: an imaging device mounted to a frame of the machine; a target configured to interact with the imaging device; and a controller operatively connected to an implement of the machine, the controller configured to receive the target data from the imaging device, interpret the target data to compute a map of a field on which the machine is operating, provide a real-time estimate of a position of the machine within the field, and provide automated input for the implement.

Description

Technical Field

[0001] This invention relates generally to working machines, and more specifically to a spatial positioning system for working machines. Background Technology

[0002] Mobile work machines can be used in heavy industrial sectors such as earthmoving, mining, and construction to move ground components, transport materials, and personnel. These machines are typically large in size and require manual operation by an operator (e.g., a driver) to perform their designated / intended tasks.

[0003] Some construction machinery (such as bulldozers) uses implements to perform various tasks on a work site. These implements may need to transport heavy loads and therefore may utilize hydraulic systems to assist in operating the mechanisms used for the implements. In particular, for earthmoving machinery, the implements may be blades used to cut the ground and move large clumps of soil to create specific slopes over large areas.

[0004] When operating machinery, knowing the machine's position on the work site is crucial for accurately positioning the blade relative to the machine, allowing for the precise movement of the optimal amount of soil. In the past, Global Positioning System (GPS) technology has been used to monitor machine location. While GPS systems are useful, they often lack precise accuracy, are too costly to deploy on small construction sites, and present difficulties in indoor environments.

[0005] U.S. Patent No. 9,481,982 discloses a method for generating scaled terrain information while operating a bulldozer. The bulldozer may include: a drive unit comprising a set of drive wheels; a motor connected to at least one drive wheel; a blade for altering the terrain surface; at least one camera for acquiring environmental images, the camera being positioned and aligned relative to the bulldozer in a known manner; and a control and processing unit. One method may include: simultaneously generating a set of image data by acquiring images of a series of terrain portions using at least one camera while moving the bulldozer, such that at least two images in the image series cover the same number of points in the terrain; and applying a Simultaneous Localization and Mapping (SLAM) algorithm or a stereo photogrammetry algorithm to the set of image data to obtain terrain data.

[0006] Given the above-mentioned shortcomings, a system is needed that can accurately locate machines in real time at the work site. Summary of the Invention

[0007] According to one aspect of the invention, a machine may be provided. The machine may include a frame, an engine supported by the frame, and a drivetrain connected to the engine and connected to a ground engagement member. The machine may include an operator's cab supported by the frame, and a controller mounted in the operator's cab for controlling machine operation. The machine may include implements operatively associated with the frame, movable relative to the frame and controlled by the controller. The machine may include an imaging device mounted to the machine, configured to interact with a target of a spatial positioning imaging system and transmit target data to the controller. The controller may be configured to interpret the target data to compute a map of the site where the machine is operating, provide a real-time estimate of the machine's position in the site, and provide automated input to the implements.

[0008] According to another aspect of the invention, a spatial positioning imaging system for a machine can be provided. The spatial positioning imaging system may include an imaging device mounted to the machine. The spatial positioning imaging system may include a target configured to interact with the imaging device. The spatial positioning imaging system may include a controller operatively connected to an implement of the machine, the controller being configured to receive target data from the imaging device, interpret the target data to compute a map of the site where the machine is operating, provide a real-time estimate of the machine's position within the site, and provide automated input to the implement.

[0009] According to another aspect of the invention, a method for spatially positioning a machine in the field can be provided. The method may include providing a machine comprising a frame, implements attached to and movable relative to the frame, an operator's cab supported by the frame, and a controller mounted in the operator's cab for controlling machine operation. The method may include providing an imaging device mounted to the machine and configured to communicate with the controller, and providing a target in the field configured to interact with the imaging device. The method may include operating the machine to a starting point in the field, establishing an origin via the controller, acquiring target data of the target through interaction with the imaging device, transmitting the target data from the imaging device to the controller, and generating a map of the field. The method may include estimating the real-time position of the machine in the field based on the map and automating the positioning of the implements relative to the frame.

[0010] These and other aspects and features of the invention will be more readily understood when read in conjunction with the accompanying drawings. Attached Figure Description

[0011] Figure 1 This is a perspective view of a machine constructed according to an embodiment of the present invention.

[0012] Figure 2This is a perspective view of the target in a spatial positioning imaging system constructed according to an embodiment of the present invention.

[0013] Figure 3 This is a perspective view of the target in a spatial positioning imaging system constructed according to an embodiment of the present invention.

[0014] Figure 4 It is a machine-operated topographic view of the scene constructed according to an embodiment of the present invention, utilizing a spatial positioning imaging system.

[0015] Figure 5 This is a flowchart describing an example sequence of steps for spatial positioning of a machine on-site according to the present invention.

[0016] The accompanying drawings illustrate one embodiment of the invention for illustrative purposes only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown herein can be employed without departing from the principles described herein. Detailed Implementation

[0017] Now refer to the attached diagram, and specifically refer to... Figure 1 The invention depicts a machine, which is generally referred to by reference numeral 10. Machine 10 is exemplarily implemented as a working machine, particularly a bulldozer. While machine 10 is depicted as a bulldozer, it should be noted that the type of machine used is merely exemplary and illustrative in nature. It should be understood that the teachings of the invention can be similarly applied to other types of working machines, including but not limited to off-highway trucks, excavators, loaders, mining vehicles, and other machines known to those skilled in the art that require precise spatial positioning.

[0018] Earthmoving machinery, particularly bulldozers, can be used in earthmoving projects to cut the ground and subsequently transport and dump soil from the cut ground from one location to another on rugged terrain. Machine 10 is supported by a frame 11. Machine 10 may include an engine 13 supported by the frame 11 for powering machine 10. While engine 13 is shown as an internal combustion engine, it may also include an electric motor, a hybrid system, or other known generator. Machine 10 may include a drivetrain 12 connected to the engine 13 and to a ground engagement member 14. Figure 1 In the machine 10 shown, the ground engagement member 14 is a wheel, but a tracked system or other known ground engagement members may also be used.

[0019] Machine 10 may include an operator's cab 17 supported by a frame 11, and a controller 20 mounted within the operator's cab 17 for controlling the operation of machine 10. Machine 10 may include implements 15 operably associated with the frame 11, which are movable relative to the frame 11 and controlled by the controller 20. Figure 1 In the machine 10 shown, the implement 15 is a blade connected to the frame 11, and further includes a hydraulic cylinder 16 for driving the blade relative to the frame. The machine 10 can be an earthmoving machine, and therefore the implement can be any of a blade, a ripper, or other ground-jointing tool for cutting the ground 50.

[0020] Machine 10 may include an imaging device 30 mounted to it. The imaging device 30 may be mounted to the frame 11 of machine 10, may be installed within the operator's cab 17 of machine 10, or may be installed at any location on machine 10 as needed. The imaging device 30 may be configured to interact with a target 40 of the spatial positioning imaging system 60 and transmit target data acquired from target 40 to controller 20. Controller 20 may be configured to interpret the target data to compute a map of the site 61 where machine 10 is operating, provide a real-time estimate of the machine 10's position within the site 61, and provide automated input to implement 15.

[0021] Figures 2 to 3 An exemplary form of a target 40 of a spatial positioning imaging system 60 is shown. In one exemplary embodiment, the imaging device 30 of the machine 10 may be a camera configured to read Quick Response (QR) codes. However, other known imaging devices may also be used to read the corresponding target encoding forms. Target 40 may be a reference target. As understood, the term "fiducial" refers to target 40 as a reference standard. Therefore, target 40 may include a sign 42 bearing a QR code 43 and located on a sign post 41. The QR code 43 may include target data interpreted by the controller 20. The target data may be relatively simple, including only the target's numerical identifier, or it may include information of a desired level of complexity. The target data may include two-dimensional spatial information of target 40 relative to a reference point, and may also include three-dimensional spatial information of target 40 relative to a reference point. Figure 2 As shown, sign 42 and signpost 41 are oriented on the ground 50, such that there is a standard height h between sign 42 and the ground 50. Therefore, multiple targets 40 are placed around the site 61, each target extending a standard height h above the ground instantaneously, allowing the controller 20 to interpret the height change of the ground 50 from one target 40 to another. Figure 3 Target 40 is shown again; label 42 has a QR code 43, but it is oriented onto bracket 44. (As shown) Figure 3As shown, the bracket 44 enables the sign 42 to remain at a standard height h above the ground 50 even when the ground 50 is tilted.

[0022] Figure 4 The spatial positioning imaging system 60 is shown in operation at the field 61. The spatial positioning imaging system 60 allows the machine 10 to precisely position itself in six axes. Figure 4 Displayed as a contour map, contour line 62 represents the elevation changes at site 61. Multiple targets 40 can be placed at various locations on site 61, enabling imaging device 30 to accurately observe the elevation changes at site 61, which are then interpreted by controller 20. Machine 10 is placed at the origin 63 of site 61, providing a reference point for machine 10 to begin its work.

[0023] Machine 10 may include additional sensors to assist controller 20 in providing a real-time estimate of the machine 10's position on site 61 by enhancing real-time estimation. Sensor 18 may be mounted to the frame of machine 10 to read operational data of machine 10. Sensor 18 may communicate with radio 19 to receive operational data. For example, sensor 18 may include an inertial measurement unit configured to measure the directional and / or rotational motion of machine 10 relative to gravity during operation. Sensor 18 may also include a distance sensor located within hydraulic cylinder 16 and connected to implement 15. This distance sensor may be configured to measure the drive distance or displacement of hydraulic cylinder 16 to determine the spatial position of implement 15 relative to the frame 11 of machine 10. Sensor 18 may also include a global positioning sensor, enabling the instantaneous positioning of machine 10 to be determined via GPS coordinates.

[0024] Industrial applicability

[0025] In practice, the teachings of this invention can be applied to many industries, including but not limited to working machines used in earthmoving, mining, agriculture, and construction. Although depicted and described in conjunction with bulldozers, such teachings can also be applied to other machines, such as off-highway trucks, excavators, loaders, mining vehicles, and other machines known to those skilled in the art that require precise spatial positioning.

[0026] Figure 5 A schematic diagram illustrating a method 100 for spatial positioning of machine 10 at site 61 is shown. In the first step 101, machine 10 is provided. Providing machine 10 includes providing a frame 11, a tooling 15 attached to and movable relative to the frame 11, an operator's cab 17 supported by the frame 11, and a controller 20 installed in the operator's cab 17 for controlling the operation of machine 10. Providing machine 10 also includes providing an imaging device 30, which is mounted to machine 10 and configured to communicate with controller 20.

[0027] In the second step 102, a target 40 is provided at the scene 61, which is configured to interact with the imaging device 30. For example... Figure 4 The example shown provides multiple targets 40 at different locations on site 61, enabling accurate mapping of the terrain changes on site 61. To create reference points, in step 3 103, machine 10 is operated to the starting point on site; in step 4 104, the operator directs controller 20 to establish origin 63.

[0028] The machine 10 is then operated, and in the process, it interacts with the imaging device 30 to acquire target data of the target 40. In step 5, 105, the imaging device 30 acquires the target data and sends it to the controller 20. The controller 20 receives the target data acquired from the imaging device 30 and interprets it. In step 6, 106, the controller 20 uses the target data as a reference to generate a map of the scene 61. Then, in step 7, 107, based on this map, the controller 20 can provide a real-time estimate of the position of the machine 10 during operation at the scene 61. Optionally, the machine 10 may be equipped with additional sensors, and in step 8, 108, the controller 20 acquires data from these additional sensors. The controller 20 can then enhance the real-time position of the machine 10 based on the additional sensor data.

[0029] Once the real-time position of machine 10 is provided, in step nine 109, controller 20 can then automate the position of implement 15 relative to frame 11. This automation may include lowering implement 15 to a fixed depth below ground level 50, and in step ten 110, may include cutting a specific profile relative to origin 63. However, this automation may also include other earthmoving techniques. Finally, in step eleven 111, machine 10 completes its work and returns implement 15 to a neutral position.

[0030] Method 100 can be applied to any machine 10, requiring only the installation of the imaging device 30, the installation of sensors, and a software update for improvement. Method 100 can also be applied to other industries and any machine that requires precise spatial positioning during operation.

[0031] Clearly, this invention is to be understood as exemplary and various changes can be made by adding, modifying, or eliminating details without departing from the fair scope of the teachings contained herein. Therefore, this invention is not limited to the specific details thereof, unless so defined in the following claims.

Claims

1. A spatial positioning imaging system for a machine, comprising: An imaging device, which is installed in the machine; The target is configured to interact with the imaging device; as well as A controller, operatively connected to the implement of the machine, is configured to receive target data from the imaging device, interpret the target data to calculate a map of the site in which the machine is operating, provide a real-time estimate of the machine’s position in the site, and provide automated input to the implement.

2. The spatial positioning imaging system of claim 1 further includes a sensor mounted to the machine, the sensor being configured to communicate with the controller and provide an additional estimate of the machine's position in the field, such that the additional estimate enhances the real-time estimate.

3. The spatial positioning imaging system according to claim 1, wherein the imaging device is a camera, and the target is a sign post containing a Quick Response (QR) code, the QR code being configured to be read by the camera.

4. The spatial positioning imaging system according to claim 1, wherein the target data further includes two-dimensional spatial information of the target relative to the origin.

5. The spatial positioning imaging system according to claim 1, wherein the target data further includes three-dimensional spatial information of the target relative to the origin.

6. A method for spatial positioning of a machine on-site, comprising: A machine is provided, the machine comprising a frame, implements attached to and movable relative to the frame, an operator's cab supported by the frame, and a controller installed in the operator's cab for controlling the operation of the machine; An imaging device is provided, which is mounted to the machine and configured to communicate with the controller; A target is provided at the scene, and the target is configured to interact with the imaging device; Operate the machine to the starting point of the site; The origin is established by directing the controller; By interacting with the imaging device, target data of the target is acquired, and the target data is sent from the imaging device to the controller; Generate a map of the site; The real-time location of the machine at the site is estimated based on the map; as well as The positioning of the machine relative to the frame is automated.

7. The method of claim 6, wherein the tool is a ground joining tool, and the step of automating the positioning of the tool further comprises positioning the ground joining tool at a fixed depth relative to the ground at the origin, the method further comprising using the ground joining tool to cut the ground at the fixed depth.

8. The method according to claim 6, further comprising: An inertial measurement unit is provided to be installed on the machine; The inertial measurement unit is used to collect the inertial measurement values ​​of the machine; The inertial measurement value is sent to the controller; as well as The machine's real-time position is enhanced based on the inertial measurement values.

9. The method according to claim 6, further comprising: A hydraulic cylinder is provided for the tool to drive the tool, and the hydraulic cylinder includes a distance sensor for measuring the driving distance of the hydraulic cylinder; The distance sensor is used to acquire the driving distance of the hydraulic cylinder; The driving distance is sent to the controller; as well as The machine's real-time position is enhanced based on the drive distance.

10. The method of claim 6, further comprising: Provide a global positioning sensor to be installed on the machine; The machine's real-time location is acquired using the aforementioned global positioning sensor; The real-time location is sent to the controller; as well as The real-time location of the machine is enhanced based on the instantaneous positioning.

Images