Dynamic image enhancement for a milling machine

Abstract

A machine for milling a road surface, such as a rotary mixer or a road planer, includes a cutting rotor that is vertically adjustable relative to a frame and housed in a rotor shroud. The milling machine can be associated with a network of vision cameras having one or more cameras positioned about the milling machine with a field of view toward the rotor shroud and a work surface. An image enhancement system can generate a reference line enhancement to be superimposed on one or more vision images obtained by the cameras and display the enhanced image on a vision display.

Description

Technical Field

[0001] This patent disclosure generally relates to a machine for milling working surfaces, such as a rotary mixer or a planer equipped with a cutting rotor, and more specifically to an imaging system for assisting in the operation of a milling machine. Background Technology

[0002] Various machines are used to remove or mill materials (such as pavement, asphalt, or concrete) from working surfaces, such as roads or similar surfaces. For example, rotary mixers and planers typically include a cylindrical rotating drum or cutting rotor supported on a frame that is in turn supported on multiple ground-engaging traction devices, such as wheels or continuous tracks. Furthermore, the cutting rotor can be vertically adjustable relative to the working surface. As the milling machine travels across the working surface, the cutting rotor lowers into the surface, and multiple toothed cutting tools 132 or picks arranged around the cylindrical surface of the cutting rotor penetrate into the top layer of the surface, breaking it apart.

[0003] To contain the chips generated during the milling process and prevent them from spreading around the milling machine, the cutting rotor is typically housed in a rotor housing, which may visually obstruct some or all of the cutting rotor. Additionally, due to the size of the milling machine, the operator's table may not be positioned to provide the most advantageous view around the machine. Therefore, in some cases, milling operations can be performed by an individual walking alongside the milling machine to observe the process and relay those observations to the machine operator.

[0004] U.S. Patent Publication No. 2019 / 0210525 ('525 Publication), entitled "Cutting Tool Visual Trajectory Representation System and Method," describes the use of one or more cameras located on a milling machine to facilitate milling operations by enhancing visibility around the machine. Images captured by the cameras can be displayed on a visual display screen accessible to the operator. Furthermore, the '525 Publication describes a computer-implemented imaging processing system that can enhance the images presented on the display screen. This disclosure relates to an improved system and method for capturing, enhancing, and presenting visual images during milling operations to assist the operator. Summary of the Invention

[0005] In one aspect, this disclosure describes a milling machine for milling a working surface similar to a road or pavement covered with asphalt. The milling machine includes a frame supported on a plurality of traction devices for traveling along a travel axis along the working surface. The frame includes a first lateral side and a second lateral side aligned with the travel axis of the milling machine. A cutting rotor, rotatably supported on the frame for milling the working surface, is shaped as a cylindrical drum, wherein the rotor axis is perpendicular to the travel axis. To accommodate the cutting rotor, a rotor housing is located on the frame and includes a first housing sidewall aligned with the first lateral side and a second housing sidewall aligned with the second lateral side. To capture images of the rotor housing and the working surface, a camera may be supported on the frame at one of the first and / or second lateral sides. The milling machine may further include an electronic controller programmed to receive the visual image from the camera; determine the position of the cutting rotor relative to one or more lateral contact lines of the working surface; generate a reference line enhancement corresponding to the lateral contact lines; and superimpose the reference line enhancement onto the visual image to produce an enhanced image.

[0006] In another aspect, this disclosure describes a method of operating a milling machine for milling a work surface similar to a road or pavement covered with asphalt. The method includes capturing a visual image comprising both a lateral side of a rotor housing housing a cutting rotor and a lateral side of the work surface. The method determines the location of one or more lateral contact lines that the cutting rotor will contact the work surface and generates a reference line enhancement corresponding to the lateral contact lines. The reference line enhancement is superimposed on the visual image to produce an enhanced image, which can be displayed on a vision display associated with the milling machine.

[0007] In another aspect of this disclosure, a control system for a milling machine having a cutting rotor with a milling working surface is described. The control system includes a first camera to capture a first visual image of a first lateral side of the milling machine, and a second camera to capture a second visual image of a second lateral side of the milling machine. The control system also includes an electronic controller configured to determine one or more lateral contact lines where the cutting rotor will contact the working surface; generate reference line enhancements corresponding to the lateral contact lines; and superimpose the reference line enhancements onto the first visual image to generate a first enhanced image, and superimpose the reference line enhancements onto the second visual image to generate a second visual image. The control system is operatively associated with a vision display to display the first enhanced image and the second enhanced image in parallel. Attached Figure Description

[0008] Figure 1 These are perspective images of a milling machine used for milling working surfaces. The milling machine is equipped with a cutting rotor housed in a rotor casing and multiple cameras that capture visual images of the milling machine and the working surface.

[0009] Figure 2 This is a schematic representation of a cutting rotor that engages with the working surface by penetrating into it, depicting one or more lateral contact points between the cutting rotor and the working surface.

[0010] Figure 3 This is a schematic block diagram of a computer-implemented image enhancement system that assists in the operation of a milling machine by enhancing visual images captured by multiple cameras.

[0011] Figure 4 This is a flowchart of a possible electronic implementation process for an image enhancement system to capture, enhance, and display visual images to assist milling operations.

[0012] Figure 5 It is a representative visual display according to this disclosure, which includes first and second displays for displaying enhanced images. Detailed Implementation

[0013] Referring now to the accompanying drawings, whereby, wherever possible, the same reference numerals denote the same features. Figure 1 The diagram illustrates a specific embodiment of a rotary mixer 100 used for road maintenance and repaving operations, as is well known to those skilled in the art. The rotary mixer 100 is configured to remove and recycle or reuse a working surface layer 102, such as pavement, concrete, asphalt, or other materials, by penetrating into and fracturing the working surface during a milling operation. The fractured material can be redeposited on the working surface 102, where it can be used as subgrade or base aggregate in subsequent paving operations. In addition to rotary mixers, this disclosure applies to other milling machines, such as road planers capable of milling and removing working surface layers, earthmoving reclaimers for mixing and transporting earth, and other machines used for working surface milling operations and similar operations in construction and agriculture.

[0014] The rotary mixer 100 may include a frame 104 that may be oriented such that the front end 106 and the rear end 108 are aligned along the machine's travel axis 110; however, since the rotary mixer 100 can operate in both forward and reverse directions, this designation is used herein for reference purposes. The frame 104 may also include a first lateral side 112 and an opposing second lateral side 114, which may correspond to the left or right side of the rotary mixer depending on the observer's orientation. Again, the first lateral side 112 and the second lateral side 114 are used herein for arbitrary reference and orientation purposes.

[0015] To support the rotary mixer 100 on the working surface 102, the frame 104 may be suspended from a plurality of ground-engaged traction devices 116. In the illustrated embodiment, the traction device 116 may be a rotatable wheel that may include rubber pneumatic tires. The wheel may be designated as a power-driven wheel to propel the rotary mixer 100, a maneuverable wheel to adjust the orientation of the rotary mixer, or a combination thereof. Another suitable embodiment of the traction device 116 includes a continuous track, such as a closed belt arranged around rollers and / or sprockets, wherein translation of the belt transports the rotary mixer 100 on the working surface 102. To vertically raise and lower the rotary mixer 100 relative to the working surface 102, the frame 104 may be coupled to the traction device 116 by a plurality of lifting columns 118. The telescopic lifting columns 118 may extend and retract independently to adjust the height, slope, and inclination of the frame 104 relative to the working surface 102. In an embodiment, the lifting column 118 may be located at the front end 106 and rear end 108 of either side 112, 114, so that the pitch, tilt and / or slope of the rotary mixer 100 can be selectively changed.

[0016] To power the traction device 116, the lifting column 118, and other systems of the rotary mixer 100, a power source such as an internal combustion engine 120 can be mounted on the frame 104. The internal combustion engine 120 can burn hydrocarbon-based fuels, such as diesel or gasoline, and convert the potential chemical energy therein into mechanical power in the form of rotational motion, which can be utilized for other useful work. The rotational output of the engine 120 can be transmitted via a crankshaft 122 extending from the engine and operatively coupled to the traction device 116 and other systems. For example, the engine 120 can be operatively coupled to and drive other power systems on the rotary mixer, such as a generator 124 generating electricity for the electrical system and a hydraulic pump 126 for pressurizing and guiding hydraulic fluids in the hydraulic system.

[0017] To engage and break up the working surface 102, the rotary mixer 100 may include a power-driven cutting rotor 130 rotatably supported by a frame 104. The cutting rotor 130 may be a drum-shaped cylindrical structure having a plurality of pick-like or toothed cutting tools 132 arranged around its cylindrical surface. As the cutting rotor 130 rotates, the cutting tools 132 impact and penetrate into the working surface 102, thereby fracturing its material. The cutting tools 132 are adapted to penetrate into the working surface 102 and remove a portion of the material as the rotary mixer 100 advances along the travel axis 110 through a process known as milling or planning. In some embodiments, the cutting tools 132 may be removed from the cutting rotor 130 for replacement if the cutting rotor wears or becomes damaged. The cutting rotor 130 may rotate about a rotor axis 134 extending between a first lateral side 112 and a second lateral side 114 of the frame 104 and generally perpendicular to the travel axis 110.

[0018] To accommodate fragmented material and debris, the cutting rotor 130 can be rotatably housed in a housing or rotor housing 136 that hangs from the frame 104 toward the working surface 102. The rotor housing 136 defines an enclosed space for the cutting rotor 130 and can be formed of welded or fastened metal sheets or plates, including a housing sidewall 138 aligned with a first lateral side 112 and another housing sidewall 138 aligned with a second lateral side 114. The rotor housing 136 and the cutting rotor 130 therein can extend across the lateral width of the rotary mixer 100. In an example of a rotary mixer used in a working surface recovery process, the rotor housing 136 can function as a mixing chamber and can be operatively associated with other systems to receive water or other materials mixed with the fragmented debris. This rotary mixing can redeposit fragments and material on the working surface 102 as the cutting rotor 130 rotates within the rotor housing 136. To drive rotation, the cutting rotor 130 can be operatively coupled to the engine 120 via a mechanical arrangement, or it can be powered by a generator 124 or a hydraulic pump 126.

[0019] To accommodate the operator, the onboard operator console 140 can be supported at an elevated position on the frame 104 to provide visibility of the work area where the milling operation is performed. The operator console 140 may include various controllers, readouts, and other input / output interfaces for monitoring and controlling the operation of the rotary mixer 100. For example, a steering mechanism 142, such as a steering wheel or joystick, may be included in the operator console 140 to turn and change direction within the rotary mixer 100. Other operator controllers may include pedals or levers to adjust the speed and / or forward-reverse direction of the rotary mixer 100. The operator console 140 may also include operator controllers for adjusting and regulating the operation of the cutting rotor 130, including parameters such as rotor speed, rotor elevation relative to the work surface 102, and depth of cut into the work surface. For visual interaction with the operator, the operator console 140 may include one or more visual displays 144, such as liquid crystal displays or similar viewing devices. In other embodiments, the rotary mixer 100 may be configured for remote operation, and some or all of the aforementioned operator controller and other input / output interfaces may be located remotely from the onboard operator console.

[0020] Even if the operator's station 140 is located in an elevated position, the visibility of the rotary mixer 100 may be limited or obstructed by obstacles. For example, since the cutting rotor 130 is located within the rotor housing 136, the interaction between the cutting rotor and the working surface 102 is necessarily blocked. Additionally, if the rotary mixer 100 is configured for remote operation, the operator may not be located in a position where they have direct first-hand visual observation of the working surface 102 and the surrounding work area. To assist the operator during milling operations, the rotary mixer 100 can be operatively associated with a vision camera network 146, which includes multiple cameras 148 or image capture devices mounted to the frame 104 or another structure of the rotary mixer. The cameras 148 can have any suitable construction and can utilize any suitable photographic technique to capture visual images. The cameras 148 can have pan, zoom, tilt, and focus capabilities and can capture still images or video. In one embodiment, the camera 148 can be a digital camera utilizing an active pixel sensor embedded in semiconductor material, but in other embodiments, the camera 148 can capture still images.

[0021] Camera 148 can be configured to provide a line of sight or field of view to locations and areas that are not visually accessible to the operator. According to one aspect of this disclosure, the first camera 148 can be configured to provide a field of view along a first lateral side 112 of the frame 104, generally along the direction of the travel axis 110. For example, camera 148 can be located on the first lateral side 112 and can be mounted on a lifting column 118 at the rear end 108 of the frame 104, facing forward and downward. The arrangement of camera 148 such that the field of view, also referred to as the viewing angle, includes the sidewalls 138 of the rotor housing 136 and the working surface 102. The field of view refers to the dimensional and angular extent of the surrounding environment that camera 148 can capture as a visual image. The field of view can be determined by the configuration of camera 148 and lenses thereon, such as wide-angle or fisheye lenses, and can be adjusted via zoom, pan, and tilt controls. In the illustrated embodiment, the field of view is indicated by dashed lines 150. To capture a similar field of view on the second lateral side 114, the second camera 148 may be located on the second lateral side at the rear end 108 of the frame 104. In other embodiments, the camera 148 may be located on the rearward side of the front end 106 of the frame 104, or at other suitable locations on the frame or other structures.

[0022] Visual images captured by multiple cameras 148 of the visual camera network 146 can be displayed on a visual display 144 associated with the operator's console 140. The operator can use images of potentially obstructed locations on the rotary mixer 100 to monitor and adjust milling operations. In this embodiment, the visual images can be displayed in real time, allowing the operator to make timely adjustments as they observe the visual images during the milling operation. The visual display 144 may include a selector switch for switching between image feeds from the different cameras 148.

[0023] refer to Figure 2 This illustrates engagement of a cutting rotor 130, housed in a rotor housing 136, relative to a working surface 102 during a milling operation. The milling operation includes different relative lifting and lowering of the cutting rotor 130 and the working surface 102 relative to a vertical axis 200. Initially, the cutting rotor 130 may be vertically positioned above the working surface 102 in a disengaged position relative to the unmilled surface 202 of the working surface 102. To engage the working surface 102, the operator actuates a lifting mechanism operably associated with the rotary mixer 100 to lower the cutting rotor 130 toward the working surface. The lifting mechanism may include a telescopic lifting column 118 connecting a frame 104 to a traction device 116, which uses hydraulic pressure to adjust the lifting and lowering of the rotary mixer 100 relative to the vertical axis 200. In other embodiments, the cutting rotor 130 may be moved independently along the vertical axis 200 relative to the frame 104. Prior to engagement, the cutting rotor 130 may be activated such that it rotates relative to a rotor axis 134.

[0024] The cutting rotor 130 can be lowered vertically so that the cutting tool 132 begins to contact the working surface 102. This point, where the cutting tool 132 initially penetrates into the unmilled surface 202, can be referred to as the scoring point or contact point of the cutting rotor. As the cutting rotor 130 is further guided vertically into and penetrates the working surface 102, the cutting tool 132 will break and separate the material of the working surface 102, thereby forming the milled surface 204. The cutting rotor 130 can be lowered into the working surface 102 until the desired cutting depth 206 is achieved. The cutting depth 206 is the difference between the unmilled surface 202 and the milled surface 204. The milling operation guides the rotary mixer 100 relative to the travel axis 110 and thus guides the cutting rotor 130 to continue, so that the cutting rotor continues to engage the working surface 102 and remove material. The cutting rotor 130 can be held at the cutting depth 206 during the milling operation, thereby forming the desired milled surface 204.

[0025] As the cutting rotor 130 penetrates into the working surface 102, the cylindrical shape of the cutting rotor can create one or more lateral contact lines where the cutting rotor contacts the working surface. Due to the arrangement of the cutting rotor 130 and the working surface 102, the lateral contact lines can extend generally parallel to the rotor axis 134 and generally perpendicular to the travel axis 110. Numerous lateral contact lines may appear during and at different stages of the milling operation. For example, when the cutting rotor 130 initially contacts the working surface 102, a single lateral contact line may exist where the cutting rotor 130 intersects the unmilled surface 202. As the cutting rotor 130 further descends into the working surface 102, the cylindrical shape of the cutting rotor removes circular segments of material from the working surface 102. These circular segments can be characterized by cleavage or chords corresponding to the unmilled surface 202 and a cutting depth 206 corresponding to the depth of that segment.

[0026] like Figure 2As shown, the milling section can produce a first lateral contact line 210 and a second lateral contact line 212, wherein the curved shape of the cutting rotor 130 intersects the unmilled surface 202 of the working surface 102. In an embodiment, the lateral contact lines 210, 212 can be estimated when the cutting rotor 130 has reached the cutting depth 206. The first lateral contact line 210 can be associated with the front end 106 of the rotary mixer 100 and considered as a front lateral contact line, and the second lateral contact line 212 can be associated with the rear end 108 of the rotary mixer 100 and considered as a rear lateral contact line. In other embodiments, the first and second lateral contact lines can be estimated at other locations where the cutting rotor 130 and the working surface 102 engage or intersect at other locations along the milled surface 204; for example, a third lateral contact line 214 can be considered as corresponding to a location on the milled surface 204 adjacent to the cutting depth 206.

[0027] Because the cutting rotor 130 is housed within a rotor housing 136 and its visibility is obstructed by the housing, the operator cannot view the first lateral contact line 210 and the second lateral contact line 212 during milling operations. Therefore, to visually assist the milling operation, the rotary mixer 100 can be operatively associated with an image enhancement system 300, which works in conjunction with a camera network 146 to generate an enhanced image of the cutting rotor 130 relative to the working surface 102. (Reference) Figure 3 The image enhancement system 300 can be implemented by an electronic controller 302, sometimes referred to as an electronic control module (ECM) or electronic control unit (ECU). The electronic controller 302 can be configured to perform image processing on the visual images acquired by the camera 148 using algorithms and computer-executed operations.

[0028] To process electronic data and execute instructions, the electronic controller 302 may include one or more microprocessors 304 or similar circuitry, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays. As explained below, the microprocessor 304 may include or be programmed to perform specific logical functions and may be configured with or associated with appropriate circuitry for such operations. The microprocessor 304 may be programmable to read and / or execute functions, steps, routines, data tables, etc., associated with the image enhancement system 300. For example, in one embodiment, the microprocessor 304 may include a specially programmed arithmetic logic unit 306 or specific circuitry for performing mathematical operations and kinematic equations to facilitate the image enhancement system 300; however, in other embodiments, the microprocessor may be a general-purpose CPU.

[0029] To store software instructions containing the image enhancement system 300, the electronic controller 302 may include system memory 308 or a similar data storage device. In various aspects, system memory 308 may be readable, writable, or a combination thereof. To facilitate the image enhancement system 300, system memory 308 may include electronically storable data (e.g., machine dimension data 310) relating to the geometry of the rotary mixer 100 and the cutting rotor 130, such as the diameter of the cutting rotor. System memory 308 may also store electronic data relating to the positioning and location of the plurality of cameras 148 (e.g., camera data 312), including the angular extension of the field of view. System memory 308 may communicate with microprocessor 304 via bus 314.

[0030] To communicate with the multiple cameras 148 associated with the camera network 146, the electronic controller 302 may include a video module or graphics unit 316, such as a video card specifically configured to receive, transmit, and / or process video and graphic images. The graphics unit 316 may include processing and data storage capabilities dedicated to processing video-specific data and may act as a communication node between the electronic controller 302 and the cameras 148 and video display 144 on the rotor mixer 100. The graphics unit 316 may communicate with the microprocessor 304 via bus 314, but in other embodiments, the functionality of the graphics unit may be integrated with the microprocessor. The graphics unit 316 can send and receive electronic data signals in computer-processable bit and byte form using the cameras 148 and the video display 144.

[0031] To obtain data regarding the milling operation, the electronic controller 302 may include an operator input / output (I / O) interface 318 that communicates with an operator controller (e.g., steering mechanism 142). Thus, the image enhancement system 300 can be notified of the direction of travel of the rotary mixer 100 relative to the travel axis 110 during the milling operation. The operator I / O interface 318 may communicate with other operator controllers, including controllers associated with, for example, the cutting rotor (e.g., rotor speed) and the lifting mechanism (e.g., cutting depth).

[0032] To obtain additional data about machine operation, the electronic controller 302 may include a system input / output (I / O) interface 320 for communicating with system settings and processes. For example, the system I / O interface 320 may communicate with sensors and controllers associated with one or more hydraulic actuators 322 of a lifting mechanism used to adjust the vertical movement of the rotary mixer 100 relative to the working surface 102. To obtain additional information about milling operations, such as the relative positions and movements of the mechanisms and components of the rotary mixer 100, the system I / O 320 may communicate with a light-sensitive visual image sensor 324 and limit switches 326 or position sensors sensitive to the relative positions of different movable elements. To communicate with one or more off-board systems, the system I / O interface 320 may be associated with a transceiver 328 for transmitting and receiving radio signals.

[0033] refer to Figure 4 An exemplary process 400 is illustrated, which can be performed by an image enhancement system 300 to generate an enhanced visual image to assist milling operations. The process 400 depicted in the flowchart for accomplishing these tasks may include a series of steps or instructions implemented as non-transitory computer-executable software code in the form of an application or program. Process 400 may begin with an image capture step 402, where a visual image 404 is captured by one or more cameras 148 associated with a camera network 146. In an embodiment, the visual image 404 may be captured individually by a first camera 148 associated with a first lateral side 112 of the rotary mixer 100 and a second camera 148 associated with a second lateral side 114 of the rotary mixer 100. Since the field of view of the camera 148 is oriented to include the rotor housing 136 (including the housing sidewall 138) and the working surface 102, both elements are included in the visual image 404. The visual image 404 may be captured in video format and processed as a real-time or on-site feed.

[0034] As described above, depending on the cutting depth and other factors, numerous lateral contact lines 210, 212 can be associated with a specific cut obtained during the milling operation. To enable the operator to customize how the visual image 404 will be displayed, in one embodiment, the process 400 may include an operator setting step 405, where the operator can input preferences regarding how the image enhancement system 300 will generate and display enhanced images of the milling operation. For example, the operator setting step 405 may receive input regarding the number and / or arrangement of lateral contact lines of interest, such as the number of lateral contact lines and the location of their contact points between the cylindrical surface of the cutting rotor and the milled surface of the working surface, the color and thickness of the reference line enhancement, etc.

[0035] In the online positioning step 406, the image enhancement system 300 may determine one or more lateral contact lines 210, 212, wherein the cutting rotor 130 physically contacts the working surface 102. The line positioning step 406 may be calculated based on various variables and parameters associated with the milling operation to determine the lateral contact lines 210, 212. For example, the line positioning step 406 may receive electronic data representing the vertical rotor rise 408 relative to the working surface 102 from a lift sensor operatively associated with a lifting mechanism. The lifting mechanism may be a lifting column 118 that connects the frame 104 to the traction device 116, or it may be a separate mechanism that vertically adjusts the cutting rotor 130 relative to the frame 104. The rotor rise 408 may correspond to a fixed or desired cutting depth 206 indicating that the cutting rotor 130 penetrates into the working surface 102. The line positioning step 406 may also receive geometric machine dimensions, including, for example, the rotor diameter and the relative height of the frame relative to the working surface, from machine dimension data 310 stored in the system memory 308 of the electronic controller 302. By applying the kinematic equations to these variables, the line position determination step 406 can calculate and resolve the intersection of the cutting rotor 130 and the working surface 102 corresponding to the first lateral contact line 210 and the second lateral contact line 212.

[0036] In generation step 410, process 400 may generate reference line enhancement 412, which may be one or more animated lines intended to represent the first lateral contact line 210 and / or the second lateral contact line 212. Reference line enhancement 412 may be a data file including the positions of the lateral contact lines as determined in line position determination step 406. In the subsequent overlay step 414, process 400 may overlay or superimpose the reference line enhancement 412 onto the visual image 404 obtained in image capture step 402. Overlay step 414 produces one or more enhanced images 416, wherein the reference line enhancement 412 is overlaid on the visual image 404. In an embodiment, enhanced images 416 may be generated separately for the visual image 404 associated with the first lateral side 112 of the rotary mixer 100 and the visual image 404 associated with the second lateral side 114 of the rotary mixer 100. The same reference line enhancement 412, which may include multiple lines related to each other, may be overlaid on two visual images 404 to generate enhanced images.

[0037] The overlay step 414 can be performed by an electronic controller 302 associated with the image enhancement system 300, or in an embodiment, the combination of the captured visual image 404 and the reference line enhancement 412 can be performed by the visual display 144. For example, the visual image 404 can be simply a real-time video feed transmitted to the visual display 144, which can receive the reference line enhancement 412 separately and overlay it onto the visual image 404.

[0038] To determine where the reference line enhancement 412 is integrated dimensionally into the visual image 404, the overlay step 414 may receive camera data 312 stored in the system memory 308 of the electronic controller 302. The overlay step 414 may calculate the relative position of the reference line enhancement 412 on the visual image 404 from the camera position and / or field of view information included in the camera data 312. The functionality of the overlay step 414 may be implemented by the arithmetic logic unit 306 of the microprocessor 304 or by the graphics unit 316.

[0039] Process 400 may include a display step 418, wherein an enhanced image 416 is displayed on a vision display 144 associated with the image enhancement system 300. For example, the graphics unit 316 may transmit data corresponding to the enhanced image 416 as a digital image file to the vision display 144. In other embodiments, the vision image 404 and the reference line enhancement 412 may be sent separately to the vision display 144, which may combine and display the enhanced image 416 including the reference line enhancement 412 to assist the operator during milling operations by indicating the position of the first and / or second lateral contact lines 210, 212 associated with the cutting rotor 130 and the working surface 102. The enhanced image 416 enables the operator to visually perceive the interface between the cutting rotor 130 and the working surface 102 that would otherwise be blocked by the rotor housing 136. In embodiments, process 400 may occur substantially in real time, thus presenting the operator with an enhanced image showing the current position and relationship of the cutting rotor 130 and the working surface 102.

[0040] In an embodiment, process 400 can be configured to adapt to or modify various adjustments during the milling operation. For example, it may be necessary to adjust the cutting depth, i.e., increase or decrease the penetration of the cutting rotor 130 into the working surface 102. Process 400 may include a cutting depth query 420 that detects adjustments to a commanded cutting depth, for example, input via an operator controller or indicated by a predetermined digital milling plan. If the cutting depth query 420 detects a command to adjust the cutting depth, process 400 may return to line position determination step 406 to reacquire data regarding rotor lifting and recalculate the first lateral contact line 210 and the second lateral contact line 212. Process 400 may then continue with subsequent steps to generate and display a new enhanced image.

[0041] In an embodiment, process 400 allows an operator to manually adjust the enhanced image 416. For example, in operator adjustment query 422, process 400 can monitor and adjust the operator's adjustments to the enhanced image. In an embodiment, the operator can adjust the position of the reference line enhancement 412 applied to the visual image 404 to account for specific situations. The operator can also use the pan, zoom, and tilt controllers to adjust the focus or field of view of the camera 148. If operator adjustment query 422 detects an operator-commanded adjustment, process 400 can return to the line position determination step 406 to reacquire any new data and perform any necessary recalculations, and then continue with subsequent steps to generate and display the new enhanced image 416. Process 400 can end with a milling operation 424, in which a rotary mixer 100 or a similar machine mills the working surface 102 to break up the material on the working surface 102.

[0042] Industrial applicability

[0043] refer to Figure 5 Furthermore, according to the preceding figures, a visual display 144 associated with the rotary mixer 100 according to this disclosure is shown, which presents enhanced images in graphic form. The visual display 144 may be a liquid crystal display or similar flat panel technology, communicating with an electronic controller 302 associated with the image enhancement system 300 via an Ethernet connection, a video graphics display (VGA) connector, or a conductive connector and port similar to those for transmitting visual images enhanced by the image enhancement system 300. In other possible embodiments, the visual display 144 may be a cathode ray tube (CRT). To present enhanced images associated with both a first camera 148 associated with a first lateral side 112 and a second camera 148 associated with a second lateral side 114, the visual display 144 may have a dual-screen configuration and may include a first display 500 and a second display 502 arranged side-by-side. The first display 500 and the second display 502 may be different, independently operable screen units with separate housings. In another embodiment, the visual display 144 may have a split-screen configuration, wherein the first display 500 and the second display 502 are associated with separate but adjacent areas on the same display unit. In various embodiments, the visual display 144 may include a display screen with different arrangements and numbers depending on the number of cameras 148 on the rotary mixer 100.

[0044] Due to the orientation and arrangement of the fields of view 150 of the first and second cameras 148, the visual image 404 captured during the image capture step 402 of process 400 may include the rotary mixer 100 (including a rotor housing 136 with housing sidewalls 138) and a portion of the working surface 102. The captured visual image 404 may be oriented forward and downward relative to the travel axis 110 to capture the front end 106 of the rotary mixer 100, but in other embodiments, the field of view of the cameras 148 may be oriented rearward to capture the rear end of the rotary mixer. In the illustrated embodiment, the working surface 102 may include paved road 510, raised curbs 512 defining the edges of the paved road 510, storm drains 514, and other common road features.

[0045] The field of view 150 and the focal points of the first and second visual images 404 captured by the first and second cameras 148, respectively, can extend substantially together, such that the visual images are aligned relative to the axis of travel 110 when displayed on the first display screen 500 and the second display screen 502. When presented side-by-side on the first display screen 500 and the second display screen 502, the first and second visual images provide a consistent view of the first lateral side 112 and the second lateral side 114 of the rotary mixer 100. To indicate the position of the cutting rotor 130 and its engagement with the working surface 102, the first enhanced image 416 on the first display screen 500 and the second enhanced image 416 on the second display screen 502 each include a reference line enhancement 412, which may include one or more reference lines corresponding to the lateral contact lines 210, 212 between the cutting rotor and the working surface. Reference line enhancements 412 on the first display screen 500 and the second display screen 502 are parallel to the rotor axis 134 and can be aligned with each other (e.g., laterally across the visual display 144) to provide continuous indication between the screens of the lateral contact lines 210, 212 of the cutting rotor 130 relative to the working surface 102. In this embodiment, the reference line enhancements 412 are indicated as dashed lines, which may, for example, highlight the lateral contact surface; however, in other embodiments, the reference line enhancements may be indicated in other ways.

[0046] As described herein, the number and arrangement of the lateral contact lines 210, 212 can vary for different cutting depths 206. Therefore, the reference line reinforcement 412 can take different forms depending on the milling operation and the operator selection described above. For example, when the plane, slope, or gradient of the rotary mixer 100 changes such that the first lateral side 112 and the second lateral side 114 do not have equal vertical heights, or when the shape of the working surface is uneven, the reference line reinforcement 412 can be skewed toward or away from each other. In this case, the lateral contact lines and corresponding reference line reinforcements may not be perfectly parallel to the rotor axis or perfectly perpendicular to the travel axis.

[0047] In this embodiment, the visual display 144 may have touchscreen capability or may be associated with other dials, knobs, or controllers to allow the operator to adjust the reference line enhancement and indication of the lateral contact lines 210, 212. For example, the operator may use input gestures to change the lateral contact lines 210, 212 relative to the curved circumferential surface of the cutting rotor 130 to customize the enhanced image 416 and the information presented therein. Additionally, the operator may be able to adjust the field of view of the camera 148, for example, by using a pan-zoom-tilt controller. In this case, process 400 can identify such adjustments and changes via a cut depth query 420 and / or an operator adjustment query 422, and the positions of the lateral contact lines 210, 212 can be recalculated, and the enhanced image 416 can be regenerated.

[0048] It should be understood that the foregoing description provides examples of the disclosed systems and techniques. However, it is conceivable that other embodiments of this disclosure may differ in detail from the foregoing examples. All references to this disclosure or its examples are intended to refer to the specific examples discussed at the time and are not intended to imply any limitation on the scope of this disclosure in a more general sense. All distinctions and adverse statements regarding certain features are intended to indicate that such features are not preferred, but are not intended to completely exclude such features from the scope of the invention unless otherwise specified.

[0049] Unless otherwise indicated herein, the descriptions of value ranges herein are intended solely as a shorthand for referring to each independent value falling within the range, and each independent value is incorporated into the specification as if described separately herein. All methods described herein may be performed in any suitable order unless otherwise indicated herein or clearly contradicted by the context.

[0050] In the context of describing the invention (especially in the context of the following claims), the terms “a,” “an,” “the,” and “at least one,” and similar pronouns, should be interpreted to cover both the singular and the plural, unless the context otherwise indicates or explicitly contradicts it. The use of the term “at least one” followed by a list of one or more items (e.g., “at least one of A and B”) should be interpreted to mean one item (A or B) selected from the listed items, or any combination of two or more of the listed items (A and B), unless the context otherwise indicates or explicitly contradicts it.

[0051] Therefore, as permitted by applicable law, this disclosure includes all modifications and equivalents to the subject matter set forth in the appended claims. Furthermore, unless otherwise indicated herein or clearly contradicted by the context, this disclosure covers any combination of the foregoing elements in all possible variations.

Claims

1. A milling machine comprising: a frame supported on a plurality of traction devices to travel along a work surface relative to a travel axis, the frame defining a first lateral side, and a second display screen of the milling machine is parallel to the travel axis of the milling machine; a cutting rotor rotatably supported on the frame for milling a work surface, the cutting rotor shaped as a cylindrical drum defining a rotor axis perpendicular to the travel axis; a rotor shroud supported on the frame to house the cutting rotor, the rotor shroud including a first shroud sidewall aligned with the first lateral side and a second shroud sidewall aligned with the second display screen; a camera supported on the frame at the first lateral side in a position to capture a visual image of the rotor shroud and the work surface; and an electronic controller programmed to receive the visual image from the camera; receive electronic data representative of vertical rotor lift relative to a work surface; receive geometric machine dimensions including rotor diameter; use the electronic data and machine dimension data and apply kinematic equations to determine a position of one or more lateral contact lines of the cutting rotor relative to the work surface, the lateral contact lines generally perpendicular to the travel axis and generally parallel to the rotor axis; generate a reference line augmentation corresponding to the lateral contact lines; and overlay the reference line augmentation on the visual image to produce an augmented image; and, the milling machine further comprising a visual display operatively associated with the electronic controller to display the augmented image, wherein the visual display allows an operator to adjust the reference line augmentation and the indicated lateral contact lines, wherein the electronic controller is capable of performing a cut depth query that is capable of detecting adjustments to a commanded cut depth, and, wherein the electronic controller is capable of identifying the adjustments through the cut depth query and / or the operator adjustment query, and is capable of recalculating the position of the lateral contact lines and regenerating the augmented image.

2. The milling machine of claim 1, wherein the lateral contact lines include a front lateral contact line oriented toward a front end of the frame and a rear lateral contact line oriented toward a rear end of the frame.

3. The milling machine of claim 2, comprising a second camera supported on the frame at the second display screen in a position to capture a second visual image of the rotor shroud and the work surface.

4. The milling machine of claim 3, wherein the electronic controller is configured to receive the second visual image from the second camera and overlay the reference line augmentation on the second visual image to generate a second augmented image.

5. The milling machine of claim 4, wherein the visual display includes a second display screen and the second augmented image is presented on the second display screen. ​ 6. The milling machine of claim 1, comprising a lift mechanism for adjusting the lift of the cutting rotor vertically relative to the working surface and a lift sensor for sensing the rotor lift relative to the working surface, and the electronic controller is configured to receive the rotor lift from the lift sensor and adjust the lateral contact line based on the rotor lift.

7. The milling machine of claim 1, wherein the electronic controller is programmed to enable selection of the number and arrangement of reference line enhancements for generating the lateral contact line.

8. The milling machine of claim 1, wherein the visual image is a video.

9. A method of operating a milling machine, comprising: capturing a visual image of a lateral side of a rotor shroud of the milling machine relative to a working surface to be milled by a cutting rotor housed in the rotor shroud; receiving electronic data representing vertical rotor lift relative to the working surface; receiving geometric machine dimensions including rotor diameter; using the electronic data and machine dimension data and applying kinematic equations to determine one or more lateral contact lines at which the cutting rotor will contact the working surface; generating a reference line enhancement corresponding to the lateral contact line; superimposing the reference line enhancement on the visual image to produce an enhanced image; and displaying the enhanced image on a visual display associated with the milling machine; wherein the visual display allows an operator to adjust the reference line enhancement and the indicated lateral contact line; and, the method includes a cut depth query that detects adjustments to a commanded cut depth; and the method further includes identifying the adjustments through the cut depth query and / or operator adjustment query and recalculating the position of the lateral contact line and regenerating the enhanced image.

Images