Method and system for configuring a machine control unit of a construction machine

Abstract

The present invention relates to a method and system for configuring a machine control unit of a construction machine. A system (70) and method for configuring a machine control unit (75) of a construction machine to perform an earthwork operation, the earthwork operation comprising a plurality of phases to be performed consecutively, the method comprising the following steps performed by a computing unit (71): receiving three-dimensional measurement data of a terrain in at least a first detection range around the construction machine; detecting, based on the three-dimensional measurement data, an element of a preceding phase of the earthwork operation; and configuring the machine control unit to at least partially automatically perform a next phase of the earthwork operation based at least on the detected element.

Description

Technical Field

[0001] This invention generally relates to a method and system for configuring a mechanical control unit for construction machinery to control earth-moving operations comprising multiple phases to be performed consecutively, such as controlling a motor grader or crawler used for ditch or road construction. Control includes using 3D sensors to detect the current state of the earth-moving operation in order to configure the mechanical control unit to at least partially control the construction machinery during the next phase of the earth-moving work. Background Technology

[0002] It is known that automated systems using construction machinery facilitate earthmoving operations by fully or partially controlling the machinery. Existing automated systems primarily focus on the final grading passes required to cut the soil surface into the desired shape. There are two distinct main applications based on whether the desired shape is positioned relative to world coordinates (“3D system”) or relative to vehicle coordinates (“2D system”). 2D systems are simpler to use. The soil shape is designed relative to the vehicle system, and the operator is entirely responsible for navigating the vehicle to ensure the system generates the correct surface. 3D systems require surface design files created using sophisticated computer systems. The machinery can then be positioned relative to the 3D plan view and cut the surface correctly using a variety of possible navigational movements.

[0003] Various measurement systems are known for their application to specific needs in the construction industry, particularly for earthmoving machinery such as excavators, graders, or bulldozers. These systems can be used with construction machinery to create digital 3D models of the machinery's surroundings, detect obstacles, and / or support control of the machinery. For example, EP3086196 A1 suggests using a camera system mounted on a bulldozer for environmental monitoring. This camera system uses SLAM or SfM algorithms to generate a 3D point cloud of the environment. JP 2019167719A2 discloses a 2D laser scanner used to generate a 3D model of the environment when an excavator equipped with the 2D laser scanner is pivoting. WO 2019 / 197064 A1 illustrates an apparatus for monitoring the environment of construction machinery using a ToF camera. The images generated in this way are used to guide the operator of the construction machinery to control its operation at the construction site based on a plan.

[0004] Advances in sensing sensors enable new interactive workflows that allow for the generation of 3D design surfaces in a simpler way than existing 2D systems. These same sensing sensors allow machinery to position itself relative to the coordinates of a design document and enable advancements in mechanical automation, allowing automated systems to be used under a wider range of operator conditions. WO 2020 / 126123 A2 discloses an example of a compact “reality capture device” comprising a laser scanner and at least one camera. Using this reality capture device, the environment can be optically scanned and measured by means of a laser scanner (e.g., using pulsed electromagnetic radiation) that emits a laser measurement beam, wherein echoes from backscattered surface points in the environment are received, and the distance to these surface points is derived and correlated with the angular emission direction of the associated laser measurement beam. This generates a 3D point cloud. For example, distance measurements can be based on the time-of-flight, shape, and / or phase of the pulse. For additional information, such as by means of an RGB or infrared camera, the laser scanner data can be combined with the camera data, particularly providing high-resolution spectral information. Reality capture devices can be mobile and configured to simultaneously provide survey and reference data. For example, at least the device's trajectory data (e.g., position and / or attitude data) is provided together with probe data (e.g., laser scanner data and / or camera data), allowing probe data from different locations of the reality capture device to be combined into a common coordinate system. Reality capture devices can be configured, for example, to create a new 3D map of the environment from the host using simultaneous localization and mapping (SLAM) capabilities. European patent application No. 21176387.5 discloses a similar reality capture device that includes an arrangement of time-of-flight (ToF) cameras instead of a laser scanner, wherein the ToF cameras are configured to jointly capture surrounding 3D point cloud data.

[0005] EP 3865895 A1 discloses a measurement system for construction machinery, which includes one or more real-world capture devices for capturing the area around the construction machinery before and during earthwork at a construction site.

[0006] Some earthmoving operations involve multiple phases to be performed consecutively, where each phase is at least partially iterated. An example of such an operation is "V-grooving," used for constructing trenches with a V-shaped profile. During successive passes of the construction machinery, the tool is continuously lowered to perform several cuts.

[0007] The aim is to provide a site surveying system for construction machinery that facilitates the operation of earthmoving work by the machinery operators, particularly by allowing a higher level of automation in the control of the machinery. Summary of the Invention

[0008] Therefore, the object of the present invention is to provide an improved system and method for configuring the mechanical control unit of construction machinery to perform earthmoving operations.

[0009] Another object of the present invention is to provide a system and method that allows for the configuration of a mechanical control unit to perform earthmoving operations at least partially automatically.

[0010] Another object of the present invention is to provide a system and method that allows for the configuration of a mechanical control unit to perform earthmoving operations at least partially automatically.

[0011] Another object of the present invention is to provide a system and method that allows a mechanical control unit to be configured to perform at least a subset of stages of an earthmoving operation, the earthmoving operation comprising multiple stages to be performed sequentially.

[0012] A first aspect of the invention relates to a system for configuring a mechanical control unit of construction machinery to perform earthmoving operations, the earthmoving operations comprising multiple stages to be performed consecutively. The system includes: a measurement system configured to capture three-dimensional measurement data of the terrain surrounding the construction machinery within at least a first detection range; a user interface configured to receive user input from an operator of the construction machinery; and a computing unit operatively coupled at least to the measurement system, the user interface, and the mechanical control unit. According to this aspect of the invention, the computing unit is configured to: detect elements of a previous stage of the earthmoving operation based on the three-dimensional measurement data; and configure the mechanical control unit to at least partially automate the next stage of the earthmoving operation based at least on the detected elements.

[0013] According to some embodiments of the system, the measurement system includes at least one measurement unit at the construction machinery, each measurement unit being configured to capture 3D point cloud data, i.e., as three-dimensional measurement data. Each measurement unit includes:

[0014] -At least one laser scanner

[0015] - Multiple ToF cameras,

[0016] - Millimeter-wave radar system, and / or

[0017] - One or more stereo camera systems.

[0018] According to some embodiments, the system includes a context camera having a known position relative to the measurement system and / or the three-dimensional measurement data, and configured to capture background image data of the terrain within the first detection range. In this case, the user interface is configured to display at least one background image to the operator of the construction machinery based on the background image data, the computing unit is configured to overlay detected elements onto the displayed background image, and to receive input from the operator related to the next stage of the earthmoving operation; and to configure the machinery control unit to also receive input from the operator. Optionally, the user interface may include a touch-sensitive display on which the background image is displayed and on which the user input is received. For example, the input from the operator may include the selection of one or more detected elements.

[0019] According to some embodiments, the construction machinery is an electric grader, and the earthmoving operation includes the construction of a ditch, wherein the stage is a pass of the electric grader along the ditch line, the pass comprising multiple cutting passes; and the detected elements include edges and / or surfaces produced by the previous cutting pass. For example, the earthmoving operation may include V-shaped trenching. Optionally, the input from the operator may include one or more of the following:

[0020] - Select the edges and / or surfaces generated by the previous cutting pass;

[0021] - Select the cutting depth for the next pass; and / or

[0022] - Choose to maintain parallel cuts for the next pass.

[0023] In one implementation, the input from the operator includes selecting the edges and surfaces generated by the previous cutting pass, and selecting to adjust the cross slope of the surface for the next pass, while keeping the selected edges as vertices of the plane to be shaped during the next pass.

[0024] According to some implementations of the system, the computing unit is configured to:

[0025] - Receive operator input indicating the desired line (e.g., spline) along which the earthmoving operation is expected to be performed;

[0026] - Locate the construction machinery based on the three-dimensional measurement data (e.g., relative to a desired line);

[0027] Position the tool at the desired line, and

[0028] - The first of the multiple stages is executed to produce detectable elements.

[0029] Optionally, the computing unit can also be configured to guide the construction machinery to the desired line.

[0030] In some implementations, the operator input includes the offset of the desired line relative to a given line, and the calculation unit is configured to calculate the desired line based on the offset. For example, if the given line is a spline, the calculation unit may be configured to calculate another spline as the desired line based on the offset. In one implementation, the given line may be a line belonging to an existing ditch on a first side of the road, and the desired line is another ditch to be constructed on the other side of the road, for example, having the same distance from the road as the ditch on the first side.

[0031] According to some embodiments of the system, the measurement system is configured to continuously capture 3D measurement data and continuously provide the captured 3D measurement data to the computing unit, wherein the computing unit is configured to continuously evaluate the received 3D measurement data to continuously detect the elements, and to continuously configure the mechanical control unit while at least partially automating the earthwork operation phase.

[0032] A second aspect of the invention relates to a construction machine, such as a grader, bulldozer, or excavator, comprising:

[0033] - Tools used to perform earthmoving operations, such as digging ditches or trenches;

[0034] - A control unit, the control unit being used to at least partially control the earthmoving operation; and

[0035] - According to the first aspect of the invention, the system is operatively coupled to or includes the control unit.

[0036] A third aspect of the invention relates to a computer-implemented method for configuring a machinery control unit for performing earthmoving operations (e.g., construction machinery according to a second aspect of the invention), the earthmoving operations comprising multiple stages to be performed sequentially. The method includes the following steps performed by a computing unit (e.g., a computing unit of the system according to a first aspect of the invention):

[0037] - Receive 3D measurement data of the terrain around the construction machinery within at least a first detection range;

[0038] - Detecting elements from the previous stage of the earthwork operation based on the 3D measurement data; and

[0039] - Configure the mechanical control unit to at least partially automate the next stage of the earthmoving operation based on the detected elements.

[0040] According to some implementations, the method further includes:

[0041] - Use a background camera with a known position relative to the 3D measurement data and / or relative to the measurement system that captures the 3D measurement data to capture background image data of the terrain within the first detection range;

[0042] - Display at least one background image to the operator of the construction machinery based on the background image data, and overlay the detected elements onto the displayed background image; and

[0043] - Receive input from the operator relating to the next stage of the earthmoving operation, in particular, wherein the input from the operator includes the selection of one or more detected elements, wherein the mechanical control unit is also configured based on the input from the operator.

[0044] According to some embodiments of the method, the construction machinery is an electric grader, and the earthmoving operation includes the construction of ditches, for example, by V-shaped trenching. In this case, the stage is a pass of the electric grader along the ditch line, the pass comprising multiple cutting passes; and the detected elements include edges and / or surfaces produced by the previous cutting pass. Optionally, the input from the operator includes one or more of the following:

[0045] - Select the edges and / or surfaces generated by the previous cutting pass;

[0046] - Choose to maintain parallel cuts in the next pass; and / or

[0047] - Select the cutting depth to use for the next pass.

[0048] In one implementation, the input from the operator includes: selecting the edges and surfaces generated by the previous cutting pass, and selecting to adjust the lateral slope of the surfaces for the next pass, while keeping the selected edges as vertices of the plane to be shaped during the next pass.

[0049] According to some implementations, the method further includes:

[0050] - Receive operator input indicating the desired line (e.g., spline) along which the earthmoving operation is to be performed;

[0051] - Locate the construction machinery based on the three-dimensional measurement data (especially relative to the desired line);

[0052] Position the tool at the desired line, and

[0053] - The first of the multiple stages is executed to produce a detectable element.

[0054] Optionally, the method may further include automatically guiding the construction machinery to the desired line. In one embodiment, the operator input includes the offset of the desired line relative to a given line, and the method includes calculating the desired line based on the offset.

[0055] According to some embodiments of the method, the 3D measurement data is continuously captured and continuously evaluated to detect the element, and the mechanical control unit is continuously configured while at least partially automating the earthwork operation.

[0056] According to some embodiments of the method, the earthwork operation includes the construction of a ditch, the desired line being a ditch line, such as a ditch edge line; the first stage is a marking pass along the ditch line, and the elements generated by the marking pass include edges and / or surfaces. Optionally, the construction machinery may be an electric grader; the earthwork operation may include V-growing; and / or the ditch line may be a ditch edge line, a ditch spline, or a ditch edge spline.

[0057] The fourth aspect of the invention relates to a computer program product comprising program code stored on a machine-readable medium or embodied by electromagnetic waves including segments of program code, and having computer-executable instructions that, when executed, particularly on a computing unit of a system according to the first aspect of the invention, perform the method according to the third aspect of the invention. Attached Figure Description

[0058] Preferred embodiments of the invention will be described more fully below with reference to the accompanying drawings, by way of example only, wherein:

[0059] Figures 1a to 1b Two exemplary embodiments of known construction machinery for performing earthmoving operations are shown;

[0060] Figure 2 An exemplary method for constructing ditches using an electric grader is illustrated;

[0061] Figure 3a and Figure 3bAn electric grader is shown as an example of construction machinery, which includes an exemplary embodiment of a construction site surveying system according to the present invention;

[0062] Figure 4 An exemplary embodiment of the system according to the present invention is shown;

[0063] Figure 5a and Figure 5b The road design includes ditches; and

[0064] Figure 6 A flowchart illustrating the steps of an exemplary embodiment of the method according to the present invention is shown. Detailed Implementation

[0065] Figure 1a and Figure 1b Two examples of known construction machinery 1 that can be equipped with the system according to the present invention are shown. Figure 1a An electric grader tool 11 is shown for digging a V-shaped trench. This is an iterative process in which the tool is continuously lowered to perform several cuts. These cuts include an initial shallow cut as a marker cut 31 and multiple subsequent deeper cuts including at least a first cut 32 and a final cut 33. Figure 1b A tracked machine (also known as a bulldozer or excavator) is shown, having a blade 11 as a tool 11 for performing earthmoving works, for example, during road construction. It is known that such construction machinery 1 is equipped with an automation system to facilitate the operator's earthmoving work by at least partially controlling the machinery during these operations.

[0066] In the construction machinery 1 shown here, namely graders and tracked machines, current automation systems typically include a slope control system that can be configured to maintain a desired lateral slope. This is a simple function that operators can use during road and ditch construction. Typical lateral slope systems control only one axis of tool movement. They act as kinematic constraints, requiring the operator to control the depth of tool 11, navigate machinery 1, and control other tool movements, such as those associated with material management.

[0067] Figure 2 Examples show the use of, for example Figure 1a The illustration shows an exemplary method for digging V-shaped trenches (“V-ditching”) with an electric grader. This method is known in itself. Currently, since the control of the grader is primarily performed manually, performing this method requires significant attention from the grader operator. Known lateral slope systems can control a single actuator among five actuators used for blade movement. Disadvantageously, this requires the operator to manage several aspects of blade control while also navigating the vehicle.

[0068] The operator of the electric grader cuts shallow markings for marking ditch lines (see also...) Figure 1a The process begins. The operator can then engage a known 2D lateral slope system, setting it to the desired ditch slope, and continue with several subsequent cutting passes. During these passes, the operator manages the load on the vehicle by adjusting the cutting depth and the angle of the blade. The blade angle is also used to move the material laterally along the blade and into the windrow. After several cutting passes, a significant amount of windrow material has been created. The operator then disengages the lateral slope system and works to first move the windrow material and then thin it out. Additional passes and windrow unfolding operations are performed until one surface of the ditch is complete. The operator can re-engage the 2D lateral slope system while cutting the bank. During these passes, soil is pushed onto the previously formed surface. Once the embankment has been formed, the operator again uses the 2D lateral slope system to clear material from the ditch. The lateral slope system is adjusted for the final surface cut. The windrow material is then unfolded and shaped to the desired height and slope.

[0069] Figure 3a and Figure 3b An electric grader is shown as an example of construction machinery 1, which includes an exemplary embodiment of a system for configuring a mechanical control unit for the construction machinery. The system includes a construction site measurement system with sensing sensors that can be included in the execution of earthmoving operations, thereby enabling a dramatic expansion of the automation used. The disclosed methods and systems provide an extension to existing 2D systems on construction machinery such as tracked machines and electric graders. This is based on the addition of sensing sensors that enhance the environmental context lacking in existing 2D systems.

[0070] like Figure 3a As shown, construction machinery 1 includes tools (shovel 11) for performing earthmoving operations. Although the machinery shown is an electric grader, it can also be any other construction machinery that can be used for earthmoving operations, such as an excavator or bulldozer. A surveying unit 2 is mounted on the cab 14 of the grader 1 as part of a construction site surveying system.

[0071] like Figure 3b As shown in more detail, the measurement unit 2 includes one or more reality capture devices (RCDs) 3a, 3b to capture three-dimensional (3D) measurement data of the surrounding (typically uneven) terrain. 3D measurement data, such as or including 3D point clouds, is captured in at least one capture direction. Preferably, this includes a direction toward the front of the construction machinery and typically at least partially matches the operator's viewing direction while operating the machinery.

[0072] In the context of this invention, RCDs specifically include sensor arrangements that capture 3D point clouds of the environment using non-contact methods. Typically, this involves determining the distance to points in the environment using time-of-flight (ToF) measurements based on light waves (e.g., light waves in the infrared spectrum), radio waves, or ultrasound. However, stereo vision sensors can provide point cloud measurements of the environment using parallax or differences in data location between left and right images. For example, each RCD may include an arrangement of at least two cameras and optional other sensors, such as laser scanners or ToF cameras. Reality capture devices 3a, 3b of one or more measurement units 2 can be configured to facilitate the generation of the same 3D point cloud of terrain for detecting obstacles and / or tracking people or animals entering hazardous areas around machinery. Measurement units 2 can be mounted on various parts of the construction machinery, such as on the tool 11, chassis 12, and / or cab 14.

[0073] The system for configuring the mechanical control unit includes a computing unit and a user interface. Preferably, the user interface can be located in or inside the cab 14, allowing the operator of the excavator 1 to use it during operation. The user interface includes a display for showing real-time images and / or a graphical user interface (GUI). Figure 3b In the exemplary embodiment shown, the user interface includes a semi-transparent display 4 positioned in front of the cab 14, allowing the operator to view the displayed content without having to take his eyes off the terrain in front of him.

[0074] The user interface includes input devices preferably located inside the cab 14, such as a touch-sensitive display (touchscreen) and an optional stylus for use with the touchscreen. The computing unit can use measurement data (RCD data) generated by the measurement unit, such as LiDAR data from a LiDAR scanner and image data from multiple cameras, to generate, for example, a 3D model of a construction site or a portion thereof, and optionally also for obstacle detection.

[0075] The construction site measurement system may also include at least one of the following components, which may optionally be housed together with the measurement unit 2 in a common housing, or together with the calculation unit and / or user interface in a common housing:

[0076] - A GNSS antenna configured to generate location data;

[0077] - An inertial measurement unit (IMU) configured to generate IMU data; and

[0078] - A cellular unit configured to transmit any data to a remote station or other vehicle, such as construction machinery or towing trucks on the same construction site.

[0079] For example, if the measurement data includes LiDAR data and image data, the image data can be used to colorize the LiDAR data and / or to optimize the reference of the LiDAR data by matching the LiDAR data with image-based generated point clouds (e.g., generated by a Visual Simultaneous Localization and Mapping (VSLAM) algorithm). Furthermore, feature tracking and / or feature recognition algorithms can help combine LiDAR data into a consistent and well-referenced global point cloud. Similarly, position data obtained using GNSS antennas and IMU data from an IMU can be used for sensor fusion to achieve higher accuracy when constructing 3D models of the terrain. VSLAM point cloud generation can also be supported on LiDAR data, particularly by introducing scale into the LiDAR data and thus increasing the stability of the algorithm. LiDAR scanners can be configured to generate LiDAR data while rotating the two axes of rotation of each scanner faster than 0.1 Hz (especially faster than 1 Hz), where the point acquisition rate is at least 300,000 points per second, and particularly at least 500,000 points per second.

[0080] LiDAR scanners and ToF cameras can capture a 3D representation of the surroundings at very high speeds. Therefore, using mobile construction machinery, coherent 3D point clouds can be generated based on SLAM (Simultaneous Localization and Mapping) algorithms, which use LiDAR or ToF data alone or in combination with image data from cameras. This localization and mapping is particularly advantageous if the construction machinery is operating under bridges or in other locations affected by GNSS signals. The SLAM algorithm can be supported by at least one IMU providing IMU data, which can be processed to stabilize the algorithm. In particular, all such fused sensor data can be processed by a Kalman filter.

[0081] Optionally, the system for configuring the machine control unit may also include at least one background camera, for example, as part of a construction site measurement system. The background camera is any camera device that generates an image providing the environmental background. Specifically, the background camera does not need to provide any data for the sensor system. However, by calculating a parallax map, the image provided by the stereo camera is suitable for use as a background camera in addition to its use as a sensor. The background camera may be housed in a common enclosure with the RCDs 3a and 3b of the measurement unit 2, at the translucent display 4, or in a common enclosure with the computing unit and user interface. The background image data can be correlated with the RCD data of the construction site measurement system using external parameters characterizing their relative positions when the background camera and RCD are mounted on the construction machinery 1. The image captured by the background camera can be displayed to the operator of the machinery on the translucent display 4, on a touchscreen, and / or on any other display of the user interface (particularly as live video).

[0082] The use of one or more RCDs (3a, 3b) installed on construction machinery enables the entire work plan to be located on site based on RCD data. If the RCD is used for navigation or mapping (e.g., a variant of SLAM), then the design documents can be located within the navigation framework. If the machinery is equipped with GNSS or another global positioning system, the RCD data can be located in global coordinates, and the design documents can also be located in the world coordinate system. The design documents can then be saved and exported for reference or later use.

[0083] Figure 4 An exemplary embodiment of the system 70 according to the present invention is illustrated. It can be at least partially mounted on construction machinery, for example, mounted on... Figure 3a and Figure 3b On an electric grader. System 70 includes a computing unit 71 operatively coupled to a user interface 73. The user interface is configured to present information to and receive input from the operator of the system. For example, the user interface 73 may be set on an electric grader. Figure 3a and Figure 3b The electric grader is located in the cab of the excavator and includes a semi-transparent screen and / or touchscreen.

[0084] The system may further include a mechanical control unit 75 operatively coupled to the computing unit 71. The mechanical control unit 75 is configured to assist the operator in performing earthmoving operations. Specifically, this includes at least partially automatically monitoring the coordinates of the tools of the construction machinery, for example, to prevent… Figure 3a and Figure 3bThe blade of the electric grader digs under a predetermined plane. Alternatively, the mechanical control unit 75 can be separately mounted on the construction machinery and connected to the system 70, so that it is operatively coupled to the computing unit 71.

[0085] System 70 may also include one or more reality capture devices (RCDs) 82 operatively coupled to computing unit 71 and at least one background camera 83. Images from the background camera 83 may be displayed on the screen (e.g., a touchscreen) of user interface 73. Alternatively, the RCDs 82 and background camera 83 may be separately mounted on construction machinery and connected to system 70 such that they are operatively coupled to computing unit 71.

[0086] The calculation unit 71 is configured to receive point cloud data of the surface of the construction site (particularly the surface around the construction machinery) from the RCD 82. The calculation unit 71 is also configured to receive user input from the user interface 73 regarding planned earthmoving operations at the construction site. The calculation unit 71 determines the 3D coordinates of the planned earthmoving operations and programs the machinery control unit 75 accordingly.

[0087] Figure 5a The final road design is shown in cross-section of the ditch 30, which includes both sides of the road crown 40. The road and / or ditch can be constructed using embodiments of the system according to the invention. For example, the road design includes values ​​for the width 41 and / or half-width 42 of the road crown 40. Regarding the ditch 30, values ​​are provided for the ditch depth 36, the ditch wall angle 38, and the ditch offset 37, which describes, for example, the distance between the bottom edge 35 of the ditch and the road center 45 or the road edge.

[0088] In order to construct a road with this cross-section using the described system, it is necessary for the system to provide spline offsets. This allows the operator to define the working direction of the road using the inner edge of the ditch 30, and then laterally offset the shoulder edges and camber 40 of the road to ensure they are parallel to the ditch line. The system should also be able to generate and follow curved splines used for guidance. This is in Figure 5b The example illustrates that the spline of the calculated ditch edge 35 follows a given spline of the road center 45. Alternatively, the spline of the first ditch edge 35 and the spline of the road can be given, and the spline of the second ditch edge 35 to be constructed on the other side of the road can be calculated. For example, the spline is calculated using an offset such that the two ditch edges 35 have the same distance to the road (e.g., to the corresponding side of the road or to the center 45 of the road).

[0089] A similar method can be used when using tracked vehicles to create slots and reduce gradation (e.g.) Figure 1b(As shown). The operator performs initial marking and cutting, then configures the system to maintain the desired slope and navigate along the desired trench. The operator is then solely responsible for managing the vehicle load and transferring it from forward to reverse and vice versa, until a stockpile scattering operation is required. In the trench easing slope method, the operator sets the marking lines and then uses the system to generate stockpiles on both sides of the tracked vehicle to form a trench. The system then guides the vehicle between the two stockpiles.

[0090] Figure 6 This is a flowchart illustrating an exemplary embodiment of a method 100 according to the present invention, namely, a method for configuring a mechanical control unit for construction machinery performing earthmoving operations, which include multiple stages to be performed consecutively. In the example described herein, the earthmoving operation is a V-shaped trenching operation (e.g., as per [reference to...]). Figure 2 (As described), and the stage is a pass along the ditch line by an electric grader or similar machinery, which includes several cutting passes. Exemplary embodiments of the system according to the invention assist the operator in performing one or more of these passes.

[0091] Besides V-shaped trenching, other implementations of the method and system can also be used in other applications involving iterative passes along the same route to construct structures, such as road construction or slope dredging. In addition to electric graders, the disclosed methods and workflows can also be used with other construction machinery, such as tracked machines and wheel loaders.

[0092] During the execution of this method, for example, one or more reality capture devices (RCDs) mounted on construction machinery are used to capture a 3D point cloud of the surface surrounding the machinery. This allows for the positioning of the construction machinery and the identification of surrounding features. Specifically, the 3D point cloud can be captured continuously. Optionally, a background camera with a known position relative to the 3D point cloud and / or relative to the measurement system capturing the 3D point cloud captures background image data of the surrounding machinery.

[0093] The illustrated method begins with a marking or registration pass, in which the soil is cut just enough to provide registration lines to aid in guiding subsequent passes. The operator may perform the marking pass 101 manually, or begin an auxiliary process 110 assisted by the mechanical control unit of the construction machinery and / or the system according to the invention.

[0094] In the latter case, during auxiliary process 110, the operator instructs on the desired line (“ditch line”) used for marking the pass, such as a straight line or spline. This instruction may include inputting coordinates or drawing a line in a background image displayed on the touchscreen. Alternatively, the operator may select a feature in the background image, such as a road or similar structure, and instruct on the offset of the desired line relative to the feature. Alternatively or additionally, a data file including coordinates may be uploaded to the system.

[0095] The system receives 111 an indication of the desired trench line as input. Since the RCD mounted on the construction machinery enables the positioning 112 of the construction machinery, the automation system can optionally be configured to guide 113 a tool (e.g., the toe of a mold plate) to the desired trench line and / or to position 114 the tool at the designated trench line and maintain that position (e.g., at least maintaining height and inclination) while the construction machinery travels along the trench line, guided by the operator or controlled by the automation system. Thus, 115 marking passes are performed fully or partially automatically. In the latter case, for example, the automation system actuates the blade to lateralize so that the tool (e.g., the toe of a mold plate) is held in the desired position while the operator controls the guidance, rounding, and rounding.

[0096] After a marking pass has been performed (manually 101 or using auxiliary process 110), subsequent passes can begin, such as a cutting process 120 that includes a first deep cut, as shown here. Using an RCD mounted to the construction machinery, the construction site surveying system can automatically detect 121 the edges and surfaces of the previous pass (i.e., the marking pass or the previous cutting pass). These edges and surfaces can be detected, for example, in image data and / or point cloud data captured by the RCD 121.

[0097] The system can then present the detected elements on a display, for example, by overlaying 122 these detected elements onto a background image displayed on a touchscreen, and allows the operator to select previously formed edges and / or surfaces to configure the automation system for the current cutting pass. For example, the operator can select the edge of a previous cut and configure the system to cut 25 mm below it in the next pass. The operator can select a previous surface and configure the system to maintain parallel cuts for subsequent passes. Cuts can also be designed to change the lateral slope of the surface, but using the previously sloping edge as the apex of the plane formed in subsequent cutting passes. 123 These selections are received as operator input on a user interface (e.g., on a touchscreen displaying a background image) to configure 124 the automation system to fully or partially automate 125 the cutting passes. The cutting process 120 can be repeated until the final pass is performed and the trench construction is complete.

[0098] Optionally, if the user interface displays a background image on a touchscreen that is also used to receive operator input (e.g., in steps 111 and 123), the input may include, or be interpreted as, a selection of pixels in the background image. Since the background camera capturing the background image data has a known position relative to the measurement system and / or 3D measurement data, the pixel selection (e.g., representing a desired ditch line) can be mapped to the 3D measurement data, for example, to the surface of a 3D terrain model generated based on the 3D measurement data. Based on this mapping, 3D coordinates on the surface can then be determined, which can be provided to the mechanical control unit for at least partial control of earthmoving operations. This is described in detail in European Patent Application No. 22180149.1.

[0099] Data captured by the RCD can also be used to monitor shed material as it forms a stockpile. The volume of material in the stockpile or the height of the stockpile can be used to indicate to the operator when an extension pass needs to be performed (see example...). Figure 2 (Steps 7 and 8 in the original text). Alternatively or additionally, the control unit may be triggered to perform extended behavior during the next iteration or to automatically perform extended iterations.

[0100] Optionally, the method may include storing information about previously identified lines and surfaces in a memory and retrieving this information in subsequent passes. For example, the method may include passes for cleaning the bottom of a ditch (see, for example...). Figure 2 (Steps 9 and 10 in the original text). In this case, if the system remembers the location of the bottom edge of the ditch, it will be helpful to the operator even when the ditch is covered with soil. This will allow the operator to cut the inner slope of the ditch, then use the inner edge of the ditch when cutting the outer slope, and then use the inner edge of the ditch again when clearing the soil from the bottom of the ditch. Correctly defining the offset using the upper edge of the ditch will be difficult, but because the system knows the location of the bottom edge relative to the upper edge, the operator can command it to guide to the hidden ditch line when clearing the soil from the ditch, which is especially useful when the ditch line is a spline.

[0101] In some implementations, the system can be configured to be self-configurable. In this mode, the operator can set the lateral slope and height for each marking pass, increasing the height until the correct depth for that pass is reached. The RCD's positioning capabilities allow for maintaining a globally consistent height while the vehicle is being guided on-site. The operator can then use incremental buttons to inform the system of the desired depth for each cutting pass. The construction site surveying system can use its positioning capabilities to determine when it is at the start of a work operation. It can then combine the positioning data with the configuration of the previous cutting pass to automatically detect and identify edges and surfaces cut by the machine. It can then automatically set the desired guide spline or mold plate toe line based on the previous pass and the depth indicated by the operator.

[0102] According to some embodiments, the method and system according to the invention can also be used to monitor the construction of V-shaped ditches. Because the system understands the workflow, it can be used to track the steps used during construction, as well as the system configuration and work products for each pass. This data provides a real-time status of the construction operation and can also be used to train new operators, specifying the desired workflow for inexperienced operators using the system.

[0103] Optionally, the system may include mechanical devices for sensing the load on the construction machinery. This may include sensing engine torque, observing changes in engine speed, hydrostatic circuit pressure and / or displacement, transmission output torque, torque converter torque, etc. In this case, the depth control of the control unit can be set to a load-related state, where the system can automatically adjust the tool height based on the detected load.

[0104] Although the invention has been illustrated above with reference to some preferred embodiments, it must be understood that many modifications and combinations of different features of the embodiments can be made. All such modifications fall within the scope of the appended claims.

Claims

1. A system (70) for configuring a mechanical control unit (75) of construction machinery (1) to perform earthmoving operations, said earthmoving operations comprising multiple stages to be performed continuously, said system (70) comprising: - A measurement system (82) configured to capture three-dimensional measurement data of the terrain around the construction machinery (1) within at least a first detection range; - Background camera (83), the background camera (83) having a known position relative to the measurement system (82) and / or the three-dimensional measurement data, and being configured to capture background image data of the terrain within the first detection range; - A user interface (73), configured to display at least one background image to the operator of the construction machinery (1) based on the background image data, and to receive (111) user input from the operator of the construction machinery (1); and - A computing unit (71), which is operatively coupled at least to the measurement system (82), the user interface (73), and the mechanical control unit (75). Its features are, The computing unit (71) is configured to: - Detect elements of the previous stage of the earthwork operation based on the three-dimensional measurement data (121); - Overlay the detected elements (122) onto the displayed background image; - Receive (123) input from the operator relating to the next stage of the earthmoving operation, the input including selection of one or more detected elements; as well as - Configure (124) the mechanical control unit (75) to at least partially automate (125) the next stage of the earthmoving operation based at least on the detected elements and the input from the operator.

2. The system (70) according to claim 1, wherein, The measurement system (82) includes at least one measurement unit (2) at the construction machinery (1), each measurement unit (2) being configured to capture 3D point cloud data, wherein each measurement unit (2) includes: - At least one laser scanner, - Multiple ToF cameras, - Millimeter-wave radar systems, and / or - One or more stereo camera systems.

3. The system (70) according to claim 1 or claim 2, wherein, - The user interface (73) includes a touch-sensitive display on which the background image is displayed and on which the user input is received.

4. The system (70) according to claim 1 or 2, wherein, The construction machinery (1) is an electric grader, and the earthwork operation includes the construction of ditches (30), wherein, - The stage is the number of passes the electric grader makes along the ditch line, the passes including multiple cutting passes; and - The detected elements include edges (35, 39) and / or surfaces generated by the previous cutting pass.

5. The system (70) according to claim 4, wherein, The earthwork operation includes V-shaped trenching, and / or the input from the operator includes: - Select the edges and / or surfaces generated by the previous cutting pass; - Select the cutting depth to use for the next pass; - Choose to maintain the parallel cuts in the next pass; and / or - Select the edges and surfaces generated by the previous cutting pass, and select to adjust the lateral slope of the surfaces for the next pass, while keeping the selected edges as vertices of the plane to be shaped during the next pass.

6. The system (70) according to claim 1 or 2, wherein, The computing unit (71) is configured to: - Receive (111) operator input indicating the desired line along which the earthmoving operation is to be performed; - Based on the three-dimensional measurement data, locate (112) the construction machinery; - Position the tool (11) (114) at the desired line, and - Perform the first of the multiple stages described in (115) to produce detectable elements.

7. The system (70) according to claim 6, wherein, - The construction machinery is positioned relative to the desired line (112). - The computing unit (71) is configured to guide the construction machinery (113) to the desired line; and / or - The desired line is a spline.

8. The system (70) according to claim 6, wherein, The operator input includes the desired line offset (37) relative to a given line, and the calculation unit (71) is configured to calculate the desired line based on the offset (37).

9. The system (70) according to claim 8, wherein, The given line is a spline, and the calculation unit (71) is configured to calculate the spline as the desired line based on the offset (37).

10. The system (70) according to claim 1 or 2, wherein, - The measurement system (82) is configured to continuously capture the three-dimensional measurement data and continuously provide the captured three-dimensional measurement data to the computing unit; - The computing unit (71) is configured to continuously evaluate the three-dimensional measurement data to detect (121) the element, and to continuously configure (124) the mechanical control unit (75) while at least partially automating (125) the earthwork operation phase.

11. A construction machine (1), the construction machine comprising: - Tool (11), said tool (11) is used to perform earthwork operations; - A mechanical control unit (75) for at least partially controlling the earthmoving operation; as well as - The system (70) according to any one of claims 1 to 10, wherein the system is operatively coupled to or includes the mechanical control unit (75).

12. The construction machinery (1) according to claim 11, wherein, The construction machinery (1) is a grader, bulldozer or excavator, and the tool (11) is used to perform earthwork operations such as digging ditches or trenches.

13. A computer-implemented method (100) for configuring a mechanical control unit (75) of a construction machine (1) according to claim 11 for performing earthmoving operations, the earthmoving operations comprising multiple stages to be performed consecutively, the method (100) comprising the following steps: - The computing unit (71) receives three-dimensional measurement data of the terrain around the construction machinery (1) within at least a first detection range; - The computing unit (71) detects (121) elements of the previous stage of the earthwork operation based on the three-dimensional measurement data; - Use a background camera (83) with a known position relative to the three-dimensional measurement data and / or relative to the measurement system (82) that captures the three-dimensional measurement data to capture background image data of the terrain within the first detection range; - The user interface (73) displays at least one background image to the operator of the construction machinery (1) based on the background image data; - The detected elements are superimposed (122) on the displayed background image by the computing unit (71); - The computing unit (71) receives (123) inputs from the operator related to the next stage of the earthwork operation, wherein the inputs from the operator include selections of one or more detected elements; - The computing unit (71) configures (124) the mechanical control unit (75) to at least partially automate (125) the next stage of the earthmoving operation based on the detected elements and the input from the operator.

14. The method (100) according to claim 13, wherein, The construction machinery (1) is an electric grader, and the earthwork operation includes the construction of ditches (30), wherein, - The stage is the number of passes the electric grader makes along the ditch line, and the number of passes includes multiple cutting passes; and - The detected elements include edges (35, 39) and / or surfaces generated by the previous cutting pass.

15. The method (100) according to claim 14, wherein, - The earthwork operation includes V-shaped trenching; - Input from the operator includes selecting the edges and / or surfaces produced by the previous cutting pass; - The input from the operator includes selecting to maintain parallel cuts for the next pass; and / or The input from the operator includes selecting the cutting depth for the next pass.

16. The method (100) according to any one of claims 13 to 15, wherein the method (100) comprises the following steps: - Receive (111) operator input indicating the desired line along which the earthmoving operation is to be performed; - Based on the three-dimensional measurement data, locate (112) the construction machinery; - Position the tool (11) (114) at the desired line, and - Perform the first of the multiple stages described in (115) to produce detectable elements.

17. The method (100) according to claim 16, wherein, The construction machinery is positioned (112) based on the three-dimensional measurement data relative to the desired line, wherein, - The method (100) includes automatically guiding (113) the construction machinery to the desired line; - The desired line is a spline; and / or - The operator input includes the offset (37) of the desired line relative to a given line, and the method includes calculating the desired line based on the offset (37).

18. The method (100) according to any one of claims 13 to 15, wherein, - Continuously capture the three-dimensional measurement data and continuously evaluate the three-dimensional measurement data to detect (121) the element, and - The mechanical control unit (75) is continuously configured (124) while at least partially automatically executing (125) the earthwork operation phase.

19. The method (100) according to any one of claims 13 to 15, wherein, - The earthwork operations include the construction of ditches (30), - The desired line is the ditch line; - The first stage is to trace along the marked sections of the ditch line, and - The elements generated by the marked iterations include edges (35, 39) and / or surfaces.

20. The method (100) according to claim 19, wherein, The desired line is the edge line of the ditch, and wherein, - The construction machinery (1) is an electric grader; - The earthwork operations include V-shaped trenching; and / or - The ditch line is a ditch edge line, a ditch spline, or a ditch edge spline.

21. A computer program product comprising program code stored on a machine-readable medium or embodied by an electromagnetic wave including a segment of program code, and the computer program product having computer-executable instructions that, when executed on a computing unit (71) of a system (70) according to any one of claims 1 to 10, perform the method (100) according to any one of claims 13 to 20.

Images