A terrain monitoring method, device, storage medium and electronic equipment

Abstract

The application provides a terrain monitoring method and device, a storage medium and an electronic device. The monitoring method comprises: determining the running conditions of the engineering machinery, the running conditions comprising a working condition, a supporting walking condition and a self-walking condition; in the case that the engineering machinery passes a track point for the first time, terrain data of the track point is determined by using a corresponding calculation method to obtain initial terrain data; in the case that the engineering machinery passes the track point multiple times and the track point has not performed work, the initial terrain data is corrected by using corrected terrain data to obtain target terrain data of the engineering machinery corresponding to the running conditions; and the target terrain data corresponding to all the running conditions is standardized and integrated to obtain target working terrain of the engineering machinery. The method realizes all-condition terrain perception of working points in the self-walking area, the supporting walking area and the non-walking area of the excavator based on the fusion of terrain data and excavator working posture data, and splices and synthesizes multi-scale elevation data.

Description

Technical Field

[0001] This application relates to the field of terrain monitoring technology, and more specifically, to a terrain monitoring method based on engineering machinery operation, a terrain monitoring device based on engineering machinery operation, a computer-readable storage medium, and an electronic device. Background Technology

[0002] Complex, unstructured, and weakly textured terrain is the primary operating environment for construction machinery (e.g., excavators) and is also a key area of ​​research in land use change identification. Perception methods for unstructured terrain environments mainly include contact perception and non-contact perception.

[0003] However, current contact-based terrain sensing technologies often focus on ground texture while neglecting differences in terrain undulations. High-precision satellite positioning suffers from significant elevation positioning errors, and differential services are costly and have limited coverage, making it difficult to achieve high-precision, low-cost terrain sensing. Non-contact sensing technologies face significant challenges in recognizing unstructured, weakly textured, and highly self-similar terrain features in excavator operating environments. Summary of the Invention

[0004] The main objective of this application is to provide a terrain monitoring method, a terrain monitoring device, a computer-readable storage medium, and an electronic device based on construction machinery operations, so as to at least solve the problem that the existing technology cannot accurately detect the terrain of construction machinery operations.

[0005] To achieve the above objectives, according to one aspect of this application, a terrain monitoring method based on construction machinery operation is provided, comprising: detecting the operating parameters of the construction machinery, and determining the operating condition of the construction machinery based on the operating parameters, wherein the operating condition includes a working condition, a support-walking condition, and a self-walking condition; the working condition is the condition in which the construction machinery is performing operations; the support-walking condition is the condition in which the loader of the construction machinery is in contact with the work surface, the excavator experiences a sudden change in pitch angle, and is not performing operations; and the self-walking condition is the condition in which the loader of the construction machinery is not in contact with the work surface and is not performing operations; when the construction machinery passes a trajectory point or a work point on its operating trajectory for the first time, the method determines the operating condition of the construction machinery and the terrain monitoring method based on the operating condition of the construction machinery and the terrain monitoring method based on the operating parameters of the construction machinery. The operating trajectory is described, and the terrain data of the trajectory points or work points are determined using a calculation method corresponding to the operating conditions of the construction machinery, thus obtaining the initial terrain data of the construction machinery corresponding to the operating conditions. If the construction machinery passes the same trajectory point or work point multiple times without having performed any work at that point, the terrain data of the trajectory point or work point is determined for each subsequent pass, except for the first pass, thus obtaining corrected terrain data. This corrected terrain data is then used to correct the initial terrain data, resulting in the target terrain data of the construction machinery corresponding to the operating conditions. Finally, all the target terrain data corresponding to the operating conditions are standardized and integrated to obtain the target operating terrain of the construction machinery.

[0006] Optionally, the operating condition is the support and travel condition. Based on the operating condition of the construction machinery and its operating trajectory, the terrain data of the trajectory point or the work point is determined using a corresponding calculation method. This includes: obtaining the coordinates of a first support point and a second support point of the construction machinery based on its operating trajectory. The first support point coordinates are the coordinates of the support point where the loader of the construction machinery contacts the work surface in the world coordinate system, and the second support point coordinates are the coordinates of the support point in the pixel coordinate system of the target image. The target image is acquired by an image acquisition device installed on the construction machinery. If the rate of change of the pitch angle of the construction machinery is greater than a preset rate of change, based on... Based on the coordinates of the first support point and the second support point, the three-dimensional coordinates of the image acquisition device in the world coordinate system are determined to obtain the first calculated coordinates; the installation height of the image acquisition device is obtained to obtain the device installation height, and the angle between the running direction of the construction machinery and the heading angle of the construction machinery is obtained to obtain the calculated angle; based on the magnitude of the calculated angle, using the first calculated coordinates, the device installation height, and the rotation matrix, the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system are determined to obtain the terrain data of the trajectory point or the working point, where the contact point between the base of the construction machinery and the working surface is the trajectory point or the working point on the running trajectory.

[0007] Optionally, based on the magnitude of the calculated angle, the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system are determined using the first calculated coordinates, the equipment installation height, and a rotation matrix to obtain the terrain data of the trajectory point or the working point. This includes: when the calculated angle is less than a preset angle, using a first distance, the first calculated coordinates, the equipment installation height, and a rotation matrix to determine the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system to obtain the terrain data of the trajectory point or the working point, wherein the first distance is the distance between the vertical body of the image acquisition device and the track drive wheel of the construction machinery; when the calculated angle is greater than or equal to the preset angle, using a second distance, the first calculated coordinates, the equipment installation height, and a rotation matrix to determine the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system to obtain the terrain data of the trajectory point or the working point, wherein the second distance is the distance between the vertical base of the image acquisition device and the outer edge of the track of the construction machinery.

[0008] Optionally, the operating condition is the self-propelled operating condition. Based on the operating condition of the construction machinery and its operating trajectory, the terrain data of the trajectory point or the work point is determined using a corresponding calculation method, including: determining the GNSS positioning data of the initial work point when the construction machinery first performs its operation as the initial three-dimensional coordinates, and determining a planar grid reference system based on the initial three-dimensional coordinates, wherein the planar grid reference system is composed of multiple planar grid cells; obtaining the average pitch angle and average roll angle of the construction machinery during its movement in each planar grid cell based on its operating trajectory; obtaining the horizontal displacement of the construction machinery, and calculating the elevation change value corresponding to the horizontal displacement based on the average pitch angle and average roll angle corresponding to each planar grid cell; integrating and accumulating all the elevation change values ​​to obtain the current elevation value, and determining the terrain data of the trajectory point or the work point based on the current elevation value.

[0009] Optionally, based on the operating trajectory of the construction machinery, obtaining the average pitch angle and average roll angle of the construction machinery during its movement in each of the planar grid cells includes: obtaining, based on the operating trajectory of the construction machinery, the unit movement step length, the unit pitch angle change value, and the unit roll angle change value of the construction machinery when it moves in each of the planar grid cells; calculating the average pitch angle corresponding to the planar grid cell based on the unit movement step length and the unit pitch angle change value; and calculating the average roll angle corresponding to the planar grid cell based on the unit movement step length and the unit roll angle change value.

[0010] Optionally, the initial terrain data is corrected using the corrected terrain data to obtain target terrain data for the engineering machinery corresponding to the operating condition. This includes: determining the difference between the corrected terrain data calculated when the engineering machinery passes the trajectory point or the work point and the corrected terrain data calculated when the engineering machinery passed the trajectory point or the work point in the previous time as a first elevation increment; calculating the absolute value of the difference between the corrected terrain data calculated when the engineering machinery passes the trajectory point or the work point in any two adjacent times to obtain multiple calculated absolute values; and correcting the initial terrain data when passing the trajectory point or the work point in two adjacent times based on the sum of all the calculated absolute values ​​and the first elevation increment to obtain target terrain data for the engineering machinery corresponding to the operating condition.

[0011] Optionally, the target terrain data corresponding to all the operating conditions are standardized and integrated to obtain the target operating terrain of the construction machinery, including: determining the resolution of the working point of the construction machinery as the target resolution; upsampling the target terrain data corresponding to the self-propelled mode to the target resolution, and downsampling the target terrain data corresponding to the support mode to the target resolution, to obtain resolution-standardized terrain data; covering the overlapping area of ​​the target terrain data corresponding to the self-propelled mode and the target terrain data corresponding to the support mode based on the resolution-standardized terrain data, to obtain data-fused terrain data; and performing edge smoothing and post-processing on the data-fused terrain data to obtain the target operating terrain of the construction machinery.

[0012] According to another aspect of this application, a terrain monitoring device based on construction machinery operation is provided, comprising: a detection unit, configured to detect the operating parameters of the construction machinery and determine the operating condition of the construction machinery based on the operating parameters, the operating condition including a working condition, a support-walking condition, and a self-walking condition, wherein the working condition is the condition in which the construction machinery is performing operations, the support-walking condition is the condition in which the loader of the construction machinery is in contact with the working surface, the excavator experiences a sudden change in pitch angle and is not performing loading / unloading operations, and the self-walking condition is the condition in which the loader of the construction machinery is not in contact with the working surface and is not performing operations; and a determination unit, configured to, when the construction machinery passes a trajectory point or a working point on its operating trajectory for the first time, determine the operating condition of the construction machinery and the operating trajectory of the construction machinery based on the operating condition of the construction machinery and the operating parameters of the construction machinery, using a method consistent with the operating parameters of the construction machinery. The calculation method corresponding to the operating conditions of the construction machinery determines the terrain data of the trajectory point or the work point, and obtains the initial terrain data of the construction machinery corresponding to the operating conditions; the correction unit is used to determine the terrain data of the trajectory point or the work point for each passing of the construction machinery except the first passing when the construction machinery passes the same trajectory point or the work point multiple times and the trajectory point or the work point has not been used for work, obtains the corrected terrain data, and uses the corrected terrain data to correct the terrain data of the two adjacent passing of the trajectory point or the work point, and obtains the target terrain data of the construction machinery corresponding to the operating conditions; the integration unit is used to standardize and integrate the target terrain data corresponding to all the operating conditions to obtain the target operating terrain of the construction machinery.

[0013] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform any of the aforementioned terrain monitoring methods based on engineering machinery operations.

[0014] According to another aspect of this application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any of the described terrain monitoring methods based on engineering machinery operations.

[0015] Applying the technical solution of this application, the above-mentioned terrain monitoring method based on construction machinery operation first determines the operating conditions of the construction machinery according to its operating parameters. These operating conditions include work conditions, support walking conditions, and self-walking conditions. Then, when the construction machinery passes a trajectory point or work point on its operating trajectory for the first time, the terrain data of the trajectory point or work point is determined using a calculation method corresponding to the operating conditions, based on the construction machinery's operating conditions and trajectory, thus obtaining initial terrain data corresponding to the operating conditions. Next, when the construction machinery passes the same trajectory point or work point multiple times without any prior work being performed at that point, the initial terrain data is corrected using corrected terrain data, resulting in target terrain data corresponding to the operating conditions. Finally, all target terrain data corresponding to the operating conditions are standardized and integrated to obtain the target operating terrain for the construction machinery. This method integrates terrain data and excavator operating posture data to achieve low-cost, high-precision dynamic perception of the terrain across all working conditions, including the excavator's self-walking area, support walking area, and non-walking area. Based on this, multi-scale elevation data is stitched together using a cross-scale terrain data fusion and reconstruction method, enabling rapid, low-cost, and high-precision acquisition of terrain change data for construction machinery operations. This solves the problem that existing technologies cannot accurately detect the operating terrain of construction machinery. Attached Figure Description

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for performing a terrain monitoring method based on engineering machinery operations, provided in an embodiment of this application, is shown.

[0018] Figure 2A schematic flowchart of a terrain monitoring method based on engineering machinery operations according to an embodiment of this application is shown;

[0019] Figure 3 A schematic flowchart of another terrain monitoring method based on engineering machinery operation provided according to an embodiment of this application is shown;

[0020] Figure 4 The illustration shows a terrain-sensing posture of construction machinery under different working conditions in a terrain monitoring method based on construction machinery operation according to an embodiment of this application.

[0021] Figure 5 A side view of the engineering machinery structure in a terrain monitoring method based on engineering machinery operations provided according to an embodiment of this application is shown;

[0022] Figure 6 A front view of the engineering machinery structure in a terrain monitoring method based on engineering machinery operations provided according to an embodiment of this application is shown;

[0023] Figure 7 A structural block diagram of a terrain monitoring device based on engineering machinery operation provided according to an embodiment of this application is shown.

[0024] The above figures include the following reference numerals:

[0025] 1. Camera; 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be used interchangeably where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] As introduced in the background section, complex unstructured, weakly textured terrain is the primary operating environment for construction machinery (excavators) (such as rugged terrain and muddy areas), and is also a key area for land use change identification research. Perception methods for unstructured terrain environments mainly include contact perception and non-contact perception.

[0030] Contact sensing uses tactile sensors such as tilt sensors, pressure sensors, and vibration sensors to collect information on three-axis attitude angles, angular velocities, pressure, and vibration. Combined with GNSS positioning, it directly or indirectly senses the texture and shape of the terrain, guiding terrain identification and classification. However, contact sensors often focus only on ground texture, neglecting differences in terrain undulations. High-precision GNSS modules achieve sub-meter accuracy in horizontal positioning, but elevation errors reach 2-5 meters or even greater. Using GNSS positioning alone is insufficient for accurately sensing terrain undulations and reconstructing the terrain. While differential positioning can significantly improve accuracy, it struggles to provide high-precision positioning in areas not covered by differential or mobile communication services, and differential services incur additional costs, hindering low-cost, large-scale applications.

[0031] Non-contact perception involves using sensors such as LiDAR and RGB-D depth cameras to acquire 3D terrain information. Procopio et al. used the difference between ground parallax and stereo parallax, calculated and fitted, to obtain a ground plane deviation threshold for terrain identification. DeepLab and SegNet models have been introduced into terrain recognition, achieving semantic segmentation of terrain categories such as ground and obstacles. Sun Hao et al. used CLIP and SAM models to achieve terrain segmentation of unlabeled data, demonstrating high accuracy and computational efficiency. These models primarily segment and classify the ground based on 3D terrain information, but they face bottlenecks in recognizing unstructured, weakly textured, and highly self-similar terrain morphology in engineering machinery (excavator) operating environments. Real-time localization and mapping (SLAM) is an effective approach for environmental modeling, mainly including visual SLAM and LiDAR SLAM. For diverse application scenarios, monocular cameras are prone to scale drift and perform poorly in complex outdoor lighting conditions, while LiDAR is limited in unstructured environments. Multi-sensor combinations and corresponding information fusion methods are often used to compensate for the shortcomings of single sensors and provide more reliable estimates. However, multi-source data fusion SLAM still suffers from problems such as poor stability of front-end odometry feature matching and insufficient accuracy of multi-source heterogeneous data fusion with the same weights in the back-end. In addition, the high cost of ordinary airborne radar, 3D radar, and radar matrices limits their large-scale application.

[0032] In summary, current contact-based terrain sensing technologies often focus on ground texture while neglecting differences in terrain undulations. GNSS positioning suffers from significant elevation positioning errors, and differential services are costly and have limited coverage, making it difficult to achieve high-precision terrain sensing. Non-contact sensing technologies face significant challenges in recognizing unstructured, weakly textured, and highly self-similar terrain features in excavator operating environments. There is an urgent need to develop a more reliable, stable, and low-cost terrain sensing technology to achieve dynamic terrain perception in engineering machinery operating environments.

[0033] To address the problem that existing technologies cannot accurately detect the terrain in which construction machinery is operating, embodiments of this application provide a terrain monitoring method, a terrain monitoring device, a computer-readable storage medium, and an electronic device based on construction machinery operations.

[0034] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0035] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a terrain monitoring method based on engineering machinery operations, according to an embodiment of the present invention. Figure 1As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0036] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the terrain monitoring method based on engineering machinery operations in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] This embodiment provides a terrain monitoring method based on engineering machinery operations that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. In addition, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0038] Figure 2This is a flowchart of a terrain monitoring method based on engineering machinery operations according to an embodiment of this application. Figure 2 and Figure 3 As shown, the method includes the following steps:

[0039] Step S201: Detect the operating parameters of the construction machinery and determine the operating conditions of the construction machinery based on the operating parameters. The operating conditions include working conditions, support and travel conditions, and self-propelled travel conditions. The working conditions are the conditions in which the construction machinery is performing operations. The support and travel conditions are the conditions in which the loader of the construction machinery is in contact with the working surface, the excavator experiences a sudden change in pitch angle, and no loading / unloading operations are performed. The self-propelled travel conditions are the conditions in which the loader of the construction machinery is not in contact with the working surface and no operations are performed.

[0040] Specifically, construction machinery can refer to excavators, bulldozers, road rollers, and other mechanical equipment. The loader of such construction machinery refers to the component that performs shoveling, transporting, unloading, and leveling of bulk materials, including excavator buckets, loader buckets, and road roller rollers. This application uses an excavator as an example. For an excavator in operation, during its walking state, the visual module of the online monitoring instrument for the working posture of construction machinery detects that the excavator's loader is not in contact with the working surface, thus identifying this. The spatial coordinates and attitude of the excavator's cab directly reflect the changes in terrain elevation. The pitch angle (θ) and roll angle (θ) between the excavator's self-moving trajectory in flat terrain areas are also measured. The continuous changes in the heading angle (ψ) directly reflect the changes in terrain aspect and slope. Terrain elevation is calculated by integrating continuous trajectory points or the aforementioned operational points, thus achieving terrain perception. For example, Figure 3 As shown, the vision module is camera 1 installed on the excavator.

[0041] Among them, excavators also include loaders. In some cases, construction machinery can also refer to loaders, that is, excavators and loaders are integrated into one structure. Construction machinery can refer to either one, but a unified calibration point is required in the calculation process. That is, if construction machinery refers to excavators, then construction machinery refers to excavators in the calculation process. If construction machinery refers to loaders, then construction machinery refers to loaders in the calculation process.

[0042] In some embodiments, Figure 4 Figures A, B, and C in the diagram represent the terrain-aware attitude of construction machinery under different working conditions. Figure 4As shown, the system monitors the excavator's travel trajectory and the loader's movements in real time. Based on these data, the system determines the excavator's real-time operating condition and then performs the following operations to obtain the terrain: When the loader touches the ground, the excavator is considered to be in working condition. The current location is taken as the working point, and the spatial coordinates of the working point are calculated based on the working point terrain monitoring steps to obtain the terrain. When the loader touches the ground and the excavator experiences a sudden change in pitch angle, the excavator is considered to be in support-walking condition. Based on the support-walking terrain monitoring steps, the excavator's spatial coordinates are calculated in reverse from the loader's spatial coordinates to obtain the terrain. When the loader does not touch the ground, the excavator is considered to be in self-walking condition. Based on the excavator's self-walking trajectory, the continuous trajectory points forming the trajectory or the aforementioned working points are integrated to solve for the terrain elevation and obtain the terrain.

[0043] Step S202: When the above-mentioned construction machinery passes through the trajectory point or work point on the running trajectory for the first time, the terrain data of the trajectory point or work point is determined by using a calculation method corresponding to the operating conditions of the above-mentioned construction machinery and the above-mentioned running trajectory of the above-mentioned construction machinery, so as to obtain the initial terrain data of the above-mentioned construction machinery corresponding to the above-mentioned operating conditions.

[0044] Specifically, by detecting the contact between the work tool and the work surface, the terrain undulations can be perceived. By estimating the spatial coordinates of the work point of the work tool, the elevation undulations of the terrain can be directly reflected, and the spatial coordinates of the excavator's work point can be determined.

[0045] Step S203: When the construction machinery passes through the same trajectory point or work point multiple times and no work has been performed at the trajectory point or work point, determine the terrain data of the trajectory point or work point for each passing time except the first time the construction machinery passes through, obtain corrected terrain data, and use the corrected terrain data to correct the initial terrain data to obtain the target terrain data of the construction machinery corresponding to the above operating conditions.

[0046] Specifically, when monitoring the excavator's travel trajectory in real time, if the travel trajectory passes through a previously recorded trajectory point or the aforementioned work point again, the travel trajectory that passes through this trajectory point or the aforementioned work point twice is taken as a loop detection unit, and the closure point detection, elevation increment calculation, and elevation closure difference allocation are executed in sequence to complete the terrain elevation loop detection correction.

[0047] Step S204: Standardize and integrate the target terrain data corresponding to all the above operating conditions to obtain the target operating terrain of the above engineering machinery.

[0048] Specifically, when the excavator finishes its work, the self-propelled and supported walking trajectories are upsampled and downsampled respectively to standardize the resolution. Then, high-precision terrain features are used to replace low-precision terrain features in the overlapping areas. Finally, post-processing, including edge smoothing, is performed to complete the monitoring.

[0049] The above embodiments address the problems of large elevation errors in existing GNSS positioning modules, high cost and limited coverage of differential services, which make it difficult to achieve high-precision and low-cost terrain perception. By fusing the three-axis attitude angles of the GNSS module and IMU module, as well as the excavator's operating attitude data, dynamic, low-cost, and high-precision terrain perception is achieved across all working conditions, including the excavator's self-propelled walking area, support walking area, and non-walking area. Furthermore, based on a cross-scale terrain data fusion and reconstruction method, multi-scale elevation data is stitched together, enabling rapid, low-cost, and high-precision acquisition of terrain change data for engineering machinery operations.

[0050] The above-described terrain monitoring method based on construction machinery operation in this application first determines the operating conditions of the construction machinery based on its operating parameters. These operating conditions include work conditions, support walking conditions, and self-walking conditions. Then, when the construction machinery passes a trajectory point or work point on its operating path for the first time, terrain data for the trajectory point or work point is determined using a calculation method corresponding to the operating conditions, based on the construction machinery's operating conditions and trajectory. This yields initial terrain data for the construction machinery corresponding to the operating conditions. Next, when the construction machinery passes the same trajectory point or work point multiple times without any prior work being performed at that point, the initial terrain data is corrected using corrected terrain data to obtain target terrain data for the construction machinery corresponding to the operating conditions. Finally, all target terrain data corresponding to the operating conditions are standardized and integrated to obtain the target operating terrain for the construction machinery. This method integrates terrain data and excavator operating posture data to achieve low-cost, high-precision dynamic perception of the terrain across all working conditions, including the excavator's self-walking area, support walking area, and non-walking area. Based on this, multi-scale elevation data is stitched together using a cross-scale terrain data fusion and reconstruction method, enabling rapid, low-cost, and high-precision acquisition of terrain change data for construction machinery operations. This solves the problem that existing technologies cannot accurately detect the operating terrain of construction machinery.

[0051] In some embodiments, the aforementioned operating condition is the aforementioned support and walking condition. Based on the operating condition of the construction machinery and the aforementioned operating trajectory of the construction machinery, the terrain data of the aforementioned trajectory points or the aforementioned work points are determined using a corresponding calculation method, including the following steps:

[0052] Step S301: Based on the above-mentioned running trajectory of the above-mentioned construction machinery, obtain the coordinates of the first support point and the second support point of the above-mentioned construction machinery. The coordinates of the first support point are the coordinates of the support point where the loader of the above-mentioned construction machinery contacts the above-mentioned working surface in the world coordinate system. The coordinates of the second support point are the coordinates of the support point in the pixel coordinate system of the target image. The target image is acquired by the image acquisition device installed on the above-mentioned construction machinery.

[0053] Step S302: When the rate of change of the pitch angle of the above-mentioned engineering machinery is greater than the preset rate of change, the three-dimensional coordinates of the above-mentioned image acquisition device in the above-mentioned world coordinate system are determined according to the coordinates of the first support point and the coordinates of the second support point, and the first calculated coordinates are obtained.

[0054] Step S303: Obtain the installation height of the image acquisition device, and obtain the angle between the running direction of the construction machinery and the heading angle of the construction machinery to obtain the calculated angle.

[0055] Step S304: Based on the magnitude of the calculated angle, using the first calculated coordinates, the equipment installation height, and the rotation matrix, determine the three-dimensional coordinates of the contact point between the base of the engineering machinery and the working surface in the world coordinate system, and obtain the terrain data of the trajectory point or the working point. The contact point between the base of the engineering machinery and the working surface is the trajectory point or the working point on the running trajectory.

[0056] Specifically, by utilizing the visual recognition of the loader's contact point and the joint criterion of IMU pitch angle abrupt changes, a reverse calculation link is established: "world coordinates of the support point → 3D coordinates of the camera → coordinates of the fuselage contact point". The principle is based on rigid body kinematics and camera projection geometry, using the fixed support point as a spatial reference to infer the true contact position of the fuselage under dynamic attitude, effectively avoiding the attitude distortion misjudgment caused by vehicle tilt in non-smooth terrain by the IMU. Its technical effect is that it is the first time that the 3D coordinates of the fuselage and ground contact point can be reconstructed with high accuracy without additional elevation sensors in complex terrains such as steep slopes and soft ground. This significantly improves the robustness and absolute accuracy of terrain perception under support walking conditions and solves the problem that traditional methods cannot obtain the real terrain in the "suspended support" state.

[0057] In some embodiments, based on the magnitude of the calculated angle, the three-dimensional coordinates of the contact point between the base of the engineering machinery and the working surface in the world coordinate system are determined using the first calculated coordinates, the equipment installation height, and the rotation matrix, thereby obtaining the terrain data of the trajectory point or the working point. This includes the following steps:

[0058] Step S3041: When the calculated angle is less than the preset angle, the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system are determined by using the first distance, the first calculated coordinates, the equipment installation height and the rotation matrix, so as to obtain the terrain data of the trajectory point or the working point. The first distance is the distance between the vertical body of the image acquisition device and the track drive wheel of the construction machinery.

[0059] Step S3042: When the calculated angle is greater than or equal to the preset angle, the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system are determined by using the second distance, the first calculated coordinates, the equipment installation height, and the rotation matrix, so as to obtain the terrain data of the trajectory point or the working point. The second distance is the distance between the vertical base of the image acquisition device and the outer edge of the track of the construction machinery.

[0060] Specifically, based on the angle between the direction of movement and the heading angle of the construction machinery, two different geometric contact models (longitudinal drive wheel contact or lateral track outer edge contact) are dynamically selected. The principle is based on the kinematic characteristics and track structure of the excavator, subdividing the single-arm support behavior into two modes: axial propulsion and lateral movement, and corresponding to different contact point geometric calculation models. The technical effect is to accurately distinguish the two typical support methods of the excavator in complex terrain, making the terrain calculation closer to the actual mechanical contact state, significantly improving the positioning accuracy of the base contact point coordinates, avoiding systematic elevation deviation caused by uniformly assuming the contact point position, and enhancing the system's adaptability in asymmetric and unstable terrain.

[0061] Among these, terrain perception in complex terrain areas is a crucial prerequisite for achieving high-precision terrain reconstruction. For steep slopes or other complex terrain areas where excavators need bucket support to traverse, the three-axis attitude of the IMU module does not match the actual slope aspect and angle. For complex terrain areas requiring support for movement, although the pitch angle (θ) and roll angle perceived by the IMU module mounted on the machine body are accurate... Parameters such as heading angle (ψ) cannot reflect the actual slope and gradient of the terrain. However, these areas have two obvious characteristics: the loader (bucket, shovel) touches the ground, and the pitch angle (θ) changes abruptly. These two parameters can be used for identification. The loader (bucket, shovel) touching the ground is identified by visual detection, and the pitch angle (θ) change is directly read by the IMU inertial measurement module. The aforementioned visual detection methods include the YOLO series model, SSD model, etc. The pitch angle (θ), that is, the pitch angle change exceeding 5°, corresponds to the roll angle (ψ). The heading angle (ψ) changes by less than 5°. At this point, the bucket used for support is relatively fixed, and the spatial position of the excavator body can be calculated in real-time based on the fixed spatial coordinates of the bucket. Specific implementation steps in some embodiments are as follows:

[0062] S 11 Determine the world coordinates of the support point of the excavating boom.

[0063] S 12 1. Inverse calculation of the world 3D coordinates of the excavator's body camera. When the excavator's boom support point touches the ground, and the body pitch angle changes while the body moves, it often corresponds to the excavator traveling up or down a steep slope. And the heading angle (ψ) is exactly in the opposite direction. As shown in Equation 1:

[0064] Equation (1);

[0065] Among them, X Wc Y Wc Z Wc X represents the world coordinates of the excavator camera. W Y W Z W Let R be the three-dimensional coordinates of the boom support point in the world coordinate system, and Z be the rotation matrix. C Scaling factor μ These are the horizontal coordinates in the pixel coordinate system. ν These are the coordinates in the vertical direction within the pixel coordinate system.

[0066] S 13 Calculation of the three-dimensional coordinates of the excavator's contact point. For example... Figure 5 and Figure 6 As shown, the camera mounting height is corrected to obtain the world coordinates of the excavator base, that is, the vertical coordinates of the camera to the base are X. Wc Y Wc Z Wc -h, where h is the height of the camera. This is combined with the distance L1 between the camera's vertical mount and the far drive wheel of the excavator's track (wheel), the distance L2 between the camera's vertical mount and the outer edge of the far drive wheel of the excavator, and the excavator's pitch angle (θ) and roll angle (θ). The coordinates of the contact point between the base and the ground are further calculated using the camera and the heading angle (ψ). In some specific implementation processes, using an excavator as the carrier, the camera can be installed at a height h above the ground, the distance L1 between the vertical base of the camera and the far-end drive wheel of the track can be 2370mm, and the distance L2 between the vertical base of the camera and the outer edge of the far-end track (wheel) of the excavator can be 1990mm.

[0067] When the angle between the direction of motion (direction of velocity or acceleration) and the heading angle (ψ) is less than 45°, it indicates that the excavator's travel direction is basically consistent with its heading, manifesting as longitudinal single-arm support and longitudinal support travel (axial propulsion mode). The contact point between the machine body and the ground surface is the track drive wheel or idler wheel, that is, the coordinates of the excavator base contact point with the ground are X. q Y q Z q As shown in Equation 2:

[0068] Equation (2);

[0069] When the angle between the direction of motion (direction of velocity or acceleration) and the heading angle (ψ) is greater than 45°, it indicates that the excavator's travel direction is inconsistent with its heading. This manifests as lateral single-arm support and lateral support travel (side-shift mode), meaning that one (side) track (wheel) is supported, and the contact point between the excavator body and the ground is the outer edge of the other (side) track (wheel). In other words, the coordinates of the excavator base's contact point with the ground are X... q Y q Z q As shown in Equation 3:

[0070] Equation (3);

[0071] In some embodiments, the aforementioned operating condition is the aforementioned self-propelled operating condition. Based on the aforementioned operating condition of the construction machinery and the aforementioned operating trajectory of the construction machinery, the terrain data of the aforementioned trajectory points or the aforementioned work points are determined using a corresponding calculation method, including the following steps:

[0072] Step S401: Determine the GNSS positioning data of the initial working point when the above-mentioned engineering machinery performs its first operation as the initial three-dimensional coordinates, and determine the planar grid reference system based on the above-mentioned initial three-dimensional coordinates. The above-mentioned planar grid reference system is composed of multiple planar grid units.

[0073] The initial three-dimensional coordinates are (X0, Y0, Z0), where Z0 is the elevation of the ground on which the excavator walks;

[0074] Step S402: Based on the above-mentioned operating trajectory of the construction machinery, obtain the average pitch angle and average roll angle of the construction machinery during its movement in each of the above-mentioned planar grid cells.

[0075] Step S403: Obtain the horizontal displacement of the above-mentioned engineering machinery, and calculate the elevation change value corresponding to the horizontal displacement based on the average pitch angle and the average roll angle corresponding to each of the above-mentioned planar grid cells.

[0076] The horizontal displacement of the aforementioned construction machinery was obtained based on GNSS positioning, with ΔX representing eastward displacement and ΔY representing northward displacement. Planar mesh cells were established using the magnitudes of ΔX and ΔY, with the values ​​of ΔX and ΔY set as the wheel (track) spacing of the excavator. Pitch angle θ and roll angle were also recorded. Continuous attitude data, including heading angle ψ;

[0077] Step S404: Integrate and accumulate all the above-mentioned elevation change values ​​to obtain the current elevation value, and determine the terrain data of the above-mentioned trajectory point or the above-mentioned operation point based on the above-mentioned current elevation value.

[0078] Specifically, under autonomous driving conditions, a terrain elevation accumulation method of "GNSS initial point + planar grid integration + attitude compensation" is adopted. The principle is to couple the GNSS horizontal displacement with the IMU pitch and roll angle changes, decompose the displacement to the body coordinate system, and then calculate the differential elevation change by combining the angle tangent relationship and integrating and accumulating it to achieve effective compensation for low-precision GNSS elevation errors. Its technical effect is that with a low-cost and widely available GNSS+IMU sensor combination, high-precision terrain reconstruction in flat terrain areas is achieved, breaking through the limitation of traditional GNSS elevation errors of 2-5m, and providing an economical, efficient and scalable terrain perception solution for large-area, long-distance engineering machinery operation paths.

[0079] In some embodiments, based on the aforementioned operating trajectory of the construction machinery, obtaining the average pitch angle and average roll angle of the construction machinery during its movement in each of the aforementioned planar grid cells includes the following steps:

[0080] Step S4021: Based on the above-mentioned operating trajectory of the above-mentioned construction machinery, obtain the unit movement step length, unit pitch angle change value and unit roll angle change value of the above-mentioned construction machinery when it moves one of the above-mentioned planar grid units.

[0081] Step S4022: Based on the above unit movement step size and the above unit pitch angle change value, calculate the average pitch angle corresponding to the above planar grid cell;

[0082] Step S4023: Calculate the average roll angle corresponding to the above-mentioned planar mesh element based on the above-mentioned unit movement step size and the above-mentioned unit roll angle change value.

[0083] Specifically, when the excavator moves autonomously, for each planar grid cell moved, the average pitch and roll angles within that grid cell are calculated based on the changes in pitch and roll angles and the corresponding step length. The excavator's displacement in the X and Y directions is decomposed into the excavator's cab orientation (determined by the heading angle ψ) and right-side direction to obtain the horizontal displacement. Finally, the elevation change during horizontal displacement is calculated based on the average pitch angle θ and average roll angle Φ of each planar grid cell. The elevation change at each moment is integrated and accumulated to obtain the current elevation.

[0084] Specifically, by performing time-weighted averaging on the IMU high-frequency attitude data using a unit movement step, spatial alignment with GNSS low-frequency trajectory points or the aforementioned operational points is achieved. The principle is to utilize the characteristic that the IMU sampling frequency is much higher than that of GNSS, and to aggregate the multi-frame attitude data within each grid cell into an equivalent average angle according to time weights, thereby eliminating integration errors caused by sampling asynchrony. The technical effect is to significantly improve the spatiotemporal consistency of the terrain elevation integration process, avoid elevation drift caused by attitude sampling jitter or delay, and make terrain reconstruction under autonomous walking conditions smoother, more stable, and repeatable, laying a high-quality data foundation for subsequent loop closure correction and multi-scale fusion.

[0085] For excavators in operation, during the excavator's walking state, the vision module of the online monitoring system for construction machinery's working posture identifies whether the excavator's loader is in contact with the work surface. The spatial coordinates and attitude of the excavator's cab directly reflect the changes in terrain elevation. The pitch angle (θ) and roll angle (θ) between the excavator's self-moving trajectory in flat terrain areas are also monitored. The continuous changes in the heading angle (ψ) directly reflect the changes in terrain aspect and slope. Terrain elevation is calculated by integrating continuous trajectory points or the aforementioned operational points, thus achieving terrain perception. Specifically, the steps are as follows:

[0086] S 21 1. Initial Data Acquisition and Initialization. Based on the excavator's movement trajectory, the initial three-dimensional coordinate point P0 (X0, Y0, Z0) is located by the GNSS module in the online monitoring instrument for the excavator's working posture at the initial work point, where Z0 is the elevation of the ground on which the excavator travels (after deducting the GNSS installation height). GNSS positioning horizontal displacement data (ΔX is the eastward displacement, ΔY is the northward displacement) is acquired. Planar mesh cells are established using the magnitudes of ΔX and ΔY, where ΔX and ΔY are appropriately set to the distance between the excavator's wheels (tracks) (high-precision GNSS positioning horizontal error can reach 1 meter; in practice, this can be adjusted according to the GNSS positioning plane accuracy). Simultaneously, the IMU inertial measurement module in the online monitoring instrument for the excavator's working posture records continuous attitude data, such as pitch angle (θ), roll angle (θ), and roll angle (θ). ), heading angle (ψ), etc.

[0087] S 22 Alignment of excavator trajectory and attitude data. Due to the high-frequency characteristics of the IMU (Inertial Measurement Unit), its data acquisition density is significantly higher than that of the GNSS positioning module. During the excavator's movement trajectory, from the center point of the previous grid to the center point of the next grid, the IMU acquires multiple sets of data. To maintain spatial calibration synchronization with the GNSS acquired data, the IMU inertial measurement data needs to be compressed and aligned. That is, for each grid point the excavator moves on the plane, its corresponding average heading angle, average pitch angle, and average roll angle can be expressed as (Equations 4-6):

[0088] Mean pitch angle: Equation (4);

[0089] Mean roll angle: Equation (5);

[0090] Mean heading angle: Equation (6);

[0091] In Equations 1 to 3, These represent the corresponding displacement segments. The monitored pitch angle value, These represent the corresponding displacement segments. The monitored roll angle, These represent the corresponding displacement segments. The monitored heading angle The average pitch angle, The average roll angle, This is the average heading angle.

[0092] Table 1 is a summary of the GNSS positioning and IMU module attitude data collected by the online monitoring instrument for the working posture of construction machinery (selected data from self-propelled areas), as shown in Table 1:

[0093] Table 1. Overview of Attitude Data

[0094]

[0095] In Table 1:

[0096] ;

[0097] ;

[0098] .

[0099] S23 1. Calculation of horizontal displacement components. The eastward (ΔX) and northward (ΔY) displacements from GNSS positioning are decomposed into the excavator's cab orientation (determined by the heading angle ψ) and lateral directions, as shown in Equation 7:

[0100] Equation (7);

[0101] in, This represents the displacement in the forward direction (negative values ​​are represented when moving backward). This indicates the direction of movement to the side (moving to the left is represented by a negative value).

[0102] In formula (7):

[0103] ;

[0104] .

[0105] S 24 Calculation of vertical height change. Using the average pitch angle ( ) and average roll angle ( (The pitch and roll angles are positive in the counterclockwise direction, and the elevation change caused by horizontal displacement is calculated.) (i.e., elevation change value) is represented as shown in Equation 8:

[0106] Equation (8);

[0107] In the corresponding embodiment:

[0108] .

[0109] Among them, tan( ) and tan( ) are respectively pitch angle ( ) and roll angle ( The tangent value of ).

[0110] S 25 Accumulated elevation by integration. The elevation of the current grid is obtained by summing Δh for each grid step and combining it with the initial elevation Z0, as shown in Equation 9:

[0111] Z = Z0 + ∑Δh (9);

[0112] Where Z is the current elevation value. In the corresponding embodiment, the elevation for operations at point 8 in Table 1 can be expressed as:

[0113] In addition, the terrain perception of the excavator's non-travel area is achieved by detecting the contact between the working tool and the working surface to sense the terrain undulations. The spatial coordinates of the working point of the working tool are then used to estimate the changes in the elevation of the terrain.

[0114] In some embodiments, the initial terrain data is corrected using the corrected terrain data to obtain the target terrain data of the engineering machinery corresponding to the operating conditions, including the following steps:

[0115] Step S2031: The difference between the corrected terrain data calculated when the above-mentioned construction machinery passes the above-mentioned trajectory point or the above-mentioned work point and the corrected terrain data calculated when the above-mentioned construction machinery passes the above-mentioned trajectory point or the above-mentioned work point is determined as the first elevation increment.

[0116] Step S2032: Calculate the absolute value of the difference between the corrected terrain data calculated when the above-mentioned engineering machinery passes through the above-mentioned trajectory point or the above-mentioned work point in any two adjacent times, and obtain multiple calculated absolute values;

[0117] Step S2033: Based on the sum of all the above-mentioned absolute values ​​and the above-mentioned first elevation increment, the terrain data of the above-mentioned two adjacent passages through the above-mentioned trajectory points or the above-mentioned work points are corrected to obtain the target terrain data of the above-mentioned engineering machinery corresponding to the above-mentioned operating conditions.

[0118] Specifically, a "loop closure detection + closure error allocation" mechanism is introduced to self-correct terrain data. The principle is based on the concept of traverse closure error adjustment in engineering surveying. By identifying repeatedly passed and undisturbed trajectory points or the aforementioned work points, the difference between the two cumulative elevation values ​​is calculated as a systematic error index, and the error is evenly distributed according to the relative weight of the elevation changes between trajectory points or the aforementioned work points. The technical effect is to effectively suppress the propagation of cumulative errors in the terrain elevation integration process, and significantly improve the overall consistency and absolute accuracy of the terrain model in long-term operation scenarios.

[0119] The elevation estimation based on the excavator's travel trajectory is a recursive process, where the elevation of the next position is calculated from the previous position. This means the error accumulates and amplifies with each trajectory point or work point. Loop closure detection is an effective method to eliminate this error, which is essentially consistent with the principle of traverse closure verification in engineering surveying. Since some trajectory points or work points in the excavator's travel trajectory may have passed over previous trajectory points or work points, a loop closure detection unit is formed when the excavator travels to a previously passed but not yet worked point—that is, when it passes over that trajectory point or work point again from the most recent point in time. Loop closure detection of terrain elevation mainly consists of the following steps:

[0120] S 31 Closure point detection. In the process of identifying closure points, firstly, the error between the GNSS-based position and a previous trajectory point or the aforementioned work point is within a single grid size. Secondly, no work has been done at that location point during the re-entry, meaning the terrain has not been modified. Thirdly, the ground slope aspect and angle at the re-entry point are within a certain error range; points where terrain deformation due to differences in surface compaction are not suitable for closure point loop detection and correction.

[0121] S 32 Elevation increment calculation. The elevation of the most recent entry into the trajectory point or the aforementioned work point and the current entry into the trajectory point or the aforementioned work point are calculated using the integral cumulative elevation calculation method. For example, if the sequence number of the most recent entry into the trajectory point or the aforementioned work point is t1, and the sequence number of the current entry into the trajectory point or the aforementioned work point is t... n The elevation of the most recent entry into this trajectory point or the aforementioned work point is obtained as Zt1, and the current elevation of the entry into this trajectory point or the aforementioned work point is Zt. n The elevation increment ΔH is represented as shown in Equation 10:

[0122] ΔH=Zt n -Zt1 Equation (10);

[0123] S 33 1. Elevation Closure Error Allocation. The closure error is allocated to all trajectory points or work points between the most recent entry point into the current trajectory point or work point and the current entry point into the current trajectory point or work point. For example, the closure error is allocated at point t... i The calculated elevation of point Z is... t,i , the tth i+1 The calculated elevation of point Z is... t,i+1 As shown in Equation 11:

[0124] Equation (11);

[0125] In equation (11), For the tth i+1 The corrected elevation of the serial number point. It is the sum of the absolute values ​​of the elevation differences of all adjacent trajectory points or aforementioned work points between the most recent entry into the trajectory point or the current entry into the trajectory point or the aforementioned work point.

[0126] In some embodiments, the target terrain data corresponding to all the above-mentioned operating conditions are standardized and integrated to obtain the target operating terrain for the above-mentioned engineering machinery, including the following steps:

[0127] Step S2041: Determine the resolution of the working point of the above-mentioned engineering machinery as the target resolution;

[0128] Step S2042: Upsample the target terrain data corresponding to the above self-propelled walking condition to the above target resolution, and downsample the target terrain data corresponding to the above supported walking condition to the above target resolution to obtain the resolution-standardized terrain data.

[0129] Step S2043: Based on the above-mentioned resolution-standardized terrain data, the overlapping area of ​​the above-mentioned target terrain data corresponding to the above-mentioned self-walking condition and the above-mentioned target terrain data corresponding to the above-mentioned support walking condition is covered to obtain the terrain data after data fusion.

[0130] Step S2044: Perform edge smoothing and post-processing on the terrain data after data fusion to obtain the target operating terrain of the engineering machinery.

[0131] Specifically, a standardized terrain reconstruction strategy based on "operation point resolution as the primary factor, cross-scale data fusion and post-processing" is proposed. The principle is to use the operation point (bucket contact point) as the highest precision benchmark, perform upsampling interpolation on the self-propelled area (low density), and downsampling aggregation on the support walking area (high density). High-resolution data is used to cover overlapping areas, and morphological filtering is combined to eliminate boundary abrupt changes. The technical effect is to achieve seamless fusion of multi-source heterogeneous and multi-resolution terrain data, which not only preserves the detail precision of the operation point, but also takes into account the continuity and smoothness of the large-scale terrain. Finally, a unified high-precision DEM (Digital Elevation Model) with engineering application value is output, providing a high-quality spatial base map for intelligent construction, earthwork calculation, and automated operation path planning.

[0132] In this section, the terrain perception grid size is the vehicle width for areas where the construction machinery travels independently (flat terrain), while the terrain perception data frequency in supporting areas (complex terrain) maintains the same high-frequency characteristics as the IMU or camera frame rate, resulting in extremely dense terrain data points. The grid size of the construction machinery's work points is consistent with the size of its loader (bucket, shovel). When multiple types of construction machinery operate together, the grid size of the work points becomes even more complex and diverse due to the varying bucket sizes. Therefore, at least three different scales of terrain perception data fusion and terrain reconstruction are ultimately involved. To achieve cross-scale terrain data fusion and reconstruction, while ensuring accuracy, the efficiency of terrain reconstruction is improved. The specific methods are as follows:

[0133] S 41Resolution standardization. Centering on the resolution of the engineering machinery's operating point, the terrain perception data for the areas where the engineering machinery travels (flat terrain areas) is upsampled to the target resolution. Upsampling methods include bilinear interpolation and cubic convolution interpolation. The high-density terrain perception data for the supporting walking areas (complex terrain areas) is downsampled to the target resolution using an average aggregation method, ultimately achieving resolution standardization.

[0134] S 42 Overlapping Area Fusion Strategy. Due to the low resolution of terrain perception data in areas where engineering machinery travels autonomously (flat terrain areas), the overlapping area fusion strategy primarily relies on terrain features for fusion. Based on time series data, high-resolution DEM data is prioritized for covering and replacing overlapping areas to ensure the accuracy of terrain representation and spatial consistency of data. During the data fusion process, high-precision terrain features are dynamically replaced with low-precision data, effectively improving the geometric accuracy of the overall terrain model.

[0135] S 43 Edge smoothing and post-processing: Gaussian filtering, median filtering, mean smoothing, and morphological closing operations are used to eliminate abrupt changes, small holes, and other anomalies at the splicing boundaries.

[0136] It should be emphasized that the above examples of the present invention are illustrative and not limiting. Therefore, the present invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of the present invention, without departing from the spirit and scope of the present invention, whether modifications or substitutions, shall also fall within the protection scope of the present invention.

[0137] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the terrain monitoring method based on engineering machinery operation of this application will be described in detail below with reference to specific embodiments.

[0138] This embodiment relates to a specific terrain monitoring method based on engineering machinery operations, such as... Figure 3 As shown, it includes the following steps:

[0139] Step 1: Monitor the excavator's travel trajectory and the loader's movements in real time, determine the excavator's real-time operating condition based on the travel trajectory and loader movements, and then perform the following operations based on the real-time operating condition to obtain the terrain:

[0140] When the loader touches the ground, it is determined that the excavator is in working condition. At this time, the current position is taken as the working point, and the spatial coordinates of the excavator working point are calculated based on the working point terrain monitoring steps to obtain the terrain.

[0141] When the loader touches the ground and the excavator experiences a sudden change in pitch angle, it is determined that the excavator is in a support walking mode. At this time, based on the support walking terrain monitoring step, the spatial coordinates of the excavator are calculated in reverse according to the spatial coordinates of the loader, thereby obtaining the terrain.

[0142] When the loader does not touch the ground, it is determined that the excavator is in self-walking mode. At this time, the continuous trajectory points or the above-mentioned work points that form the trajectory are integrated according to the self-walking trajectory of the excavator to solve the terrain elevation and obtain the terrain.

[0143] Step 2: When monitoring the excavator's travel trajectory in real time, if the travel trajectory passes through the recorded trajectory point or the above-mentioned work point again, the travel trajectory that passes through this trajectory point or the above-mentioned work point twice is taken as a loop detection unit, and the closure point detection, elevation increment calculation and elevation closure difference allocation are executed in sequence to complete the terrain elevation loop detection correction.

[0144] Step 3: When the excavator finishes its work, the walking trajectories of self-propelled walking and supported walking are upsampled and downsampled respectively to standardize the resolution. Then, high-precision terrain features are used to replace low-precision terrain features in the overlapping area. Finally, post-processing including edge smoothing is performed to complete the monitoring.

[0145] Step 3 above includes:

[0146] Step 501: Using the resolution of the engineering machinery operation point as the target resolution, the terrain perception data of the engineering machinery self-moving area, i.e., the flat terrain area, is upsampled to the target resolution; the high-density terrain perception data of the supporting walking area, i.e., the complex terrain area, is downsampled to the target resolution, and finally the resolution is standardized.

[0147] Step 502: Use high-resolution DEM data to cover and replace overlapping areas to ensure the accuracy of terrain representation and spatial consistency of data.

[0148] Step 503 employs methods including Gaussian filtering, median filtering, mean smoothing, and morphological closing operations to eliminate abrupt changes, small holes, and other anomalies at the splicing boundaries.

[0149] This application also provides a terrain monitoring device based on engineering machinery operations. It should be noted that this terrain monitoring device based on engineering machinery operations can be used to execute the terrain monitoring method based on engineering machinery operations provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0150] The following describes the terrain monitoring device based on engineering machinery operation provided in the embodiments of this application.

[0151] Figure 7 This is a schematic diagram of a terrain monitoring device based on engineering machinery operations according to an embodiment of this application. Figure 7 As shown, the device includes a detection unit 10, a determination unit 20, a calibration unit 30, and an integration unit 40. The detection unit 10 detects the operating parameters of the construction machinery and determines the operating condition of the construction machinery based on the operating parameters. The operating conditions include a working condition, a support-walking condition, and a self-walking condition. The working condition is when the construction machinery is performing work. The support-walking condition is when the loader of the construction machinery is in contact with the work surface but not performing work. The self-walking condition is when the loader of the construction machinery is not in contact with the work surface and is not performing work. The determination unit 20, when the construction machinery passes a trajectory point or work point on the operating trajectory for the first time, determines the operating condition of the construction machinery based on the operating condition and the operating trajectory of the construction machinery, using a method similar to the above... The calculation method corresponding to the operating conditions of the construction machinery determines the terrain data of the aforementioned trajectory points or work points, and obtains the initial terrain data of the construction machinery corresponding to the aforementioned operating conditions; the correction unit 30 is used to determine the terrain data of the aforementioned trajectory points or work points each time the construction machinery passes through them (except for the first time), when the construction machinery passes through the same trajectory points or work points multiple times and no work has been performed at the aforementioned trajectory points or work points, obtains corrected terrain data, and uses the corrected terrain data to correct the aforementioned initial terrain data, thereby obtaining the target terrain data of the construction machinery corresponding to the aforementioned operating conditions; the integration unit 40 is used to standardize and integrate the target terrain data corresponding to all the aforementioned operating conditions to obtain the target operating terrain of the construction machinery.

[0152] The aforementioned terrain monitoring device based on construction machinery operation of this application includes a detection unit, a determination unit, a correction unit, and an integration unit. The detection unit is used to determine the operating conditions of the construction machinery based on its operating parameters. The operating conditions include work conditions, support walking conditions, and self-walking conditions. The determination unit is used to determine the terrain data of the trajectory point or work point when the construction machinery passes the trajectory point or work point on its operating trajectory for the first time, based on the operating conditions and the operating trajectory of the construction machinery, using a calculation method corresponding to the operating conditions of the construction machinery, to obtain the initial terrain data of the construction machinery corresponding to the operating conditions. The correction unit is used to correct the initial terrain data using corrected terrain data when the construction machinery passes the same trajectory point or work point multiple times and the trajectory point or work point has not been used for operation, to obtain the target terrain data of the construction machinery corresponding to the operating conditions. The integration unit is used to standardize and integrate the target terrain data corresponding to all operating conditions to obtain the target operating terrain of the construction machinery. This device integrates terrain data and excavator operating posture data to achieve low-cost, high-precision dynamic perception of the terrain under all working conditions, including the excavator's self-walking area, support walking area, and non-walking area. Based on this, it uses a cross-scale terrain data fusion and reconstruction method to stitch together multi-scale elevation data, enabling rapid, low-cost, and high-precision acquisition of terrain change data for construction machinery operations. This solves the problem that existing technologies cannot accurately detect the operating terrain of construction machinery.

[0153] In some optional embodiments, the above-mentioned operating condition is the above-mentioned support walking condition. The determining unit includes a first acquisition module, a first determining module, a second acquisition module, and a second determining module. The first acquisition module is used to acquire the coordinates of the first support point and the second support point of the construction machinery based on the above-mentioned operating trajectory of the construction machinery. The first support point coordinates are the coordinates of the support point where the loader of the construction machinery contacts the working surface in the world coordinate system, and the second support point coordinates are the coordinates of the support point in the pixel coordinate system of the target image. The target image is acquired by the image acquisition device installed on the construction machinery. The first determining module is used to determine the coordinates of the first support point based on the above-mentioned first support point coordinates when the rate of change of the pitch angle of the construction machinery is greater than a preset rate of change. The first calculated coordinates are obtained by using the coordinates of the second support point and the image acquisition device in the world coordinate system. The second acquisition module obtains the installation height of the image acquisition device and the angle between the operating direction and the heading angle of the construction machinery. The second determination module, based on the calculated angle, uses the first calculated coordinates, the installation height, and a rotation matrix to determine the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system, obtaining the terrain data of the trajectory point or the working point. The contact point between the base of the construction machinery and the working surface is the trajectory point or the working point on the operating trajectory. This method achieves, for the first time, high-precision reconstruction of the three-dimensional coordinates of the contact point between the machine body and the ground without additional elevation sensors in complex terrains such as steep slopes and soft ground.

[0154] In some optional embodiments, the second determining module includes a first determining submodule and a second determining submodule. The first determining submodule is used to determine the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system when the calculated angle is less than a preset angle, using a first distance, the first calculated coordinates, the equipment installation height, and a rotation matrix, to obtain the terrain data of the trajectory point or the working point. The first distance is the distance between the vertical body of the image acquisition device and the track drive wheel of the construction machinery. The second determining submodule is used to determine the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system when the calculated angle is greater than or equal to the preset angle, using a second distance, the first calculated coordinates, the equipment installation height, and a rotation matrix, to obtain the terrain data of the trajectory point or the working point. The second distance is the distance between the vertical base of the image acquisition device and the outer edge of the track of the construction machinery. This accurately distinguishes between two typical support methods of excavators in complex terrain, making the terrain calculation closer to the actual mechanical contact state and significantly improving the positioning accuracy of the base contact point coordinates.

[0155] In some optional embodiments, the above-mentioned operating condition is the above-mentioned self-propelled operating condition. The determining unit includes a third determining module, a third acquiring module, a fourth acquiring module, and a fourth determining module. The third determining module is used to determine the GNSS positioning data of the initial working point when the above-mentioned engineering machinery performs its first operation as the initial three-dimensional coordinates, and to determine a planar grid reference system based on the above-mentioned initial three-dimensional coordinates. The planar grid reference system is composed of multiple planar grid cells. The third acquiring module is used to obtain the average pitch angle and average roll angle of the above-mentioned engineering machinery during the movement process of each of the above-mentioned planar grid cells based on the above-mentioned operating trajectory of the above-mentioned engineering machinery. The fourth acquiring module is used to obtain the horizontal displacement of the above-mentioned engineering machinery, and to calculate the elevation change value corresponding to the horizontal displacement based on the above-mentioned average pitch angle and average roll angle corresponding to each of the above-mentioned planar grid cells. The fourth determining module is used to integrate and accumulate all the above-mentioned elevation change values ​​to obtain the current elevation value, and to determine the terrain data of the above-mentioned trajectory point or the above-mentioned working point based on the above-mentioned current elevation value. High-precision (centimeter-level) terrain reconstruction in flat terrain areas is achieved.

[0156] In some optional embodiments, the third acquisition module includes a first acquisition submodule, a first calculation submodule, and a second calculation submodule. The first acquisition submodule is used to acquire, based on the operating trajectory of the engineering machinery, the unit movement step length, the unit pitch angle change value, and the unit roll angle change value when the engineering machinery moves one of the planar grid cells. The first calculation submodule is used to calculate the average pitch angle corresponding to the planar grid cell based on the unit movement step length and the unit pitch angle change value. The second calculation submodule is used to calculate the average roll angle corresponding to the planar grid cell based on the unit movement step length and the unit roll angle change value. This significantly improves the spatiotemporal consistency of the terrain elevation integration process and avoids elevation drift caused by attitude sampling jitter or delay.

[0157] In some optional embodiments, the correction unit includes a fifth determining module, a calculation module, and a correction module. The fifth determining module is used to determine the difference between the corrected terrain data calculated when the construction machinery passes the trajectory point or the work point in the current instance and the corrected terrain data calculated when the construction machinery passed the trajectory point or the work point in the previous instance, as the first elevation increment. The calculation module is used to calculate the absolute value of the difference between the corrected terrain data calculated when the construction machinery passes the trajectory point or the work point in any two adjacent instances, obtaining multiple calculated absolute values. The correction module is used to correct the initial terrain data when the construction machinery passes the trajectory point or the work point in two adjacent instances based on the sum of all the calculated absolute values ​​and the first elevation increment, to obtain the target terrain data of the construction machinery corresponding to the operating conditions. This significantly improves the overall consistency and absolute accuracy of the terrain model in long-term operation scenarios.

[0158] In some optional embodiments, the integration unit includes a sixth determining module, a first processing module, a data fusion module, and a second processing module. The sixth determining module is used to determine the resolution of the working point of the engineering machinery as the target resolution. The first processing module is used to upsample the target terrain data corresponding to the self-propelled walking condition to the target resolution and downsample the target terrain data corresponding to the supported walking condition to the target resolution, obtaining resolution-standardized terrain data. The data fusion module is used to cover the overlapping area of ​​the target terrain data corresponding to the self-propelled walking condition and the target terrain data corresponding to the supported walking condition based on the resolution-standardized terrain data, obtaining data-fused terrain data. The second processing module is used to perform edge smoothing and post-processing on the data-fused terrain data to obtain the target working terrain of the engineering machinery. This approach preserves the detail accuracy of the working point while also considering the continuity and smoothness of the large-scale terrain, ultimately outputting a unified high-precision DEM (Digital Elevation Model) with engineering application value.

[0159] The aforementioned terrain monitoring device based on engineering machinery operations includes a processor and a memory. The aforementioned detection units are all stored as program units in the memory, and the processor executes these program units to achieve the corresponding functions. All of the aforementioned modules reside in the same processor; alternatively, the modules may be located in different processors in any combination.

[0160] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem of inaccurate detection of the working terrain for construction machinery in existing technologies.

[0161] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0162] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the terrain monitoring method based on engineering machinery operations.

[0163] This invention provides a processor for running a program, wherein the program executes the terrain monitoring method based on engineering machinery operations.

[0164] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements at least the steps of a terrain monitoring method based on engineering machinery operations. The device described herein may be a server, PC, PAD, mobile phone, etc.

[0165] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform a program that initializes a terrain monitoring method with at least one step based on engineering machinery operations.

[0166] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0167] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0168] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0169] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0170] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0171] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0172] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0173] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0175] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0176] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0177] 1) The terrain monitoring method based on construction machinery operation described in this application first determines the operating conditions of the construction machinery based on its operating parameters. The operating conditions include work conditions, support walking conditions, and self-walking conditions. Then, when the construction machinery passes a trajectory point or work point on its operating trajectory for the first time, the terrain data of the trajectory point or work point is determined using a calculation method corresponding to the operating conditions, based on the operating conditions and the operating trajectory of the construction machinery, thus obtaining the initial terrain data of the construction machinery corresponding to the operating conditions. Then, when the construction machinery passes the same trajectory point or work point multiple times and no work has been performed at the trajectory point or work point, the initial terrain data is corrected using corrected terrain data to obtain the target terrain data of the construction machinery corresponding to the operating conditions. Finally, the target terrain data corresponding to all operating conditions are standardized and integrated to obtain the target operating terrain of the construction machinery. This method integrates terrain data and excavator operating posture data to achieve low-cost, high-precision dynamic perception of the terrain across all working conditions, including the excavator's self-walking area, support walking area, and non-walking area. Based on this, multi-scale elevation data is stitched together using a cross-scale terrain data fusion and reconstruction method, enabling rapid, low-cost, and high-precision acquisition of terrain change data for construction machinery operations. This solves the problem that existing technologies cannot accurately detect the operating terrain of construction machinery.

[0178] 2) The terrain monitoring device based on construction machinery operation described in this application includes a detection unit, a determination unit, a correction unit, and an integration unit. The detection unit is used to determine the operating conditions of the construction machinery based on its operating parameters. The operating conditions include work conditions, support walking conditions, and self-walking conditions. The determination unit is used to determine the terrain data of the trajectory point or work point when the construction machinery passes the trajectory point or work point on its operating trajectory for the first time, based on the operating conditions and the operating trajectory of the construction machinery, using a calculation method corresponding to the operating conditions of the construction machinery, to obtain the initial terrain data of the construction machinery corresponding to the operating conditions. The correction unit is used to correct the initial terrain data using corrected terrain data when the construction machinery passes the same trajectory point or work point multiple times and the trajectory point or work point has not been used for operation, to obtain the target terrain data of the construction machinery corresponding to the operating conditions. The integration unit is used to standardize and integrate the target terrain data corresponding to all operating conditions to obtain the target operating terrain of the construction machinery. This device integrates terrain data and excavator operating posture data to achieve low-cost, high-precision dynamic perception of the terrain under all working conditions, including the excavator's self-walking area, support walking area, and non-walking area. Based on this, it uses a cross-scale terrain data fusion and reconstruction method to stitch together multi-scale elevation data, enabling rapid, low-cost, and high-precision acquisition of terrain change data for construction machinery operations. This solves the problem that existing technologies cannot accurately detect the operating terrain of construction machinery.

[0179] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A terrain monitoring method based on engineering machinery operations, characterized in that, include: The operating parameters of the construction machinery are detected, and the operating conditions of the construction machinery are determined based on the operating parameters. The operating conditions include working conditions, support and travel conditions, and self-propelled travel conditions. The working conditions are when the construction machinery is performing operations. The support and travel conditions are when the loader of the construction machinery is in contact with the working surface, the excavator experiences a sudden change in pitch angle, and is not performing operations. The self-propelled travel conditions are when the loader of the construction machinery is not in contact with the working surface and is not performing operations. When the construction machinery passes a trajectory point or work point on its operating trajectory for the first time, the terrain data of the trajectory point or work point is determined by a calculation method corresponding to the operating conditions of the construction machinery, based on the operating conditions of the construction machinery and the operating trajectory of the construction machinery, so as to obtain the initial terrain data of the construction machinery corresponding to the operating conditions. When the construction machinery passes through the same trajectory point or work point multiple times and no work has been performed at the trajectory point or work point, the terrain data of the trajectory point or work point is determined for each passage of the construction machinery except for the first passage, corrected terrain data is obtained, and the corrected terrain data is used to correct the initial terrain data to obtain the target terrain data of the construction machinery corresponding to the operating conditions. The target terrain data corresponding to all the operating conditions are standardized and integrated to obtain the target operating terrain of the engineering machinery; The operating condition is the supporting walking condition. Based on the operating condition of the construction machinery and its running trajectory, the terrain data of the trajectory points or the work points are determined using a corresponding calculation method, including: Based on the operating trajectory of the construction machinery, the coordinates of the first support point and the second support point of the construction machinery are obtained. The coordinates of the first support point are the coordinates of the support point where the loader of the construction machinery contacts the working surface in the world coordinate system. The coordinates of the second support point are the coordinates of the support point in the pixel coordinate system of the target image. The target image is acquired by the image acquisition device installed on the construction machinery. When the rate of change of the pitch angle of the engineering machinery is greater than the preset rate of change, the three-dimensional coordinates of the image acquisition device in the world coordinate system are determined based on the coordinates of the first support point and the coordinates of the second support point, and the first calculated coordinates are obtained. The installation height of the image acquisition device is obtained, and the angle between the running direction of the construction machinery and the heading angle of the construction machinery is obtained to calculate the angle. Based on the magnitude of the calculated angle, using the first calculated coordinates, the equipment installation height, and the rotation matrix, the three-dimensional coordinates of the contact point between the base of the engineering machinery and the working surface in the world coordinate system are determined, and the terrain data of the trajectory point or the working point is obtained. The contact point between the base of the engineering machinery and the working surface is the trajectory point or the working point on the running trajectory.

2. The terrain monitoring method according to claim 1, characterized in that, Based on the magnitude of the calculated angle, using the first calculated coordinates, the equipment installation height, and the rotation matrix, the three-dimensional coordinates of the contact point between the base of the engineering machinery and the working surface in the world coordinate system are determined, obtaining the terrain data of the trajectory point or the working point, including: When the calculated angle is less than the preset angle, the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system are determined by using the first distance, the first calculated coordinates, the equipment installation height and the rotation matrix, so as to obtain the terrain data of the trajectory point or the working point. The first distance is the distance between the vertical body of the image acquisition device and the track drive wheel of the construction machinery. When the calculated angle is greater than or equal to the preset angle, the three-dimensional coordinates of the contact point between the base of the engineering machinery and the working surface in the world coordinate system are determined by using the second distance, the first calculated coordinates, the equipment installation height, and the rotation matrix, so as to obtain the terrain data of the trajectory point or the working point. The second distance is the distance between the vertical base of the image acquisition device and the outer edge of the track of the engineering machinery.

3. The terrain monitoring method according to claim 1, characterized in that, The operating condition is the self-propelled operating condition. Based on the operating condition of the construction machinery and its operating trajectory, the terrain data of the trajectory points or the work points are determined using a corresponding calculation method, including: The GNSS positioning data of the initial working point when the engineering machinery performs its first operation is determined as the initial three-dimensional coordinates, and a planar grid reference system is determined based on the initial three-dimensional coordinates. The planar grid reference system is composed of multiple planar grid units. Based on the operating trajectory of the construction machinery, the average pitch angle and average roll angle of the construction machinery during its movement in each planar grid cell are obtained; The horizontal displacement of the engineering machinery is obtained, and the elevation change value corresponding to the horizontal displacement is calculated based on the average pitch angle and the average roll angle corresponding to each of the planar grid cells. Integrate and accumulate all the elevation changes to obtain the current elevation value, and determine the terrain data of the trajectory point or the work point based on the current elevation value.

4. The terrain monitoring method according to claim 3, characterized in that, Based on the operating trajectory of the construction machinery, the average pitch angle and average roll angle of the construction machinery during its movement in each planar grid cell are obtained, including: Based on the operating trajectory of the construction machinery, obtain the unit movement step, unit pitch angle change, and unit roll angle change of the construction machinery when it moves one of the planar grid cells. The average pitch angle corresponding to the planar grid cell is calculated based on the unit movement step size and the unit pitch angle change value. The average roll angle corresponding to the planar mesh cell is calculated based on the unit movement step size and the unit roll angle change value.

5. The terrain monitoring method according to claim 1, characterized in that, The initial terrain data is corrected using the corrected terrain data to obtain target terrain data for the engineering machinery corresponding to the operating conditions, including: The difference between the corrected terrain data calculated when the engineering machinery passes the trajectory point or the work point in the current instance and the corrected terrain data calculated when the engineering machinery passes the trajectory point or the work point in the previous instance is determined as the first elevation increment; The absolute value of the difference between the corrected terrain data obtained when the engineering machinery passes through the trajectory point or the work point twice in any two adjacent times is calculated, resulting in multiple calculated absolute values; Based on the sum of all the calculated absolute values ​​and the first elevation increment, the initial terrain data when passing the trajectory point or the work point twice consecutively is corrected to obtain the target terrain data of the engineering machinery corresponding to the operating conditions.

6. The terrain monitoring method according to claim 1, characterized in that, The target terrain data corresponding to all the operating conditions are standardized and integrated to obtain the target operating terrain for the engineering machinery, including: The resolution of the working point of the engineering machinery is determined as the target resolution; The target terrain data corresponding to the self-propelled walking condition is upsampled to the target resolution, and the target terrain data corresponding to the supported walking condition is downsampled to the target resolution to obtain resolution-standardized terrain data; Based on the resolution-normalized terrain data, the overlapping areas of the target terrain data corresponding to the self-propelled walking condition and the target terrain data corresponding to the supported walking condition are covered to obtain the fused terrain data. The fused terrain data is subjected to edge smoothing and post-processing to obtain the target operating terrain for the engineering machinery.

7. A terrain monitoring device based on engineering machinery operations, characterized in that, include: The detection unit is used to detect the operating parameters of the construction machinery and determine the operating conditions of the construction machinery based on the operating parameters. The operating conditions include working conditions, support and travel conditions, and self-propelled travel conditions. The working conditions are the conditions in which the construction machinery is performing operations. The support and travel conditions are the conditions in which the loader of the construction machinery is in contact with the working surface, the excavator experiences a sudden change in pitch angle, and is not performing operations. The self-propelled travel conditions are the conditions in which the loader of the construction machinery is not in contact with the working surface and is not performing operations. The determining unit is used to determine the terrain data of the trajectory point or the work point based on the operating conditions of the construction machinery and the operating trajectory of the construction machinery, using a calculation method corresponding to the operating conditions of the construction machinery, and obtain the initial terrain data of the construction machinery corresponding to the operating conditions. The correction unit is used to determine the terrain data of the trajectory point or the work point for each passing of the construction machinery except the first passing, when the construction machinery passes the same trajectory point or the work point multiple times and the trajectory point or the work point has not performed any work, obtain corrected terrain data, and use the corrected terrain data to correct the initial terrain data to obtain the target terrain data of the construction machinery corresponding to the operating conditions. An integration unit is used to standardize and integrate the target terrain data corresponding to all the operating conditions to obtain the target operating terrain of the engineering machinery. The operating condition is the support walking condition. The determining unit includes a first acquisition module, a first determining module, a second acquisition module, and a second determining module. The first acquisition module is used to acquire the coordinates of the first support point and the second support point of the construction machinery according to the running trajectory of the construction machinery. The first support point coordinates are the coordinates of the support point where the loader of the construction machinery contacts the working surface in the world coordinate system. The second support point coordinates are the coordinates of the support point in the pixel coordinate system of the target image. The target image is acquired by the image acquisition device installed on the construction machinery. The first determining module is used to determine the three-dimensional coordinates of the image acquisition device in the world coordinate system based on the coordinates of the first support point and the coordinates of the second support point when the rate of change of the pitch angle of the construction machinery is greater than a preset rate of change, thereby obtaining the first calculated coordinates; the second acquiring module is used to acquire the installation height of the image acquisition device, thereby obtaining the device installation height, and to acquire the angle between the running direction of the construction machinery and the heading direction of the construction machinery, thereby obtaining the calculated angle; the second determining module is used to determine the three-dimensional coordinates of the contact point between the base of the construction machinery and the working surface in the world coordinate system based on the magnitude of the calculated angle, using the first calculated coordinates, the device installation height, and the rotation matrix, thereby obtaining the terrain data of the trajectory point or the working point, wherein the contact point between the base of the construction machinery and the working surface is the trajectory point or the working point on the running trajectory.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the terrain monitoring method based on engineering machinery operation as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing terrain monitoring based on engineering machinery operations as described in any one of claims 1 to 6.

Images