Methods, apparatus, and engineering equipment for detecting operational terrain

Abstract

This disclosure provides a method, apparatus, and engineering equipment for detecting work terrain, relating to the field of computer technology, and particularly to the fields of autonomous driving and unmanned engineering operations. The implementation scheme is as follows: Point cloud data of the work area collected by a lidar at the current time is acquired. The work area is divided into multiple grids, each of which has a corresponding height value. The point cloud data includes the three-dimensional coordinates of multiple sampling points. For any grid: based on the three-dimensional coordinates of the multiple sampling points, an input point of the grid is determined from the multiple sampling points; based on the height coordinates of the input point and the height value of the grid, the type of the input point is determined, including noise points and ground points; and in response to determining that the input point is a ground point, the height value of the grid is updated based on the height coordinates of the input point.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to the fields of autonomous driving and unmanned engineering operation technology. Specifically, it relates to a method and apparatus for detecting work terrain, electronic equipment, computer-readable storage medium, computer program product, and engineering equipment for detecting work terrain. Background Technology

[0002] Construction equipment refers to equipment used in construction operations (such as mining, building construction, pipeline laying, etc.), such as excavators, electric shovels, bulldozers, cranes, road rollers, etc. Typically, construction equipment is operated on-site by workers.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] This disclosure provides a method and apparatus for detecting work terrain, an electronic device, a computer-readable storage medium, a computer program product, and engineering equipment for detecting work terrain.

[0005] According to one aspect of this disclosure, a method for detecting operational terrain is provided, comprising: acquiring point cloud data of an operational area collected by a lidar at a current time, the operational area being divided into multiple grids, each of the multiple grids having a corresponding height value, the point cloud data including the three-dimensional coordinates of multiple sampling points; for any grid among the multiple grids: determining an input point of the grid from the multiple sampling points based on the three-dimensional coordinates of the multiple sampling points; determining the type of the input point based on the height coordinates of the input point and the height value of the grid, the type including noise points and ground points; and updating the height value of the grid based on the height coordinates of the input point in response to determining that the input point is a ground point.

[0006] According to one aspect of this disclosure, a work terrain detection device is provided, comprising: an acquisition module configured to acquire point cloud data of a work area collected by a lidar at a current time, the work area being divided into multiple grids, each of the multiple grids having a corresponding height value, the point cloud data including the three-dimensional coordinates of multiple sampling points; a first determination module configured to, for any grid among the multiple grids, determine an input point of the grid based on the three-dimensional coordinates of the multiple sampling points; a second determination module configured to determine the type of the input point based on the height coordinates of the input point and the height value of the grid, the type including noise points and ground points; and an update module configured to, in response to determining that the input point is a ground point, update the height value of the grid based on the height coordinates of the input point.

[0007] According to one aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0008] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the above-described method.

[0009] According to one aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method.

[0010] According to one aspect of this disclosure, an engineering device is provided, including the aforementioned electronic device.

[0011] According to one or more embodiments of this disclosure, the accuracy of operational terrain detection can be improved.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0014] Figure 1A schematic diagram of an exemplary system in which various methods described herein may be implemented according to embodiments of this disclosure is shown;

[0015] Figure 2 A flowchart of a working terrain detection method according to an embodiment of the present disclosure is shown;

[0016] Figure 3 A flowchart illustrating the process of updating grid height values ​​according to an embodiment of the present disclosure is shown;

[0017] Figure 4 A schematic diagram illustrating the height value update process for different grid cells according to embodiments of the present disclosure is shown;

[0018] Figure 5 A schematic diagram of a terrain detection process according to an embodiment of the present disclosure is shown;

[0019] Figure 6 A schematic diagram of terrain detection results (elevation map) according to an embodiment of the present disclosure is shown;

[0020] Figure 7A , 7B A schematic diagram showing terrain detection results under low dust conditions according to an embodiment of the present disclosure is illustrated;

[0021] Figure 8A , 8B A schematic diagram showing terrain detection results under conditions of high dust levels according to an embodiment of the present disclosure is illustrated.

[0022] Figure 9 A structural block diagram of a working terrain detection device according to an embodiment of the present disclosure is shown; and

[0023] Figure 10 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0026] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0027] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0028] Construction sites (also known as engineering work sites) are typically outdoors in harsh environments. The increasing intelligence and automation of construction equipment is a current development trend. Construction equipment uses sensors mounted on it to perceive its surroundings and utilizes this information for autonomous, unmanned operation, thereby improving the safety and efficiency of construction work.

[0029] In unmanned operation scenarios involving ground operations, engineering equipment needs to detect the terrain in real time (i.e., the undulation of the ground, that is, the height values ​​of different ground areas) and plan and carry out operations based on the terrain.

[0030] Among related technologies, the commonly used methods for detecting operational terrain include the following three:

[0031] The first method uses 3D surface reconstruction techniques (such as Poisson surface reconstruction) to reconstruct the ground point cloud data collected by lidar, generating a grid model of the ground to obtain the operational terrain. This method is computationally intensive and slow, making real-time terrain detection impossible.

[0032] The second method divides the work area into multiple grids and uses point cloud data collected by LiDAR over a period of time to calculate the average height of the point cloud within each grid, which is then used as the terrain height of that grid to generate a grid elevation map. However, dust is commonly present at engineering work sites (e.g., ground dust, smoke, etc.), and this dust can be detected by LiDAR, resulting in a large number of noisy points in the point cloud data. This method directly uses the average height of the point cloud as the ground height, which has low accuracy and does not meet the real-time requirements of engineering operations.

[0033] The third method uses a depth camera to acquire multiple frames of environmental depth images. Based on each frame, a Kalman filter algorithm is used to update the ground height value of each grid cell in real time, generating a grid elevation map. This method has good real-time performance, but its noise reduction effect for dust is not ideal. In addition, the effective range of the depth camera is relatively short (i.e., the depth image is only accurate when the object is close to the depth camera), which is not accurate enough for large engineering equipment, resulting in low accuracy in terrain detection.

[0034] In view of the above problems, the embodiments of this disclosure provide a method for detecting work terrain to achieve accurate and real-time detection of work terrain.

[0035] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0036] Figure 1 A schematic diagram of an exemplary system 100 in which the various methods and apparatus described herein can be implemented according to embodiments of this disclosure is shown. Reference Figure 1 The system 100 includes engineering equipment 110, server 120, and one or more communication networks 130 that couple engineering equipment 110 to server 120.

[0037] The engineering equipment 110 can be any equipment used in engineering operations (such as mineral mining, building construction, pipeline laying, etc.), including but not limited to excavators, electric shovels, bulldozers, cranes, road rollers, etc. In embodiments of this disclosure, the engineering equipment 110 may include electronic devices according to embodiments of this disclosure and / or be configured to perform methods according to embodiments of this disclosure.

[0038] Server 120 may run one or more services or software applications that enable the execution of operational terrain detection methods. In some embodiments, server 120 may also provide other services or software applications that may include non-virtual and virtual environments. Figure 1 In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. A user of engineering device 110 may sequentially interact with server 120 using one or more client applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0039] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0040] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0041] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from engineering equipment 110. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of engineering equipment 110.

[0042] Network 130 can be any type of network well known to those skilled in the art, and can support data communication using any of a variety of available protocols (including, but not limited to, TCP / IP, SNA, IPX, etc.). By way of example only, one or more networks 110 can be Ethernet, satellite communication networks, local area networks (LANs), Ethernet-based networks, token ring, wide area networks (WANs), the Internet, virtual networks, virtual private networks (VPNs), intranets, extranets, public switched telephone networks (PSTNs), infrared networks, wireless networks (including, for example, Bluetooth, Wi-Fi), and / or any combination of these with other networks.

[0043] System 100 may also include one or more databases 150. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 150 may be used to store information such as audio files and video files. Databases 150 may reside in various locations. For example, a database used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. Databases 150 may be of different types. In some embodiments, the database used by server 120 may be a relational database or a non-relational database. One or more of these databases may store, update, and retrieve data from and from the databases in response to commands.

[0044] In some embodiments, one or more of the databases 150 may also be used by an application to store application data. The databases used by the application may be of different types, such as key-value stores, object stores, or regular stores supported by a file system.

[0045] Engineering equipment 110 may include sensors 111 for sensing the surrounding environment. Sensors 111 may include one or more of the following: a visual camera, an infrared camera, an ultrasonic sensor, a millimeter-wave radar, and a lidar (LiDAR). Different sensors can provide different detection accuracies and ranges. Multiple identical sensors can be used to expand the detection range. Cameras can be installed in front of, behind, or at other locations on the engineering equipment. Visual cameras can capture the situation inside and outside the engineering equipment in real time and present it to personnel. Furthermore, by analyzing the images captured by the visual cameras, information such as the equipment environment and the operating status of other engineering equipment can be obtained. Infrared cameras can capture objects in night vision conditions. Ultrasonic sensors can be installed around the engineering equipment to measure the distance of environmental objects from the equipment using the strong directionality of ultrasound. Millimeter-wave radar can be installed in front of, behind, or at other locations on the engineering equipment to measure the distance of environmental objects from the equipment using the characteristics of electromagnetic waves. LiDAR can be installed in front of, behind, or at other locations on the engineering equipment to detect object edges and shape information, thereby enabling object recognition and tracking. Due to the Doppler effect, the radar device can also measure the speed changes of the engineering equipment and moving objects.

[0046] Engineering equipment 110 may also include a communication device 112. The communication device 112 may include a satellite positioning module capable of receiving satellite positioning signals (e.g., BeiDou, GPS, GLONASS, and GALILEO) from satellite 141 and generating coordinates based on these signals. The communication device 112 may also include a Real-Time Kinematic (RTK) module, capable of real-time centimeter-level positioning in the field, improving positioning accuracy and operational efficiency. The communication device 112 may also include a module for communicating with a mobile communication base station 142. The mobile communication network can implement any suitable communication technology, such as current or emerging wireless communication technologies (e.g., 5G technology) such as GSM / GPRS, CDMA, and LTE. When the engineering equipment is an engineering vehicle (e.g., excavator, bulldozer, etc.), the communication device 112 may also have a vehicle-to-everything (V2X) module, configured to enable vehicle-to-the-world communication, such as vehicle-to-vehicle (V2V) communication with other engineering vehicles 143 and vehicle-to-infrastructure (V2I) communication with infrastructure 144. Furthermore, the communication device 112 may also have a module configured to communicate with user terminals 145 (including but not limited to smartphones, tablets, or wearable devices such as watches) via, for example, an IEEE 802.11 standard wireless LAN or Bluetooth. Using the communication device 112, the engineering equipment 110 can also access the server 120 via network 130.

[0047] Engineering equipment 110 may also include a control device 113. The control device 113 may include a processor, such as a central processing unit (CPU) or graphics processing unit (GPU), or other dedicated processors, that communicates with various types of computer-readable storage devices or media. In the case where engineering equipment 110 is an engineering vehicle, the control device 113 may include an autonomous driving system for automatically controlling various actuators in the vehicle. The autonomous driving system is configured to control the powertrain, steering system, and braking system of engineering equipment 110 (not shown) in response to inputs from multiple sensors 111 or other input devices via multiple actuators to control acceleration, steering, and braking respectively, without human intervention or with limited human intervention. Some processing functions of the control device 113 can be implemented via cloud computing. For example, some processing can be performed using an onboard processor while other processing can be performed using cloud computing resources. The control device 113 may be configured to perform methods according to this disclosure. Furthermore, the control device 113 may be implemented as an example of an electronic device on the engineering equipment side (client) according to this disclosure.

[0048] Figure 1 The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0049] Figure 2 A flowchart of a working terrain detection method 200 according to an embodiment of the present disclosure is shown. Method 200 is typically used in engineering equipment (e.g., Figure 1 The engineering equipment 110 shown is executed locally, thereby avoiding data transmission and improving the real-time performance and computational efficiency of the terrain detection. Understandably, in some cases, method 200 can also be performed on a server (e.g., Figure 1 The execution is performed at server 120 (as shown). That is, the execution entity of each step of method 200 is usually... Figure 1 The engineering equipment 110 shown (specifically, it may be the control device 113 in the engineering equipment 110), or it may be Figure 1 Server 120 is shown in the image.

[0050] like Figure 2 As shown, method 200 includes steps 210-240.

[0051] In step 210, point cloud data of the work area collected by the lidar at the current time is acquired. The work area is divided into multiple grids, and each grid has a corresponding height value. The point cloud data includes the three-dimensional coordinates of multiple sampling points.

[0052] In step 220, for any grid among multiple grids, the input point of the grid is determined from the multiple sampling points based on the three-dimensional coordinates of the multiple sampling points.

[0053] In step 230, the type of input point is determined based on the height coordinates of the input point and the height value of the grid. The types include noise points and ground points.

[0054] In step 240, in response to determining that the input point is a ground point, the height value of the raster is updated based on the height coordinates of the input point.

[0055] According to embodiments of this disclosure, point cloud data collected by LiDAR can be acquired in real time, and based on the three-dimensional coordinates of each sampling point, the sampling point (i.e., input point) corresponding to each grid for updating the grid height value can be determined. Furthermore, by determining whether the input point of the grid is a noise point or a ground point, and updating the grid height value only based on the height coordinates of the ground point, noise points can be effectively filtered out, improving the accuracy of terrain detection and achieving accurate and real-time operational terrain detection.

[0056] The steps of method 200 are described in detail below.

[0057] In step 210, the point cloud data of the work area collected by the lidar at the current time is acquired.

[0058] As mentioned earlier, lidar can be fixed in front of, to the side of, or in other locations on engineering equipment, and multiple lidar units can be installed. Lidar emits laser beams into the surrounding environment to collect point cloud data of the surrounding environment.

[0059] The point cloud data acquired by the lidar includes a large number of sampling points. Each sampling point includes three-dimensional coordinates (x, y, z) representing its spatial position. The x and y coordinates represent the horizontal position of the sampling point, and the z coordinate represents its vertical position, i.e., its altitude. In the embodiments of this disclosure, the z coordinate is referred to as the altitude coordinate. Typically, each sampling point also includes the laser reflection intensity and its attitude angles relative to the lidar (including yaw, pitch, and roll angles).

[0060] Typically, the original three-dimensional coordinates of the sampling points acquired by lidar are three-dimensional coordinates in the lidar coordinate system. According to some embodiments, in order to facilitate the detection of the working terrain and generate a grid elevation map of the ground, it is necessary to transform the three-dimensional coordinates in the lidar coordinate system to the world coordinate system (such as the UTM coordinate system, WGS84 coordinate system, etc.). That is, the three-dimensional coordinates in step 210 refer to the three-dimensional coordinates in the world coordinate system.

[0061] According to some embodiments, the engineering equipment housing the lidar is equipped with a positioning module, such as an RTK module. The RTK module can collect its own pose information in real time, including its three-dimensional coordinates and attitude angles (including yaw, pitch, and roll angles) in the world coordinate system.

[0062] Based on the pose information acquired by RTK, point cloud data in the LiDAR coordinate system can be transformed to the world coordinate system. Specifically, the positions of the RTK module and the LiDAR relative to the engineering equipment can be calibrated. Based on the pose information acquired by the RTK module, the transformation relationship between the RTK coordinate system and the world coordinate system can be obtained (the transformation relationship typically includes translation vectors and transformation matrices). Based on the calibrated positions of the RTK module and the LiDAR relative to the engineering equipment, the transformation relationship between the LiDAR coordinate system and the RTK coordinate system can be obtained. Based on the transformation relationship between the LiDAR coordinate system and the RTK coordinate system, and the transformation relationship between the RTK coordinate system and the world coordinate system, the transformation relationship between the LiDAR coordinate system and the world coordinate system can be obtained. Based on the loop relationship between the LiDAR coordinate system and the world coordinate system, the 3D coordinates in the LiDAR coordinate system can be transformed to the world coordinate system.

[0063] LiDAR can continuously emit laser beams to continuously collect point cloud data of the surrounding environment. According to some embodiments, to facilitate real-time detection of the work terrain, the LiDAR transmits the collected point cloud data back to the control device of the engineering equipment at regular time intervals. Correspondingly, the control device of the engineering equipment acquires the point cloud data collected by the LiDAR at regular time intervals. In the embodiments of this disclosure, each frame of point cloud data transmitted back by the LiDAR can be referred to as "one frame of point cloud data." According to some embodiments, the time interval for the LiDAR to transmit the point cloud data back to the device can be set to a small value, such as 0.1 seconds, thereby improving the real-time performance and accuracy of the work terrain detection.

[0064] According to some embodiments, point cloud data is obtained by scanning the work area with a lidar using a non-repeating scanning path. It should be noted that the non-repeating scanning path in this embodiment refers to the scanning path not repeating within each time interval, not that the scanning path is not repeated throughout the entire operation of the lidar. Non-repeating scanning can collect dense point cloud data corresponding to different locations over time without moving the lidar (while common 360° repetitive scanning can only obtain dense point clouds by moving the lidar), improving the stability and accuracy of data acquisition, thereby enhancing the accuracy of terrain detection.

[0065] In addition, the number of laser emitters required by non-repetitive scanning lidar (the number of lines is usually in the single digits) is usually much smaller than that of 360° repetitive scanning lidar (which usually needs to reach 64, 128 or more lines in order to collect denser point cloud data), thus greatly reducing costs.

[0066] It is understandable that the scanning range of LiDAR is typically large. Considering that the working area of ​​engineering equipment can only be the area that the equipment can reach, according to some embodiments, the working area in step 210 can be set as an area of ​​a certain size centered on the location of the engineering equipment (the location of the engineering equipment can be obtained, for example, through an RTK module), such as a 100m*100m area. Further, the working area is divided into multiple grids, each with a fixed size (e.g., 0.2m*0.2m). In the embodiments of this disclosure, the size of a single grid can be referred to as the resolution of the working area. Based on the above embodiments, by acquiring the location of the engineering equipment in real time and determining the working area based on the location of the engineering equipment, processing only the point cloud data within the working area can reduce the amount of data to be processed, thereby improving the efficiency and real-time performance of the terrain detection.

[0067] In the embodiments of this disclosure, the position of each grid is indexed by horizontal and vertical coordinates (x-coordinate, y-coordinate), and terrain information is recorded by storing the height value of the grid.

[0068] In step 220, the input point of each grid is determined from the multiple sampling points based on the three-dimensional coordinates of the multiple sampling points.

[0069] According to some embodiments, the input point for each grid can be determined using the following steps: based on the three-dimensional coordinates of multiple sampling points, a set of sampling points corresponding to the grid is determined, the set of sampling points including sampling points located within the grid; and the sampling point with the largest height coordinate in the set of sampling points is used as the input point for the grid. The input point is used to update the height value of the grid. Depending on the actual situation, the set of sampling points corresponding to the grid can include any number of sampling points. For example, in some cases, the set of sampling points for the grid can include one or more sampling points. In other cases, the set of sampling points for the grid can also be empty (null).

[0070] It is understandable that different grids correspond to different numbers of sampling points and different height distributions at the same time; the same grid also corresponds to different numbers of sampling points and different height distributions at different times. This variation makes terrain reconstruction difficult. Based on the above embodiment, by using the sampling point with the largest height coordinate in the sampling point set as the input point, the stability of the input data can be guaranteed. Furthermore, by discarding other sampling points, computational efficiency can be improved, storage space can be saved, and real-time terrain detection can be achieved. At the same time, the above embodiment can also avoid the interference caused by multiple height values ​​returned when the LiDAR scans a vertical object.

[0071] According to other embodiments, the average height coordinates of each sampling point in the sampling point set can also be used as the input point of the grid.

[0072] In step 230, the type of input point is determined based on the height coordinates of the input point and the height value of the grid. The types of input points include noise points and ground points.

[0073] According to some embodiments, an input point is determined to be a noise point in response to the difference between the height coordinates of the input point and the height value of the grid being greater than a first threshold. The first threshold can be set, for example, to 0.5m. It is understood that since dust floats above the ground, the height of noise points is typically higher than the ground level (i.e., the height value of the grid). By comparing the height coordinates of the input point with the height value of the grid, noise points can be quickly identified.

[0074] According to some embodiments, an input point is determined to be a ground point in response to the following conditions being met: the difference between the height coordinate of the input point and the height value of the grid is less than or equal to the first threshold (condition one); or the difference between the height coordinate of the input point and the height value of the grid is greater than the first threshold, and the number of input points identified as noise points in the grid within a preset time period is greater than or equal to a second threshold (condition two). The preset time period can be set to, for example, 1 second, and the second threshold can be set to, for example, 5 seconds.

[0075] In the above embodiments, under condition one, ground points can be quickly and accurately identified by comparing the height coordinates of the input point with the height value of the grid when the ground height within the grid decreases, remains constant, or increases slowly (the increase per unit time is less than a first threshold). Under condition two, ground points can be quickly and accurately identified when the ground height within the grid increases rapidly (the increase per unit time is greater than or equal to a first threshold).

[0076] According to some embodiments, each grid has a corresponding noise point container and a ground point container, which are used to store the most recent historical noise point and historical ground point, respectively. The noise point container and the ground point container can be implemented as any data structure such as an array, linked list, or set.

[0077] Accordingly, according to some embodiments, method 200 further includes: in response to determining that an input point is a noise point, adding the input point to the noise point container corresponding to the grid; and in response to determining that the number of noise points in the noise point container is greater than or equal to a third threshold, deleting the noise point with the earliest acquisition time in the noise point container. Thus, the noise point container stores only a small number (less than the third threshold) of historical noise points most recent to the current time, saving storage space and enabling real-time terrain detection.

[0078] According to some embodiments, method 200 further includes: in response to determining that the input point is a ground point, adding the input point to the ground point container corresponding to the grid; and in response to determining that the number of ground points in the ground point container is greater than or equal to a fourth threshold, deleting the ground point with the earliest acquisition time from the ground point container. Thus, the ground point container stores only a small number (less than the fourth threshold) of historical ground points most recent to the current time, saving storage space, reducing computational load, improving the calculation efficiency of grid height values, and enabling real-time terrain detection.

[0079] According to some embodiments, the third threshold and the fourth threshold can be set to the same value, such as 5. Furthermore, the values ​​of the third threshold and the fourth threshold can be the same as the second threshold, for example, all three being 5.

[0080] In step 240, in response to step 230 determining that the input point is a ground point, the height value of the raster is updated based on the height coordinates of the input point. Specifically, the height value of the raster can be updated to the average height coordinates of all ground points in the ground point container. Based on the above embodiment, by using the average height coordinates of ground points (i.e., all ground points in the ground point container) over a recent period as the height value of the raster, the accuracy and real-time performance of the operational terrain detection can be improved.

[0081] It should be noted that steps 210-240 above describe the process of updating the height values ​​of each grid cell in real time. Whenever the LiDAR returns a frame of point cloud data, steps 210-240 are executed to update the height values ​​of each grid cell.

[0082] It is understood that method 200 may also include an initialization process for the height values ​​of each grid cell. According to some embodiments, the initial value of the grid cell's height value can be set to the height coordinates of the first input point of that grid cell. Specifically, after determining the input point of a grid cell in step 220, if the current height value of the grid cell is null, meaning that the grid cell currently has no corresponding historical point cloud data, then the height value of the grid cell is initialized to the height coordinates of the current input point. If the current height value of the grid cell is not null, then the height value of the grid cell is updated through steps 230 and 240 described above.

[0083] Figure 3 A flowchart illustrating a process 300 for updating grid height values ​​according to an embodiment of the present disclosure is shown. Figure 3 As shown, process 300 includes steps 310-360. In steps 310-360, the steps shown in parallelogram boxes are data input / output steps, the steps shown in rectangle boxes are data processing steps, and the steps shown in diamond boxes are judgment steps.

[0084] like Figure 3 As shown, in step 310, the three-dimensional coordinates (x, y, z) of the input point P and the acquisition time t of the input point P are obtained.

[0085] Subsequently, in step 320, it is determined whether the difference between the height coordinate z (i.e., Pz) of the input point P and the current height value of the grid is greater than 0.5m (i.e., the first threshold), that is, whether Pz > height + 0.5m is satisfied.

[0086] If step 320 determines no, then step 340 is executed, the input point P is added to the ground point container ground as a ground point, and the acquisition time t of the input point P is stored.

[0087] If step 320 is correct, then step 330 is executed, adding the input point P as a noise point to the noise point container fog. Subsequently, step 350 is executed to determine whether the number of noise points added to the fog container within 1 second (i.e., the preset time) reaches 5 (i.e., the third threshold).

[0088] If step 350 determines that the condition is yes, then step 340 is executed, adding the input point P as a ground point to the ground point container and storing the acquisition time t of the input point P. Subsequently, step 360 is further executed.

[0089] If step 350 determines otherwise, then proceed to step 360.

[0090] In step 360, the height value of the grid is updated to the average height coordinates of all ground points in the ground point container, i.e., height = average height of all ground points in the ground.

[0091] Figure 4 A schematic diagram illustrating the height value update process for different grids according to embodiments of the present disclosure is shown. Figure 4 Each curve in the graph corresponds to a grid.

[0092] In each graph, the horizontal axis represents the number of times the input point P(x,y,z) is entered into the grid (corresponding to time), and the vertical axis represents the height value (in meters). Solid lines represent the height coordinate (z-coordinate) of the input point P, and dashed lines represent the grid height. Figure 4 As can be seen, the method of this disclosure can effectively filter abrupt dust noise (see graphs grid_1, grid_2, and grid_3), while also responding promptly to normal ground height increases (e.g., material feeding) (see graph grid_4). Furthermore, as shown in graphs grid_5 and grid_6, the embodiments of this disclosure can effectively stabilize the grid height values ​​even when the height coordinates of sampling points in certain areas oscillate due to non-repetitive scanning.

[0093] Figure 5 A schematic diagram of a terrain detection process 500 according to an embodiment of the present disclosure is shown. Figure 5 As shown, process 500 includes steps 510-570. In steps 510-570, the steps shown in parallelogram boxes are data input / output steps, and the steps shown in rectangle boxes are data processing steps.

[0094] In step 510, pose information is acquired through the RTK module to determine the position of the engineering equipment.

[0095] In step 520, a certain size area (e.g., 100m*100m) centered on the engineering equipment is taken as the work area, and the work area is divided into multiple grids of fixed size (e.g., 0.2m*0.2m) to obtain an initialized elevation map.

[0096] In steps 530 and 540, the single-frame point cloud data (relative to the lidar coordinate system) collected by the lidar at the current time and the pose information collected by the RTK module are obtained respectively.

[0097] In step 550, based on the pose information acquired by the RTK module, the point cloud data obtained in step 530 is converted to the world coordinate system, and the grid corresponding to each sampling point is determined based on the three-dimensional coordinates of each sampling point. Thus, the set of sampling points corresponding to each grid can be obtained. Further, for each grid, the sampling point with the largest height coordinate in the set of sampling points corresponding to that grid is used as the input point of that grid.

[0098] In step 560, the raster point cloud data is denoised, that is, the input points of the raster are determined to be noise points or ground points.

[0099] Subsequently, in step 570, the height value of the grid is updated based on the determination result of step 560. Specifically, if step 560 determines that the input point is a ground point, the height value of the grid is updated based on the height coordinates of that ground point.

[0100] It is understandable that steps 530-570 can be executed repeatedly to achieve real-time and dynamic detection of the working terrain.

[0101] Figure 6 A schematic diagram of the operational terrain detection results (elevation map) according to an embodiment of the present disclosure is shown. Figure 6 Each mesh vertex corresponds to a grid cell in the raster elevation map, and different grayscale values ​​represent different height values. The height value of each mesh vertex corresponds to the height value of the raster cell, and the height values ​​of other locations (such as the edges and faces of each mesh) are obtained through interpolation.

[0102] According to embodiments of this disclosure, accurate work terrain detection can still be achieved even in the presence of a large amount of dust in the environment.

[0103] Figure 7A , 7B A schematic diagram illustrating terrain detection results under low-dust conditions according to an embodiment of the present disclosure is shown. Figure 7A , 7B As shown, when there is less dust in the environment, there are fewer noise points in the point cloud data, and the point cloud data 710 is basically consistent with the generated raster elevation map 720.

[0104] Figure 8A , 8B A schematic diagram illustrating terrain detection results under dusty conditions according to an embodiment of the present disclosure is shown. Figure 8A , 8B As shown, in environments with high levels of dust, point cloud data contains a large number of noisy points 810. The method according to this embodiment can effectively filter out these noisy points 810, generating a raster elevation map 830 based solely on ground point cloud data 820, thus achieving accurate terrain detection for the work area.

[0105] According to embodiments of this disclosure, a working terrain detection device is also provided. Figure 9 A structural block diagram of a working terrain detection device 900 according to an embodiment of the present disclosure is shown. Figure 9 As shown, the device 900 includes:

[0106] The acquisition module 910 is configured to acquire point cloud data of the work area collected by the lidar at the current time, wherein the work area is divided into multiple grids, each of the multiple grids has a corresponding height value, and the point cloud data includes the three-dimensional coordinates of multiple sampling points.

[0107] The first determining module 920 is configured to, for any grid among the plurality of grids, determine the input point of the grid from the plurality of sampling points based on the three-dimensional coordinates of the plurality of sampling points;

[0108] The second determining module 930 is configured to determine the type of the input point based on the height coordinates of the input point and the height value of the grid, wherein the type includes noise points and ground points; and

[0109] The update module 940 is configured to update the height value of the grid based on the height coordinates of the input point in response to determining that the input point is a ground point.

[0110] According to embodiments of this disclosure, point cloud data collected by LiDAR can be acquired in real time, and based on the three-dimensional coordinates of each sampling point, the sampling point (i.e., input point) corresponding to each grid for updating the grid height value can be determined. Furthermore, by determining whether the input point of the grid is a noise point or a ground point, and updating the grid height value only based on the height coordinates of the ground point, noise points can be effectively filtered out, improving the accuracy of terrain detection and achieving accurate and real-time terrain detection.

[0111] According to some embodiments, point cloud data is obtained by the lidar scanning the work area along a non-repeating scanning path.

[0112] According to some embodiments, the first determining module 920 includes: a first determining unit configured to determine a set of sampling points corresponding to the grid based on the three-dimensional coordinates of the plurality of sampling points, the set of sampling points including sampling points located within the grid; and a second determining unit configured to take the sampling point with the largest height coordinate in the set of sampling points as the input point.

[0113] According to some embodiments, the second determining module 930 is further configured to: determine the input point as a noise point in response to determining that the difference between the height coordinate and the height value is greater than a first threshold.

[0114] According to some embodiments, the second determining module 930 is further configured to: determine the input point as a ground point in response to determining that any of the following conditions are met: the difference between the height coordinate and the height value is less than or equal to the first threshold; or the difference between the height coordinate and the height value is greater than the first threshold, and the number of input points of the grid determined as noise points within a preset time period is greater than or equal to a second threshold.

[0115] According to some embodiments, the device 900 further includes: a first adding module configured to add the input point to the noise point container corresponding to the grid in response to determining that the input point is a noise point; and a first deleting module configured to delete the noise point with the earliest acquisition time in the noise point container in response to determining that the number of noise points in the noise point container is greater than or equal to a third threshold.

[0116] According to some embodiments, the device 900 further includes: a second adding module configured to add the input point to the ground point container corresponding to the grid in response to determining that the input point is a ground point; and a second deleting module configured to delete the ground point with the earliest acquisition time in the ground point container in response to determining that the number of ground points in the ground point container is greater than or equal to a fourth threshold.

[0117] According to some embodiments, the update module 940 is further configured to update the height value of the grid to the average height coordinates of various ground points in the ground point container.

[0118] According to some embodiments, the initial value of the grid height is the height coordinate of the first input point of the grid.

[0119] It should be understood that Figure 9 The various modules or units of the apparatus 900 shown can be connected to the reference. Figure 2 The steps in method 200 described correspond to each other. Therefore, the operations, features, and advantages described above for method 200 also apply to apparatus 900 and its included modules and units. For the sake of brevity, some operations, features, and advantages will not be repeated here.

[0120] While specific functions have been discussed above with reference to particular modules, it should be noted that the functions of the various modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. For example, the first determining module 920 and the second determining module 930 described above can be combined into a single module in some embodiments.

[0121] It should also be understood that this article can describe various technologies in the general context of software and hardware components or program modules. The above regarding... Figure 9 The various modules described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. For example, in some embodiments, one or more of modules 910-940 can be implemented together in a System on Chip (SoC). The SoC may include an integrated circuit chip (which includes a processor (e.g., a Central Processing Unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or one or more components of other circuitry) and may optionally execute received program code and / or include embedded firmware to perform functions.

[0122] According to embodiments of this disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described operational terrain detection method.

[0123] According to embodiments of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to cause the computer to perform the above-described operational terrain detection method.

[0124] According to embodiments of this disclosure, a computer program product is also provided, including a computer program, wherein the computer program implements the above-described operational terrain detection method when executed by a processor.

[0125] According to embodiments of this disclosure, an engineering device for detecting operational terrain is also provided, including the aforementioned electronic device.

[0126] refer to Figure 10 The present invention describes a structural block diagram of an electronic device 1000 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0127] like Figure 10 As shown, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of the electronic device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0128] Multiple components in electronic device 1000 are connected to I / O interface 1005, including: input unit 1006, output unit 1007, storage unit 1008, and communication unit 1009. Input unit 1006 can be any type of device capable of inputting information to electronic device 1000. Input unit 1006 can receive input digital or character information and generate key signal input related to user settings and / or function control of electronic device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 1007 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1008 may include, but is not limited to, disk and optical disk. Communication unit 1009 allows electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth. TM Equipment, 802.11 equipment, Wi-Fi equipment, WiMAX equipment, cellular communication equipment and / or the like.

[0129] The computing unit 1001 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute method 200 by any other suitable means (e.g., by means of firmware).

[0130] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0131] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0132] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0133] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0134] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0135] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0136] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0137] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A method for terrain detection in engineering operations, used to generate a raster elevation map of the ground in real time, the method comprising: The point cloud data of the work area collected by the lidar at the current time is acquired. The work area is divided into multiple grids, each of which has a corresponding height value. The point cloud data includes the three-dimensional coordinates of multiple sampling points. For any one of the plurality of grids: Based on the three-dimensional coordinates of the multiple sampling points, a set of sampling points corresponding to the grid is determined, and the set of sampling points includes sampling points located within the grid. The sampling point with the largest height coordinate in the set of sampling points is used as the input point of the grid, and the input point is used to update the height value of the grid. Based on the height coordinates of the input point and the height value of the grid, the type of the input point is determined, wherein the type includes noise points and ground points; and In response to determining that the input point is a ground point, the height value of the grid is updated based on the height coordinates of the input point.

2. The method according to claim 1, wherein, The point cloud data is obtained by scanning the work area with the lidar according to a non-repeating scanning path.

3. The method according to claim 1, wherein, Determining the type of the input point based on its height coordinates and the height value of the grid includes: In response to determining that the difference between the height coordinate and the height value is greater than a first threshold, the input point is determined to be a noise point.

4. The method according to claim 3, wherein, Determining the type of the input point based on its height coordinates and the height value of the grid includes: The input point is determined to be a ground point in response to the determination that any of the following conditions are met: The difference between the height coordinate and the height value is less than or equal to the first threshold; or The difference between the height coordinate and the height value is greater than the first threshold, and the number of input points of the grid that are identified as noise points within a preset time period is greater than or equal to the second threshold.

5. The method according to claim 1, further comprising: In response to determining that the input point is a noise point, the input point is added to the noise point container corresponding to the grid. as well as In response to determining that the number of noise points in the noise point container is greater than or equal to a third threshold, the noise point with the earliest acquisition time in the noise point container is deleted.

6. The method according to claim 1, further comprising: In response to determining that the input point is a ground point, the input point is added to the ground point container corresponding to the grid. as well as In response to determining that the number of ground points in the ground point container is greater than or equal to a fourth threshold, the ground point with the earliest acquisition time in the ground point container is deleted.

7. The method according to claim 6, wherein, In response to determining that the input point is a ground point, updating the height value of the raster based on the height coordinates of the input point includes: The height value of the grid is updated to the average height coordinates of all ground points in the ground point container.

8. The method according to claim 1, wherein, The initial value of the grid height is the height coordinate of the first input point of the grid.

9. An engineering operation terrain detection device for generating a grid elevation map of the ground in real time, the device comprising: The acquisition module is configured to acquire point cloud data of the work area collected by the lidar at the current time, wherein the work area is divided into multiple grids, each of the multiple grids has a corresponding height value, and the point cloud data includes the three-dimensional coordinates of multiple sampling points; The first determining module is configured to, for any grid among the plurality of grids, determine a set of sampling points corresponding to the grid based on the three-dimensional coordinates of the plurality of sampling points, the set of sampling points including sampling points located within the grid; and to use the sampling point with the largest height coordinate in the set of sampling points as the input point of the grid, the input point being used to update the height value of the grid. The second determining module is configured to determine the type of the input point based on the height coordinates of the input point and the height value of the grid, wherein the type includes noise points and ground points; and The update module is configured to update the height value of the raster based on the height coordinates of the input point in response to determining that the input point is a ground point.

10. The apparatus according to claim 9, wherein, The point cloud data is obtained by scanning the work area with the lidar according to a non-repeating scanning path.

11. The apparatus according to claim 9, wherein, The second determining module is further configured as follows: In response to determining that the difference between the height coordinate and the height value is greater than a first threshold, the input point is determined to be a noise point.

12. The apparatus according to claim 11, wherein, The second determining module is further configured as follows: The input point is determined to be a ground point in response to the determination that any of the following conditions are met: The difference between the height coordinate and the height value is less than or equal to the first threshold; or The difference between the height coordinate and the height value is greater than the first threshold, and the number of input points of the grid that are identified as noise points within a preset time period is greater than or equal to the second threshold.

13. The apparatus of claim 9, further comprising: The first adding module is configured to add the input point to the noise point container corresponding to the grid in response to determining that the input point is a noise point; as well as The first deletion module is configured to delete the noise point with the earliest acquisition time in the noise point container in response to determining that the number of noise points in the noise point container is greater than or equal to a third threshold.

14. The apparatus of claim 9, further comprising: The second adding module is configured to add the input point to the ground point container corresponding to the grid in response to determining that the input point is a ground point; as well as The second deletion module is configured to delete the ground point with the earliest acquisition time in the ground point container in response to determining that the number of ground points in the ground point container is greater than or equal to a fourth threshold.

15. An electronic device comprising: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

17. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method of any one of claims 1-8.

18. An engineering device comprising the electronic device as claimed in claim 15.

Images