Point cloud denoising method, device and system, mechanical equipment and computer program product

Abstract

The invention relates to a point cloud denoising method, device and system, mechanical equipment and a computer program product, and relates to the technical field of intelligent unmanned engineering machinery. The point cloud denoising method comprises the following steps: constructing a first point cloud map of mechanical equipment in a digging and loading operation scene in real time according to radar point cloud data and sensor data; performing flying dust denoising on the first point cloud map to obtain a second point cloud map; and performing material denoising on the second point cloud map to obtain a third point cloud map.

Description

Technical Field

[0001] The present disclosure relates to the technical field of intelligent unmanned engineering machinery, and in particular to a point cloud denoising method, a point cloud denoising device, a point cloud denoising system, mechanical equipment, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of industrialization, the intelligent unmanned operation of mechanical equipment (construction machinery) has attracted widespread attention. When mechanical equipment is excavating and loading, it often encounters problems such as harsh working environment and high working intensity of mechanical equipment, which can easily lead to safety hazards. In addition, the difficulty of hiring people and high costs will also limit the industry. As an important part of the intelligent unmanned operation of mechanical equipment, the point cloud map is mainly used to construct a three-dimensional point cloud map of the real scene for mechanical equipment to perform excavation and loading operations without human intervention. Summary of the invention

[0003] The inventors of the present disclosure have discovered that the above-mentioned related technologies have the following problems: when mechanical equipment is in a digging and loading operation, how to ensure the accuracy of the unmanned digging and loading operation performed by the mechanical equipment according to the point cloud map.

[0004] In view of this, the present disclosure proposes a point cloud denoising method, which uses radar point cloud data and sensor data to construct a first point cloud map of mechanical equipment in the excavation and loading operation scene in real time, and obtains a second point cloud map by performing dust denoising on the first point cloud map, and then obtains a third point cloud map by performing material denoising on the second point cloud map. Considering that mechanical equipment will generate dust and spilled materials during the excavation and loading operation, a more accurate point cloud map is obtained through dust denoising and material denoising, thereby ensuring the accuracy of the unmanned excavation and loading operation of the mechanical equipment according to the point cloud map.

[0005] According to some embodiments of the first aspect of the present disclosure, a point cloud denoising method is provided, including: constructing a first point cloud map of mechanical equipment in an excavation and loading operation scenario in real time based on radar point cloud data and sensor data; performing dust denoising on the first point cloud map to obtain a second point cloud map; and performing material denoising on the second point cloud map to obtain a third point cloud map.

[0006] In some embodiments, performing dust denoising on the first point cloud map to obtain the second point cloud map includes: performing dust denoising on the first point cloud map according to the intensity of the point cloud in the first point cloud map to obtain the second point cloud map.

[0007] In some embodiments, dust denoising is performed on the first point cloud map according to the intensity of the point cloud in the first point cloud map to obtain a second point cloud map, including: determining a point cloud intensity threshold according to the type of dust; and filtering out point clouds in the first point cloud map whose intensity is less than the point cloud intensity threshold according to the point cloud intensity threshold to obtain a second point cloud map.

[0008] In some embodiments, performing dust denoising on the first point cloud map to obtain a second point cloud map includes: performing dust denoising on the first point cloud map to obtain a second point cloud map according to the intensity of the point cloud in the first point cloud map and the distance between the point cloud coordinates in the first point cloud map and the coordinates of the radar in the first point cloud map.

[0009] In some embodiments, dust denoising is performed on the first point cloud map according to the intensity of the point cloud in the first point cloud map and the distance between the point cloud coordinates in the first point cloud map and the coordinates of the radar in the first point cloud map to obtain the second point cloud map, including: determining the point cloud intensity threshold according to the type of dust; filtering out the point clouds in the first point cloud map whose intensity is less than the point cloud intensity threshold according to the point cloud intensity threshold to obtain a first intermediate point cloud map; filtering out isolated point clouds in the first intermediate point cloud map according to the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map to obtain the second point cloud map.

[0010] In some embodiments, determining the point cloud intensity threshold according to the type of dust includes: when the type of dust is determined, determining the point cloud intensity threshold according to the type of dust, wherein the first point cloud map is a stacked frame point cloud map.

[0011] In some embodiments, determining the point cloud intensity threshold according to the type of dust includes: determining a first point cloud intensity threshold according to the type of dust; filtering out point clouds in the first point cloud map whose intensity is less than the first point cloud intensity threshold according to the first point cloud intensity threshold, to obtain a second intermediate point cloud map; sorting the point clouds in the second intermediate point cloud map according to the intensity of the point clouds in the second intermediate point cloud map; statistically analyzing the intensity distribution of the point clouds in the second intermediate point cloud map to determine the second point cloud intensity threshold, wherein the second point cloud intensity threshold is the median of the intensity distribution or the lower third quantile of the intensity distribution; determining the point cloud intensity threshold according to the size relationship between the first point cloud intensity threshold and the second point cloud intensity threshold.

[0012] In some embodiments, based on the size relationship between the first point cloud intensity threshold and the second point cloud intensity threshold, determining the point cloud intensity threshold includes: when the second point cloud intensity threshold is greater than or equal to the first point cloud intensity threshold, based on the first point cloud intensity threshold, filtering out point clouds in the first point cloud map whose intensity is less than the first point cloud intensity threshold to obtain a third intermediate point cloud map; sorting the point clouds in the third intermediate point cloud map based on the intensity of the point clouds in the third intermediate point cloud map; counting the intensity distribution of the point clouds in the third intermediate point cloud map; and adjusting the first point cloud intensity threshold based on the intensity distribution of the point clouds in the third intermediate point cloud map to determine the point cloud intensity threshold.

[0013] In some embodiments, the first point cloud intensity threshold is adjusted according to the intensity distribution of the point cloud in the third intermediate point cloud map, and determining the point cloud intensity threshold includes: when there are dust point clouds exceeding the first number threshold at the material point cloud in the third intermediate point cloud map, reducing the first point cloud intensity threshold until there are dust point clouds not exceeding the first number threshold at the material point cloud and the number of material point clouds is greater than or equal to the second number threshold; when the number of material point clouds is less than the second number threshold, increasing the first point cloud intensity threshold until there are dust point clouds not exceeding the first number threshold at the material point cloud and the number of material point clouds is greater than or equal to the second number threshold; when there are dust point clouds not exceeding the first number threshold at the material point cloud in the third intermediate point cloud map and the number of material point clouds is greater than or equal to the second number threshold, determining the first point cloud intensity threshold as the point cloud intensity threshold.

[0014] In some embodiments, determining the point cloud intensity threshold based on the size relationship between the first point cloud intensity threshold and the second point cloud intensity threshold also includes: when the second point cloud intensity threshold is less than the first point cloud intensity threshold, determining the second point cloud intensity threshold as the point cloud intensity threshold.

[0015] In some embodiments, determining the point cloud intensity threshold according to the type of dust includes: when the type of dust is uncertain, measuring the intensity of the dust by radar to obtain a dust intensity measurement value; determining the point cloud intensity threshold according to the dust intensity measurement value, wherein the first point cloud map is a stacked frame point cloud map.

[0016] In some embodiments, determining a point cloud intensity threshold according to the dust intensity measurement value includes: determining a third point cloud intensity threshold according to the dust intensity measurement value; filtering out point clouds in the first point cloud map whose intensity is less than the third point cloud intensity threshold according to the third point cloud intensity threshold, to obtain a fourth intermediate point cloud map; sorting the point clouds in the fourth intermediate point cloud map according to the intensity distribution of the point clouds in the fourth intermediate point cloud map; statistically analyzing the intensity distribution of the point clouds in the fourth intermediate point cloud map to determine a fourth point cloud intensity threshold, wherein the fourth point cloud intensity threshold is the median of the intensity distribution or the lower one-third quantile of the intensity distribution; determining the point cloud intensity threshold according to the size relationship between the third point cloud intensity threshold and the fourth point cloud intensity threshold.

[0017] In some embodiments, determining the point cloud intensity threshold according to the size relationship between the third point cloud intensity threshold and the fourth point cloud intensity threshold includes: when the fourth point cloud intensity threshold is greater than or equal to the third point cloud intensity threshold, filtering out point clouds in the first point cloud map whose intensity is less than the third point cloud intensity threshold according to the third point cloud intensity threshold to obtain a fifth intermediate point cloud map; sorting the point clouds in the fifth intermediate point cloud map according to the intensity of the point clouds in the fifth intermediate point cloud map; and calculating the intensity distribution of the point clouds in the fifth intermediate point cloud map; adjusting the third point cloud intensity threshold according to the intensity distribution of the point clouds in the fifth intermediate point cloud map to determine the point cloud intensity threshold.

[0018] In some embodiments, the third point cloud intensity threshold is adjusted according to the intensity distribution of point clouds in the fifth intermediate point cloud map, and determining the point cloud intensity threshold includes: when there are dust point clouds exceeding the first number threshold at the material point cloud in the fifth intermediate point cloud map, reducing the third point cloud intensity threshold until there are dust point clouds not exceeding the first number threshold at the material point cloud and the number of material point clouds is greater than or equal to the second number threshold; when the number of material point clouds is less than the second number threshold, increasing the third point cloud intensity threshold until there are dust point clouds not exceeding the first number threshold at the material point cloud and the number of material point clouds is greater than or equal to the second number threshold; when there are dust point clouds not exceeding the first number threshold at the material point cloud in the fifth intermediate point cloud map and the number of material point clouds is greater than or equal to the second number threshold, determining the third point cloud intensity threshold as the point cloud intensity threshold.

[0019] In some embodiments, determining the point cloud intensity threshold based on the size relationship between the third point cloud intensity threshold and the fourth point cloud intensity threshold also includes: when the fourth point cloud intensity threshold is less than the third point cloud intensity threshold, determining the fourth point cloud intensity threshold as the point cloud intensity threshold.

[0020] In some embodiments, filtering out isolated point clouds in the first intermediate point cloud map according to the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map to obtain the second point cloud map includes: dividing the point cloud in the first intermediate point cloud map that is farthest from the coordinates of the radar in the first intermediate point cloud map and the area between the coordinates of the radar in the first intermediate point cloud map into multiple areas according to the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map; assigning different denoising coefficients to different areas, wherein the denoising coefficient of the area close to the coordinates of the radar in the first intermediate point cloud map is greater than the denoising coefficient of the area far from the coordinates of the radar in the first intermediate point cloud map; when the number of point clouds in the target area of ​​multiple areas is less than or equal to the denoising coefficient of the target area, determining the point cloud in the target area as an isolated point cloud; filtering out the isolated point cloud to obtain the second point cloud map.

[0021] In some embodiments, the radar point cloud data includes at least one of the point cloud time, point cloud data volume, frame point cloud and dust intensity measurement value, and the sensor data includes at least one of the time of the current orientation, rotation angle, pitch angle, roll angle and rotation speed state.

[0022] In some embodiments, performing material denoising on the second point cloud map to obtain a third point cloud map includes: performing material denoising on the second point cloud map according to a bucket position of the mechanical equipment to obtain a third point cloud map.

[0023] In some embodiments, performing material denoising on the second point cloud map according to the bucket position of the mechanical equipment to obtain a third point cloud map includes: synchronizing the bucket position with the second point cloud map according to the point cloud time; determining a denoising area according to the bucket position and the bucket size of the mechanical equipment; and performing material denoising on the denoising area in the second point cloud map to obtain a third point cloud map.

[0024] In some embodiments, performing material denoising on the second point cloud map to obtain a third point cloud map includes: performing material denoising on the second point cloud map according to a bucket position of the mechanical equipment and a working status of the mechanical equipment to obtain a third point cloud map.

[0025] In some embodiments, material denoising is performed on the second point cloud map according to the bucket position of the mechanical equipment and the working status of the mechanical equipment to obtain a third point cloud map, including: synchronizing the bucket position and the working status of the mechanical equipment with the second point cloud map according to the point cloud time; determining the denoising area according to the bucket position and the bucket size of the mechanical equipment; performing material denoising on the denoising area in the second point cloud map to obtain a sixth intermediate point cloud map; determining the critical area of ​​the working boundary according to the working boundary of the bucket and the critical area threshold of the working boundary, wherein the working boundary includes at least one of an excavation boundary and an unloading boundary; determining whether the bucket leaves the critical area of ​​the working boundary according to the working status of the mechanical equipment, wherein the working status of the mechanical equipment includes at least one of an excavation state, a rotation speed state, and an unloading state; when the bucket leaves the critical area of ​​the working boundary, clearing the point cloud in the critical area of ​​the sixth intermediate point cloud map to obtain a third point cloud map.

[0026] In some embodiments, material denoising is performed on the second point cloud map according to the bucket position of the mechanical equipment and the working status of the mechanical equipment to obtain the third point cloud map, which also includes: real-time monitoring of the bucket position of the mechanical equipment and the working status of the mechanical equipment when the bucket has not left the critical area of ​​the working boundary.

[0027] In some embodiments, based on radar point cloud data and sensor data, real-time construction of a first point cloud map of mechanical equipment in an excavation and loading operation scenario includes: extracting point clouds in the on-vehicle coordinate system of mechanical equipment at different orientations based on radar point cloud data at different orientations; converting the point cloud space in the on-vehicle coordinate system of mechanical equipment at different orientations to the off-vehicle coordinate system of the mechanical equipment to obtain point clouds in the off-vehicle coordinate system of the mechanical equipment at different orientations; and splicing the point clouds in the off-vehicle coordinate system of mechanical equipment at different orientations based on sensor data to construct a first point cloud map of the mechanical equipment.

[0028] According to some embodiments of the second aspect of the present disclosure, a point cloud denoising device is provided, including: a construction unit, configured to construct in real time a first point cloud map of mechanical equipment in an excavation and loading operation scenario based on radar point cloud data and sensor data; a dust denoising unit, configured to perform dust denoising on the first point cloud map to obtain a second point cloud map; and a material denoising unit, configured to perform material denoising on the second point cloud map to obtain a third point cloud map.

[0029] In some embodiments, the dust denoising unit is further configured to perform dust denoising on the first point cloud map according to the intensity of the point cloud in the first point cloud map to obtain a second point cloud map.

[0030] In some embodiments, the dust denoising unit is further configured to determine a point cloud intensity threshold based on the type of dust; based on the point cloud intensity threshold, filter out point clouds in the first point cloud map whose intensity is less than the point cloud intensity threshold to obtain a second point cloud map.

[0031] In some embodiments, the dust denoising unit is also configured to perform dust denoising on the first point cloud map based on the intensity of the point cloud in the first point cloud map and the distance between the point cloud coordinates in the first point cloud map and the coordinates of the radar in the first point cloud map to obtain a second point cloud map.

[0032] In some embodiments, the dust denoising unit is also configured to determine a point cloud intensity threshold based on the type of dust; based on the point cloud intensity threshold, filter out point clouds in the first point cloud map whose intensity is less than the point cloud intensity threshold to obtain a first intermediate point cloud map; based on the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map, filter out isolated point clouds in the first intermediate point cloud map to obtain a second point cloud map.

[0033] In some embodiments, the dust denoising unit is further configured to determine a point cloud intensity threshold according to the type of dust when the type of dust is determined, wherein the first point cloud map is a stacked frame point cloud map.

[0034] In some embodiments, the dust denoising unit is further configured to determine a first point cloud intensity threshold according to the type of dust; filter out point clouds in the first point cloud map whose intensity is less than the first point cloud intensity threshold according to the first point cloud intensity threshold, to obtain a second intermediate point cloud map; sort the point clouds in the second intermediate point cloud map according to the intensity of the point clouds in the second intermediate point cloud map; statistically analyze the intensity distribution of the point clouds in the second intermediate point cloud map to determine the second point cloud intensity threshold, wherein the second point cloud intensity threshold is the median of the intensity distribution or the lower third quantile of the intensity distribution; determine the point cloud intensity threshold according to the size relationship between the first point cloud intensity threshold and the second point cloud intensity threshold.

[0035] In some embodiments, the dust denoising unit is also configured to, when the second point cloud intensity threshold is greater than or equal to the first point cloud intensity threshold, filter out point clouds in the first point cloud map whose intensity is less than the first point cloud intensity threshold according to the first point cloud intensity threshold, to obtain a third intermediate point cloud map; sort the point clouds in the third intermediate point cloud map according to the intensity of the point clouds in the third intermediate point cloud map; count the intensity distribution of the point clouds in the third intermediate point cloud map; adjust the first point cloud intensity threshold according to the intensity distribution of the point clouds in the third intermediate point cloud map, and determine the point cloud intensity threshold.

[0036] In some embodiments, the dust denoising unit is further configured to, when there are dust point clouds exceeding a first quantity threshold at the material point cloud in the third intermediate point cloud map, reduce the first point cloud intensity threshold until there are dust point clouds not exceeding the first quantity threshold at the material point cloud and the number of material point clouds is greater than or equal to the second quantity threshold; when the number of material point clouds is less than the second quantity threshold, increase the first point cloud intensity threshold until there are dust point clouds not exceeding the first quantity threshold at the material point cloud and the number of material point clouds is greater than or equal to the second quantity threshold; when there are dust point clouds not exceeding the first quantity threshold at the material point cloud in the third intermediate point cloud map and the number of material point clouds is greater than or equal to the second quantity threshold, determine the first point cloud intensity threshold as the point cloud intensity threshold.

[0037] In some embodiments, the dust denoising unit is further configured to determine the second point cloud intensity threshold as the point cloud intensity threshold when the second point cloud intensity threshold is less than the first point cloud intensity threshold.

[0038] In some embodiments, the dust denoising unit is also configured to measure the intensity of dust by radar to obtain a dust intensity measurement value when the type of dust is uncertain; determine the point cloud intensity threshold based on the dust intensity measurement value, wherein the first point cloud map is a stacked frame point cloud map.

[0039] In some embodiments, the dust denoising unit is further configured to determine a third point cloud intensity threshold based on the measured value of the dust intensity; filter out point clouds in the first point cloud map whose intensity is less than the third point cloud intensity threshold based on the third point cloud intensity threshold to obtain a fourth intermediate point cloud map; sort the point clouds in the fourth intermediate point cloud map based on the intensity distribution of the point clouds in the fourth intermediate point cloud map; count the intensity distribution of the point clouds in the fourth intermediate point cloud map to determine the fourth point cloud intensity threshold, wherein the fourth point cloud intensity threshold is the median of the intensity distribution or the lower third quantile of the intensity distribution; determine the point cloud intensity threshold based on the size relationship between the third point cloud intensity threshold and the fourth point cloud intensity threshold.

[0040] In some embodiments, the dust denoising unit is also configured to, when the fourth point cloud intensity threshold is greater than or equal to the third point cloud intensity threshold, filter out point clouds in the first point cloud map whose intensity is less than the third point cloud intensity threshold according to the third point cloud intensity threshold, to obtain a fifth intermediate point cloud map; sort the point clouds in the fifth intermediate point cloud map according to the intensity of the point clouds in the fifth intermediate point cloud map; count the intensity distribution of the point clouds in the fifth intermediate point cloud map; adjust the third point cloud intensity threshold according to the intensity distribution of the point clouds in the fifth intermediate point cloud map, and determine the point cloud intensity threshold.

[0041] In some embodiments, the dust denoising unit is further configured to, when there are dust point clouds exceeding the first quantity threshold at the material point cloud in the fifth intermediate point cloud map, reduce the third point cloud intensity threshold until there are dust point clouds not exceeding the first quantity threshold at the material point cloud and the number of material point clouds is greater than or equal to the second quantity threshold; when the number of material point clouds is less than the second quantity threshold, increase the third point cloud intensity threshold until there are dust point clouds not exceeding the first quantity threshold at the material point cloud and the number of material point clouds is greater than or equal to the second quantity threshold; when there are dust point clouds not exceeding the first quantity threshold at the material point cloud in the fifth intermediate point cloud map and the number of material point clouds is greater than or equal to the second quantity threshold, determine the third point cloud intensity threshold as the point cloud intensity threshold.

[0042] In some embodiments, the dust denoising unit is further configured to determine the fourth point cloud intensity threshold as the point cloud intensity threshold when the fourth point cloud intensity threshold is less than the third point cloud intensity threshold.

[0043] In some embodiments, the dust denoising unit is further configured to divide the area between the point cloud in the first intermediate point cloud map that is farthest from the coordinates of the radar in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map into multiple areas according to the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map; assign different denoising coefficients to different areas, wherein the denoising coefficient of the area close to the coordinates of the radar in the first intermediate point cloud map is greater than the denoising coefficient of the area far from the coordinates of the radar in the first intermediate point cloud map; when the number of point clouds in the target area of ​​multiple areas is less than or equal to the denoising coefficient of the target area, determine the point cloud in the target area as an isolated point cloud; filter out the isolated point cloud to obtain a second point cloud map.

[0044] In some embodiments, the radar point cloud data includes at least one of the point cloud time, point cloud data volume, frame point cloud and dust intensity measurement value, and the sensor data includes at least one of the time of the current orientation, rotation angle, pitch angle, roll angle and rotation speed state.

[0045] In some embodiments, the material denoising unit is further configured to perform material denoising on the second point cloud map according to the bucket position of the mechanical equipment to obtain a third point cloud map.

[0046] In some embodiments, the material denoising unit is also configured to synchronize the bucket position with the second point cloud map according to the point cloud time; determine the denoising area according to the bucket position and the bucket size of the mechanical equipment; perform material denoising on the denoising area in the second point cloud map to obtain a third point cloud map.

[0047] In some embodiments, the material denoising unit is further configured to perform material denoising on the second point cloud map according to the bucket position of the mechanical equipment and the working status of the mechanical equipment to obtain a third point cloud map.

[0048] In some embodiments, the material denoising unit is also configured to synchronize the bucket position and the working status of the mechanical equipment with the second point cloud map according to the point cloud time; determine the denoising area according to the bucket position and the bucket size of the mechanical equipment; perform material denoising on the denoising area in the second point cloud map to obtain a sixth intermediate point cloud map; determine the critical area of ​​the working boundary according to the working boundary of the bucket and the critical area threshold of the working boundary, wherein the working boundary includes at least one of the excavation boundary and the unloading boundary; determine whether the bucket leaves the critical area of ​​the working boundary according to the working status of the mechanical equipment, wherein the working status of the mechanical equipment includes at least one of the excavation state, the rotation speed state and the unloading state; when the bucket leaves the critical area of ​​the working boundary, clear the point cloud in the critical area of ​​the sixth intermediate point cloud map to obtain a third point cloud map.

[0049] In some embodiments, the material denoising unit is further configured to monitor the bucket position and working status of the mechanical equipment in real time when the bucket does not leave the critical area of ​​the working boundary.

[0050] In some embodiments, the construction unit is also configured to extract point clouds in the on-vehicle coordinate system of mechanical equipment at different orientations based on radar point cloud data at different orientations; convert the point cloud space in the on-vehicle coordinate system of mechanical equipment at different orientations to the off-vehicle coordinate system of the mechanical equipment to obtain point clouds in the off-vehicle coordinate system of mechanical equipment at different orientations; and splice the point clouds in the off-vehicle coordinate system of mechanical equipment at different orientations according to sensor data to construct a first point cloud map of the mechanical equipment.

[0051] According to some embodiments of the third aspect of the present disclosure, a point cloud denoising device is provided, comprising: a memory and a processor coupled to the memory, wherein the processor is configured to execute the point cloud denoising method in any one of the above embodiments based on instructions stored in the memory.

[0052] According to some embodiments of the fourth aspect of the present disclosure, a point cloud denoising system is provided, comprising: a point cloud denoising device in any one of the above embodiments; a radar configured to collect radar point cloud data and send it to the point cloud denoising device; and a sensor configured to collect sensor data and send it to the point cloud denoising device.

[0053] According to some embodiments of the fifth aspect of the present disclosure, there is provided a mechanical device, comprising: the point cloud denoising system in any one of the above embodiments.

[0054] According to some embodiments of the sixth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the point cloud denoising method in any of the above embodiments is implemented.

[0055] According to some embodiments of the seventh aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the point cloud denoising method in any of the above embodiments.

[0056] In the above embodiment, the first point cloud map of the mechanical equipment in the digging and loading operation scene is constructed in real time through radar point cloud data and sensor data, and the second point cloud map is obtained by performing dust denoising on the first point cloud map, and then the third point cloud map is obtained by performing material denoising on the second point cloud map. Considering that the mechanical equipment will generate dust and spilled materials during the digging and loading operation, a more accurate point cloud map is obtained through dust denoising and material denoising, thereby ensuring the accuracy of the mechanical equipment in performing unmanned digging and loading operations according to the point cloud map. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0058] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings.

[0059] Figure 1 A flowchart showing some embodiments of the point cloud denoising method of the present disclosure.

[0060] Figure 2 Schematic diagrams showing some embodiments of the positional relationship between the mechanical device and the point cloud of the present disclosure.

[0061] Figure 3 Schematic diagrams showing some embodiments of the dust denoising method disclosed in the present invention.

[0062] Figure 4 Schematic diagrams showing some embodiments of the material denoising method disclosed in the present invention.

[0063] Figure 5 Schematic diagrams showing other embodiments of the positional relationship between a mechanical device and a point cloud according to the present disclosure.

[0064] Figure 6 Schematic diagrams showing some embodiments of the point cloud denoising apparatus disclosed herein.

[0065] Figure 7 Schematic diagrams showing other embodiments of the point cloud denoising device disclosed in the present invention.

[0066] Figure 8Schematic diagrams showing some embodiments of the point cloud denoising system of the present disclosure.

[0067] Fig. 9 Schematic diagrams showing some embodiments of the mechanical apparatus of the present disclosure.

[0068] Fig.10 Schematic diagrams showing other embodiments of the point cloud denoising method disclosed in the present invention. DETAILED DESCRIPTION

[0069] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0070] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0071] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0072] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0073] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0074] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0075] Excavators are important engineering machinery among mechanical equipment, and are widely used in construction, mining and other engineering construction industries. Excavators often encounter problems such as harsh operating environment and high workload for operators, which can easily lead to safety hazards. In addition, the difficulty in hiring and high costs will also limit the development of the industry. Therefore, the development of excavators with unmanned operation functions has huge industry demand and practical significance.

[0076] As an important part of unmanned excavator operation, point cloud map is mainly used to construct three-dimensional point cloud map of real scene for unmanned excavator to plan excavation and loading tasks.

[0077] A typical working condition of an unmanned excavator is to rotate on a relatively fixed platform to perform excavation and loading operations. For this typical working condition, only a small range of work surface updates need to be implemented during the excavation process. Related systems and methods mostly use multi-line laser radars with more than 16 lines to establish point cloud maps through SLAM (Simultaneous Localization and Mapping). There is a large amount of data redundancy and calculation, and the dust generated during the excavation and loading operations and the spillage of materials from the bucket are not taken into account when building the map.

[0078] Therefore, in the scenario where mechanical equipment is performing excavation and loading operations, how to ensure the accuracy of unmanned excavation and loading operations performed by the mechanical equipment based on the point cloud map is a problem that needs to be solved.

[0079] If the dust point cloud and material point cloud in the point cloud map are not cleared, the dust point cloud and material point cloud will remain in the point cloud map, causing the mechanical equipment (i.e., the control program in the mechanical equipment) to be unable to perform excavation and unloading operations normally. Therefore, it is necessary to denoise the dust point cloud and material point cloud to ensure that a clean point cloud map is provided to the control program, so that the mechanical equipment can successfully complete the excavation and loading operations, thereby ensuring the accuracy of the unmanned excavation and loading operations of the mechanical equipment according to the point cloud map.

[0080] In response to the above problems, the present disclosure proposes a point cloud denoising method, which uses radar point cloud data and sensor data to construct a first point cloud map of mechanical equipment in the excavation and loading operation scene in real time, and obtains a second point cloud map by performing dust denoising on the first point cloud map, and then obtains a third point cloud map by performing material denoising on the second point cloud map. Considering that mechanical equipment will generate dust and spilled materials during the excavation and loading operation, a more accurate point cloud map is obtained through dust denoising and material denoising, which ensures the accuracy of unmanned excavation and loading operations of mechanical equipment based on point cloud maps, as follows.

[0081] Figure 1 A flowchart showing some embodiments of the point cloud denoising method of the present disclosure.

[0082] like Figure 1 As shown, the point cloud denoising method includes steps 110 to 130, and the point cloud denoising method is performed by a point cloud denoising device.

[0083] In step 110, a first point cloud map of the mechanical equipment in the excavation and loading operation scenario is constructed in real time based on the radar point cloud data and the sensor data.

[0084] In some embodiments, the radar point cloud data includes at least one of the point cloud time, point cloud data volume, frame point cloud and dust intensity measurement value, and the sensor data includes at least one of the time of the current orientation, rotation angle, pitch angle, roll angle and rotation speed state.

[0085] In some embodiments, the scanning mode of the radar is a petal scanning mode.

[0086] In some embodiments, the point cloud denoising device first receives low-frequency (e.g., 10 Hz) radar point cloud data and high-frequency (e.g., 100 Hz) sensor data, and then constructs a first point cloud map of the mechanical equipment in the excavation and loading operation scenario in real time based on the radar point cloud data and the sensor data.

[0087] In some embodiments, first, based on the radar point cloud data at different orientations, point clouds in the on-vehicle coordinate system of the mechanical equipment at different orientations are extracted; then, the point cloud space in the on-vehicle coordinate system of the mechanical equipment at different orientations is converted to the off-vehicle coordinate system of the mechanical equipment to obtain the point clouds in the off-vehicle coordinate system of the mechanical equipment at different orientations (assuming that the mechanical equipment (such as an excavator) does not move, then the off-vehicle coordinates of the mechanical equipment are stationary coordinates); finally, based on the sensor data, the point clouds in the off-vehicle coordinate system of the mechanical equipment at different orientations are spliced ​​to construct the first point cloud map of the mechanical equipment (the splicing here refers to the automatic splicing of the point clouds in the stationary coordinates as the excavator rotates).

[0088] The radar point cloud data is used to extract the point clouds of the mechanical equipment in different positions in the on-vehicle coordinate system, and then the point cloud in the on-vehicle coordinate system is converted to the off-vehicle coordinate system, and finally the point cloud in the off-vehicle coordinate system is spliced ​​according to the sensor data to form a first point cloud map. By converting the point cloud in the on-vehicle coordinate system into the point cloud in the off-vehicle coordinate system, it is convenient to splice the point clouds in different positions later, so that a more accurate first point cloud map can be obtained. By splicing the point cloud in the off-vehicle coordinate system through sensor data, a first point cloud map corresponding to the actual working conditions can be obtained. In addition, the radar point cloud data and sensor data are real-time data. By constructing the first point cloud map based on the radar point cloud data and sensor data, the point cloud map can be constructed in real time.

[0089] For example, radar point cloud data is connected to the point cloud denoising device through the UDP (User Datagram Protocol) network protocol, and sensor data is connected to the point cloud denoising device through the CAN (Controller Area Network) bus protocol.

[0090] Figure 2 Schematic diagrams showing some embodiments of the positional relationship between the mechanical device and the point cloud of the present disclosure.

[0091] like Figure 2 As shown in Figure 1, when mechanical equipment (such as an excavator) excavates materials and unloads them into a mining truck, dust is generated during excavation and unloading, such as Figure 2 As shown in 21 (dust) in Figure 2, the material in the bucket may be scattered along the way in the excavation area and during the rotation process, and may also form scattered materials when unloading in the unloading area, such as Figure 2 As shown in Figure 22, the spilled material generally falls on the area directly below the bucket.

[0092] like Figure 2 As shown, the mechanical equipment (such as an excavator) includes a boom 23 and a domain controller 24. A laser radar 25 is installed on the top of the cab of the mechanical equipment, and a turntable attitude sensor 26 is installed on the turntable of the mechanical equipment. Figure 2 As shown, a loading coordinate system (Oup) is constructed at the loading turntable of the mechanical equipment, and a disembarking coordinate system (Odown) is constructed at the disembarking place of the mechanical equipment.

[0093] like Figure 2 As shown, the data of the laser radar 25 is point cloud data (Pointcloud), and the data of the laser radar is connected to the domain controller 24 through the UDP network protocol. The data of the turntable attitude sensor 26 is Euler angle (rpy), and the data of the turntable attitude sensor is connected to the domain controller 24 through the CAN bus protocol.

[0094] In step 120, dust denoising is performed on the first point cloud map to obtain a second point cloud map.

[0095] In some embodiments, according to the intensity of the point cloud in the first point cloud map, dust denoising is performed on the first point cloud map to obtain a second point cloud map. Specifically, the method includes steps (111) and (112).

[0096] In step (111), a point cloud intensity threshold is determined according to the type of dust.

[0097] In step (112), based on the point cloud intensity threshold, point clouds in the first point cloud map whose intensity is less than the point cloud intensity threshold are filtered out to obtain a second point cloud map.

[0098] By performing dust denoising on the first point cloud map according to the point cloud intensity threshold, an accurate point cloud map is obtained, which ensures the accuracy of unmanned excavation and loading operations performed by mechanical equipment based on the point cloud map.

[0099] In some embodiments, dust denoising is performed on the first point cloud map according to the intensity of the point cloud in the first point cloud map and the distance between the point cloud coordinates in the first point cloud map and the coordinates of the radar in the first point cloud map to obtain a second point cloud map. Specifically, the method includes steps (121) to (123).

[0100] In step (121), a point cloud intensity threshold is determined according to the type of dust.

[0101] In step (122), based on the point cloud intensity threshold, point clouds in the first point cloud map whose intensity is less than the point cloud intensity threshold are filtered out to obtain a first intermediate point cloud map.

[0102] In step (123), isolated point clouds in the first intermediate point cloud map are filtered out according to the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map to obtain a second point cloud map.

[0103] By performing dust denoising on the first point cloud map based on the point cloud intensity threshold and the distance between the point cloud coordinates in the first intermediate point cloud map and the radar coordinates in the first intermediate point cloud map, a more accurate point cloud map is obtained, thereby ensuring the accuracy of unmanned excavation and loading operations performed by mechanical equipment based on the point cloud map.

[0104] In some embodiments, based on the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map, the area between the point cloud in the first intermediate point cloud map that is farthest from the coordinates of the radar in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map is divided into multiple areas; different denoising coefficients are assigned to different areas, wherein the denoising coefficient of the area close to the coordinates of the radar in the first intermediate point cloud map is greater than the denoising coefficient of the area far from the coordinates of the radar in the first intermediate point cloud map; when the number of point clouds in the target area of ​​multiple areas is less than or equal to the denoising coefficient of the target area, the point cloud in the target area is determined as an isolated point cloud; the isolated point cloud is filtered out to obtain a second point cloud map.

[0105] In other words, the radar scanning line angle is fixed, and the collected point cloud will generally show that the point cloud close to the radar coordinates will be denser, and the point cloud far from the radar coordinates will be sparser. If some low-intensity point clouds are filtered out from the dense point cloud, isolated point clouds will remain.

[0106] By assigning different denoising coefficients to different areas and taking into account the characteristics of radar petal-like scanning, isolated point clouds can be filtered out more accurately to obtain a more accurate point cloud map, thereby ensuring the accuracy of unmanned excavation and loading operations performed by mechanical equipment based on the point cloud map.

[0107] Regarding step (111) and step (121), there are mainly two situations: one is that the type of dust is determined, and the other is that the type of dust is uncertain, as described below.

[0108] If the type of dust is determined, the point cloud intensity threshold is determined according to the type of dust, wherein the first point cloud map is a stacked frame point cloud map. The stacked frame point cloud refers to the point cloud maps corresponding to different materials.

[0109] In some embodiments, a first point cloud intensity threshold is determined according to the type of dust; based on the first point cloud intensity threshold, point clouds in the first point cloud map whose intensity is less than the first point cloud intensity threshold are filtered out to obtain a second intermediate point cloud map; based on the intensity of the point clouds in the second intermediate point cloud map, the point clouds in the second intermediate point cloud map are sorted; the intensity distribution of the point clouds in the second intermediate point cloud map is statistically analyzed to determine a second point cloud intensity threshold, wherein the second point cloud intensity threshold is the median of the intensity distribution or the lower third quantile of the intensity distribution; the point cloud intensity threshold is determined based on the size relationship between the first point cloud intensity threshold and the second point cloud intensity threshold.

[0110] If the second point cloud intensity threshold is greater than or equal to the first point cloud intensity threshold, perform the following operations: according to the first point cloud intensity threshold, filter out the point clouds in the first point cloud map whose intensity is less than the first point cloud intensity threshold to obtain a third intermediate point cloud map; according to the intensity of the point clouds in the third intermediate point cloud map, sort the point clouds in the third intermediate point cloud map; count the intensity distribution of the point clouds in the third intermediate point cloud map; according to the intensity distribution of the point clouds in the third intermediate point cloud map, adjust the first point cloud intensity threshold to determine the point cloud intensity threshold.

[0111] If there are dust point clouds exceeding the first quantity threshold at the material point cloud in the third intermediate point cloud map, the first point cloud intensity threshold is reduced until there are dust point clouds exceeding the first quantity threshold at the material point cloud and the number of material point clouds is greater than or equal to the second quantity threshold.

[0112] If the number of material point clouds is less than the second number threshold, the first point cloud intensity threshold is increased until there are dust point clouds that do not exceed the first number threshold at the material point cloud and the number of material point clouds is greater than or equal to the second number threshold.

[0113] If there are dust point clouds that do not exceed the first quantity threshold at the material point cloud in the third intermediate point cloud map and the number of material point clouds is greater than or equal to the second quantity threshold, the first point cloud intensity threshold is determined as the point cloud intensity threshold.

[0114] By adjusting the first point cloud intensity threshold in real time according to the denoising effect of the point cloud map to obtain a more appropriate point cloud intensity threshold, more accurate denoising of the point cloud map is achieved, and a more accurate point cloud map is obtained, thereby ensuring the accuracy of unmanned excavation and loading operations performed by mechanical equipment according to the point cloud map.

[0115] If the second point cloud intensity threshold is less than the first point cloud intensity threshold, the second point cloud intensity threshold is determined as the point cloud intensity threshold.

[0116] If the type of dust is uncertain, the intensity of the dust is measured by radar to obtain a dust intensity measurement value, and then the point cloud intensity threshold is determined based on the dust intensity measurement value, wherein the first point cloud map is a stacked frame point cloud map.

[0117] In some embodiments, a third point cloud intensity threshold is determined based on the measured value of dust intensity; based on the third point cloud intensity threshold, point clouds in the first point cloud map whose intensity is less than the third point cloud intensity threshold are filtered out to obtain a fourth intermediate point cloud map; based on the intensity distribution of point clouds in the fourth intermediate point cloud map, point clouds in the fourth intermediate point cloud map are sorted; the intensity distribution of point clouds in the fourth intermediate point cloud map is statistically analyzed to determine a fourth point cloud intensity threshold, wherein the fourth point cloud intensity threshold is the median of the intensity distribution or the lower one-third quantile of the intensity distribution; the point cloud intensity threshold is determined based on the size relationship between the third point cloud intensity threshold and the fourth point cloud intensity threshold. Those skilled in the art should understand that the fourth point cloud intensity threshold being the median of the intensity distribution or the lower one-third quantile of the intensity distribution is only used as an example, and the fourth point cloud intensity threshold can also be set to other values ​​according to the intensity distribution.

[0118] If the fourth point cloud intensity threshold is greater than or equal to the third point cloud intensity threshold, then according to the third point cloud intensity threshold, filter out the point clouds in the first point cloud map whose intensity is less than the third point cloud intensity threshold to obtain the fifth intermediate point cloud map; sort the point clouds in the fifth intermediate point cloud map according to the intensity of the point clouds in the fifth intermediate point cloud map; count the intensity distribution of the point clouds in the fifth intermediate point cloud map; adjust the third point cloud intensity threshold according to the intensity distribution of the point clouds in the fifth intermediate point cloud map to determine the point cloud intensity threshold.

[0119] If there are dust point clouds exceeding the first quantity threshold at the material point cloud in the fifth intermediate point cloud map, reduce the third point cloud intensity threshold until there are dust point clouds exceeding the first quantity threshold at the material point cloud and the number of material point clouds is greater than or equal to the second quantity threshold.

[0120] If the number of material point clouds is less than the second number threshold, the third point cloud intensity threshold is increased until there are dust point clouds at the material point clouds that do not exceed the first number threshold and the number of material point clouds is greater than or equal to the second number threshold.

[0121] If there are dust point clouds not exceeding the first quantity threshold at the material point cloud in the fifth intermediate point cloud map and the quantity of the material point clouds is greater than or equal to the second quantity threshold, the third point cloud intensity threshold is determined as the point cloud intensity threshold.

[0122] By adjusting the third point cloud intensity threshold in real time according to the denoising effect of the point cloud map to obtain a more appropriate point cloud intensity threshold, more accurate denoising of the point cloud map is achieved, and a more accurate point cloud map is obtained, thereby ensuring the accuracy of unmanned excavation and loading operations performed by mechanical equipment according to the point cloud map.

[0123] If the fourth point cloud intensity threshold is less than the third point cloud intensity threshold, the fourth point cloud intensity threshold is determined as the point cloud intensity threshold.

[0124] By sorting the point cloud according to the material strength characteristics, filtering out the dust from the point cloud map according to the strength, forming the remaining dust into an isolated point cloud, and performing statistical filtering on the isolated point cloud according to the distance, the dust denoising effect is significantly improved.

[0125] Intensity filtering and distance statistical filtering are used to deal with the dust generated during excavation and unloading. Through field experiments, it is found that the point cloud intensity value of dust is significantly lower than the point cloud intensity value of ground materials. Therefore, the point cloud with intensity lower than the intensity filtering threshold can be filtered out by setting the intensity filtering threshold (i.e., the point cloud intensity threshold). At the same time, it is also found that different materials have different intensity characteristics. Therefore, the overlapping frame point cloud is sorted by intensity to automatically find the appropriate intensity filtering threshold. The lightly filtered point cloud will form some isolated outliers, and the area close to the radar coordinates in the point cloud map will be slightly dense, and the area far from the radar coordinates in the point cloud map will be slightly sparse. Therefore, the distance statistical filtering method is used to remove noise from the point cloud after intensity filtering, as follows.

[0126] Figure 3 Schematic diagrams showing some embodiments of the dust denoising method disclosed in the present invention.

[0127] like Figure 3 As shown, the dust denoising method includes steps 310 to 380.

[0128] In step 310, the filtering of dust generated when the bucket is excavated and unloaded is started.

[0129] In step 320, it is determined that the point cloud currently requiring denoising is a stacked frame point cloud.

[0130] Overlapping point clouds can form dense point clouds, and the number of overlapping frames can be set from 10 to 50.

[0131] In step 330, an intensity filter threshold is set.

[0132] For dry bulk earthwork scenes, the intensity filter threshold can be set to 18 to 23, and for bulk earthwork scenes, the intensity filter threshold can be set to 22 to 27.

[0133] In some embodiments, if the type of dust is certain and the filtering mode is a threshold mode, the intensity filtering threshold is directly determined according to the type of dust; if the type of dust is uncertain and the filtering mode is a mixed mode, it is necessary to obtain the dust intensity measurement value based on the radar to determine the intensity filtering threshold.

[0134] In step 340, a filtering mode is entered.

[0135] In step 350, the overlapping frame point clouds are sorted according to the intensity of the point clouds in the point cloud map to obtain intensity feature values.

[0136] The intensity distribution of the point cloud is statistically analyzed by sorting the overlapping frame point clouds.

[0137] In some embodiments, the intensity feature value may be, for example, a median value of the intensity distribution of the statistical point cloud or a minimum value of the lower third quantile.

[0138] In step 360, the minimum value among the intensity filter threshold and the intensity feature value is determined.

[0139] In step 370 , the minimum value is determined as the target threshold, and point clouds in the point cloud map whose intensity is less than the target threshold are filtered out.

[0140] Whether to re-execute step 330 or execute step 380 is determined according to the effect of the point cloud map after filtering.

[0141] If the effect of the point cloud map after filtering is poor, the intensity filter threshold is adjusted to obtain a better filtering effect.

[0142] It should be noted that if the dust that currently needs to be filtered out has been filtered out during the previous operation process, there is no need to perform steps 330 to 360. In step 370, the filtering threshold applied in the previous operation process is set as the target threshold to filter out the point clouds in the point cloud map whose intensity is less than the target threshold.

[0143] In step 380 , the point cloud map is filtered based on distance statistics.

[0144] According to the distance between the point cloud in the point cloud map and the radar coordinates of the radar in the point cloud map, the isolated point cloud in the filtered point cloud map is determined, and then the isolated point cloud is filtered out.

[0145] Specifically, according to the distance between the point cloud in the point cloud map and the radar coordinates of the radar in the point cloud map, the area between the point cloud and the radar coordinates is divided into multiple areas, and different areas are assigned different denoising coefficients, wherein the denoising coefficient of the area close to the radar coordinates in the point cloud map is greater than the denoising coefficient of the area far from the radar coordinates in the point cloud map.

[0146] In some embodiments, the calculation formula of the denoising coefficient is as shown in formula (1), which is specifically as follows.

[0147] MeanK = round(DR) / 5×10 (1)

[0148] Where D represents the side length of the excavation area, which is generally set to 15 to 30 meters. R represents the distance between the point cloud and the radar coordinates, R<=D, and round refers to rounding calculation.

[0149] In step 130, material denoising is performed on the second point cloud map to obtain a third point cloud map.

[0150] In some embodiments, the second point cloud map is subjected to material denoising according to the bucket position of the mechanical equipment to obtain a third point cloud map. Specifically, the method includes steps (211) to (213).

[0151] In step (211), the bucket position is time-synchronized with the second point cloud map according to the point cloud time.

[0152] In step (212), a denoising area is determined based on the bucket position and the bucket size of the mechanical equipment.

[0153] In some embodiments, the size of the bucket refers to the length, width and height of the bucket, and the position of the bucket refers to the vertical distance from the center of the bucket to the ground and the coordinates of the bucket.

[0154] In step (213), material denoising is performed on the denoised area in the second point cloud map to obtain a third point cloud map.

[0155] By denoising the point cloud map based on the position of the bucket and the prior knowledge that the spilled material is located under the bucket, and taking into account the possibility of spilled material in the excavation and loading operations, a more accurate point cloud map is obtained, ensuring the accuracy of unmanned excavation and loading operations performed by mechanical equipment based on the point cloud map.

[0156] Based on the prior knowledge that the spilled material is located under the bucket, BBOX filtering is performed on the denoising area based on the position and size of the bucket, so that the material spilled by the bucket during the rotary unloading process can be quickly cleared.

[0157] In some embodiments, the second point cloud map is subjected to material denoising according to the bucket position of the mechanical equipment and the working state of the mechanical equipment to obtain a third point cloud map. Specifically, the method includes steps (221) to (226).

[0158] In step (221), the bucket position and the working status of the mechanical equipment are synchronized with the second point cloud map according to the point cloud time.

[0159] In step (222), a denoising area is determined based on the bucket position and the bucket size of the mechanical equipment.

[0160] In step (223), material denoising is performed on the denoised area in the second point cloud map to obtain a sixth intermediate point cloud map.

[0161] In step (224), a critical area of ​​the working boundary is determined according to the working boundary of the bucket and a critical area threshold of the working boundary, wherein the working boundary includes at least one of an excavation boundary and an unloading boundary.

[0162] In step (225), it is determined whether the bucket has left the critical area of ​​the working boundary according to the working state of the mechanical equipment, wherein the working state of the mechanical equipment includes at least one of an excavation state, a rotation speed state, and an unloading state.

[0163] In step (226), when the bucket leaves the critical area of ​​the working boundary, the point cloud in the critical area in the sixth intermediate point cloud map is cleared to obtain a third point cloud map.

[0164] In some embodiments, when the bucket does not leave the critical area of ​​the working boundary, the bucket position of the mechanical equipment and the working status of the mechanical equipment are monitored in real time.

[0165] By real-time detection of the bucket position and working status of the mechanical equipment, it is possible to clear the point cloud of the critical area in a timely manner once the bucket leaves the critical area of ​​the working boundary, and to obtain a more accurate point cloud map in a timely manner, thus ensuring the accuracy of the unmanned excavation and loading operations of the mechanical equipment based on the point cloud map.

[0166] Based on the position of the bucket, the prior knowledge that the spilled material is located under the bucket, and the working status of the mechanical equipment, the point cloud map is denoised. Here, not only the denoised area under the bucket is denoised, but also the critical area of ​​the working boundary of the bucket is denoised. Considering that materials may spill in the digging and loading operation, and that materials may spill during the digging or unloading process, a more accurate point cloud map is obtained, ensuring the accuracy of the unmanned digging and loading operation of the mechanical equipment based on the point cloud map.

[0167] By clearing the point cloud of critical areas according to the working status of the mechanical equipment, the denoising effect of the point cloud corresponding to the material spilled by the bucket is significantly improved.

[0168] For the materials spilled during the bucket rotation unloading process, according to the characteristic that they will be spilled under the bucket, based on the prior knowledge of the bucket position, bounding box (BBOX) filtering is performed with the bucket as the center. For the point cloud near the working boundary, since the bucket position data and the status data of the mechanical equipment cannot be completely synchronized with the point cloud data or the bucket position is relatively high, after the bucket rotates out of the boundary, the spilled materials have not yet fallen into place and continue to fall in the air. According to the set critical area threshold, the critical area of ​​the working boundary is determined. When it is judged that the bucket leaves the critical area, all point clouds in the critical area are cleared to complete the denoising of the spilled materials in the critical area.

[0169] Figure 4 Schematic diagrams showing some embodiments of the material denoising method disclosed in the present invention.

[0170] like Figure 4 As shown, the material denoising method includes steps 410 to 480.

[0171] In step 410, filtering of materials spilled during bucket rotation unloading is initiated.

[0172] In step 420, the bucket position and the working status of the mechanical equipment are synchronized with the point cloud map according to the point cloud time.

[0173] In some embodiments, the synchronization error may be set to 1 to 10 milliseconds.

[0174] In step 430 , a critical zone threshold of the working boundary of the bucket is set.

[0175] In some embodiments, the critical region threshold can be set to 1 to 2 degrees to the left and right of the working boundary.

[0176] In step 440, a denoising area is determined according to the bucket position and the bucket size of the mechanical equipment. The denoising area is determined by the length (L) of the bucket, the width (W) of the bucket, the height (H) of the bucket, and the distance (Hpadding) from the center of the bucket to the ground vertically. Figure 5 shown.

[0177] Figure 5 Schematic diagrams showing other embodiments of the positional relationship between a mechanical device and a point cloud according to the present disclosure.

[0178] like Figure 5 As shown, H is the height of the bucket in the vertical direction, and Hpadding refers to the distance from the center of the bucket vertically to the ground.

[0179] In step 450 , it is determined whether the current point cloud is a point cloud in a denoising area.

[0180] If it is a point cloud in the denoised area, then step 480 is directly executed; if it is not a point cloud in the denoised area, then step 460 is executed.

[0181] In step 460 , it is determined whether the state of the mechanical equipment is in a critical area leaving the working boundary.

[0182] In some embodiments, the state of the mechanical equipment includes at least one of an excavation state, a rotation speed state, and an unloading state. When the bucket leaves the critical area, the excavation state 3 indicates that the bucket is about to leave the boundary of the excavator area and rotate from the excavation area to the unloading area, and the unloading state 5 indicates that the bucket is about to leave the boundary of the unloading area and rotate from the unloading area to the excavation area; or the rotation speed state on both sides of the boundary is used to determine whether to leave the critical area.

[0183] If the state of the mechanical equipment is out of the critical area of ​​the working boundary, step 470 is executed.

[0184] In step 470 , the point clouds in the critical area in the point cloud map are cleared.

[0185] In step 480, material denoising is performed on the point cloud in the denoising area, wherein the material denoising is BBOX filtering, that is, cuboid area filtering.

[0186] Figure 6 Schematic diagrams showing some embodiments of the point cloud denoising apparatus disclosed herein.

[0187] like Figure 6 As shown, the point cloud denoising device 60 includes a construction unit 61 , a dust denoising unit 62 and a material denoising unit 63 .

[0188] The construction unit 61 is configured to construct a first point cloud map of the mechanical equipment in the excavation and loading operation scenario in real time according to the radar point cloud data and the sensor data.

[0189] In some embodiments, the radar point cloud data includes at least one of the point cloud time, point cloud data volume, frame point cloud and dust intensity measurement value, and the sensor data includes at least one of the time of the current orientation, rotation angle, pitch angle, roll angle and rotation speed state.

[0190] In some embodiments, the construction unit 61 is also configured to extract point clouds in the on-vehicle coordinate system of mechanical equipment at different orientations based on radar point cloud data at different orientations; convert the point cloud space in the on-vehicle coordinate system of mechanical equipment at different orientations to the off-vehicle coordinate system of the mechanical equipment to obtain point clouds in the off-vehicle coordinate system of mechanical equipment at different orientations; and splice the point clouds in the off-vehicle coordinate system of mechanical equipment at different orientations according to sensor data to construct a first point cloud map of the mechanical equipment.

[0191] The dust denoising unit 62 is configured to perform dust denoising on the first point cloud map to obtain a second point cloud map.

[0192] In some embodiments, the dust denoising unit 62 is further configured to perform dust denoising on the first point cloud map according to the intensity of the point cloud in the first point cloud map to obtain a second point cloud map.

[0193] In some embodiments, the dust denoising unit 62 is further configured to determine a point cloud intensity threshold according to the type of dust; based on the point cloud intensity threshold, filter out point clouds in the first point cloud map whose intensity is less than the point cloud intensity threshold to obtain a second point cloud map.

[0194] In some embodiments, the dust denoising unit 62 is also configured to perform dust denoising on the first point cloud map according to the intensity of the point cloud in the first point cloud map and the distance between the point cloud coordinates in the first point cloud map and the coordinates of the radar in the first point cloud map to obtain a second point cloud map.

[0195] In some embodiments, the dust denoising unit 62 is also configured to determine a point cloud intensity threshold based on the type of dust; based on the point cloud intensity threshold, filter out point clouds in the first point cloud map whose intensity is less than the point cloud intensity threshold to obtain a first intermediate point cloud map; based on the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map, filter out isolated point clouds in the first intermediate point cloud map to obtain a second point cloud map.

[0196] In some embodiments, the dust denoising unit 62 is further configured to determine a point cloud intensity threshold according to the type of dust when the type of dust is determined, wherein the first point cloud map is a stacked frame point cloud map.

[0197] In some embodiments, the dust denoising unit 62 is also configured to determine a first point cloud intensity threshold according to the type of dust; according to the first point cloud intensity threshold, filter out point clouds in the first point cloud map whose intensity is less than the first point cloud intensity threshold to obtain a second intermediate point cloud map; according to the intensity of the point clouds in the second intermediate point cloud map, sort the point clouds in the second intermediate point cloud map; statistically analyze the intensity distribution of the point clouds in the second intermediate point cloud map to determine the second point cloud intensity threshold, wherein the second point cloud intensity threshold is the median of the intensity distribution or the lower third quantile of the intensity distribution; determine the point cloud intensity threshold based on the size relationship between the first point cloud intensity threshold and the second point cloud intensity threshold.

[0198] In some embodiments, the dust denoising unit 62 is also configured to, when the second point cloud intensity threshold is greater than or equal to the first point cloud intensity threshold, filter out point clouds in the first point cloud map whose intensity is less than the first point cloud intensity threshold according to the first point cloud intensity threshold, to obtain a third intermediate point cloud map; sort the point clouds in the third intermediate point cloud map according to the intensity of the point clouds in the third intermediate point cloud map; count the intensity distribution of the point clouds in the third intermediate point cloud map; adjust the first point cloud intensity threshold according to the intensity distribution of the point clouds in the third intermediate point cloud map, and determine the point cloud intensity threshold.

[0199] In some embodiments, the dust denoising unit 62 is further configured to, when there are dust point clouds exceeding a first quantity threshold at the material point cloud in the third intermediate point cloud map, reduce the first point cloud intensity threshold until there are dust point clouds not exceeding the first quantity threshold at the material point cloud and the number of material point clouds is greater than or equal to the second quantity threshold; when the number of material point clouds is less than the second quantity threshold, increase the first point cloud intensity threshold until there are dust point clouds not exceeding the first quantity threshold at the material point cloud and the number of material point clouds is greater than or equal to the second quantity threshold; when there are dust point clouds not exceeding the first quantity threshold at the material point cloud in the third intermediate point cloud map and the number of material point clouds is greater than or equal to the second quantity threshold, determine the first point cloud intensity threshold as the point cloud intensity threshold.

[0200] In some embodiments, the dust denoising unit 62 is further configured to determine the second point cloud intensity threshold as the point cloud intensity threshold when the second point cloud intensity threshold is less than the first point cloud intensity threshold.

[0201] In some embodiments, the dust denoising unit 62 is also configured to measure the intensity of dust by radar to obtain a dust intensity measurement value when the type of dust is uncertain; determine the point cloud intensity threshold based on the dust intensity measurement value, wherein the first point cloud map is a stacked frame point cloud map.

[0202] In some embodiments, the dust denoising unit 62 is also configured to determine a third point cloud intensity threshold based on the measured value of the dust intensity; filter out point clouds in the first point cloud map whose intensity is less than the third point cloud intensity threshold based on the third point cloud intensity threshold to obtain a fourth intermediate point cloud map; sort the point clouds in the fourth intermediate point cloud map based on the intensity distribution of the point clouds in the fourth intermediate point cloud map; count the intensity distribution of the point clouds in the fourth intermediate point cloud map to determine the fourth point cloud intensity threshold, wherein the fourth point cloud intensity threshold is the median of the intensity distribution or the lower one-third quantile of the intensity distribution; determine the point cloud intensity threshold based on the size relationship between the third point cloud intensity threshold and the fourth point cloud intensity threshold.

[0203] In some embodiments, the dust denoising unit 62 is also configured to, when the fourth point cloud intensity threshold is greater than or equal to the third point cloud intensity threshold, filter out point clouds in the first point cloud map whose intensity is less than the third point cloud intensity threshold according to the third point cloud intensity threshold, to obtain a fifth intermediate point cloud map; sort the point clouds in the fifth intermediate point cloud map according to the intensity of the point clouds in the fifth intermediate point cloud map; count the intensity distribution of the point clouds in the fifth intermediate point cloud map; adjust the third point cloud intensity threshold according to the intensity distribution of the point clouds in the fifth intermediate point cloud map, and determine the point cloud intensity threshold.

[0204] In some embodiments, the dust denoising unit 62 is further configured to, when there are dust point clouds exceeding the first quantity threshold at the material point cloud in the fifth intermediate point cloud map, reduce the third point cloud intensity threshold until there are dust point clouds not exceeding the first quantity threshold at the material point cloud and the number of material point clouds is greater than or equal to the second quantity threshold; when the number of material point clouds is less than the second quantity threshold, increase the third point cloud intensity threshold until there are dust point clouds not exceeding the first quantity threshold at the material point cloud and the number of material point clouds is greater than or equal to the second quantity threshold; when there are dust point clouds not exceeding the first quantity threshold at the material point cloud in the fifth intermediate point cloud map and the number of material point clouds is greater than or equal to the second quantity threshold, determine the third point cloud intensity threshold as the point cloud intensity threshold.

[0205] In some embodiments, the dust denoising unit 62 is further configured to determine the fourth point cloud intensity threshold as the point cloud intensity threshold when the fourth point cloud intensity threshold is less than the third point cloud intensity threshold.

[0206] In some embodiments, the dust denoising unit 62 is also configured to divide the area between the point cloud in the first intermediate point cloud map that is farthest from the coordinates of the radar in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map into multiple areas according to the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map; assign different denoising coefficients to different areas, wherein the denoising coefficient of the area close to the coordinates of the radar in the first intermediate point cloud map is greater than the denoising coefficient of the area far from the coordinates of the radar in the first intermediate point cloud map; when the number of point clouds in the target area of ​​multiple areas is less than or equal to the denoising coefficient of the target area, determine the point cloud in the target area as an isolated point cloud; filter out the isolated point cloud to obtain a second point cloud map.

[0207] The material denoising unit 63 is configured to perform material denoising on the second point cloud map to obtain a third point cloud map.

[0208] In some embodiments, the material denoising unit 63 is further configured to perform material denoising on the second point cloud map according to the bucket position of the mechanical equipment to obtain a third point cloud map.

[0209] In some embodiments, the material denoising unit 63 is also configured to synchronize the bucket position with the second point cloud map according to the point cloud time; determine the denoising area according to the bucket position and the bucket size of the mechanical equipment; perform material denoising on the denoising area in the second point cloud map to obtain a third point cloud map.

[0210] In some embodiments, the material denoising unit 63 is further configured to perform material denoising on the second point cloud map according to the bucket position of the mechanical equipment and the working status of the mechanical equipment to obtain a third point cloud map.

[0211] In some embodiments, the material denoising unit 63 is also configured to synchronize the bucket position and the working status of the mechanical equipment with the second point cloud map according to the point cloud time; determine the denoising area according to the bucket position and the bucket size of the mechanical equipment; perform material denoising on the denoising area in the second point cloud map to obtain a sixth intermediate point cloud map; determine the critical area of ​​the working boundary according to the working boundary of the bucket and the critical area threshold of the working boundary, wherein the working boundary includes at least one of the excavation boundary and the unloading boundary; determine whether the bucket leaves the critical area of ​​the working boundary according to the working status of the mechanical equipment, wherein the working status of the mechanical equipment includes at least one of the excavation state, the rotation speed state and the unloading state; when the bucket leaves the critical area of ​​the working boundary, clear the point cloud in the critical area of ​​the sixth intermediate point cloud map to obtain a third point cloud map.

[0212] In some embodiments, the material denoising unit 63 is further configured to monitor the bucket position and working status of the mechanical equipment in real time when the bucket does not leave the critical area of ​​the working boundary.

[0213] In the above embodiment, the first point cloud map of the mechanical equipment in the digging and loading operation scene is constructed in real time through radar point cloud data and sensor data, and the second point cloud map is obtained by performing dust denoising on the first point cloud map, and then the third point cloud map is obtained by performing material denoising on the second point cloud map. Considering that the mechanical equipment will generate dust and spilled materials during the digging and loading operation, a more accurate point cloud map is obtained through dust denoising and material denoising, thereby ensuring the accuracy of the mechanical equipment in performing unmanned digging and loading operations according to the point cloud map.

[0214] Figure 7 Schematic diagrams showing other embodiments of the point cloud denoising device disclosed in the present invention.

[0215] like Figure 7 As shown, the point cloud denoising device 70 of this embodiment includes: a memory 71 and a processor 72 coupled to the memory 71 , and the processor 72 is configured to execute the point cloud denoising method in any one of the aforementioned embodiments based on instructions stored in the memory 71 .

[0216] The memory 71 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0217] The point cloud denoising device 70 may also include an input / output interface 73, a network interface 74, a storage interface 75, etc. These interfaces 73, 74, 75, the memory 71, and the processor 72 may be connected, for example, via a bus 76. The input / output interface 73 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, a touch screen, a microphone, and a speaker. The network interface 74 provides a connection interface for various networked devices. The storage interface 75 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0218] In the above embodiment, the first point cloud map of the mechanical equipment in the digging and loading operation scene is constructed in real time through radar point cloud data and sensor data, and the second point cloud map is obtained by performing dust denoising on the first point cloud map, and then the third point cloud map is obtained by performing material denoising on the second point cloud map. Considering that the mechanical equipment will generate dust and spilled materials during the digging and loading operation, a more accurate point cloud map is obtained through dust denoising and material denoising, thereby ensuring the accuracy of the mechanical equipment in performing unmanned digging and loading operations according to the point cloud map.

[0219] Figure 8Schematic diagrams showing some embodiments of the point cloud denoising system of the present disclosure.

[0220] like Figure 8 As shown, the point cloud denoising system 80 includes a point cloud denoising device 81 , a radar 82 and a sensor 83 .

[0221] The point cloud denoising device 81 is configured as the point cloud denoising device in any of the above embodiments.

[0222] In some embodiments, the point cloud denoising device 81 may be, for example, a domain controller in a mechanical device.

[0223] For example, a domain controller is a computing device used for data collection and computing processing.

[0224] The radar 82 is configured to collect radar point cloud data and send it to the point cloud denoising device.

[0225] In some embodiments, radar 82 may be, for example, a lidar.

[0226] For example, the laser radar is installed on the top of the cab of mechanical equipment (such as an excavator) with the installation direction facing forward. The installation angle of the radar is generally selected according to the expected excavation and loading tasks to collect environmental point clouds.

[0227] The sensor 83 is configured to collect sensor data and send it to the point cloud denoising device.

[0228] In some embodiments, the sensor 83 may be, for example, a turntable attitude sensor.

[0229] For example, the turntable attitude sensor is installed on the turntable of the upper vehicle of mechanical equipment (such as an excavator) to measure the turntable's rotation angle, rotation angular velocity, pitch angle, and roll angle.

[0230] In the above embodiment, the first point cloud map of the mechanical equipment in the digging and loading operation scene is constructed in real time through radar point cloud data and sensor data, and the second point cloud map is obtained by performing dust denoising on the first point cloud map, and then the third point cloud map is obtained by performing material denoising on the second point cloud map. Considering that the mechanical equipment will generate dust and spilled materials during the digging and loading operation, a more accurate point cloud map is obtained through dust denoising and material denoising, thereby ensuring the accuracy of the mechanical equipment in performing unmanned digging and loading operations according to the point cloud map.

[0231] Fig. 9 Schematic diagrams showing some embodiments of the mechanical apparatus of the present disclosure.

[0232] like Fig. 9 As shown, the mechanical device 90 includes a point cloud denoising system 91 .

[0233] The point cloud denoising system 91 is configured as the point cloud denoising system in any of the above embodiments.

[0234] In the above embodiment, the first point cloud map of the mechanical equipment in the digging and loading operation scene is constructed in real time through radar point cloud data and sensor data, and the second point cloud map is obtained by performing dust denoising on the first point cloud map, and then the third point cloud map is obtained by performing material denoising on the second point cloud map. Considering that the mechanical equipment will generate dust and spilled materials during the digging and loading operation, a more accurate point cloud map is obtained through dust denoising and material denoising, thereby ensuring the accuracy of the mechanical equipment in performing unmanned digging and loading operations according to the point cloud map.

[0235] Fig.10 Schematic diagrams showing other embodiments of the point cloud denoising method disclosed in the present invention.

[0236] like Fig.10 As shown, the point cloud denoising method includes steps 1001 to 1004.

[0237] In step 1001, point cloud denoising begins.

[0238] In step 1002, dust generated when the bucket of the mechanical equipment is excavating and unloading is filtered out (or denoised).

[0239] In step 1003, materials spilled when the bucket of the mechanical equipment is digging and unloading are filtered out (denoising).

[0240] like Fig.10 As shown, the dust generated by the mechanical equipment during the excavation and loading operation and the materials scattered by the bucket are taken in sequence. Step 1002 can be executed by the dust filtering module when the bucket is excavating and unloading, and step 1003 can be executed by the material filtering module when the bucket is rotating and unloading. The dust and the scattered materials are filtered out by steps 1002 and 1003, and the construction point cloud map of the excavation and loading area is denoised to form a clean construction point cloud map of the excavation and loading area.

[0241] In step 1004 , point cloud denoising is terminated.

[0242] In the above embodiment, the first point cloud map of the mechanical equipment in the digging and loading operation scene is constructed in real time through radar point cloud data and sensor data, and the second point cloud map is obtained by performing dust denoising on the first point cloud map, and then the third point cloud map is obtained by performing material denoising on the second point cloud map. Considering that the mechanical equipment will generate dust and spilled materials during the digging and loading operation, a more accurate point cloud map is obtained through dust denoising and material denoising, thereby ensuring the accuracy of the mechanical equipment in performing unmanned digging and loading operations according to the point cloud map.

[0243] In some embodiments, a computer program product is protected, including a computer program or an instruction, which implements the above-mentioned point cloud denoising method when the computer program or instruction is executed by a processor. The computer program product includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by the CPU, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0244] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0245] So far, the point cloud denoising method, device, system, mechanical equipment and computer program product disclosed in the present invention have been described in detail. In order to avoid obscuring the concept of the present invention, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.

[0246] The method and system of the present disclosure may be implemented in many ways. For example, the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0247] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A point cloud denoising method, comprising: Based on radar point cloud data and sensor data, the first point cloud map of the mechanical equipment in the excavation and loading operation scenario is constructed in real time; Perform dust denoising on the first point cloud map to obtain a second point cloud map; Material denoising is performed on the second point cloud map to obtain a third point cloud map.

2. The point cloud denoising method according to claim 1, wherein: The performing dust denoising on the first point cloud map to obtain a second point cloud map comprises: According to the intensity of the point cloud in the first point cloud map, dust denoising is performed on the first point cloud map to obtain a second point cloud map.

3. The point cloud denoising method according to claim 2, wherein: The step of performing dust denoising on the first point cloud map according to the intensity of the point cloud in the first point cloud map to obtain the second point cloud map comprises: Determine the point cloud intensity threshold based on the type of dust; According to the point cloud intensity threshold, point clouds in the first point cloud map whose intensity is less than the point cloud intensity threshold are filtered out to obtain the second point cloud map.

4. The point cloud denoising method according to claim 1, wherein: The performing dust denoising on the first point cloud map to obtain a second point cloud map comprises: According to the intensity of the point cloud in the first point cloud map and the distance between the point cloud coordinates in the first point cloud map and the coordinates of the radar in the first point cloud map, dust denoising is performed on the first point cloud map to obtain the second point cloud map.

5. The point cloud denoising method according to claim 4, wherein: The step of performing dust denoising on the first point cloud map according to the intensity of the point cloud in the first point cloud map and the distance between the point cloud coordinates in the first point cloud map and the coordinates of the radar in the first point cloud map to obtain the second point cloud map comprises: Determine the point cloud intensity threshold based on the type of dust; According to the point cloud intensity threshold, point clouds with intensities less than the point cloud intensity threshold in the first point cloud map are filtered out to obtain a first intermediate point cloud map; According to the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map, isolated point clouds in the first intermediate point cloud map are filtered out to obtain the second point cloud map.

6. The point cloud denoising method according to claim 3 or 5, wherein: Determining the point cloud intensity threshold according to the type of dust includes: When the type of the dust is determined, the point cloud intensity threshold is determined according to the type of the dust, wherein the first point cloud map is a stacked frame point cloud map.

7. The point cloud denoising method according to claim 6, wherein: Determining the point cloud intensity threshold according to the type of dust includes: Determining a first point cloud intensity threshold according to the type of dust; According to the first point cloud intensity threshold, point clouds having intensities less than the first point cloud intensity threshold in the first point cloud map are filtered out to obtain a second intermediate point cloud map; sorting the point clouds in the second intermediate point cloud map according to the intensities of the point clouds in the second intermediate point cloud map; Counting the intensity distribution of the point cloud in the second intermediate point cloud map, and determining a second point cloud intensity threshold, wherein the second point cloud intensity threshold is the median of the intensity distribution or the lower one-third quantile of the intensity distribution; The point cloud intensity threshold is determined according to a magnitude relationship between the first point cloud intensity threshold and the second point cloud intensity threshold.

8. The point cloud denoising method according to claim 7, wherein: The determining the point cloud intensity threshold according to the magnitude relationship between the first point cloud intensity threshold and the second point cloud intensity threshold comprises: When the second point cloud intensity threshold is greater than or equal to the first point cloud intensity threshold, point clouds having intensities less than the first point cloud intensity threshold in the first point cloud map are filtered out according to the first point cloud intensity threshold to obtain a third intermediate point cloud map; sorting the point clouds in the third intermediate point cloud map according to the intensities of the point clouds in the third intermediate point cloud map; Counting the intensity distribution of the point cloud in the third intermediate point cloud map; The first point cloud intensity threshold is adjusted according to the intensity distribution of the point cloud in the third intermediate point cloud map to determine the point cloud intensity threshold.

9. The point cloud denoising method according to claim 8, wherein: The step of adjusting the first point cloud intensity threshold according to the intensity distribution of the point cloud in the third intermediate point cloud map to determine the point cloud intensity threshold comprises: When there are dust point clouds exceeding a first quantity threshold at the material point cloud in the third intermediate point cloud map, reducing the first point cloud intensity threshold until there are dust point clouds exceeding the first quantity threshold at the material point cloud and the quantity of the material point clouds is greater than or equal to the second quantity threshold; When the number of the material point clouds is less than the second number threshold, increasing the first point cloud intensity threshold until there are dust point clouds not exceeding the first number threshold at the material point clouds and the number of the material point clouds is greater than or equal to the second number threshold; When there are dust point clouds not exceeding the first quantity threshold at the material point cloud in the third intermediate point cloud map and the number of the material point clouds is greater than or equal to the second quantity threshold, the first point cloud intensity threshold is determined as the point cloud intensity threshold.

10. The point cloud denoising method according to claim 8, wherein: The determining the point cloud intensity threshold according to the magnitude relationship between the first point cloud intensity threshold and the second point cloud intensity threshold further comprises: When the second point cloud intensity threshold is less than the first point cloud intensity threshold, the second point cloud intensity threshold is determined as the point cloud intensity threshold.

11. The point cloud denoising method according to claim 3 or 5, wherein: Determining the point cloud intensity threshold according to the type of dust includes: When the type of the dust is uncertain, the intensity of the dust is measured by radar to obtain a measured value of the intensity of the dust; The point cloud intensity threshold is determined according to the dust intensity measurement value, wherein the first point cloud map is a stacked frame point cloud map.

12. The point cloud denoising method according to claim 11, wherein: Determining the point cloud intensity threshold according to the dust intensity measurement value includes: Determining a third point cloud intensity threshold according to the dust intensity measurement value; According to the third point cloud intensity threshold, point clouds having intensities less than the third point cloud intensity threshold in the first point cloud map are filtered out to obtain a fourth intermediate point cloud map; sorting the point clouds in the fourth intermediate point cloud map according to the intensity distribution of the point clouds in the fourth intermediate point cloud map; Counting the intensity distribution of the point cloud in the fourth intermediate point cloud map, and determining a fourth point cloud intensity threshold, wherein the fourth point cloud intensity threshold is the median of the intensity distribution or the lower one-third quantile of the intensity distribution; The point cloud intensity threshold is determined according to a magnitude relationship between the third point cloud intensity threshold and the fourth point cloud intensity threshold.

13. The point cloud denoising method according to claim 12, wherein: The determining the point cloud intensity threshold according to the magnitude relationship between the third point cloud intensity threshold and the fourth point cloud intensity threshold comprises: When the fourth point cloud intensity threshold is greater than or equal to the third point cloud intensity threshold, point clouds having intensities less than the third point cloud intensity threshold in the first point cloud map are filtered out according to the third point cloud intensity threshold to obtain a fifth intermediate point cloud map; sorting the point clouds in the fifth intermediate point cloud map according to the intensities of the point clouds in the fifth intermediate point cloud map; Counting the intensity distribution of the point cloud in the fifth intermediate point cloud map; According to the intensity distribution of the point cloud in the fifth intermediate point cloud map, the third point cloud intensity threshold is adjusted to determine the point cloud intensity threshold.

14. The point cloud denoising method according to claim 13, wherein: The step of adjusting the third point cloud intensity threshold according to the intensity distribution of the point cloud in the fifth intermediate point cloud map to determine the point cloud intensity threshold comprises: When there are dust point clouds exceeding a first quantity threshold at the material point cloud in the fifth intermediate point cloud map, reducing the third point cloud intensity threshold until there are dust point clouds exceeding the first quantity threshold at the material point cloud and the quantity of the material point clouds is greater than or equal to the second quantity threshold; When the number of the material point clouds is less than the second number threshold, increasing the third point cloud intensity threshold until there are dust point clouds not exceeding the first number threshold at the material point clouds and the number of the material point clouds is greater than or equal to the second number threshold; When there are dust point clouds not exceeding the first quantity threshold at the material point cloud in the fifth intermediate point cloud map and the number of the material point clouds is greater than or equal to the second quantity threshold, the third point cloud intensity threshold is determined as the point cloud intensity threshold.

15. The point cloud denoising method according to claim 13, wherein: The determining the point cloud intensity threshold according to the magnitude relationship between the third point cloud intensity threshold and the fourth point cloud intensity threshold further comprises: When the fourth point cloud intensity threshold is less than the third point cloud intensity threshold, the fourth point cloud intensity threshold is determined as the point cloud intensity threshold.

16. The point cloud denoising method according to claim 5, wherein: The filtering out isolated point clouds in the first intermediate point cloud map according to the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map to obtain the second point cloud map comprises: According to the distance between the point cloud coordinates in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map, dividing the area between the point cloud in the first intermediate point cloud map that is farthest from the coordinates of the radar in the first intermediate point cloud map and the coordinates of the radar in the first intermediate point cloud map into a plurality of areas; Allocating different denoising coefficients to different areas, wherein a denoising coefficient of an area close to the coordinates of the radar in the first intermediate point cloud map is greater than a denoising coefficient of an area far from the coordinates of the radar in the first intermediate point cloud map; When the number of point clouds in the target area of ​​the multiple areas is less than or equal to the denoising coefficient of the target area, determining the point cloud in the target area as an isolated point cloud; The isolated point cloud is filtered out to obtain the second point cloud map.

17. The point cloud denoising method according to any one of claims 1 to 5, wherein: The radar point cloud data includes at least one of point cloud time, point cloud data volume, frame point cloud and dust intensity measurement value, and the sensor data includes at least one of current orientation time, rotation angle, pitch angle, roll angle and rotation speed state.

18. The point cloud denoising method according to claim 17, wherein: The performing material denoising on the second point cloud map to obtain a third point cloud map comprises: The second point cloud map is subjected to material denoising according to the bucket position of the mechanical equipment to obtain the third point cloud map.

19. The point cloud denoising method according to claim 18, wherein: The performing material denoising on the second point cloud map according to the bucket position of the mechanical equipment to obtain the third point cloud map comprises: Synchronizing the bucket position with the second point cloud map according to the point cloud time; Determining a denoising area according to the bucket position and the bucket size of the mechanical equipment; Material denoising is performed on the denoising area in the second point cloud map to obtain a third point cloud map.

20. The point cloud denoising method according to claim 17, wherein: The performing material denoising on the second point cloud map to obtain a third point cloud map comprises: The second point cloud map is subjected to material denoising according to the bucket position of the mechanical equipment and the working status of the mechanical equipment to obtain the third point cloud map.

21. The point cloud denoising method according to claim 20, wherein: The performing material denoising on the second point cloud map according to the bucket position of the mechanical equipment and the working state of the mechanical equipment to obtain the third point cloud map comprises: According to the point cloud time, the bucket position and the working state of the mechanical equipment are synchronized with the second point cloud map; Determining a denoising area according to the bucket position and the bucket size of the mechanical equipment; Performing material denoising on the denoised area in the second point cloud map to obtain a sixth intermediate point cloud map; determining a critical area of ​​the working boundary according to a working boundary of the bucket and a critical area threshold of the working boundary, wherein the working boundary includes at least one of an excavation boundary and an unloading boundary; Determining whether the bucket leaves the critical area of ​​the working boundary according to the working state of the mechanical equipment, wherein the working state of the mechanical equipment includes at least one of an excavation state, a rotation speed state, and an unloading state; When the bucket leaves the critical area of ​​the working boundary, the point cloud in the critical area in the sixth intermediate point cloud map is cleared to obtain the third point cloud map.

22. The point cloud denoising method according to claim 21, wherein: The performing material denoising on the second point cloud map according to the bucket position of the mechanical equipment and the working state of the mechanical equipment to obtain the third point cloud map further comprises: When the bucket does not leave the critical area of ​​the working boundary, the bucket position of the mechanical equipment and the working status of the mechanical equipment are monitored in real time.

23. The point cloud denoising method according to any one of claims 1 to 5, wherein: The real-time construction of a first point cloud map of the mechanical equipment in the excavation and loading operation scenario based on the radar point cloud data and the sensor data includes: Extracting point clouds of the mechanical equipment in the vehicle coordinate system at different orientations according to the radar point cloud data at different orientations; Converting the point cloud space of the mechanical equipment in the different orientations in the on-vehicle coordinate system to the off-vehicle coordinate system of the mechanical equipment to obtain the point cloud of the mechanical equipment in the different orientations in the off-vehicle coordinate system; According to the sensor data, the point clouds of the mechanical equipment in the different orientations in the vehicle coordinate system are spliced ​​to construct the first point cloud map of the mechanical equipment.

24. A point cloud denoising device, comprising: A construction unit is configured to construct a first point cloud map of the mechanical equipment in a digging and loading operation scenario in real time according to the radar point cloud data and the sensor data; a dust denoising unit, configured to perform dust denoising on the first point cloud map to obtain a second point cloud map; The material denoising unit is configured to perform material denoising on the second point cloud map to obtain a third point cloud map.

25. A point cloud denoising device, comprising: Memory; and A processor coupled to the memory, wherein the processor is configured to execute the point cloud denoising method according to any one of claims 1 to 23 based on instructions stored in the memory.

26. A point cloud denoising system, comprising: The point cloud denoising device as claimed in claim 24 or 25; A radar, configured to collect the radar point cloud data and send it to the point cloud denoising device; The sensor is configured to collect the sensor data and send it to the point cloud denoising device.

27. A mechanical device comprising: The point cloud denoising system as claimed in claim 26.

28. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the point cloud denoising method according to any one of claims 1 to 23 is implemented.

29. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the point cloud denoising method according to any one of claims 1 to 23 is implemented.

Images