Point cloud map updating method and apparatus, and engineering vehicle and storage medium

Abstract

Provided in the present disclosure are a point cloud map updating method and apparatus, and an engineering vehicle and a storage medium. The point cloud map updating method comprises: loading an initial global point cloud map on the basis of the current position of an engineering vehicle; performing region division on the global point cloud map, so as to obtain a plurality of local point cloud maps; determining a region where each of a plurality of vehicle-mounted radars is currently located; using the plurality of vehicle-mounted radars to update, at a first preset period, the local point cloud map corresponding to an excavation and loading region, and to update, at a second preset period, the local point cloud map corresponding to a non-excavation-and-loading region; and updating the global point cloud map on the basis of updating the results of the plurality of local point cloud maps.

Description

Point cloud map updating method and device, engineering vehicle and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure is based on and claims priority to an application with CN application number 202311718181.7 and filing date December 13, 2023. The disclosure content of this CN application is hereby incorporated into this disclosure as a whole. Technical Field

[0003] The present disclosure relates to the field of engineering machinery control, and in particular to a point cloud map updating method and device, an engineering vehicle, and a storage medium. Background Art

[0004] As an important part of intelligent unmanned operations, point cloud maps mainly serve to build three-dimensional models of real scenes for unmanned excavators to plan excavation and loading tasks.

[0005] The perception system of an unmanned excavator consists of multiple radars. During excavation and loading operations, the excavator rotates on a relatively fixed step most of the time to perform excavation and loading operations, and sometimes moves the working position within a small range. For this typical working condition, the perception system needs to establish a three-dimensional point cloud map of the surrounding environment in real time and meet the following requirements: (1) When the excavator goes down the platform for refueling or maintenance and then goes back on the platform, or when the excavator changes the working area, it can automatically load the previously created point cloud map; (2) When the bucket performs an excavation operation in the excavation area, or when the bucket performs a loading operation on top of the truck, the point cloud map of the excavation and loading area can be automatically loaded. The geometric shape of the material will change, requiring the perception system to be able to dynamically update the point cloud map of the excavation area in real time; (3) When the non-excavation area changes due to side wall material landslides or other equipment entering the site for crushing and cleaning operations, the perception system is required to be able to update the map; (4) When multiple radars are turned on, the perception system is required to simultaneously perform multiple radar real-time mapping updates; (5) When the excavator continues to repeat the rotation and excavation operations, the perception system radar continues to repeat the three-dimensional surrounding environment mapping, and the point cloud is repeatedly superimposed. It is required to reasonably limit the size of the point cloud map and ensure the uniformity of the map.

[0006] Summary of the Invention

[0007] In a first aspect of the present disclosure, a point cloud map updating method is provided, comprising: loading an initial global point cloud map according to a current position of an engineering vehicle; dividing the global point cloud map into regions to obtain a plurality of local point cloud maps; determining the region in which each of a plurality of on-board radars is currently located; using the plurality of on-board radars, updating the local point cloud map corresponding to the excavation and loading area at a first predetermined period, and updating the local point cloud map corresponding to the non-excavation and loading area at a second predetermined period; and updating the global point cloud map according to the update results of the plurality of local point cloud maps.

[0008] In some embodiments, updating the local point cloud map corresponding to the excavation and loading area at a first predetermined period includes: generating a local point cloud map corresponding to the excavation and loading area at the first predetermined period; determining the area where the bucket in the engineering vehicle is located; when the engineering vehicle is performing excavation and loading operations, clearing the local point cloud map of the area where the bucket is located; and regenerating the local point cloud map of the area where the bucket is located.

[0009] In some embodiments, determining the area where the bucket of the engineering vehicle is located includes: according to the current position P of the bucket w_bucket Determine the area where the bucket is located.

[0010] In some embodiments, the current position P of the bucket w_bucket The current posture R of the excavator carrier in the engineering vehicle w_b and the current position P w_b , and the position P of the bucket in the excavator carrier coordinate system b_bucket Sure.

[0011] In some embodiments, the current position P of the bucket w_bucket is the current posture R of the excavator carrier w_b and the position P of the bucket in the excavator carrier coordinate system b_bucket The product of the current position P of the excavator carrier w_b The sum value of .

[0012] In some embodiments, determining the area where the bucket of the engineering vehicle is located includes: dividing the local point cloud map into spatial grids; and determining the area where the bucket is located based on the grid where the bucket is currently located.

[0013] In some embodiments, updating the local point cloud map corresponding to the non-excavation area at a second predetermined period includes: updating the local point cloud map corresponding to the non-excavation area in the background at the second predetermined period; extracting key frame features of the local point cloud map corresponding to the non-excavation area, and extracting real-time point cloud features; performing feature change detection based on the key frame features and the real-time point cloud features; and updating the point cloud map for areas where the feature change amount is greater than a predetermined threshold.

[0014] In some embodiments, extracting key frame features of the local point cloud map corresponding to the non-excavation area includes: dividing the local point cloud map corresponding to the non-excavation area into multiple cells of a predetermined size; performing feature decomposition on each cell to determine whether the feature type of each cell is a surface feature; performing variance decomposition on the feature covariance of multiple cells with surface feature types to obtain a rotation matrix; using the rotation matrix to transform the direction of the features of each cell to obtain a rotation-invariant feature direction; using the feature direction to extract key frames from each cell; and establishing a feature histogram for the extracted key frames.

[0015] In some embodiments, performing feature decomposition on each cell to determine whether the feature type of each cell is a surface feature includes: performing feature decomposition on each cell to obtain a first cell eigenvalue λ1, a second cell eigenvalue λ2, and a third cell eigenvalue λ3, where λ1≥λ2≥λ3; when the second cell eigenvalue λ2 is much larger than the third cell eigenvalue λ3, determining that the feature of each cell is a surface feature.

[0016] In some embodiments, obtaining the rotation matrix includes: performing variance decomposition on the characteristic covariance to obtain the characteristic vector V d ; According to the feature vector V d The rotation matrix R is obtained.

[0017] In some embodiments, the feature vector V d =[V d1 V d2 V d3 ]; the rotation matrix is ​​R = [V d1 V d2 V d1 ×V d2 ].

[0018] In some embodiments, the center of the initial global point cloud map is the current position P of the excavator carrier in the engineering vehicle. w_b , the radius of the initial global point cloud map is the operating radius of the engineering vehicle.

[0019] In some embodiments, the current position P of the excavator carrier w_b The position P of the excavator positioning sensor in the global coordinate system w_g and Posture R w_g , the position P of the excavator carrier coordinate system relative to the positioning antenna g_b Sure.

[0020] In some embodiments, the current position P of the excavator carrier w_b Position the excavator sensor in the global coordinate system R w_g The position P of the excavator carrier coordinate system relative to the positioning antenna g_b The product of the position P of the excavator positioning sensor in the global coordinate system w_g The sum value of .

[0021] In a second aspect of the present disclosure, a point cloud map updating device is provided, comprising: a first processing module configured to load an initial global point cloud map according to a current position of an engineering vehicle; a second processing module configured to divide the global point cloud map into regions to obtain a plurality of local point cloud maps; a third processing module configured to determine the region where each of a plurality of vehicle-mounted radars is currently located; a fourth processing module configured to use the plurality of vehicle-mounted radars to update the local point cloud map corresponding to the excavation and loading area at a first predetermined period, and to update the local point cloud map corresponding to the non-excavation and loading area at a second predetermined period; and a fifth processing module configured to update the global point cloud map according to the update results of the plurality of local point cloud maps.

[0022] In a third aspect of the present disclosure, a point cloud map updating device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute the method as described in any of the above embodiments based on instructions stored in the memory.

[0023] In a fourth aspect of the present disclosure, an engineering vehicle is provided, comprising a point cloud map updating device as described in any of the above embodiments.

[0024] In some embodiments, the construction vehicle comprises an excavator.

[0025] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0026] According to a sixth aspect of an embodiment of the present disclosure, a computer program is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0027] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0029] FIG1 is a schematic diagram of an excavation and loading operation scene according to an embodiment of the present disclosure;

[0030] FIG2 is a schematic diagram of a flow chart of a point cloud map updating method according to an embodiment of the present disclosure;

[0031] FIG3 is a flow chart of a point cloud map updating method according to another embodiment of the present disclosure;

[0032] FIG4 is a flow chart of a point cloud map updating method according to another embodiment of the present disclosure;

[0033] FIG5 is a schematic structural diagram of a point cloud map updating device according to an embodiment of the present disclosure;

[0034] FIG6 is a schematic structural diagram of a point cloud map updating device according to another embodiment of the present disclosure;

[0035] FIG7 is a schematic structural diagram of an engineering vehicle according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0037] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0038] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0039] Technologies, methods and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, such technologies, methods and equipment should be considered part of the authorization specification.

[0040] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0041] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0042] The inventors noticed that the point cloud map update solution in the related art cannot update the point cloud map of the excavation area in a timely manner when the excavation area scene changes, and cannot meet the needs of the excavation operation scene.

[0043] Based on this, the present disclosure provides a point cloud map update solution, which can timely update the point cloud map of the excavation and loading area when the excavation and loading area scene changes, effectively meeting the needs of the excavation and loading operation scene.

[0044] Figure 1 is a schematic diagram of an excavation and loading operation scenario according to one embodiment of the present disclosure. As shown in Figure 1, an excavator 11 comprises a traveling device, a slewing device, an arm device, and a bucket. During excavation and loading, the excavator 11 is parked on a loading platform 13 near a sidewall material 33. The slewing device rotates the bucket 12 to the excavation area 31, and the arm device controls the bucket excavation. After the bucket is excavated, the arm device is raised, and the slewing device rotates the bucket material to the loading area 32, where it is loaded into the bucket of a transport truck 14. This excavation and loading process continues in a reciprocating cycle until the transport truck is full.

[0045] The excavator is equipped with multiple LiDAR sensors 21-25 for mapping the sidewall material and loading areas. It is also equipped with a GPS (Global Positioning System) / IMU (Inertial Measurement Unit) for global positioning, loading the initial point cloud map based on the global position, and performing global point cloud map updates.

[0046] The point cloud map of the sidewall material area represents the distribution and shape of the material to be excavated. It provides the excavator with information such as the material's location, shape, and size, which is used for excavation trajectory planning. If the shape of the material changes during a bucket excavation, the point cloud mapping system should be able to promptly update the point cloud map of the material area to facilitate trajectory planning for the next excavation.

[0047] If the side wall material area changes due to non-excavation actions, such as side wall material sliding, rockfall in the transport truck parking area, rockfall in the excavator driving area, etc., the point cloud mapping update system should be able to update the material area shape in the background (if it slides into the current excavation and loading area, it will be updated in real time).

[0048] The loading area point cloud map represents the distribution and shape of materials within the transport truck. This provides the excavator with information such as where to load materials and the shape and size of the loaded materials, allowing for loading trajectory planning. If the shape of the materials within the truck changes during a loading operation, the point cloud mapping system should be able to promptly update the loading area point cloud map shape to facilitate trajectory planning for the next loading operation.

[0049] The field of view of a single laser radar is limited. The field of view of the radar shown in the figure is 77 degrees. During excavation and loading operations, the excavator generally rotates only within a small range to improve loading efficiency. Therefore, the vehicle body can be equipped with multiple laser radars to compensate for the radar's scanning blind spots. As an embodiment, Figure 1 shows five laser radars 21-25, one on each side in front of the excavator, and one on the left, right, and tail of the excavator.

[0050] Figure 2 is a flow chart of a point cloud map updating method according to an embodiment of the present disclosure. In some embodiments, the following point cloud map updating method is executed by a point cloud map updating device.

[0051] In step 201 , an initial global point cloud map is loaded according to the current position of the engineering vehicle.

[0052] It should be noted here that when the excavator goes down the platform to refuel or return to the platform after maintenance, or when the excavator changes the working area, the GPS / IMU positioning sensor of the excavator is obtained through GPS in the global coordinate system. w_g The current position P of the excavator carrier can be obtained by the following formula (1): w_b , the position T of each radar in the global coordinate system can be obtained by the following formula (2): w_l (R w_l ,P w_l ), the position P of the bucket in the global coordinate system can be obtained by the following formula (3): w_bucket .P w_b =R w_g *P g_b +P w_g (1) T w_l =T w_g *T g_b *T b_i *T i_l (2)

[0053] Among them, T w_g (R w_g ,P w_g ) is the position of the GPS positioning sensor of the excavator in the global coordinate system, T g_b (R g_b ,P g_b ) is the position of the excavator carrier coordinate system relative to the GPS positioning antenna, T b_i (R b_i ,P b_i ) is the position of the IMU coordinate system relative to the GPS positioning antenna coordinate system, T i_l (R i_l ,P i_l ) is the position of the laser radar coordinate system relative to the IMU coordinate system, T b_bucket (R b_bucket ,P b_bucket ) is the position of the bucket in the excavator carrier coordinate system.

[0054] It should be noted that if the excavator already has a point cloud map of the area, it will load the point cloud map with P w_b Point cloud map of the corresponding area within the excavator's operating radius as the center.

[0055] If a point cloud map does not exist for the area, a point cloud map is generated using radar.

[0056] The global map of the excavation area is loaded only once each time the system is powered on and initialized.

[0057] In some embodiments, the center of the initial global point cloud map is the current position P of the excavator carrier in the engineering vehicle. w_b ,The radius of the initial global point cloud map is the operating radius of the engineering vehicle.

[0058] In some embodiments, the current position P of the excavator carrier w_b The position P of the excavator positioning sensor in the global coordinate system w_g and Posture R w_g , the position P of the excavator carrier coordinate system relative to the positioning antenna g_b Sure.

[0059] In some embodiments, as shown in formula (1), the current position P of the excavator carrier is w_b Position the excavator sensor in the global coordinate system R w_g The position P of the excavator carrier coordinate system relative to the positioning antenna g_b The product of the position of the excavator positioning sensor in the global coordinate system P w_g The sum value of .

[0060] In step 202 , the global point cloud map is divided into regions to obtain a plurality of local point cloud maps.

[0061] It should be noted here that when the laser radar creates or updates the point cloud map, it uses the current frame point cloud and the local map point cloud to calculate the position increment T of the laser radar relative to the local map. l_incre ; Combine the pose increment with the initial pose T w_l The position and posture of the lidar under the global map are obtained by cyclic multiplication and update. According to the global position and posture of the lidar, the point cloud under the lidar coordinates is transformed to the global map coordinate system, and the global point cloud is obtained by accumulation.

[0062] To improve point cloud registration efficiency, the global map of the excavation and loading area is divided into regions, taking into account the fact that excavators mostly rotate. As an example, as shown in Figure 1, eight regions 1-8 are divided, centered on the excavator carrier and rotating 360 degrees. These eight regions are fixed and do not change with the excavator's rotation. A kdtree is constructed for the point clouds in each of these eight regions, which serves as the search tree for point cloud registration.

[0063] It should be made clear that the shape of the point cloud in these eight areas is not fan-shaped, but is determined by the actual shape accumulation results of the point cloud scanned and registered in real time by the lidar in the corresponding area.

[0064] In step 203 , the area where each of the plurality of vehicle-mounted radars is currently located is determined.

[0065] It should be noted that when the excavator is rotating to dig and load, the real-time position of each radar determines the local map area in which it is currently located. As shown in Figure 1, radar 21 is located in area 3, radar 22 is located in area 4, radar 23 is located in areas 5 / 6, radar 24 is located in areas 7 / 8, and radar 25 is located in area 1 / 2. The corresponding critical areas can be further restricted by the rotation angle threshold.

[0066] In step 204 , a local point cloud map corresponding to the excavation and loading area is updated at a first predetermined period using multiple vehicle-mounted radars.

[0067] In some implementations, the step of updating the local point cloud map corresponding to the excavation area is shown in FIG3 .

[0068] In step 301, a local point cloud map corresponding to the excavation area is generated in a first predetermined period.

[0069] According to the excavator's action state, the excavator performed an excavation operation. As shown in Figure 1, the excavator performed an excavation operation in areas 1 and 8. A loading operation was performed in areas 3 and 4.

[0070] Due to excavation or loading operations, the shape of the material changes, and the local map of the area will no longer be available. The subsequent real-time radar point cloud cannot be aligned with the original local map point cloud. Therefore, it is necessary to directly clear the local map of the area where the bucket is located.

[0071] As an embodiment, the first predetermined period is 20 ms.

[0072] In step 302 , the area where the bucket of the construction vehicle is located is determined.

[0073] In some embodiments, based on the current position P of the bucket w_bucket Determine the area where the bucket will be located.

[0074] For example, the current position P of the bucket w_bucket The current posture R of the excavator carrier in the engineering vehicle w_b and the current position P w_b , and the bucket position P in the excavator carrier coordinate system b_bucket Sure.

[0075] For example, as shown in formula (3), the current position P of the bucket is w_bucket is the current posture R of the excavator carrier w_b and the bucket position P in the excavator carrier coordinate system b_bucket The product of the current position P of the excavator carrier w_b The sum value of .

[0076] In another embodiment, the local point cloud map is divided into spatial grids, and the area where the bucket is located is determined based on the grid where the bucket is currently located.

[0077] In step 303 , when the engineering vehicle is performing excavation and loading operations, the local point cloud map of the area where the bucket is located is cleared.

[0078] It should be noted here that if the engineering vehicle has not been excavated or loaded (the shape of the material will not change) and is only rotating, the point cloud will not be cleared according to the bucket position.

[0079] In step 304 , a local point cloud map of the area where the bucket is located is regenerated.

[0080] The lidar point cloud registration method is used for the area corresponding to the bucket to regenerate the local point cloud map.

[0081] Returning to FIG. 2 , in step 205 , the local point cloud map corresponding to the non-excavation area is updated at a second predetermined period using multiple vehicle-mounted radars.

[0082] In some implementations, the step of updating the local point cloud map corresponding to the non-excavation area is shown in FIG4 .

[0083] In step 401, the background updates the local point cloud map corresponding to the non-excavation area at a second predetermined period.

[0084] It should be noted that in non-excavation areas, the material shape generally does not change unless there is material sliding off the sidewall, rockfall in the truck parking area, or rockfall in the excavator driving area. To reduce system operating resources, local map updates in non-excavation areas are run on a separate thread.

[0085] In some embodiments, the second predetermined period is 500 ms.

[0086] In some embodiments, a keyframe feature matching method is adopted to update the local map area with shape changes.

[0087] In step 402 , key frame features of the local point cloud map corresponding to the non-excavation area are extracted, and real-time point cloud features are extracted.

[0088] In some embodiments, extracting key frame features of a local point cloud map corresponding to a non-excavation area includes the following steps:

[0089] 1) Divide the local point cloud map corresponding to the non-excavation area into a plurality of cells of a predetermined size.

[0090] For example, the size of a cell is 0.2m.

[0091] The local point cloud map contains all the cubic cells. In order to quickly find the cells, a hash table and octree are built in the local point cloud map, and the cells are searched through the octree.

[0092] 2) Perform feature decomposition on each cell to determine whether the feature type of each cell is a face feature.

[0093] For example, perform eigendecomposition on a cell to obtain a first cell eigenvalue λ1, a second cell eigenvalue λ2, and a third cell eigenvalue λ3, where λ1 ≥ λ2 ≥ λ3. If the second cell eigenvalue λ2 is much larger than the third cell eigenvalue λ3, the feature type of the cell is determined to be a face feature.

[0094] In addition, when the first cell characteristic value λ2 is much greater than the second cell characteristic value λ3, the feature type of the cell is determined to be a line feature.

[0095] 3) Perform variance decomposition on the feature covariance of multiple cells with face feature types to obtain a rotation matrix.

[0096] In some embodiments, the feature covariance is subjected to variance decomposition to obtain three eigenvalues ​​λ d1 ,λd2 ,λ d3 , and the corresponding eigenvector V d , where λ d1 ≥λ d2 ≥λ d3 , V d =[V d1 V d2 V d3 ].

[0097] Next, according to the eigenvector V d Get the rotation matrix R.

[0098] For example, the rotation matrix R = [V d1 V d2 V d1 ×V d2 ].

[0099] 4) Use the rotation matrix to transform the direction of each cell feature to obtain the rotation-invariant feature direction.

[0100] 5) Extract key frames for each cell using feature directions.

[0101] 6) Create a feature histogram for the extracted key frames.

[0102] It should be noted that a feature histogram is established for the extracted real-time point cloud features.

[0103] In step 403, feature change detection is performed based on the key frame features and the real-time point cloud features.

[0104] For example, the Euclidean distance between key frame features and real-time point cloud features is used to determine whether the point cloud has changed.

[0105] In step 404, the point cloud map is updated for areas where the feature change is greater than a predetermined threshold.

[0106] It should be noted that for areas where the feature change is not greater than a predetermined threshold, there is no need to update the point cloud map.

[0107] Returning to FIG. 2 , in step 206 , the global point cloud map is updated according to the update results of the multiple local point cloud maps.

[0108] The global point cloud is divided into spatial grids, the overlapping areas of each network are screened, and the point clouds in the blank areas are supplemented to obtain an updated global map.

[0109] In the point cloud map updating method provided in the above embodiment of the present disclosure, the point cloud map of the excavation and loading area is updated in a timely manner when the scene of the excavation and loading area changes, thereby effectively meeting the needs of the excavation and loading operation scene.

[0110] FIG5 is a schematic diagram of the structure of a point cloud map updating device according to an embodiment of the present disclosure. As shown in FIG5 , the point cloud map updating device includes a first processing module 51 , a second processing module 52 , a third processing module 53 , a fourth processing module 54 and a fifth processing module 55 .

[0111] The first processing module 51 is configured to load an initial global point cloud map according to the current position of the engineering vehicle.

[0112] In some embodiments, the center of the initial global point cloud map is the current position P of the excavator carrier in the engineering vehicle. w_b ,The radius of the initial global point cloud map is the operating radius of the engineering vehicle.

[0113] In some embodiments, the current position P of the excavator carrier w_b The position P of the excavator positioning sensor in the global coordinate system w_g and Posture R w_g , the position P of the excavator carrier coordinate system relative to the positioning antenna g_b Sure.

[0114] In some embodiments, as shown in formula (1), the current position P of the excavator carrier is w_b Position the excavator sensor in the global coordinate system R w_g The position P of the excavator carrier coordinate system relative to the positioning antenna g_b The product of the position of the excavator positioning sensor in the global coordinate system P w_g The sum value of .

[0115] The second processing module 52 is configured to divide the global point cloud map into regions to obtain a plurality of local point cloud maps.

[0116] The third processing module 53 is configured to determine an area where each of the plurality of vehicle-mounted radars is currently located.

[0117] The fourth processing module 54 is configured to use multiple vehicle-mounted radars to update the local point cloud map corresponding to the excavation and loading area at a first predetermined period, and to update the local point cloud map corresponding to the non-excavation and loading area at a second predetermined period.

[0118] In some embodiments, the fourth processing module 54 is configured to execute the steps described in any embodiment of FIG3 to update the local point cloud map corresponding to the excavation area at a first predetermined period. The fourth processing module 54 is further configured to execute the steps described in any embodiment of FIG4 to update the local point cloud map corresponding to the non-excavation area at a second predetermined period.

[0119] The fifth processing module 55 is configured to update the global point cloud map according to the update results of the multiple local point cloud maps.

[0120] FIG6 is a schematic diagram of the structure of a point cloud map updating device according to another embodiment of the present disclosure. As shown in FIG6 , the point cloud map updating device includes a memory 61 and a processor 62 .

[0121] The memory 61 is used to store instructions. The processor 62 is coupled to the memory 61 . The processor 62 is configured to execute the method involved in any one of the embodiments in FIG. 2 to FIG. 4 based on the instructions stored in the memory.

[0122] As shown in Figure 6, the point cloud map updating device also includes a communication interface 63 for exchanging information with other devices. At the same time, the point cloud map updating device also includes a bus 64, through which the processor 62, the communication interface 63, and the memory 61 communicate with each other.

[0123] The memory 61 may include high-speed RAM memory or non-volatile memory, such as at least one disk storage device. The memory 61 may also be a memory array. The memory 61 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules.

[0124] Furthermore, the processor 62 may be a central processing unit (CPU), or may be an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the present disclosure.

[0125] The present disclosure also relates to a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method involved in any one of the embodiments in Figures 2 to 4 is implemented.

[0126] FIG7 is a schematic diagram of the structure of an engineering vehicle according to an embodiment of the present disclosure. As shown in FIG7 , the engineering vehicle 70 includes a point cloud map updating device 71 , which is the point cloud map updating device involved in any one of the embodiments in FIG5 or FIG6 .

[0127] In some embodiments, the construction vehicle 70 comprises an excavator.

[0128] By implementing the above embodiments of the present disclosure, the following beneficial effects can be achieved:

[0129] (1) When the excavator moves off the workbench, returns to the workbench after maintenance, or when the excavator changes the work area, the previously created point cloud map can be automatically loaded;

[0130] (2) When the bucket performs an excavation operation in the excavation area, or when the bucket performs a loading operation on top of the truck, the shape of the material in the excavation and loading area will change, and the perception system can dynamically update the point cloud map of the excavation and loading area in real time;

[0131] (3) When the non-excavation area changes due to sidewall material landslide or other equipment entering the site for crushing and cleaning operations, the perception system can update the map in the background;

[0132] (4) When multiple radars are turned on, the perception system can simultaneously perform real-time mapping updates on multiple radars;

[0133] (5) When the excavator continuously rotates and performs digging operations, the perception system radar continuously builds a three-dimensional map of the surrounding environment, and the point cloud is repeatedly superimposed, which can reasonably limit the size of the point cloud map and ensure the uniformity of the map.

[0134] In some embodiments, the functional units described above may be implemented as general-purpose processors, programmable logic controllers (PLC), digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof, for performing the functions described in the present disclosure.

[0135] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0136] The description of the present disclosure is provided for purposes of illustration and description and is not intended to be exhaustive or to limit the disclosure to the disclosed form. Many modifications and variations will be apparent to those skilled in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure and to enable those skilled in the art to understand the present disclosure and design various embodiments with various modifications suitable for specific applications.

Claims

1. A point cloud map updating method, comprising: Load the initial global point cloud map according to the current position of the engineering vehicle; Dividing the global point cloud map into regions to obtain a plurality of local point cloud maps; Determine a region where each of the plurality of vehicle-mounted radars is currently located; Using the multiple vehicle-mounted radars, the local point cloud map corresponding to the excavation and loading area is updated at a first predetermined period, and the local point cloud map corresponding to the non-excavation and loading area is updated at a second predetermined period; The global point cloud map is updated according to the update results of the multiple local point cloud maps.

2. The method according to claim 1, wherein: Updating the local point cloud map corresponding to the excavation and loading area at a first predetermined period includes: Generating a local point cloud map corresponding to the excavation and loading area in the first predetermined period; determining the area where the bucket in the engineering vehicle is located; When the engineering vehicle is performing excavation and loading operations, clearing a local point cloud map of the area where the bucket is located; Regenerate a local point cloud map of the area where the bucket is located.

3. The method according to claim 2, wherein: Determining the area where the bucket in the engineering vehicle is located includes: According to the current position P of the bucket w_bucket The area in which the bucket is located is determined.

4. The method according to claim 3, wherein: The current position P of the bucket w_bucket The current posture R of the excavator carrier in the engineering vehicle w_b and the current position P w_b , and the position P of the bucket in the coordinate system of the excavator carrier b_bucket Sure.

5. The method according to claim 4, wherein: The current position P of the bucket w_bucket is the current posture R of the excavator carrier w_b and the position P of the bucket in the coordinate system of the excavator carrier b_bucket The product of the current position P of the excavator carrier w_b The sum value of .

6. The method according to claim 2, wherein: Determining the area where the bucket in the engineering vehicle is located includes: Dividing the local point cloud map into spatial grids; The area where the bucket is located is determined according to the grid where the bucket is currently located.

7. The method according to claim 1, wherein: Updating the local point cloud map corresponding to the non-excavation area at a second predetermined period includes: In a second predetermined period, the background updates the local point cloud map corresponding to the non-excavation area; Extract key frame features of the local point cloud map corresponding to the non-excavation area, and extract real-time point cloud features; Perform feature change detection according to the key frame features and the real-time point cloud features; For areas where the feature change is greater than a predetermined threshold, the point cloud map is updated.

8. The method according to claim 7, wherein: The key frame features of the local point cloud map corresponding to the non-excavation area are extracted, including: Dividing the local point cloud map corresponding to the non-excavation area into a plurality of cells of a predetermined size; Performing feature decomposition on each cell to determine whether the feature type of each cell is a face feature; Perform variance decomposition on the feature covariance of multiple cells with face feature types to obtain a rotation matrix; Using the rotation matrix to transform the direction of the feature of each cell to obtain a rotationally invariant feature direction; Extracting key frames from each cell using the characteristic direction; Create feature histograms for the extracted key frames.

9. The method according to claim 8, wherein: Performing feature decomposition on each cell to determine whether the feature type of each cell is a face feature includes: Performing eigendecomposition on each of the cells to obtain a first cell eigenvalue λ1, a second cell eigenvalue λ2, and a third cell eigenvalue λ3, wherein λ1≥λ2≥λ3; When the second cell characteristic value λ2 is much greater than the third cell characteristic value λ3, it is determined that the feature of each cell is a surface feature.

10. The method according to claim 8, wherein: The rotation matrix is ​​obtained: The feature covariance is subjected to variance decomposition to obtain the feature vector V d ; According to the feature vector V d The rotation matrix R is obtained.

11. The method according to claim 10, wherein: The feature vector V d =[V d1 V d2 V d3 ]; The rotation matrix is ​​R = [V d1 V d2 V d1 ×V d2 ].

12. The method according to any one of claims 1 to 11, wherein: The center of the initial global point cloud map is the current position P of the excavator carrier in the engineering vehicle. w_b , the radius of the initial global point cloud map is the operating radius of the engineering vehicle.

13. The method according to claim 12, wherein: The current position P of the excavator carrier w_b The position P of the excavator positioning sensor in the global coordinate system w_g and posture R w_g , the position P of the excavator carrier coordinate system relative to the positioning antenna g_b Sure.

14. The method according to claim 13, wherein: The current position P of the excavator carrier w_b The posture R of the excavator positioning sensor in the global coordinate system w_g The position P of the excavator carrier coordinate system relative to the positioning antenna g_b The product of the position P of the excavator positioning sensor in the global coordinate system w_g The sum value of .

15. A point cloud map updating device, comprising: A first processing module is configured to load an initial global point cloud map according to the current position of the engineering vehicle; A second processing module is configured to divide the global point cloud map into regions to obtain a plurality of local point cloud maps; A third processing module is configured to determine a region where each of the plurality of vehicle-mounted radars is currently located; A fourth processing module is configured to use the multiple vehicle-mounted radars to update the local point cloud map corresponding to the excavation area at a first predetermined period, and to update the local point cloud map corresponding to the non-excavation area at a second predetermined period; The fifth processing module is configured to update the global point cloud map according to the update results of the multiple local point cloud maps.

16. A point cloud map updating device, comprising: Memory; A processor is coupled to the memory, and the processor is configured to execute the method according to any one of claims 1 to 14 based on instructions stored in the memory.

17. An engineering vehicle, comprising the point cloud map updating device as claimed in claim 15 or 16.

18. The engineering vehicle according to claim 17, wherein: The engineering vehicle includes an excavator.

19. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method according to any one of claims 1 to 14 is implemented.

20. A computer program comprising computer instructions, wherein the computer instructions implement the method according to any one of claims 1 to 14 when executed by a processor.

Images