Point cloud map construction method and system, domain controller and excavator

Abstract

The invention provides a point cloud map construction method and system, a domain controller and an excavator. The point cloud map construction method comprises the steps of constructing a first point cloud map according to point cloud data acquired by a laser radar and attitude data acquired by a turntable attitude sensor under the condition that an excavator turntable is in a static state, and extracting a static image of a target object from the first point cloud map; constructing a second point cloud map according to the point cloud data acquired by the laser radar and the attitude data acquired by the turntable attitude sensor under the condition that the excavator turntable rotates towards the first direction, and extracting a first double image of the target object from the second point cloud map; under the condition that the excavator turntable rotates towards a second direction, constructing a third point cloud map according to the point cloud data acquired by the laser radar and the attitude data acquired by the turntable attitude sensor, and extracting a second ghosting image of the target object from the third point cloud map; and eliminating the ghosting in the second point cloud map by using the still image and the first ghosting, and eliminating the ghosting in the third point cloud map by using the still image and the second ghosting.

Description

Technical Field

[0001] The present disclosure relates to the technical field of intelligent unmanned engineering machinery, and in particular to a point cloud map construction method and system, a domain controller and an excavator. Background Art

[0002] Excavators are important engineering machinery, widely used in construction, mining and other engineering construction industries. Excavators often encounter problems such as harsh operating environment and high workload for operators, which can easily lead to safety hazards. In addition, the difficulty in hiring and high costs will also limit the industry. Therefore, the development of excavators with unmanned operation functions has huge industry demand and practical significance.

[0003] As an important part of unmanned excavator operation, point cloud map is mainly used to construct three-dimensional point cloud map of real scene for unmanned excavator to plan excavation and loading tasks. Summary of the invention

[0004] The inventors noticed that a typical working condition of an unmanned excavator is to perform excavation and loading operations by rotating on a relatively fixed platform. In this process, due to the time asynchrony between the laser radar installed on the excavator and the turntable attitude sensor, ghosting occurs in the generated point cloud map.

[0005] Based on this, the present disclosure provides a point cloud map construction method that can effectively eliminate ghosting in the point cloud map.

[0006] In a first aspect of the present disclosure, a point cloud map construction method is provided, which is executed by a domain controller, including: when an excavator turntable is in a stationary state, a first point cloud map is constructed based on point cloud data collected by a laser radar and posture data collected by a turntable posture sensor, and a static image of a target object is extracted from the first point cloud map; when the excavator turntable rotates in a first direction, a second point cloud map is constructed based on point cloud data collected by the laser radar and posture data collected by the turntable posture sensor, and a first ghost of the target object is extracted from the second point cloud map; when the excavator turntable rotates in a second direction opposite to the first direction, a third point cloud map is constructed based on point cloud data collected by the laser radar and posture data collected by the turntable posture sensor, and a second ghost of the target object is extracted from the third point cloud map; the static image and the first ghost are used to eliminate ghosts in the second point cloud map, and the static image and the second ghost are used to eliminate ghosts in the third point cloud map.

[0007] In some embodiments, the use of the still image and the first ghost to eliminate ghosts in the second point cloud map includes: generating a first posture transformation matrix from the first ghost to the still image; and using the first posture transformation matrix to align the second point cloud map with the first point cloud map to eliminate ghosts in the second point cloud map.

[0008] In some embodiments, generating a first posture transformation matrix from the first ghost to the still image includes: using an iterative closest point ICP registration algorithm to register the first ghost with the still image to generate the first posture transformation matrix.

[0009] In some embodiments, the use of the still image and the second ghost to eliminate ghosts in the third point cloud map includes: generating a second posture transformation matrix from the second ghost to the still image; and using the second posture transformation matrix to align the third point cloud map with the first point cloud map to eliminate ghosts in the third point cloud map.

[0010] In some embodiments, generating a second posture transformation matrix from the second ghost to the still image includes: using an ICP registration algorithm to register the second ghost with the still image to generate the second posture transformation matrix.

[0011] In some embodiments, the point cloud data collected by the laser radar includes frame start time, frame length and frame point cloud data; the attitude data collected by the turntable attitude sensor includes roll angle, pitch angle, rotation angle and rotation speed.

[0012] In some embodiments, the above method also includes: determining the position of the target object; wherein, determining the position of the target object includes: controlling a specified bucket tooth in the bucket of the excavator to touch the top surface of the target object, wherein the target object is set within the construction radius of the bucket of the excavator; recording the position of the tooth tip of the specified bucket tooth as the position of the target object.

[0013] In some embodiments, the designated tooth comprises a center tooth of the bucket.

[0014] In some embodiments, the target object comprises a cylinder; and controlling a designated bucket tooth in the bucket of the excavator to touch a top surface of the target object comprises: controlling the middle bucket tooth to touch a central area of ​​the top surface of the target object.

[0015] In some embodiments, extracting the still image of the target object from the first point cloud map includes: extracting the still image from the first point cloud map according to the position of the target object.

[0016] In some embodiments, extracting the first ghost of the target object from the second point cloud map includes: extracting the first ghost from the second point cloud map according to the position of the target object.

[0017] In some embodiments, extracting the second ghost image of the target object from the third point cloud map includes: extracting the second ghost image from the third point cloud map according to the position of the target object.

[0018] In a second aspect of the present disclosure, a domain controller is provided, including: a first processing module, configured to construct a first point cloud map according to point cloud data collected by a laser radar and posture data collected by a turntable posture sensor when the excavator turntable is in a stationary state, and extract a still image of a target object from the first point cloud map; a second processing module, configured to construct a second point cloud map according to point cloud data collected by the laser radar and posture data collected by the turntable posture sensor when the excavator turntable rotates in a first direction, and extract a first ghost of the target object from the second point cloud map; a third processing module, configured to construct a third point cloud map according to point cloud data collected by the laser radar and posture data collected by the turntable posture sensor when the excavator turntable rotates in a second direction opposite to the first direction, and extract a second ghost of the target object from the third point cloud map; a fourth processing module, configured to eliminate ghosts in the second point cloud map using the still image and the first ghost, and eliminate ghosts in the third point cloud map using the still image and the second ghost.

[0019] In a third aspect of the present disclosure, a domain controller is provided, comprising: a memory; and a processor, coupled to the memory, the processor being configured to execute, based on instructions stored in the memory, a point cloud map construction method as described in any of the above embodiments.

[0020] In a fourth aspect of the present disclosure, a point cloud map construction system is provided, comprising: a domain controller as described in any of the above embodiments; a laser radar configured to collect point cloud data and send the point cloud data to the domain controller; a turntable attitude sensor configured to collect attitude data of an excavator turntable relative to a dismounted part of the excavator, and send the attitude data to the domain controller.

[0021] In some embodiments, the laser radar is installed on the top of the cab of the excavator in a predetermined installation direction, and the installation angle of the laser radar is determined by a predetermined work task; the turntable attitude sensor is installed at the turntable of the excavator.

[0022] In some embodiments, the posture data includes a roll angle, a pitch angle, a slew angle, and a slew speed.

[0023] In a fifth aspect of the present disclosure, an excavator is provided, comprising: a point cloud map construction system as described in any of the above embodiments.

[0024] In a sixth aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0025] In a seventh aspect of the present disclosure, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0026] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0028] Figure 1 A schematic diagram of a point cloud map construction system according to an embodiment of the present disclosure;

[0029] Figure 2 A schematic diagram of a point cloud map ghosting according to an embodiment of the present disclosure;

[0030] Figure 3 A schematic diagram of a point cloud map construction method according to an embodiment of the present disclosure;

[0031] Figure 4 A schematic diagram of a process of constructing a point cloud map according to another embodiment of the present disclosure;

[0032] Figure 5 A schematic diagram of the structure of a domain controller according to an embodiment of the present disclosure;

[0033] Figure 6 A schematic diagram of the structure of a domain controller according to another embodiment of the present disclosure;

[0034] Figure 7 This is a schematic diagram of the structure of a domain controller according to another embodiment of the present disclosure;

[0035] Figure 8 A schematic diagram of the structure of a point cloud map construction system according to an embodiment of the present disclosure;

[0036] Fig. 9 The figure is a schematic structural diagram of an excavator according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0038] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0039] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0040] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0041] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0042] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0043] The inventors have noticed that there are several related technologies for point cloud map construction.

[0044] Related technology 1: The system obtains point cloud information of the earthwork material area through lidar equipment, and divides the lidar point cloud into plane points and corner points according to the curvature calculation method. By integrating IMU (Inertial Measurement Unit) information and point cloud registration method, it constructs multiple back-end constraints to achieve three-dimensional mapping of the material area of ​​the construction scene.

[0045] Related technology 2: By detecting the bumpy state of the vehicle, the point cloud data is divided into a non-bumpy state and a bumpy state, and the point cloud data in the non-bumpy state is optimized, ultimately achieving global optimization and point cloud mapping.

[0046] Related technology 3: By integrating lidar, inertial navigation and vision to achieve mapping and positioning, it can meet the mapping needs of unknown indoor and outdoor environments and the autonomous positioning of robots.

[0047] Related technology 4: Using a distributed framework based on Kalman filters to achieve simultaneous calibration, positioning, and mapping of multiple lidars. This implementation distributes intensive calculations to smaller computing devices or resources dedicated to each lidar and eliminates single point failure issues.

[0048] It should be noted that the above-mentioned related technologies do not specifically consider the scene characteristics of typical excavator operations, namely repeated movements in a small range and continuous changes in the surrounding environment, and the map ghosting phenomenon caused by the inherent time asynchrony between the laser radar and the on-board posture sensor in the unmanned excavator rotation mapping. For the typical working conditions of the excavator rotation operation, the above-mentioned existing technologies have the following shortcomings:

[0049] Related technology 1: The system realizes 3D mapping of material areas in construction scenes by fusing IMU information and LiDAR point cloud registration methods, but there is no relevant measure for the problem of re-synchronization caused by time asynchrony between IMU and LiDAR.

[0050] Related technology 2: This algorithm improves the positioning accuracy and mapping quality of vehicles on bumpy roads by detecting the bumpy state of the vehicle. However, there is basically no bumping during the rotation of the excavator, so it is not suitable for excavator operation scenarios.

[0051] Related technology 3: For the problem of low initial pose accuracy and large ghosting in point cloud maps, short-term real-time repair can be performed, but it cannot solve the ghosting problem during the continuous mapping process.

[0052] Related technology 4: Implementing multi-radar external parameter calibration and synchronous mapping through a distributed framework. However, external parameter calibration requires the collection of point and surface features, which is not very effective because the materials in the excavation and loading area change at any time.

[0053] Based on this, the present disclosure provides a point cloud map construction method, which can effectively eliminate ghosting in the point cloud map and improve the accuracy of the three-dimensional point cloud map.

[0054] like Figure 1 As shown, the laser radar 11 is installed on the top of the cab of the excavator 10, with the installation direction facing forward, and is configured to select the installation angle of the laser radar 11 according to the predetermined work task to collect environmental point cloud data.

[0055] In some embodiments, the laser radar 11 links the point cloud data to the domain controller 13 via a UDP (User Datagram Protocol) link.

[0056] The turntable attitude sensor 12 is installed at the turntable of the vehicle, and is used to measure the attitude data of the turntable relative to the vehicle. For example, the attitude data is rpy (Euler angle), that is, roll angle (roll), pitch angle (pitch), yaw angle (yaw), and yaw rate. The data is linked to the domain controller 13 through the CAN bus protocol.

[0057] The domain controller 13 is a computing device used to process the point cloud data reported by the laser radar 11 and the attitude data reported by the turntable attitude sensor 12 .

[0058] For example, the domain controller 11 receives low-frequency (e.g., 10 Hz) lidar point cloud data (including frame start time, frame length, frame point cloud) and high-frequency (e.g., 100 Hz) turntable attitude sensor data (including time, rotation angle, pitch angle, roll angle, and rotation speed), removes the point cloud data of the boom 14 in the lidar on-vehicle coordinate system to obtain the point cloud data of the material 15, and transforms the point cloud data of the material 15 to the excavator off-vehicle coordinate through spatial transformation (assuming that the excavator is not moving, and this off-vehicle coordinate is a stationary coordinate), and automatically splices the point cloud generated with the movement of the excavator in the stationary coordinate to construct a point cloud map of the excavator operation scene.

[0059] It should be noted here that when the turntable is stationary, the constructed point cloud map includes a static image of the target object, such as Figure 2 When the turntable rotates to the left, the constructed point cloud map includes the ghost of the target object, as shown in Figure 2 When the turntable rotates to the right, the constructed point cloud map includes the ghost of the target object, as shown in Figure 3 As shown in the left shadow bar in the figure, the accuracy of the 3D point cloud map is reduced due to the existence of ghost images in the point cloud map.

[0060] Figure 3 The following is a flow chart of a method for constructing a point cloud map according to an embodiment of the present disclosure. In some embodiments, the following method for constructing a point cloud map is executed by a domain controller, including steps 31-34.

[0061] In step 31, when the excavator turntable is in a stationary state, a first point cloud map is constructed based on the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor, and a static image of the target object is extracted from the first point cloud map.

[0062] In some embodiments, the point cloud data collected by the laser radar includes frame start time, frame length and frame point cloud data, and the attitude data collected by the turntable attitude sensor includes roll angle, pitch angle, rotation angle and rotation speed.

[0063] In step 32, when the excavator turntable rotates in the first direction, a second point cloud map is constructed based on the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor, and a first ghost image of the target object is extracted from the second point cloud map.

[0064] For example, the first direction is leftward or leftward.

[0065] In step 33, when the excavator turntable rotates in a second direction opposite to the first direction, a third point cloud map is constructed based on the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor, and a second ghost of the target object is extracted from the third point cloud map.

[0066] For example, if the first direction is to the left, the second direction is to the right. Conversely, if the first direction is to the right, the second direction is to the left.

[0067] In step 34 , ghosts in the second point cloud map are eliminated using the still image and the first ghost, and ghosts in the third point cloud map are eliminated using the still image and the second ghost.

[0068] In some embodiments, the step of eliminating ghosts in the second point cloud map using the still image and the first ghost includes the following.

[0069] 1) Generate the first pose transformation matrix from the first ghost image to the still image.

[0070] For example, the first ghost image is registered with the still image using an ICP (Iterative Closest Point) registration algorithm to generate a first posture transformation matrix T_L2O.

[0071] 2) Using the first posture transformation matrix, align the second point cloud map with the first point cloud map to eliminate ghosting in the second point cloud map.

[0072] For example, the second point cloud map is transformed from the vehicle coordinate system to the vehicle disembarkation coordinate system using the spatial transformation shown in formula (1) so as to align the first point cloud map.

[0073] (1)

[0074] In formula (1), is the point cloud data of the second point cloud map in the vehicle coordinate system. It is the point cloud data of the second point cloud map in the vehicle disembarkation coordinate system.

[0075] In some embodiments, the step of eliminating ghosts in the third point cloud map using the still image and the second ghost includes the following.

[0076] 1) Generate a second pose transformation matrix from the second ghost image to the still image.

[0077] For example, the second ghost image is registered with the still image using an ICP registration algorithm to generate a second posture transformation matrix T_R2O.

[0078] 2) Using the second posture transformation matrix, align the third point cloud map with the first point cloud map to eliminate ghosting in the third point cloud map.

[0079] For example, the third point cloud map is transformed from the vehicle coordinate system to the vehicle disembarkation coordinate system using the spatial transformation shown in formula (2) so as to align the first point cloud map.

[0080] (2)

[0081] In formula (2), is the point cloud data of the third point cloud map in the vehicle coordinate system. It is the point cloud data of the third point cloud map in the vehicle coordinate system.

[0082] In some embodiments, the position of the target object may be determined in advance so that an image of the target object can be quickly extracted from point cloud data near the position.

[0083] In some embodiments, the step of determining the location of the target object includes the following.

[0084] 1) Controlling a designated bucket tooth in the bucket of the excavator to touch the top surface of a target object, wherein the target object is set within a construction radius of the bucket of the excavator.

[0085] For example, specify the tooth to be the center tooth of a bucket.

[0086] For example, the target object is a cylinder.

[0087] 2) Record the position of the tooth tip of the specified bucket tooth as the position of the target object.

[0088] In some embodiments, when the target object is a cylinder, the middle teeth of the bucket are controlled to touch the central area of ​​the top surface of the target object, so that the target object can be obtained more accurately.

[0089] For example, a still image is extracted from the first point cloud map according to the position of the target object.

[0090] For another example, according to the position of the target object, the first ghost is extracted from the second point cloud map.

[0091] For another example, according to the position of the target object, the second ghost image is extracted from the third point cloud map.

[0092] In the point cloud map construction method provided in the above-mentioned embodiment of the present disclosure, point cloud maps are respectively established when the turntable is stationary, turning left and turning right, and the still image, the first ghost and the second ghost of the target object are respectively extracted. The corresponding posture transformation matrix is ​​determined according to the still image, the first ghost and the second ghost of the target object, so that according to the posture transformation matrix, the point cloud maps established when the turntable is turning left and turning right respectively are aligned with the point cloud map established when the turntable is stationary, thereby effectively eliminating ghosts in the point cloud map.

[0093] In some embodiments, the step of constructing a point cloud map based on the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor is as follows: Figure 4 shown.

[0094] 1) The laser radar sends the collected point cloud data to the domain controller, and the turntable attitude sensor sends the collected attitude data to the domain controller.

[0095] In some embodiments, the point cloud data collected by the laser radar is environmental point cloud data, and the point cloud data collected by the laser radar includes frame start time, frame length and frame point cloud data.

[0096] In some embodiments, the attitude data collected by the turntable attitude sensor includes a rotation angle, a rotation angular velocity, a pitch angle, and a roll angle.

[0097] 2) Perform time compensation on the attitude data collected by the turntable attitude sensor.

[0098] For example, the timestamp of the attitude data collected by the turntable attitude sensor and received at the same time is set as the first timestamp, and the timestamp of the point cloud data collected by the lidar is set as the second timestamp. The time deviation is determined based on the difference between the second timestamp and the first timestamp, and then the turntable attitude sensor time is compensated according to the time deviation.

[0099] For example, the timestamp of the attitude data collected by the turntable attitude sensor received by the domain controller at the same time is , the timestamp of receiving the lidar point cloud data is Due to the different working processes and signal transmission methods of sensors, there is generally a time deviation between the two timestamps. , the system time compensation of the turntable attitude sensor time is performed through formula (3).

[0100] (3)

[0101] 3) Perform point cloud preprocessing on the point cloud data collected by the lidar.

[0102] For example, point cloud preprocessing may include at least one of the following.

[0103] 3.1) Perform validity filtering on the cloud data collected by the lidar points to remove NAN (non-numeric data).

[0104] 3.2) Perform range filtering based on the excavator’s operating radius and only retain point cloud data within the operating radius.

[0105] 3.3) According to the installation external parameters of the laser radar , the laser radar point cloud data in the laser radar coordinate system Convert to the excavator coordinate system to obtain the laser radar point cloud data of the excavator coordinate system .

[0106] 3.4) According to the range of the boom relative to the first direction (Y direction) of the vehicle coordinate system, extract the boom point cloud data Cloud_Oup_boom to retain the material point cloud data.

[0107] 4) Split the point cloud data into frames.

[0108] For example, according to the attitude sensor data frequency Point cloud data frequency The ratio of is used to determine the number of splits n, where n is a natural number greater than 1, and the single-frame point cloud is split into n parts.

[0109] In some embodiments, each frame of point cloud data after splitting is divided into n equal parts. After splitting, the frequency of the point cloud data is the same as the frequency of the gesture sensor data.

[0110] 5) Time synchronization of the time-compensated posture data and the point cloud data after frame point cloud splitting.

[0111] 6) Perform speed compensation on the time-synchronized cloud data and attitude data.

[0112] For example, according to the rotational angular velocity of the turntable attitude sensor , time error , to compensate for the rotation angle.

[0113] For example, according to formula (4), the rotation angle Compensation is performed to obtain a more accurate position of the current radar frame relative to the vehicle.

[0114] (4)

[0115] Through the above-mentioned time compensation and speed compensation, the accuracy of the point cloud map can be effectively improved.

[0116] 7) Spatial synchronization.

[0117] According to the rotation angle of the turntable attitude sensor after speed compensation, and the pitch angle and roll angle of the attitude sensor after time synchronization, the transformation matrix of the excavator's onboard coordinate system relative to the excavator's offboard coordinate system is constructed. According to the transformation matrix and the laser radar point cloud data in the excavator's onboard coordinate system, the laser radar point cloud data in the excavator's offboard coordinate system is determined.

[0118] For example, based on the yaw angle after speed compensation and the roll angle and pitch angle of the attitude sensor after time synchronization, the transformation matrix of the boarding car relative to the disembarking car is constructed. , as shown in formula (5).

[0119] (5)

[0120] Next, according to formula (6), the laser radar is obtained in the vehicle coordinate system: , transform to the off-vehicle coordinate system, and obtain the point cloud relative to the excavator off-vehicle coordinate system .

[0121] (6)

[0122] 8) Stitching point cloud maps.

[0123] The generated material point cloud data is automatically spliced ​​in the stationary excavator undercarriage coordinate system as the excavator moves, constructing a point cloud map of the excavator rotation operation scene.

[0124] Figure 5 FIG. 1 is a schematic diagram of the structure of a domain controller according to an embodiment of the present disclosure. Figure 5 As shown, the domain controller includes a first processing module 51 , a second processing module 52 , a third processing module 53 and a fourth processing module 54 .

[0125] The first processing module 51 is configured to construct a first point cloud map based on the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor when the excavator turntable is in a stationary state, and extract a still image of the target object from the first point cloud map.

[0126] In some embodiments, the point cloud data collected by the laser radar includes frame start time, frame length and frame point cloud data, and the attitude data collected by the turntable attitude sensor includes roll angle, pitch angle, rotation angle and rotation speed.

[0127] The second processing module 52 is configured to construct a second point cloud map based on the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor when the excavator turntable rotates in the first direction, and extract the first ghost image of the target object from the second point cloud map.

[0128] For example, the first direction is leftward or leftward.

[0129] The third processing module 53 is configured to construct a third point cloud map based on the point cloud data collected by the lidar and the posture data collected by the turntable posture sensor when the excavator turntable rotates in a second direction opposite to the first direction, and extract a second ghost image of the target object from the third point cloud map.

[0130] For example, if the first direction is to the left, the second direction is to the right. Conversely, if the first direction is to the right, the second direction is to the left.

[0131] The fourth processing module 54 is configured to eliminate ghosts in the second point cloud map by using the still image and the first ghost, and to eliminate ghosts in the third point cloud map by using the still image and the second ghost.

[0132] In some embodiments, the operation of eliminating ghosts in the second point cloud map using the still image and the first ghost includes the following.

[0133] 1) Generate the first pose transformation matrix from the first ghost image to the still image.

[0134] For example, the first ghost is registered with the still image using an ICP registration algorithm to generate a first posture transformation matrix T_L2O.

[0135] 2) Using the first posture transformation matrix, align the second point cloud map with the first point cloud map to eliminate ghosting in the second point cloud map.

[0136] For example, the second point cloud map is transformed from the vehicle coordinate system to the vehicle disembarkation coordinate system using the spatial transformation shown in formula (1) so as to align the first point cloud map.

[0137] In some embodiments, the step of eliminating ghosts in the third point cloud map using the still image and the second ghost includes the following.

[0138] 1) Generate a second pose transformation matrix from the second ghost image to the still image.

[0139] For example, the second ghost image is registered with the still image using an ICP registration algorithm to generate a second posture transformation matrix T_R2O.

[0140] 2) Using the second posture transformation matrix, align the third point cloud map with the first point cloud map to eliminate ghosting in the third point cloud map.

[0141] For example, the third point cloud map is transformed from the vehicle coordinate system to the vehicle disembarkation coordinate system using the spatial transformation shown in formula (2) so as to align the first point cloud map.

[0142] Figure 6A schematic diagram of the structure of a domain controller according to another embodiment of the present disclosure. Figure 6 and Figure 5 The difference is that in Figure 6 In the illustrated embodiment, the domain controller further includes a fifth processing module 55 .

[0143] The fifth processing module 55 is configured to determine the position of the target object so as to quickly extract the image of the target object from the point cloud data near the position.

[0144] In some embodiments, the operation of the fifth processing module 55 to determine the position of the target object includes the following contents.

[0145] 1) Controlling a designated bucket tooth in the bucket of the excavator to touch the top surface of a target object, wherein the target object is set within a construction radius of the bucket of the excavator.

[0146] For example, specify the tooth to be the center tooth of a bucket.

[0147] For example, the target object is a cylinder.

[0148] 2) Record the position of the tooth tip of the specified bucket tooth as the position of the target object.

[0149] In some embodiments, when the target object is a cylinder, the middle teeth of the bucket are controlled to touch the central area of ​​the top surface of the target object, so that the target object can be obtained more accurately.

[0150] Accordingly, the first processing module 51 is configured to extract a still image from the first point cloud map according to the position of the target object. The second processing module 52 is configured to extract a first ghost image from the second point cloud map according to the position of the target object. The third processing module 53 is configured to extract a second ghost image from the third point cloud map according to the position of the target object.

[0151] Figure 7 The diagram is a structural diagram of a domain controller according to another embodiment of the present disclosure.

[0152] like Figure 7 As shown, the domain controller 70 may be in the form of a general-purpose computing device and includes a memory 71, a processor 72, and a bus 73 for connecting different system components.

[0153] The memory 71 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application, a boot loader, and other programs. The system memory may include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium may store, for example, instructions of a corresponding embodiment of at least one point cloud map construction method being executed. The non-volatile storage medium includes, but is not limited to, a disk memory, an optical memory, a flash memory, and the like.

[0154] The processor 72 may be implemented by a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistors, and other discrete hardware components. Accordingly, each module such as an acquisition module, a calculation module, and an adjustment module may be implemented by a central processing unit (CPU) running instructions in a memory that execute corresponding steps, or may be implemented by a dedicated circuit that executes corresponding steps.

[0155] For example, the processor 72 is configured to execute instructions stored in the memory to implement the following Figure 3 The method of any one of the embodiments.

[0156] The bus 73 may use any of a variety of bus structures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0157] The interfaces 74, 75, 76 of the domain controller 70, the memory 71 and the processor 72 may be connected via a bus 73. The input / output interface 74 may provide a connection interface for input / output devices such as a display, a mouse, and a keyboard. The network interface 75 may provide a connection interface for various networked devices. The storage interface 76 may provide a connection interface for external storage devices such as a floppy disk, a USB flash drive, and an SD card.

[0158] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames can be implemented by computer-readable program instructions.

[0159] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0160] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to work in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0161] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0162] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the following is implemented: Figure 3 The method of any one of the embodiments.

[0163] The present disclosure also provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processor, the following is achieved: Figure 3 The method of any one of the embodiments.

[0164] Figure 8 FIG. 1 is a schematic diagram of the structure of a point cloud map construction system according to an embodiment of the present disclosure. Figure 8 As shown, the point cloud map construction system includes a domain controller 81, a laser radar 82, and a turntable attitude sensor 83. The domain controller 81 is Figures 5 to 7 A domain controller is involved in any one of the embodiments.

[0165] The laser radar 82 is configured to collect point cloud data and send the point cloud data to the domain controller 81 .

[0166] For example, the laser radar 82 is a petal-type scanning laser radar.

[0167] In some embodiments, the laser radar 82 is installed on the top of the cab of the excavator in a predetermined installation direction, and the installation angle of the laser radar 82 is determined by the predetermined working task.

[0168] For example, the installation direction of the laser radar 82 is ahead, that is, the installation direction of the laser radar 82 is the same as the direction of the cab of the excavator.

[0169] The turntable posture sensor 83 is configured to collect posture data of the excavator turntable relative to the undercarriage part of the excavator, and send the posture data to the domain controller 81.

[0170] In some embodiments, the turntable attitude sensor 83 is installed at the excavator turntable.

[0171] In some embodiments, the attitude data includes roll angle, pitch angle, slew angle, and slew speed.

[0172] Fig. 9 FIG. 1 is a schematic diagram of the structure of an excavator according to an embodiment of the present disclosure. Fig. 9 As shown, the excavator 90 includes a point cloud map construction system 91. The point cloud map construction system 91 is Figure 8 A point cloud map construction system is provided in any one of the embodiments.

[0173] By implementing the above-mentioned embodiments of the present disclosure, ghosting in the point cloud map can be effectively eliminated and the accuracy of the three-dimensional point cloud map can be improved.

[0174] In some embodiments, the functional unit described above may be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof, for performing the functions described in the present disclosure.

[0175] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0176] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A point cloud map construction method, executed by a domain controller, comprising: When the excavator turntable is in a stationary state, a first point cloud map is constructed according to the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor, and a static image of the target object is extracted from the first point cloud map; When the excavator turntable rotates in a first direction, constructing a second point cloud map according to the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor, and extracting a first ghost image of the target object from the second point cloud map; When the excavator turntable rotates in a second direction opposite to the first direction, a third point cloud map is constructed according to the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor, and a second ghost image of the target object is extracted from the third point cloud map; The still image and the first ghost are used to eliminate ghosts in the second point cloud map, and the still image and the second ghost are used to eliminate ghosts in the third point cloud map.

2. The point cloud map construction method according to claim 1, wherein: The using the still image and the first ghost to eliminate the ghost in the second point cloud map comprises: generating a first posture transformation matrix from the first ghost image to the still image; The second point cloud map is aligned with the first point cloud map by using the first posture transformation matrix to eliminate ghosting in the second point cloud map.

3. The point cloud map construction method according to claim 2, wherein: The generating of a first posture transformation matrix from the first ghost image to the still image comprises: The first ghost is registered with the still image using an iterative closest point ICP registration algorithm to generate the first posture transformation matrix.

4. The point cloud map construction method according to claim 1, wherein: The using the still image and the second ghost to eliminate the ghost in the third point cloud map comprises: generating a second posture transformation matrix from the second ghost image to the still image; The third point cloud map is aligned with the first point cloud map by using the second posture transformation matrix to eliminate ghosting in the third point cloud map.

5. The point cloud map construction method according to claim 4, wherein: Generating a second posture transformation matrix from the second ghost image to the still image comprises: The second ghost is registered with the still image using an ICP registration algorithm to generate the second posture transformation matrix.

6. The point cloud map construction method according to claim 1, wherein: The point cloud data collected by the laser radar includes frame start time, frame length and frame point cloud data; The attitude data collected by the turntable attitude sensor include roll angle, pitch angle, rotation angle and rotation speed.

7. The point cloud map construction method according to any one of claims 1 to 6, further comprising: determining the location of the target object; Wherein, determining the position of the target object comprises: Controlling a designated bucket tooth in the bucket of the excavator to touch the top surface of the target object, wherein the target object is set within the construction radius of the bucket of the excavator; The position of the tooth tip of the designated bucket tooth is recorded as the position of the target object.

8. The point cloud map construction method according to claim 7, wherein: The designated teeth include middle teeth of the bucket.

9. The point cloud map construction method according to claim 7, wherein: The target object comprises a cylinder; The controlling of the designated bucket teeth in the bucket of the excavator to touch the top surface of the target object comprises: The middle bucket tooth is controlled to touch the central area of ​​the top surface of the target object.

10. The point cloud map construction method according to claim 7, wherein: The step of extracting a still image of the target object from the first point cloud map comprises: The still image is extracted from the first point cloud map according to the position of the target object.

11. The point cloud map construction method according to claim 7, wherein: Extracting the first ghost of the target object from the second point cloud map comprises: The first ghost is extracted from the second point cloud map according to the position of the target object.

12. The point cloud map construction method according to claim 7, wherein: Extracting the second ghost of the target object from the third point cloud map comprises: The second ghost is extracted from the third point cloud map according to the position of the target object.

13. A domain controller, comprising: The first processing module is configured to construct a first point cloud map based on the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor when the turntable of the excavator is in a stationary state, and extract a still image of the target object from the first point cloud map; A second processing module is configured to construct a second point cloud map according to the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor when the turntable of the excavator rotates in the first direction, and extract a first ghost image of the target object from the second point cloud map; A third processing module is configured to construct a third point cloud map according to the point cloud data collected by the laser radar and the posture data collected by the turntable posture sensor when the turntable of the excavator rotates in a second direction opposite to the first direction, and extract a second ghost image of the target object from the third point cloud map; The fourth processing module is configured to use the still image and the first ghost to eliminate the ghost in the second point cloud map, and use the still image and the second ghost to eliminate the ghost in the third point cloud map.

14. A domain controller, comprising: Memory; A processor is coupled to the memory, and the processor is configured to execute the point cloud map construction method according to any one of claims 1 to 12 based on the instructions stored in the memory.

15. A point cloud map construction system, comprising: The domain controller according to claim 13 or 14; a laser radar, configured to collect point cloud data and send the point cloud data to the domain controller; The turntable posture sensor is configured to collect posture data of the excavator turntable relative to the undercarriage part of the excavator and send the posture data to the domain controller.

16. The point cloud map construction system according to claim 15, wherein: The laser radar is installed on the top of the cab of the excavator in a predetermined installation direction, and the installation angle of the laser radar is determined by the predetermined working task; The turntable attitude sensor is installed on the turntable of the excavator.

17. The point cloud map construction system according to claim 15, wherein: The posture data includes a roll angle, a pitch angle, a rotation angle and a rotation speed.

18. An excavator, comprising: A point cloud map construction system as described in any one of claims 15 to 17.

19. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method according to any one of claims 1 to 12 is implemented.

20. A computer program product, comprising computer instructions, wherein when the computer instructions are executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images