Working machine 720-degree panoramic point cloud splicing method and device and working machine

Abstract

The invention provides an operation machine 720-degree panoramic point cloud splicing method and device and an operation machine, and belongs to the technical field of detection. The method comprises the following steps: acquiring initial point cloud data in each direction; converting the initial point cloud data into an operation machine coordinate system and splicing the initial point cloud data to obtain a 360-degree look-around point cloud; according to the 360-degree look-around point cloud at the current moment and the 360-degree look-around point cloud at the previous moment, determining an upper point cloud and a lower point cloud of the operation machine; and splicing the upper point cloud, the lower point cloud and the 360-degree panoramic point cloud at the current moment to obtain a 720-degree panoramic point cloud. According to the method, the 720-degree panoramic point cloud obtained through splicing can cover the upper portion and the lower portion of the operation machine, an operator can conveniently observe obstacles above and below the operation machine and stones, pits and gullies below the operation machine, accident potential is reduced, and safety is improved. Meanwhile, according to the method, an existing radar sensing system on the operation machine can be utilized, and hardware cost does not need to be increased.

Description

Technical Field

[0001] The present invention relates to the technical field of detection, and particularly to a 720° panoramic point cloud stitching method for a working machine, a 720° panoramic point cloud stitching device for a working machine, and a working machine. Background Art

[0002] Working machines are an important part of the equipment industry. Generally speaking, mechanical equipment necessary for comprehensive mechanized construction projects such as earthwork construction projects, road surface construction and maintenance, mobile lifting and loading operations, and various building projects is called a working machine.

[0003] Due to their large volume, working machines have many visual blind spots. Especially when reversing, turning, or operating in a narrow space, it is easy to cause collision accidents. In order to minimize the accidents caused by these visual blind spots as much as possible, various detection systems have emerged. These detection systems detect data such as images and point clouds in the visual blind spots and display them so that operators can observe the images in the blind spots.

[0004] Although many working machines on the market have already been equipped with a 360° surround view system, the camera is easily affected by light and cannot judge the depth of field, making it difficult to meet the requirements for the working environment perception of working machines. Radar can well make up for the deficiencies of the camera. It is not affected by external environmental light, can identify the positions and distances of various obstacles, has a certain anti-interference ability, and can work stably under extreme weather conditions (such as night, haze, and dust).

[0005] However, existing radar perception systems generally can only detect the point clouds around the working machine, but cannot detect the conditions under the working machine. Operators cannot observe the obstacles, stones, potholes, or gullies under the chassis and the body, and there are still potential accident hazards. Summary of the Invention

[0006] The purpose of the embodiments of the present invention is to provide a 720° panoramic point cloud stitching method, device, and working machine for a working machine. This method obtains the 360° surround view point cloud by stitching the surrounding point clouds, and then determines the upper and lower point clouds of the working machine according to the 360° surround view point cloud at the current moment and the 360° surround view point cloud at the previous moment. The lower point cloud, the upper point cloud, and the 360° surround view point cloud at the current moment are stitched to obtain a 720° panoramic point cloud. This panoramic point cloud can cover the upper and lower parts of the working machine, facilitating operators to observe the obstacles above and below the working machine, as well as the stones, potholes, and gullies below, reducing potential accident hazards and improving safety. At the same time, this method can utilize the existing radar perception system on the working machine without increasing the hardware cost.

[0007] To achieve the above object, a first aspect of the present invention provides a method for 720° panoramic point cloud stitching of a construction machine, and the method for 720° panoramic point cloud stitching of the construction machine includes:

[0008] Obtain the initial point cloud data in each direction;

[0009] Convert the initial point cloud data to the coordinate system of the construction machine and perform stitching to obtain a 360° panoramic view point cloud;

[0010] Determine the upper point cloud and the lower point cloud of the construction machine according to the 360° panoramic view point cloud at the current moment and the 360° panoramic view point cloud at the previous moment;

[0011] Stitch the upper point cloud, the lower point cloud and the 360° panoramic view point cloud at the current moment to obtain a 720° panoramic point cloud.

[0012] According to the above technical means, this method obtains the point clouds around and stitches them to obtain a 360° panoramic view point cloud, and then determines the upper point cloud and the lower point cloud of the construction machine according to the 360° panoramic view point cloud at the current moment and the 360° panoramic view point cloud at the previous moment. The lower point cloud, the upper point cloud and the 360° panoramic view point cloud at the current moment are stitched to obtain a 720° panoramic point cloud. This panoramic point cloud can cover the upper and lower parts of the construction machine, facilitating the operator to observe the obstacles above and below the construction machine, as well as the stones, potholes and gullies below, reducing potential accident hazards and enhancing safety. At the same time, this method can utilize the existing radar sensing system on the construction machine without increasing the hardware cost.

[0013] In some feasible embodiments, determining the upper point cloud and the lower point cloud of the construction machine according to the 360° panoramic view point cloud at the current moment and the 360° panoramic view point cloud at the previous moment includes:

[0014] Register the 360° panoramic view point cloud at the current moment with the 360° panoramic view point cloud at the previous moment to obtain a transformation matrix of the 360° panoramic view point cloud at the previous moment relative to the 360° panoramic view point cloud at the current moment;

[0015] Convert the 360° panoramic view point cloud at the previous moment to the coordinate system of the construction machine at the current moment according to the transformation matrix to obtain a transformed point cloud;

[0016] Intercept the upper point cloud and the lower point cloud of the construction machine from the transformed point cloud according to the structural parameters of the construction machine.

[0017] According to the above technical means, by registering the 360° panoramic point clouds at two adjacent moments, the transformation matrix between the 360° panoramic point clouds at two adjacent moments can be quickly determined. Based on this transformation matrix, the 360° panoramic point cloud at the previous moment can be transformed into the working machine coordinate system at the current moment, and the transformed point cloud that can cover the upper and lower parts of the working machine can be obtained. In this way, the upper point cloud and the lower point cloud of the working machine can be intercepted from the transformed point cloud according to the structural parameters of the working machine, without adding new point cloud acquisition devices to specifically collect the upper point cloud and the lower point cloud, effectively controlling the hardware cost.

[0018] In some feasible embodiments, registering the 360° panoramic point cloud at the current moment with the 360° panoramic point cloud at the previous moment to obtain the transformation matrix of the 360° panoramic point cloud at the previous moment relative to the 360° panoramic point cloud at the current moment includes:

[0019] Projecting the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment onto the XY plane of the working machine coordinate system for image matching to obtain matching pixel points;

[0020] Mapping and restoring the matching pixel points to point clouds to obtain the point cloud set at the current moment and the point cloud set at the previous moment;

[0021] Using the normal distribution algorithm to match the point cloud set at the previous moment and the point cloud set at the current moment to obtain the initial pose transformation matrix;

[0022] Using the iterative closest point algorithm to match the point cloud set at the previous moment and the point cloud set at the current moment to optimize the initial pose transformation matrix and obtain the transformation matrix of the 360° panoramic point cloud at the previous moment relative to the 360° panoramic point cloud at the current moment.

[0023] In some feasible embodiments, intercepting the upper point cloud and the lower point cloud of the working machine from the transformed point cloud according to the structural parameters of the working machine includes:

[0024] Determining the XY coordinates of the working machine projected onto the XY plane of the working machine coordinate system according to the structural parameters of the working machine;

[0025] Determining the Z coordinates of the upper surface and the lower surface of the working machine on the Z axis according to the structural parameters of the working machine;

[0026] Intercepting the upper point cloud and the lower point cloud of the working machine from the transformed point cloud according to the XY coordinates and the Z coordinates.

[0027] According to the above technical means, the XY coordinates of the construction machine projected onto the XY plane of the construction machine coordinate system can be confirmed based on the structural parameters of the construction machine. The XY coordinates of the point clouds corresponding to the upper and lower parts of the construction machine are consistent with the XY coordinates of the projection of the construction machine in the construction machine coordinate system. The Z coordinate of the point cloud located above the construction machine is greater than the Z coordinate corresponding to the upper surface of the construction machine, and the Z coordinate of the point cloud located below the construction machine is less than the Z coordinate corresponding to the lower surface of the construction machine. Thus, the upper point cloud and the lower point cloud of the construction machine can be intercepted from the converted point cloud, and the upper point cloud and the lower point cloud can be quickly intercepted by using the spatial relationship between each point cloud and the construction machine.

[0028] In some feasible embodiments, the initial point cloud data is converted to the construction machine coordinate system and stitched to obtain a 360° panoramic point cloud, including:

[0029] Taking the projection of the rotation center of the construction machine on the ground as the origin, the forward direction as the X axis, the vertically upward direction as the Z axis, the horizontal left direction as the Y axis according to the right-hand rule, and the horizontal right direction as the Y axis according to the left-hand rule, a construction machine coordinate system is established;

[0030] According to the installation positions of the respective point cloud acquisition devices on the construction machine, the rotation angles of the point cloud acquisition devices around the construction machine coordinate system and the translation vectors of the point cloud acquisition devices relative to the construction machine coordinate system are determined, and the point cloud acquisition devices are used to acquire the initial point cloud data;

[0031] According to the rotation angles, the rotation matrix of the initial point cloud data relative to the construction machine coordinate system is determined;

[0032] According to the initial point cloud data, the rotation matrix, and the translation vector, the initial point cloud data is converted into the construction machine coordinate system;

[0033] The initial point cloud data converted into the construction machine coordinate system is stitched to obtain a 360° panoramic point cloud.

[0034] According to the above technical means, according to the installation positions of the respective point cloud acquisition devices on the construction machine, the rotation angles of the point cloud acquisition devices around the construction machine coordinate system and the translation vectors of the point cloud acquisition devices relative to the construction machine coordinate system can be determined. Then, by rotating and translating the initial point cloud data, the initial point cloud data can be all converted into the construction machine coordinate system. In this way, the initial point cloud in each direction uses the unified construction machine coordinate system, and stitching will not go wrong due to inconsistent coordinate systems, affecting the operator's observation of the surrounding environment of the construction machine.

[0035] In some feasible embodiments, the step of converting the initial point cloud data into the construction machine coordinate system according to the initial point cloud data, the rotation matrix, and the translation vector includes:

[0036] Calculate the product of the rotation matrix and the initial point cloud data to obtain the rotated initial point cloud data;

[0037] Translate the rotated initial point cloud data according to the translation vector to obtain the transformed initial point cloud data.

[0038] In some feasible embodiments, determining the upper point cloud and the lower point cloud of the working machine according to the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment includes:

[0039] Register the 360° panoramic point cloud at the current moment with the 360° panoramic point cloud at the previous moment to obtain the transformation matrix of the 360° panoramic point cloud at the current moment relative to the 360° panoramic point cloud at the previous moment;

[0040] Transform the working machine into the working machine coordinate system at the previous moment according to the transformation matrix;

[0041] Intercept the upper point cloud and the lower point cloud of the working machine from the 360° panoramic point cloud at the previous moment according to the structural parameters of the working machine.

[0042] According to the above technical means, by registering the 360° panoramic point clouds at two adjacent moments, the transformation matrix between the 360° panoramic point clouds at two adjacent moments can be quickly determined. Based on this transformation matrix, the working machine can be transformed into the working machine coordinate system at the previous moment, and then the upper point cloud and the lower point cloud of the working machine can be intercepted from the 360° panoramic point cloud at the previous moment according to the structural parameters of the working machine, without adding new point cloud acquisition equipment to specifically collect the upper point cloud and the lower point cloud, effectively controlling the hardware cost.

[0043] In some feasible embodiments, intercepting the upper point cloud and the lower point cloud of the working machine from the 360° panoramic point cloud at the previous moment according to the structural parameters of the working machine includes:

[0044] Determine the XY coordinates of the projection of the working machine onto the XY plane of the working machine coordinate system at the previous moment according to the structural parameters of the working machine;

[0045] Determine the Z coordinates of the upper surface and the lower surface of the working machine on the Z axis according to the structural parameters of the working machine;

[0046] Intercept the upper point cloud and the lower point cloud of the working machine from the 360° panoramic point cloud at the previous moment according to the XY coordinates and the Z coordinates.

[0047] According to the above technical means, the XY coordinates of the construction machine projected onto the XY plane of the construction machine coordinate system can be confirmed based on the structural parameters of the construction machine. The XY coordinates of the point clouds corresponding to the upper and lower parts of the construction machine are the same as the XY coordinates of the conversion points in the coordinate system at the previous moment. The Z coordinate of the point cloud located above the construction machine is greater than the Z coordinate corresponding to the upper surface of the construction machine, and the Z coordinate of the point cloud located below the construction machine is less than the Z coordinate corresponding to the lower surface of the construction machine. Thus, the upper point cloud and the lower point cloud of the construction machine can be intercepted from the 360° panoramic point cloud at the previous moment, and the upper point cloud and the lower point cloud can be quickly intercepted by using the spatial relationship between each point cloud and the construction machine.

[0048] The second aspect of the present application provides a 720° panoramic point cloud splicing device for a construction machine, and the 720° panoramic point cloud splicing device for the construction machine includes:

[0049] A data acquisition unit for acquiring initial point cloud data in each direction;

[0050] A panoramic view splicing unit for converting the initial point cloud data into the construction machine coordinate system and performing splicing to obtain a 360° panoramic point cloud;

[0051] A point cloud interception unit for determining the upper point cloud and the lower point cloud of the construction machine according to the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment;

[0052] A panoramic view splicing unit for splicing the upper point cloud, the lower point cloud and the 360° panoramic point cloud at the current moment to obtain a 720° panoramic point cloud.

[0053] According to the above technical means, the data acquisition unit in the device acquires the point clouds around, and the panoramic view splicing unit splices them to obtain a 360° panoramic point cloud. Then, the point cloud interception unit determines the upper point cloud and the lower point cloud of the construction machine according to the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment. The panoramic view splicing unit splices the lower point cloud, the upper point cloud and the 360° panoramic point cloud at the current moment to obtain a 720° panoramic point cloud. This panoramic point cloud can cover the upper and lower parts of the construction machine, facilitating the operator to observe the obstacles above and below the construction machine, as well as the stones, potholes and gullies below, reducing potential accident hazards and enhancing safety. At the same time, the device can utilize the existing radar sensing system on the construction machine without increasing the hardware cost.

[0054] The third aspect of the present application provides a construction machine, and the construction machine includes a control system that applies the 720° panoramic point cloud splicing method for the construction machine.

[0055] Through the above technical solution, the method obtains the point clouds around to splice into a 360° panoramic point cloud, and then determines the upper and lower point clouds of the working machine according to the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment. The lower point cloud, the upper point cloud and the 360° panoramic point cloud at the current moment are spliced into a 720° panoramic point cloud, which can cover the upper and lower parts of the working machine, facilitating the operator to observe the obstacles above and below the working machine, as well as the stones, potholes and gullies below, reducing potential accident hazards and enhancing safety. At the same time, the method can utilize the existing radar sensing system on the working machine without increasing the hardware cost.

[0056] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the specification, and are used together with the following specific embodiments to explain the embodiments of the present invention, but do not constitute a limitation to the embodiments of the present invention. In the drawings:

[0058] Figure 1 is a flowchart of a method for splicing a 720° panoramic point cloud of a working machine provided by an embodiment of the present invention;

[0059] Figure 2 is a block diagram of a device for splicing a 720° panoramic point cloud of a working machine provided by an embodiment of the present invention;

[0060] Figure 3 is a schematic diagram of the installation of a laser radar of an excavator provided by an embodiment of the present invention;

[0061] Figure 4 is a block diagram of a hardware system implemented by a method for splicing a 720° panoramic point cloud of a working machine provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] The following describes in detail the specific embodiments of the present invention with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining and understanding the present invention, and are not used to limit the present invention.

[0063] Embodiment 1

[0064] Figure 1 is a flowchart of a method for splicing a 720° panoramic point cloud of a working machine provided by an embodiment of the present invention. As Figure 1 shown, the method for splicing a 720° panoramic point cloud of the working machine includes:

[0065] S1: Obtain the initial point cloud data in each direction.

[0066] In the embodiments of the present application, point cloud data refers to a set of vectors in a three-dimensional coordinate system. Each vector in the set is recorded in the form of a point, and each point contains at least three-dimensional coordinates. The point cloud data in the present application is a set of spatial data collected by a radar system, and the initial point cloud data is the unprocessed point cloud data collected by the radar system. In order to achieve 360° panoramic point cloud stitching, the point clouds around the construction machine need to be applied. Therefore, the initial point cloud in the present application at least includes the point clouds in the front, rear, left, and right directions of the construction machine, where the forward direction refers to the forward direction of the construction machine.

[0067] The method of the present application can directly obtain the initial point cloud data in each direction from the radar system through a data interface, or obtain it from the original control system of the construction machine.

[0068] S2: Convert the initial point cloud data to the construction machine coordinate system and perform stitching to obtain a 360° panoramic point cloud.

[0069] The coordinate system of the point cloud data collected by each point cloud acquisition device in the radar system is related to the installation position of the corresponding point cloud acquisition device. Point cloud acquisition devices installed at different positions on the construction machine have different coordinate systems. Therefore, in order to facilitate stitching, the initial point cloud data needs to be converted to the construction machine coordinate system first.

[0070] In some feasible embodiments, converting the initial point cloud data to the construction machine coordinate system and performing stitching to obtain a 360° panoramic point cloud includes:

[0071] Taking the projection of the rotation center of the construction machine on the ground as the origin, the forward direction as the X-axis, vertically upward as the Z-axis, horizontally to the left according to the right-hand rule as the Y-axis, and horizontally to the right according to the left-hand rule as the Y-axis, a construction machine coordinate system is established.

[0072] According to the installation positions of the respective point cloud acquisition devices on the construction machine, determine the rotation angles of the point cloud acquisition devices around the construction machine coordinate system and the translation vectors of the point cloud acquisition devices relative to the construction machine coordinate system. The point cloud acquisition devices are used to collect the initial point cloud data. In the embodiments of the present application, the rotation angles of the point cloud acquisition devices around the construction machine coordinate system include: the angle γ of the point cloud acquisition device rotating around the X-axis of the construction machine coordinate system, the angle β of the point cloud acquisition device rotating around the Y-axis of the construction machine coordinate system, and the angle α of the point cloud acquisition device rotating around the Z-axis of the construction machine coordinate system. Each point cloud acquisition device corresponds to a set of rotation angles.

[0073] Determine the rotation matrix of the initial point cloud data relative to the construction machine coordinate system according to the rotation angles. In some feasible embodiments, determine the rotation matrix of the initial point cloud data relative to the construction machine coordinate system according to trigonometric function relationships.

[0074] Convert the initial point cloud data to the working machine coordinate system according to the initial point cloud data, the rotation matrix, and the translation vector.

[0075] Stitch the initial point cloud data that has been converted to the working machine coordinate system to obtain a 360° panoramic point cloud.

[0076] According to the above technical means, the rotation angle of the point cloud acquisition device around the working machine coordinate system and the translation vector of the point cloud acquisition device relative to the working machine coordinate system can be determined according to the installation positions of the point cloud acquisition devices on the working machine. Then, by rotating and translating the initial point cloud data, all the initial point cloud data can be converted into the working machine coordinate system. In this way, the initial point cloud in each direction uses the unified working machine coordinate system, and stitching will not go wrong due to inconsistent coordinate systems, affecting the operator's observation of the surrounding environment of the working machine.

[0077] In some feasible embodiments, the step of converting the initial point cloud data to the working machine coordinate system according to the initial point cloud data, the rotation matrix, and the translation vector includes:

[0078] Calculate the product of the initial point cloud data and the rotation matrix to obtain the rotated initial point cloud data;

[0079] Translate the rotated initial point cloud data according to the translation vector to obtain the converted initial point cloud data. The converted initial point cloud data is expressed as:

[0080] P i ' = R i P i + t i , i ∈ [1, 2, 3,..., n];

[0081] where P i represents the point cloud collected by the i-th point cloud acquisition device, P i ' represents the point cloud collected by the i-th point cloud acquisition device that has been converted to the working machine coordinate system, R i represents the rotation matrix of the i-th point cloud acquisition device relative to the working machine coordinate system, t i represents the translation vector of the i-th point cloud acquisition device relative to the working machine coordinate system, and n represents the number of point cloud acquisition devices.

[0082] The rotation matrix R i is expressed as:

[0083]

[0084] where α i represents the angle of rotation of the i-th point cloud acquisition device around the Z-axis of the working machine coordinate system; βi represents the angle of rotation of the i-th point cloud acquisition device around the Y-axis of the working machine coordinate system; γ i represents the angle of rotation of the i-th point cloud acquisition device around the X-axis of the working machine coordinate system.

[0085] The initial point cloud data after being transformed into the working machine coordinate system is stitched using existing point cloud stitching methods, which will not be elaborated in this application.

[0086] S3: Determine the upper point cloud and the lower point cloud of the working machine based on the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment.

[0087] In some feasible embodiments, determining the upper point cloud and the lower point cloud of the working machine based on the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment includes:

[0088] Register the 360° panoramic point cloud at the current moment with the 360° panoramic point cloud at the previous moment to obtain the transformation matrix of the 360° panoramic point cloud at the previous moment relative to the 360° panoramic point cloud at the current moment. In this embodiment, the transformation matrix of the 360° panoramic point cloud at the previous moment relative to the 360° panoramic point cloud at the current moment is denoted as the first transformation matrix, and the first transformation matrix is based on the 360° panoramic point cloud at the current moment.

[0089] Transform the 360° panoramic point cloud at the previous moment into the working machine coordinate system at the current moment according to the first transformation matrix to obtain the transformed point cloud;

[0090] Intercept the upper point cloud and the lower point cloud of the working machine from the transformed point cloud according to the structural parameters of the working machine.

[0091] According to the above technical means, by registering the 360° panoramic point clouds at two adjacent moments, the first transformation matrix between the 360° panoramic point clouds at two adjacent moments can be quickly determined. Based on the first transformation matrix, the 360° panoramic point cloud at the previous moment can be transformed into the working machine coordinate system at the current moment to obtain the transformed point cloud that can cover the upper and lower parts of the working machine. In this way, the upper point cloud and the lower point cloud of the working machine can be intercepted from the transformed point cloud without adding new point cloud acquisition devices to specifically collect the upper point cloud and the lower point cloud, effectively controlling the hardware cost.

[0092] In some feasible embodiments, registering the 360° panoramic point cloud at the current moment with the 360° panoramic point cloud at the previous moment to obtain the transformation matrix of the 360° panoramic point cloud at the previous moment relative to the 360° panoramic point cloud at the current moment includes:

[0093] Project the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment onto the XY plane of the working machine coordinate system for image matching to obtain matching pixel points; each pair of matching pixel points includes a pixel point projected from the 360° panoramic point cloud at the current moment and a pixel point projected from the 360° panoramic point cloud at the previous moment. The image matching uses existing image matching technologies, which are not limited in this application. In this way, the same partial pixel points can be determined from the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment, reducing the computational complexity of subsequent point cloud matching and accelerating point cloud matching.

[0094] Map and restore the matching pixel points to point clouds to obtain the point cloud set at the current moment and the point cloud set at the previous moment; after mapping and restoring each pair of matching pixel points, a point cloud data from the 360° panoramic point cloud at the current moment and a point cloud data from the 360° panoramic point cloud at the previous moment will be obtained. The set composed of the point cloud data from the 360° panoramic point cloud at the current moment is the point cloud set at the current moment. Similarly, the set composed of the point cloud data from the 360° panoramic point cloud at the previous moment is the point cloud set at the previous moment.

[0095] Use the normal distribution algorithm to match the point cloud set at the previous moment and the point cloud set at the current moment to obtain an initial pose transformation matrix. The normal distribution algorithm divides the point cloud into multiple network units and calculates the statistical characteristics (such as mean and covariance matrix) within each unit, thereby quickly estimating the initial pose transformation matrix between the two point clouds and filtering out the point pairs that cannot be matched.

[0096] Use the iterative closest point algorithm to match the point cloud set at the previous moment and the point cloud set at the current moment to optimize the initial pose transformation matrix and obtain the transformation matrix of the 360° panoramic point cloud at the previous moment relative to the 360° panoramic point cloud at the current moment. The iterative closest point algorithm uses the initial pose transformation matrix as the initial value, iteratively searches for the correspondence between each point in the remaining point cloud at the previous moment and the nearest neighbor point in the remaining point cloud at the current moment, and minimizes the distance between these corresponding points to optimize the pose transformation matrix.

[0097] In some feasible embodiments, intercept the upper point cloud and the lower point cloud of the working machine from the transformed point cloud according to the structural parameters of the working machine, including:

[0098] Determine the XY coordinates of the working machine projected onto the XY plane of the working machine coordinate system at the current moment according to the structural parameters of the working machine;

[0099] Determine the Z coordinates of the upper surface and the lower surface of the working machine on the Z axis according to the structural parameters of the working machine;

[0100] Intercept the upper point cloud and lower point cloud of the working machine from the transformed point cloud according to the XY coordinates and Z coordinates.

[0101] According to the above technical means, the XY coordinates of the working machine projected onto the XY plane of the working machine coordinate system can be confirmed based on the structural parameters of the working machine. The XY coordinates of the point clouds corresponding to the upper and lower parts of the working machine are consistent with the XY coordinates of the projection of the working machine in the working machine coordinate system. The Z coordinate of the point cloud located above the working machine is greater than the Z coordinate corresponding to the upper surface of the working machine, and the Z coordinate of the point cloud located below the working machine is less than the Z coordinate corresponding to the lower surface of the working machine. Thus, the upper point cloud and lower point cloud of the working machine can be intercepted from the transformed point cloud, and the upper point cloud and lower point cloud can be quickly intercepted by using the spatial relationship between each point cloud and the working machine.

[0102] In practical applications, since it is difficult to correspond the point density on the working machine one by one with the coordinates of the point cloud, therefore, the XY coordinates of each point on the contour of the working machine can be determined according to the projection of the contour of the working machine on the XY plane of the working machine coordinate system. The X coordinate range and Y coordinate range are determined according to the XY coordinates of each point on the contour, and then the point cloud data in the transformed point cloud whose X coordinate belongs to the X coordinate range and Y coordinate belongs to the Y coordinate range is intercepted. Then, according to the Z coordinates of the upper surface and lower surface of the working machine on the Z axis, the upper point cloud and lower point cloud of the working machine are intercepted respectively. In this way, not too much calculation is required, and there is no need to unify the point density on the working machine.

[0103] S4: Stitch the upper point cloud, lower point cloud and the 360° panoramic point cloud at the current moment to obtain a 720° panoramic point cloud.

[0104] According to the above technical means, this method obtains the surrounding point clouds and stitches them to obtain a 360° panoramic point cloud. Then, the upper point cloud and lower point cloud of the working machine are determined according to the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment. The lower point cloud, upper point cloud and the 360° panoramic point cloud at the current moment are stitched to obtain a 720° panoramic point cloud. This panoramic point cloud can cover the upper and lower parts of the working machine, facilitating the operator to observe the obstacles above and below the working machine, as well as the stones, potholes and gullies below, reducing potential accident hazards and improving safety. At the same time, this method can utilize the existing radar sensing system on the working machine without increasing the hardware cost.

[0105] In the embodiment of the present application, the stitched 720° panoramic point cloud needs to be displayed in the working machine. For the convenience of the operator to view, the 720° panoramic point cloud is combined with the dynamic model of the working machine for display.

[0106] The second aspect of the present application provides a 720° panoramic point cloud stitching device for a working machine, as Figure 2As shown, the 720° panoramic point cloud stitching device for the working machine includes:

[0107] A data acquisition unit for acquiring initial point cloud data in each direction;

[0108] A panoramic stitching unit for converting the initial point cloud data into the coordinate system of the working machine and performing stitching to obtain a 360° panoramic point cloud;

[0109] A point cloud intercepting unit for determining the upper point cloud and the lower point cloud of the working machine based on the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment;

[0110] A full - view stitching unit for stitching the upper point cloud, the lower point cloud and the 360° panoramic point cloud at the current moment to obtain a 720° panoramic point cloud.

[0111] According to the above technical means, the data acquisition unit in the device acquires the point clouds around, and the panoramic stitching unit stitches them to obtain a 360° panoramic point cloud. Then, the point cloud intercepting unit determines the upper point cloud and the lower point cloud of the working machine based on the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment. The full - view stitching unit stitches the lower point cloud, the upper point cloud and the 360° panoramic point cloud at the current moment to obtain a 720° panoramic point cloud. This panoramic point cloud can cover the upper and lower parts of the working machine, facilitating the operator to observe the obstacles above and below the working machine, as well as the stones, potholes and gullies below, reducing potential accident hazards and enhancing safety. At the same time, the device can utilize the existing radar sensing system on the working machine without increasing the hardware cost.

[0112] In some feasible embodiments, the point cloud intercepting unit includes:

[0113] A first registration module for registering the 360° panoramic point cloud at the current moment with the 360° panoramic point cloud at the previous moment to obtain a transformation matrix of the 360° panoramic point cloud at the previous moment relative to the 360° panoramic point cloud at the current moment. In this embodiment, the transformation matrix of the 360° panoramic point cloud at the previous moment relative to the 360° panoramic point cloud at the current moment is denoted as the first transformation matrix, and the first transformation matrix is based on the 360° panoramic point cloud at the current moment.

[0114] A first transformation module for transforming the 360° panoramic point cloud at the previous moment into the coordinate system of the working machine at the current moment according to the first transformation matrix to obtain a transformed point cloud;

[0115] A first interception module for intercepting the upper point cloud and the lower point cloud of the working machine from the transformed point cloud according to the structural parameters of the working machine.

[0116] According to the above technical means, the first registration module can quickly determine the transformation matrix between the 360° panoramic point clouds at two adjacent moments by registering the 360° panoramic point clouds at two adjacent moments. Based on this transformation matrix, the first transformation module can transform the 360° panoramic point cloud at the previous moment into the operation machinery coordinate system at the current moment to obtain the transformed point cloud that can cover the upper and lower parts of the operation machinery. In this way, the first interception module can intercept the upper point cloud and the lower point cloud of the operation machinery from the transformed point cloud according to the structural parameters of the operation machinery, without adding new point cloud acquisition equipment to specifically collect the upper point cloud and the lower point cloud, effectively controlling the hardware cost.

[0117] In some feasible embodiments, the first registration module is specifically configured to:

[0118] Project the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment onto the XY plane of the operation machinery coordinate system for image matching to obtain matching pixel points; each pair of matching pixel points includes a pixel point projected from the 360° panoramic point cloud at the current moment and a pixel point projected from the 360° panoramic point cloud at the previous moment. Image matching uses existing image matching technologies, which are not limited in this application. In this way, the same partial pixel points can be determined from the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment, reducing the computational amount of subsequent point cloud matching and accelerating point cloud matching.

[0119] Map and restore the matching pixel points to point clouds to obtain the point cloud set at the current moment and the point cloud set at the previous moment; after mapping and restoring each pair of matching pixel points, a point cloud data from the 360° panoramic point cloud at the current moment and a point cloud data from the 360° panoramic point cloud at the previous moment will be obtained. The set composed of the point cloud data from the 360° panoramic point cloud at the current moment is the point cloud set at the current moment, and similarly, the set composed of the point cloud data from the 360° panoramic point cloud at the previous moment is the point cloud set at the previous moment.

[0120] Use the normal distribution algorithm to match the point cloud set at the previous moment and the point cloud set at the current moment to obtain the initial pose transformation matrix;

[0121] Use the iterative closest point algorithm to match the point cloud set at the previous moment and the point cloud set at the current moment to optimize the initial pose transformation matrix to obtain the transformation matrix of the 360° panoramic point cloud at the current moment relative to the 360° panoramic point cloud at the previous moment.

[0122] Embodiment 2

[0123] This embodiment provides a method for stitching 720° panoramic point clouds of a working machine, and the method for stitching 720° panoramic point clouds of the working machine includes:

[0124] S1: Obtain the initial point cloud data in each direction;

[0125] S2: Convert the initial point cloud data to the working machine coordinate system and splice it to obtain a 360° panoramic view point cloud.

[0126] S3: Determine the upper point cloud and the lower point cloud of the working machine based on the 360° panoramic view point cloud at the current moment and the 360° panoramic view point cloud at the previous moment.

[0127] In some feasible embodiments, determining the upper point cloud and the lower point cloud of the working machine based on the 360° panoramic view point cloud at the current moment and the 360° panoramic view point cloud at the previous moment includes:

[0128] Register the 360° panoramic view point cloud at the current moment with the 360° panoramic view point cloud at the previous moment to obtain the transformation matrix of the 360° panoramic view point cloud at the current moment relative to the 360° panoramic view point cloud at the previous moment. In this embodiment, the transformation matrix of the 360° panoramic view point cloud at the current moment relative to the 360° panoramic view point cloud at the previous moment is denoted as the second transformation matrix, and the second transformation matrix is based on the 360° panoramic view point cloud at the previous moment. The acquisition method of the second transformation matrix is the same as that of the first transformation matrix.

[0129] Convert the working machine to the working machine coordinate system at the previous moment according to the second transformation matrix;

[0130] Intercept the upper point cloud and the lower point cloud of the working machine from the 360° panoramic view point cloud at the previous moment according to the structural parameters of the working machine.

[0131] According to the above technical means, by registering the 360° panoramic view point clouds at two adjacent moments, the second transformation matrix between the 360° panoramic view point clouds at two adjacent moments can be quickly determined. Based on the second transformation matrix, the working machine can be converted to the working machine coordinate system at the previous moment, and then the upper point cloud and the lower point cloud of the working machine can be intercepted from the 360° panoramic view point cloud at the previous moment according to the structural parameters of the working machine, without adding new point cloud acquisition equipment to specifically collect the upper point cloud and the lower point cloud, effectively controlling the hardware cost.

[0132] In some feasible embodiments, intercepting the upper point cloud and the lower point cloud of the working machine from the 360° panoramic view point cloud at the previous moment according to the structural parameters of the working machine includes:

[0133] Determine the XY coordinates of the working machine projected onto the XY plane of the working machine coordinate system at the previous moment according to the structural parameters of the working machine;

[0134] Determine the Z coordinates of the upper surface and the lower surface of the working machine on the Z axis according to the structural parameters of the working machine;

[0135] Intercept the upper point cloud and lower point cloud of the working machine from the 360° panoramic point cloud at the previous moment according to the XY coordinates and Z coordinates.

[0136] According to the above technical means, the XY coordinates of the working machine projected onto the XY plane of the working machine coordinate system can be confirmed according to the structural parameters of the working machine. The XY coordinates of the corresponding point clouds above and below the working machine are consistent with the XY coordinates of the conversion points in the coordinate system at the previous moment. The Z coordinate of the point cloud located above the working machine is greater than the Z coordinate corresponding to the upper surface of the working machine, and the Z coordinate of the point cloud located below the working machine is less than the Z coordinate corresponding to the lower surface of the working machine. Thus, the upper point cloud and lower point cloud of the working machine can be intercepted from the 360° panoramic point cloud at the previous moment, and the interception of the upper point cloud and lower point cloud can be quickly realized by using the spatial relationship between each point cloud and the working machine.

[0137] In practical applications, since it is difficult to make the point-taking density on the working machine correspond one by one with the coordinates of the point cloud, therefore, the XY coordinates of each point on the contour of the working machine in the working machine coordinate system at the previous moment can be determined, and then the X coordinate range and Y coordinate range can be determined according to the XY coordinates of each point. Then, the point cloud data in the 360° panoramic point cloud at the previous moment whose X coordinate belongs to the X coordinate range and Y coordinate belongs to the Y coordinate range is intercepted. Then, according to the Z coordinates of the upper surface and lower surface of the working machine on the Z axis, the upper point cloud of the working machine and the lower point cloud of the working machine are intercepted respectively. In this way, too much calculation is not required, and the point-taking density on the working machine does not need to be unified.

[0138] S4: Stitch the upper point cloud, lower point cloud and the 360° panoramic point cloud at the current moment to obtain a 720° panoramic point cloud.

[0139] According to the above technical means, this method obtains the surrounding point clouds and stitches them to obtain a 360° panoramic point cloud. Then, according to the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment, the upper point cloud and lower point cloud of the working machine are determined. The lower point cloud, upper point cloud and the 360° panoramic point cloud at the current moment are stitched to obtain a 720° panoramic point cloud. This panoramic point cloud can cover the upper and lower parts of the working machine, facilitating the operator to observe the obstacles above and below the working machine, as well as the stones, potholes and gullies below, reducing potential accident hazards and improving safety. At the same time, this method can utilize the existing radar sensing system on the working machine without increasing the hardware cost.

[0140] This application also provides a 720° panoramic point cloud stitching device for a working machine, and the 720° panoramic point cloud stitching device for the working machine includes:

[0141] A data acquisition unit for acquiring initial point cloud data in each direction;

[0142] The panoramic stitching unit is used to convert the initial point cloud data into the working machine coordinate system and perform stitching to obtain a 360° panoramic point cloud;

[0143] The point cloud intercepting unit is used to determine the upper point cloud and the lower point cloud of the working machine according to the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment;

[0144] The panoramic stitching unit is used to stitch the upper point cloud, the lower point cloud and the 360° panoramic point cloud at the current moment to obtain a 720° panoramic point cloud.

[0145] According to the above technical means, the data acquisition unit in the device acquires the surrounding point clouds, and the panoramic stitching unit stitches them to obtain a 360° panoramic point cloud. Then, the point cloud intercepting unit determines the upper point cloud and the lower point cloud of the working machine according to the 360° panoramic point cloud at the current moment and the 360° panoramic point cloud at the previous moment. The panoramic stitching unit stitches the lower point cloud, the upper point cloud and the 360° panoramic point cloud at the current moment to obtain a 720° panoramic point cloud. This panoramic point cloud can cover the upper and lower parts of the working machine, facilitating the operator to observe the obstacles above and below the working machine, as well as the stones, potholes and gullies below, reducing potential accident hazards and improving safety. At the same time, the device can utilize the existing radar sensing system on the working machine without increasing the hardware cost.

[0146] In some feasible embodiments, the point cloud intercepting unit includes:

[0147] The second registration module is used to register the 360° panoramic point cloud at the current moment with the 360° panoramic point cloud at the previous moment to obtain the transformation matrix of the 360° panoramic point cloud at the current moment relative to the 360° panoramic point cloud at the previous moment;

[0148] The second transformation module is used to transform the working machine into the working machine coordinate system at the previous moment according to the transformation matrix;

[0149] The second interception module is used to intercept the upper point cloud and the lower point cloud of the working machine from the 360° panoramic point cloud at the previous moment according to the transformed point.

[0150] According to the above technical means, the second registration module can quickly determine the transformation matrix between the 360° panoramic point clouds at two adjacent moments by registering the 360° panoramic point clouds at two adjacent moments. Based on this transformation matrix, the projection points corresponding to the working machine can be transformed into the working machine coordinate system at the previous moment. Thus, the second interception module intercepts the upper point cloud and the lower point cloud of the working machine from the 360° panoramic point cloud at the previous moment without adding new point cloud acquisition equipment to specifically collect the upper point cloud and the lower point cloud, effectively controlling the hardware cost.

[0151] In some feasible embodiments, the second intercepting module is specifically configured to:

[0152] Determine the XY coordinates of the construction machine projected onto the XY plane of the construction machine coordinate system according to the structural parameters of the construction machine;

[0153] Determine the Z coordinates of the upper surface and the lower surface of the construction machine on the Z axis according to the structural parameters of the construction machine;

[0154] Intercept the upper point cloud and the lower point cloud of the construction machine from the 360° panoramic point cloud at the previous moment according to the XY coordinates and the Z coordinates.

[0155] According to the above technical means, the XY coordinates of the construction machine projected onto the XY plane of the construction machine coordinate system can be confirmed according to the structural parameters of the construction machine. The XY coordinates of the point clouds corresponding to the upper and lower parts of the construction machine are the same as the XY coordinates of the conversion points in the coordinate system at the previous moment. The Z coordinate of the point cloud located above the construction machine is greater than the Z coordinate corresponding to the upper surface of the construction machine, and the Z coordinate of the point cloud located below the construction machine is less than the Z coordinate corresponding to the lower surface of the construction machine. Therefore, the upper point cloud and the lower point cloud of the construction machine can be intercepted from the 360° panoramic point cloud at the previous moment, and the upper point cloud and the lower point cloud can be quickly intercepted by using the spatial relationship between each point cloud and the construction machine.

[0156] The third aspect of the present application provides a construction machine, and the construction machine includes a control system that applies the 720° panoramic point cloud stitching method for the construction machine described above.

[0157] Taking the construction machine as an excavator as an example, the 720° panoramic point cloud stitching method for the construction machine of the present application will be further described below.

[0158] In this embodiment, the radar system of the excavator uses lidar to collect point cloud data. The selected lidar has a horizontal field of view angle range of 0 to 360° and a vertical field of view angle range of 0 to 90°. As Figure 3 shown, a lidar is installed at the left front corner and the right front corner of the excavator body, as well as on the left side, the right side, and the rear. The lidar is vertically outward. Installing two lidars in the front can avoid blind spots caused by the shovel arm blocking the lidar field of view during panoramic stitching. A total of five lidars collect point clouds for stitching to obtain a 360° panoramic point cloud.

[0159] When performing 720° point cloud stitching, first obtain the point clouds collected by each lidar for 360° panoramic point cloud stitching. Register the 360° panoramic point cloud at the previous moment with the 360° panoramic point cloud at the current moment to obtain the point clouds directly above and below the excavator body. Stitch the point clouds directly above and below the excavator body with the 360° panoramic point cloud to obtain the 720° panoramic point cloud. The 720° panoramic point cloud is combined with the dynamic model for display, and the attitude of the dynamic model is updated synchronously according to the body rotation angle and the rotation angles of the boom, arm, and bucket. As Figure 4 shown, the point cloud processing host in the excavator control system obtains the initial point cloud data from five lidars at the front left, front right, left, right, and rear, and processes it. The excavator controller obtains the body rotation angle from the body rotation center encoder, and obtains the inclination angles of each part from the inclination sensors of the body, boom, arm, and bucket, and updates the attitude of the dynamic model according to the body rotation angle and the inclination angles of each part. The display screen shows the stitched 720° point cloud and the dynamic model. The excavator dynamic model is in the middle of the panoramic point cloud. If the model can only be in a static state, then when the surrounding point cloud rotates, it is impossible to tell whether it is the body rotating or the chassis rotating just by looking at the point cloud. Therefore, when the excavator body rotates, it is necessary to rotate the body of the model together and keep the body facing directly forward. The boom joint, arm joint, and bucket joint can also move synchronously in the model. The 720° panoramic view and the excavator dynamic model support viewing from different perspectives, can be dragged up, down, left, right, forward, and backward, and support zooming in and out.

[0160] The embodiment of the present invention also provides a machine-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above-mentioned 720° panoramic point cloud stitching method for construction machinery is implemented.

[0161] Those skilled in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program is stored in a storage medium, including several instructions to enable a single-chip microcomputer, chip, or processor to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical disks, etc., which can store program codes.

[0162] The optional embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. In addition, it should be noted that, among the various specific technical features described in the above specific embodiments, they can be combined in any appropriate manner without conflict. To avoid unnecessary repetition, the embodiments of the present invention will not separately describe various possible combination manners.

[0163] In addition, any combination can be made among various different embodiments of the present invention as long as it does not violate the idea of the embodiments of the present invention, and it should equally be regarded as the content disclosed by the embodiments of the present invention.

Claims

1. A method for 720° panoramic point cloud stitching of a working machine, characterized in that, The method for 720° panoramic point cloud stitching of the working machine includes: Obtain the initial point cloud data in each direction; Convert the initial point cloud data to the working machine coordinate system and perform stitching to obtain a 360° panoramic view point cloud; Determine the upper point cloud and the lower point cloud of the working machine based on the 360° panoramic view point cloud at the current moment and the 360° panoramic view point cloud at the previous moment; Stitch the upper point cloud, the lower point cloud and the 360° panoramic view point cloud at the current moment to obtain a 720° panoramic point cloud.

2. The 720° panoramic point cloud stitching method for construction machinery according to claim 1, characterized in that, Determine the upper point cloud and the lower point cloud of the working machine based on the 360° panoramic view point cloud at the current moment and the 360° panoramic view point cloud at the previous moment, including: Register the 360° panoramic view point cloud at the current moment with the 360° panoramic view point cloud at the previous moment to obtain the transformation matrix of the 360° panoramic view point cloud at the previous moment relative to the 360° panoramic view point cloud at the current moment; Convert the 360° panoramic view point cloud at the previous moment to the working machine coordinate system at the current moment according to the transformation matrix to obtain the transformed point cloud; Intercept the upper point cloud and the lower point cloud of the working machine from the transformed point cloud according to the structural parameters of the working machine.

3. The method for 720° panoramic point cloud stitching of the construction machine according to claim 2, wherein, Register the 360° panoramic view point cloud at the current moment with the 360° panoramic view point cloud at the previous moment to obtain the transformation matrix of the 360° panoramic view point cloud at the previous moment relative to the 360° panoramic view point cloud at the current moment, including: Project the 360° panoramic view point cloud at the current moment and the 360° panoramic view point cloud at the previous moment onto the XY plane of the working machine coordinate system for image matching to obtain the matching pixel points; Map and restore the matching pixel points to point clouds to obtain the point cloud set at the current moment and the point cloud set at the previous moment; Use the normal distribution algorithm to match the point cloud set at the previous moment and the point cloud set at the current moment to obtain the initial pose transformation matrix; Use the iterative closest point algorithm to match the point cloud set at the previous moment and the point cloud set at the current moment to optimize the initial pose transformation matrix to obtain the transformation matrix of the 360° panoramic view point cloud at the previous moment relative to the 360° panoramic view point cloud at the current moment.

4. The method for 720° panoramic point cloud stitching of the construction machine according to claim 2, characterized in that, Intercept the upper point cloud and the lower point cloud of the working machine from the transformed point cloud according to the structural parameters of the working machine, including: Determine the XY coordinates of the working machine projected onto the XY plane of the working machine coordinate system according to the structural parameters of the working machine; Determine the Z coordinates of the upper surface and the lower surface of the working machine on the Z axis according to the structural parameters of the working machine; Intercept the upper point cloud and the lower point cloud of the working machine from the transformed point cloud according to the XY coordinates and the Z coordinates.

5. The method for 720° panoramic point cloud stitching of a construction machine according to claim 1, wherein Convert the initial point cloud data to the working machine coordinate system and perform stitching to obtain a 360° panoramic view point cloud, including: Establish a working machine coordinate system with the projection of the rotation center of the working machine on the ground as the origin, the forward direction as the X axis, vertically upward as the Z axis, horizontally to the left according to the right-hand rule as the Y axis, and horizontally to the right according to the left-hand rule as the Y axis; Determine the rotation angle of the point cloud acquisition device around the working machine coordinate system and the translation vector of the point cloud acquisition device relative to the working machine coordinate system according to the installation position of each point cloud acquisition device on the working machine, and the point cloud acquisition device is used to acquire the initial point cloud data; Determine the rotation matrix of the initial point cloud data relative to the working machine coordinate system according to the rotation angle; Convert the initial point cloud data to the working machine coordinate system according to the initial point cloud data, the rotation matrix, and the translation vector; Stitch the initial point cloud data converted to the working machine coordinate system to obtain a 360° panoramic view point cloud.

6. The method for 720° panoramic point cloud stitching of the construction machine according to claim 5, wherein, The step of converting the initial point cloud data to the working machine coordinate system according to the initial point cloud data, the rotation matrix, and the translation vector includes: Calculate the product of the rotation matrix and the initial point cloud data to obtain the rotated initial point cloud data; Translate the rotated initial point cloud data according to the translation vector to obtain the converted initial point cloud data.

7. The method for 720° panoramic point cloud stitching of a construction machine according to claim 1, characterized in that, Determine the upper point cloud and the lower point cloud of the working machine according to the 360° panoramic view point cloud at the current moment and the 360° panoramic view point cloud at the previous moment, including: Register the 360° panoramic view point cloud at the current moment with the 360° panoramic view point cloud at the previous moment to obtain the transformation matrix of the 360° panoramic view point cloud at the current moment relative to the 360° panoramic view point cloud at the previous moment; Convert the working machine to the working machine coordinate system at the previous moment according to the transformation matrix; Intercept the upper point cloud and the lower point cloud of the working machine from the 360° panoramic view point cloud at the previous moment according to the structural parameters of the working machine.

8. The method for 720° panoramic point cloud stitching of the construction machine according to claim 7, characterized in that, The step of intercepting the upper point cloud and the lower point cloud of the working machine from the 360° panoramic view point cloud at the previous moment according to the structural parameters of the working machine includes: Determine the XY coordinates of the working machine projected onto the XY plane of the working machine coordinate system according to the structural parameters of the working machine; Determine the Z coordinates of the upper surface and the lower surface of the working machine on the Z axis according to the structural parameters of the working machine; Intercept the upper point cloud and the lower point cloud of the working machine from the 360° panoramic view point cloud at the previous moment according to the XY coordinates and the Z coordinates.

9. A 720° panoramic point cloud stitching device for a working machine, characterized in that, The 720° panoramic view point cloud stitching device for the working machine includes: A data acquisition unit for acquiring initial point cloud data in each direction; A panoramic view stitching unit for converting the initial point cloud data to the working machine coordinate system and stitching it to obtain a 360° panoramic view point cloud; A point cloud intercepting unit for determining the upper point cloud and the lower point cloud of the working machine according to the 360° panoramic view point cloud at the current moment and the 360° panoramic view point cloud at the previous moment; A panoramic stitching unit for stitching the upper point cloud, the lower point cloud, and the 360° panoramic view point cloud at the current moment to obtain a 720° panoramic view point cloud.

10. An earthmoving machine, characterized in that, The working machine includes a control system, and the control system applies the 720° panoramic view point cloud stitching method for the working machine according to any one of claims 1-8.

Images