An arm modeling method, electronic device and working machine of a working machine

Abstract

The application relates to a method for modeling a boom of a working machine, an electronic device and the working machine. The method comprises: determining a first edge point set of a combined boom based on point cloud data of the combined boom; the first edge point set is used to represent a set of projected points of first boundary points of the combined boom in a first projection plane; determining a hinged position of a first boom and a second boom based on a multi-straight line fitting result of the first edge point set; dividing the point cloud data of the combined boom based on the hinged position to obtain a target edge point set of a target boom; performing three-dimensional curve fitting based on the target edge point set to obtain a contour curve of the target boom; and constructing a structure model of the target boom based on the contour curve of the target boom and size information. The point cloud data of the combined boom is divided by the hinged position of the first boom and the second boom, the edge point sets of the booms can be accurately distinguished, the structure model of the target boom is established based on this, and the modeling accuracy of the combined boom can be improved.

Description

Technical Field

[0001] This application relates to the field of construction machinery technology, specifically to a boom modeling method for construction machinery, electronic equipment, and construction machinery. Background Technology

[0002] Combined booms have a wide operating range and complex structure, creating multiple blind spots for operators and greatly increasing the risk of collisions with surrounding equipment, buildings, and other spatial obstacles. The accuracy of combined boom modeling directly impacts the reliability of collision risk prediction.

[0003] Currently, most structural models of combined booms are created manually using finite element software. However, combined booms have various combination forms, and manual modeling relies entirely on the operator's subjective experience, making it difficult to reproduce the true spatial structure of the boom. In addition, when changing the boom combination method or adapting to new working conditions, the model needs to be rebuilt, and repeated adjustments to the model will further amplify the modeling error, resulting in poor modeling accuracy of combined booms. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a boom modeling method, electronic device, and operating machinery. The method utilizes the hinge position to divide the point cloud data of the combined boom, accurately distinguishing the point cloud data of each boom in the combined boom, and establishing a structural model of the target boom based on this, thereby improving the modeling accuracy of the combined boom.

[0005] To address the aforementioned technical problems, this application provides a method for modeling the boom of a work machinery, wherein the work machinery includes a combined boom, the combined boom comprising a first boom and a second boom hinged to the first boom; the method includes: Based on the point cloud data of the combined boom, a first edge point set of the combined boom is determined; wherein, the first edge point set is used to characterize the set of projection points of the first boundary points of the combined boom on the first projection plane; the projection of the combined boom on the first projection plane has an angle. Based on the multi-line fitting results of the first edge point set, the hinge position between the first boom and the second boom is determined; The point cloud data of the combined boom is divided based on the hinge position to obtain the target edge point set of the target boom; wherein, the target boom is used to represent at least one of the first boom and the second boom; The contour curve of the target boom is obtained by performing three-dimensional curve fitting based on the target edge point set; Based on the contour curve and dimensional information of the target boom, a structural model of the target boom is constructed.

[0006] In some embodiments, determining the hinge position between the first boom and the second boom based on the multi-line fitting results of the first edge point set includes: Based on the first set of edge points, multiple straight line fitting is performed to obtain the first straight line of the first boom on the first projection plane and the second straight line of the second boom on the first projection plane; The hinge position is determined based on the intersection of the first straight line and the second straight line.

[0007] In some embodiments, the point cloud data of the combined boom includes multiple discrete points of the combined boom; determining the hinge position based on the intersection of the first straight line and the second straight line includes: A second set of target points is obtained by filtering target projection points located in a first region from the first set of target points; wherein, the first set of target points is used to characterize the set of projection points of multiple discrete points of the combined boom on the first projection plane; the first region is the region centered on the intersection point; For each target projection point in the second target point set, a second region corresponding to the target projection point is determined, and the number of projection points in the second target point set located in the second region is counted to obtain the number of points corresponding to the target projection point; the second region is the region centered on the target projection point; The position of the target projection point corresponding to the maximum number of points is determined as the hinge position.

[0008] In some embodiments, the point cloud data of the combined boom includes a plurality of discrete points of the combined boom; determining a first set of edge points of the combined boom based on the point cloud data of the combined boom includes: Discrete sampling is performed on the target line segment to obtain multiple sampling points of the target line segment; the target line segment is the line connecting the root of the first boom and the end of the second boom; For each sampling point of the target line segment, the discrete point where the distance between the combined boom and the sampling point in the first direction satisfies the first condition is determined as the first boundary point; wherein, the first direction is the direction perpendicular to the target line segment in the first projection plane; the first condition is used to indicate the closest or farthest distance.

[0009] In some embodiments, before determining the first set of edge points of the combined boom based on the point cloud data of the combined boom, the method further includes: Obtain the first point cloud data of the environment in which the combined boom is located; Point cloud data matching the third region is filtered from the first point cloud data to obtain the second point cloud data; wherein, the third region is the area away from the ground in the environmental region divided based on the target line segment; the target line segment is the line connecting the root of the first boom and the end of the second boom; Based on the second point cloud data, the point cloud data of the combined boom is determined.

[0010] In some embodiments, the step of dividing the point cloud data of the combined boom based on the hinge position to obtain the target edge point set of the target boom includes: Based on the hinge position, the first edge point set of the combined boom is divided to obtain the first edge point set of the target boom; Based on the point cloud data of the combined boom, a second edge point set of the combined boom is determined; the second edge point set is used to characterize the set of projection points of the second boundary points of the combined boom on the second projection plane; the second projection plane intersects with the first projection plane; The target edge point set is determined based on the first edge point set of the target boom and the second edge point set of the combined boom.

[0011] In some embodiments, dividing the first edge point set of the combined boom based on the hinge position to obtain the first edge point set of the target boom includes: Based on the hinge position, projection points located in the fourth region are selected from the first edge points of the combined boom; the fourth region is used to characterize the region where the distance from the hinge position in the direction of the target common edge is less than a first distance; the target common edge is the common edge of the first projection plane and the second projection plane; For each projection point in the fourth region, in response to the distance between the projection point and the hinge position in a third direction being greater than a second distance, the projection point is removed from the first edge point of the combined boom; the third direction is the direction perpendicular to the target common edge in the first projection plane; The third edge point set of the combined boom is divided based on the hinge position to obtain the first edge point set of the target boom; the third edge point set includes the remaining projection points in the first edge point set of the combined boom.

[0012] In some embodiments, the point cloud data of the combined boom includes a plurality of discrete points of the combined boom; determining the second edge point set of the combined boom based on the point cloud data of the combined boom includes: For each sampling point in the target line segment, the discrete point where the positional relationship between the combined boom and the sampling point in the second direction satisfies the second condition is determined as the second boundary point; the target line segment is the line connecting the root of the first boom and the end of the second boom. The second condition is that it is located on the target side of the target line segment and is the farthest from the sampling point; the second direction is the direction perpendicular to the target line segment in the second projection plane.

[0013] In some embodiments, the step of performing three-dimensional curve fitting based on the target edge point set to obtain the contour curve of the target boom includes: Based on the target edge point set and target constraints, a three-dimensional curve fitting is performed to obtain the contour curve of the target boom; wherein, the target constraints include at least one of the following: The contour curve passes through the hinge position; The difference between the fitted value and the measured value of the first boundary point of the target boom at the third-party upward position is within the target difference range; the target edge point set includes the measured value of the first boundary point of the target boom at the third-party upward position.

[0014] In some embodiments, constructing a structural model of the target boom based on its contour curve and dimensional information includes: For each curve segment of the contour curve, a target sampling density corresponding to the target curvature is determined based on the target curvature of the curve segment; in the correspondence, the sampling density is positively correlated with the curvature; the curve segment is sampled based on the target sampling density to obtain the sampling points of the curve segment; For each sampling point of the contour curve, based on the size information of the target boom, the sampling point is offset along a third direction and a fourth direction to obtain the envelope of the target boom and the sampling point; the fourth direction is the direction perpendicular to the target common edge in the second projection plane; the target common edge is the common edge of the first projection plane and the second projection plane; Based on the envelope corresponding to each sampling point, a structural model of the target boom is generated.

[0015] In some embodiments, after constructing the structural model of the target boom, the method further includes: Based on the sampling points of the contour curve, the structural model is divided into voxel elements to obtain multiple voxel elements of the target boom; For each of the aforementioned volume elements, a collision analysis is performed on the volume element to obtain the collision analysis result of the volume element; Based on the collision analysis results of each volume element, the collision analysis results of the target boom are determined.

[0016] This application also provides an electronic device, including a storage medium and a controller, wherein a computer program is stored on the storage medium, and the computer program, when executed by the controller, implements the steps of the method described above.

[0017] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0018] This application also provides a working machine, including a combined boom, a point cloud acquisition device, and the aforementioned electronic equipment.

[0019] This application discloses a boom modeling method, electronic device, and operating machinery. The method includes: determining a first edge point set of the combined boom based on point cloud data; wherein the first edge point set represents the set of projection points of the first boundary points of the combined boom on a first projection plane; the projection of the combined boom on the first projection plane has an angle; determining the hinge position of the first boom and the second boom based on the multi-line fitting results of the first edge point set; dividing the point cloud data of the combined boom based on the hinge position to obtain a target edge point set of the target boom; wherein the target boom represents at least one boom among the first boom and the second boom; performing three-dimensional curve fitting based on the target edge point set of the target boom to obtain the contour curve of the target boom; and constructing a structural model of the target boom based on the contour curve and dimensional information of the target boom. The technical solution of this application utilizes the hinge position of the first boom and the second boom to divide the point cloud data of the combined boom, which can accurately distinguish the edge point sets of each boom, and establish a structural model of the target boom based on this, thereby improving the modeling accuracy of the combined boom. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a method for modeling the boom of a work machine according to an embodiment.

[0021] Figure 2 This is a schematic diagram showing the installation location of a point cloud acquisition device according to one embodiment.

[0022] Figure 3 This is a schematic diagram of the coordinate system of a point cloud acquisition device according to one embodiment.

[0023] Figure 4 This is a schematic diagram of area division according to one embodiment.

[0024] Figure 5 This is a schematic diagram of the structure of an electronic device according to one embodiment. Detailed Implementation

[0025] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings. Preferred embodiments of this application are shown in the drawings. However, this application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. In this application, "each" includes one or more items.

[0027] Figure 1 This is a flowchart illustrating a method for modeling the boom of a work machinery according to one embodiment. The work machinery includes a combined boom, which comprises a first boom and a second boom hinged to the first boom. Figure 1 As shown, the boom modeling method of this application for operating machinery, applied to electronic devices, may include the following steps: S1, Based on the point cloud data of the combined boom, determine the first edge point set of the combined boom; wherein, the first edge point set is used to characterize the set of projection points of the first boundary points of the combined boom on the first projection plane; the projection of the combined boom on the first projection plane has an angle; S2, Based on the multi-line fitting results of the first edge point set, determine the hinge position between the first boom and the second boom; S3, the point cloud data of the combined boom is divided based on the articulation position to obtain the target edge point set of the target boom; wherein, the target boom is used to represent at least one of the first boom and the second boom; S4. Perform three-dimensional curve fitting based on the target edge point set to obtain the contour curve of the target boom; S5. Based on the contour curve and dimensional information of the target boom, construct the structural model of the target boom.

[0028] The point cloud data of the combined boom can include multiple discrete points of the combined boom. The first boundary point of the combined boom can be a discrete point located on the first boundary of the combined boom; the first boundary can be the outer surface of the combined boom perpendicular to the first projection plane. In actual implementation, the discrete point of the combined boom located on the first boundary can be extracted from the multiple discrete points of the combined boom based on the boundary features of the combined boom, thus obtaining the first boundary point; alternatively, the first boundary point of the boom can be determined based on the distance between the discrete point of the combined boom and the target line segment in the first projection plane. For specific implementation details, please refer to the relevant description below.

[0029] Optionally, the angled projection of the combined boom onto the first projection plane indicates that the angle between the projections of the first boom and the second boom on the first projection plane is not zero. Considering that the first boom and the second boom are hinged, when the projection of the combined boom onto the first projection plane has an angle, a first straight line of the first boom and a second straight line of the second boom can be fitted based on the first set of edge points. In actual implementation, the hinge position of the first boom and the second boom can be determined based on the intersection of the first straight line and the second straight line; see the relevant description below for details.

[0030] After determining the hinge position of the first boom and the second boom, the first edge point set of the combined boom can be divided based on the hinge position to obtain the first edge point set of the target boom. Then, the target edge point set can be determined based on the first edge point set of the target boom. For specific implementation, please refer to the relevant description below.

[0031] Next, parametric 3D curve fitting methods such as B-spline and non-uniform rational B-spline (NURBS) can be used to fit the boundary points of the target edge point set to obtain the contour curve of the target boom. Then, based on the size and contour information of the target boom, the envelope of the target boom can be generated to obtain the structural model of the target boom.

[0032] Thus, considering the hinge connection between the first and second booms, and given the angled projection of the combined boom onto the first projection plane, the hinge positions of the first and second booms can be accurately located based on the multi-line fitting results of the first edge point set of the combined boom. Then, using the hinge position as the segmentation datum, the point cloud data of the combined boom is divided, achieving accurate segmentation of the edge point sets of the first and second booms. This effectively avoids mutual interference between boundary points of different booms during the modeling process. Finally, relying on the segmented, unmixed target edge point set, contour curve fitting and model construction are performed, restoring the true structure of the target boom and thus improving the modeling accuracy of the combined boom.

[0033] Optionally, the combined boom can be mounted on the main body of the work machinery, with the base of the first boom as the mounting point. A point cloud acquisition device is used to collect point cloud data of the combined boom. This device can be mounted on the main body of the work machinery, or at the base of the first boom, or at the end of the second boom. There can be one or more point cloud acquisition devices. If there are multiple devices, the point cloud data collected by each device can be fused first, and then the point cloud data of the combined boom can be extracted from the collected point cloud data.

[0034] Optionally, the first projection plane can be a coordinate plane in the target coordinate system, which can be the coordinate system of the point cloud acquisition device, the coordinate system of the operating machinery, or the world coordinate system. In one embodiment, the X-axis direction in the target coordinate system can be parallel to the length direction of the combined boom, the Y-axis direction can be parallel to the thickness direction of the combined boom, and the Z-axis direction can be parallel to the width direction of the combined boom. In actual implementation, the target coordinate system can also be translated based on the above coordinate axis directions, or rotated based on one of the coordinate axis directions, as long as the requirements of projection and fitting calculation are met.

[0035] Please combine Figure 2 and Figure 3 The point cloud acquisition device can be installed at the root A of the first boom, oriented parallel to the length of the combined boom, to acquire point cloud data towards the end B of the second boom. Its point cloud acquisition range can be distributed in a three-dimensional cone shape, completely covering the combined boom. Taking the target coordinate system as the point cloud acquisition device coordinate system as an example, in the point cloud acquisition device coordinate system, its origin O can be located at the root A of the first boom, the X-axis can be parallel to the orientation of the point cloud acquisition device, the Z-axis can be parallel to the mast C (the width direction of the combined boom) installed at the root A of the first boom, and the Y-axis can be parallel to the thickness direction of the combined boom. The aforementioned first projection plane can be the XOZ coordinate plane in the point cloud acquisition device coordinate system, so that the projection of the combined boom onto the first projection plane has an angle.

[0036] In some embodiments, step S2, determining the hinge position between the first boom and the second boom based on the multi-line fitting results of the first edge point set, includes: Based on the first set of edge points, multiple straight lines are fitted to obtain the first straight line of the first boom on the first projection plane and the second straight line of the second boom on the first projection plane. The hinge position is determined based on the intersection of the first and second straight lines.

[0037] Please combine Figure 2 and Figure 3The aforementioned first boundary can be the outer surface of the combined boom perpendicular to the XOZ plane; the aforementioned first boundary points can be discrete points located on this outer surface; the aforementioned first edge point set can include the projection points of the discrete points on the outer surface onto the XOZ coordinate plane. In actual implementation, algorithms such as Random Sample Consensus (RANSAC) and Hough transform can be used to perform multi-line fitting on the first edge point set to obtain the first straight line L1 of the first boom in the XOZ coordinate plane and the second straight line L2 of the second boom in the XOZ plane.

[0038] Next, the intersection of the first straight line L1 and the second straight line L1 can be determined as the hinge position between the first boom and the second boom. Considering that the edges of the first boom and the second boom are not straight, there may be a deviation between the above intersection point and the actual hinge position. To eliminate this deviation, the position of the hinge component used to connect the first boom and the second boom can be further located based on the position of the intersection point, and the position of the hinge component can be determined as the hinge position between the first boom and the second boom.

[0039] Thus, considering the hinged characteristics of the first boom and the second boom, and given that the projection of the combined boom on the first projection plane has an angle, the hinged positions of the first boom and the second boom can be accurately located based on the multi-line fitting results of the first edge point set of the combined boom.

[0040] Considering the point cloud acquisition scenario of the combined boom, the point cloud of the articulated components exhibits a localized high-density distribution. Although the intersection points of the lines obtained through multi-line fitting may deviate from the actual articulation positions, they are located in the same local region, and the position of the articulated component largely coincides with the actual articulation position. Based on this, the region where the articulated component is located can be first locked based on the line intersection points. Then, using the density center positioning principle, the position of the articulated component can be accurately located within this region, and the position of the articulated component can be determined as the articulation position between the first boom and the second boom, thus completing the articulation position optimization.

[0041] In some embodiments, determining the hinge position based on the intersection of the first and second straight lines includes: The target projection points located in the first region are selected from the first target point set to obtain the second target point set; wherein, the first target point set is used to represent the set of projection points of multiple discrete points of the combined boom on the first projection plane; the first region is the region centered on the intersection point; For each target projection point in the second target point set, determine the second region corresponding to the target projection point, and count the number of projection points in the second target point set located in the second region to obtain the number of points corresponding to the target projection point; the second region is the region centered on the target projection point. The position of the target projection point corresponding to the maximum number of points is determined as the hinge position.

[0042] The extent of the first region can be determined based on the dimensions of the combined boom, and the extent of the second region can be determined based on the dimensions of the hinge. The shapes of the first and second regions can be circular, rectangular, etc. Optionally, the dimensions can include at least one type of dimension parameter selected from width, thickness, length, and radius. In different embodiments of this application, the dimension information mentioned can be the same type of dimension parameter or different types of dimension parameters.

[0043] Please combine Figure 2 and Figure 3 First, a circular region with the intersection point p and a search radius r1 can be constructed in the XOZ coordinate plane as the first region. Then, the positions of the projection points of multiple discrete points of the combined boom in the XOZ coordinate plane can be compared with the first region, and the target projection points located in the first region can be selected from the first target point set to obtain a local point cloud set Q, which is the aforementioned second target point set. Optionally, the radius r1 can be set with reference to the width information of the combined boom.

[0044] Next, for each target projection point in the local point cloud set Q, a circular region with a search radius of r2 centered on the target projection point can be constructed in the XOZ coordinate plane as a second region. Then, the number of projection points in the local point cloud set Q located in the second region is counted to obtain the density feature value of the target projection point, i.e., the number of points corresponding to the target projection point. Optionally, the radius r2 can be set with reference to the width information of the hinge.

[0045] Finally, the position of the target projection point with the largest density feature value in the local point cloud set Q can be determined as the position of the hinge; and the position of the hinge can be determined as the hinge position between the first boom and the second boom.

[0046] Thus, by first utilizing the characteristic that the intersection point of the lines obtained by multi-line fitting is in the same local area as the position of the hinge, the range for determining the position of the hinge can be narrowed down to the first area; then, by utilizing the characteristic that the point cloud of the hinge presents a local high-density distribution, the position of the hinge can be accurately determined based on the density feature value of each projection point in the first area, thereby improving the accuracy of determining the hinge position of the first boom and the second boom.

[0047] In some embodiments, step S1, determining a first set of edge points of the combined boom based on the point cloud data of the combined boom, includes: Discrete sampling is performed on the target line segment to obtain multiple sampling points of the target line segment; the target line segment is the line connecting the root of the first boom and the end of the second boom. For each sampling point of the target line segment, the discrete point where the distance between the combined boom and the sampling point in the first direction satisfies the first condition is determined as the first boundary point; wherein, the first direction is the direction perpendicular to the target line segment in the first projection plane; the first condition is used to indicate the closest or farthest distance.

[0048] Please combine Figure 2 and Figure 3 The first direction can be the Z-axis direction. In actual implementation, the target line segment AB can be discretized first to obtain multiple sampling points; then, for each sampling point on the target line segment AB, the Z-axis component of the three-dimensional spatial distance between each discrete point of the combined boom and the sampling point can be calculated to obtain the distance between the sampling point and each discrete point in the Z-axis direction; then, the distance between the sampling point and each discrete point in the Z-axis direction can be compared, and the closest or farthest discrete point can be determined as the first boundary point; finally, the projection point of the first boundary point in the XOZ coordinate plane can be determined as the projection point of the first edge point set.

[0049] Thus, by using the distance in the direction perpendicular to the target line segment within the first projection plane as the selection criterion for the first boundary point, and determining the corresponding first boundary point by sampling point along the target line segment, the interference of other irrelevant discrete points on the first boundary point can be effectively eliminated, thereby improving the accuracy of the first edge point set.

[0050] Considering that the end of the second boom typically suspends slings, hooks, and other lifting components, the point cloud acquisition equipment within its acquisition range may collect point cloud data not only of the combined boom but also of the lifting components. To ensure the accuracy of the combined boom's point cloud data, the acquired point cloud data needs to be filtered to eliminate interference from the lifting component's point cloud data.

[0051] In some embodiments, before step S1, determining the first edge point set of the combined boom based on the point cloud data of the combined boom, the method further includes: Acquire the first point cloud data of the environment in which the combined boom is located; The second point cloud data is obtained by filtering out the point cloud data that matches the third region from the first point cloud data; the third region is the area on the side of the environment that is far from the ground, which is divided based on the target line segment; the target line segment is the line connecting the root of the first boom and the end of the second boom. Based on the second point cloud data, the point cloud data of the combined boom is determined.

[0052] Considering that the end of a combined boom typically bends downwards due to factors such as gravity or load, point cloud data of the combined boom can be filtered out through region division. As shown in Figure 4, a target line segment can be constructed using the root A of the first boom and the end B of the second boom. Then, the environmental region D above the target line segment can be defined as the third region, and discrete points located in environmental region D can be filtered from the first point cloud data to obtain the second point cloud data. The second point cloud data can then be directly determined as the point cloud data of the combined boom. Alternatively, the interference of obstacle point clouds can be considered, and the third point cloud data can be further filtered to remove obstacle point clouds, thus obtaining the point cloud data of the combined boom.

[0053] In this way, the interference of the point cloud data of the sling components on the point cloud data of the combined boom can be effectively eliminated, ensuring the accuracy of the final point cloud data of the combined boom.

[0054] Considering that the point cloud acquisition equipment, within its acquisition range, may collect point cloud data not only of the combined boom but also of obstacles such as other machinery or buildings, it is necessary to filter the point cloud data acquired by the acquisition equipment to eliminate interference from obstacle point cloud data in order to ensure the accuracy of the combined boom's point cloud data.

[0055] In some embodiments, before step S1, determining the first edge point set of the combined boom based on the point cloud data of the combined boom, the method further includes: Acquire the first point cloud data of the environment in which the combined boom is located; Point cloud data matching the target area is selected from the first point cloud data to obtain the third point cloud data; the target area is the area in the second projection plane formed by offsetting to at least one side of the target line segment with the target line segment as the baseline; the range of the target area is determined based on the size information of the combined boom; the second projection plane intersects with the first projection plane; Based on the third point cloud data, the point cloud data of the combined boom is determined.

[0056] Optionally, the intersection of the second projection plane and the first projection plane means that the angle between the second projection plane and the first projection plane is not zero. To simplify the calculation process, the second projection plane is perpendicular to the first projection plane. In the above target coordinate system, the second projection plane can be a coordinate plane perpendicular to the first projection plane. Please refer to... Figure 2 and Figure 3The second projection plane mentioned above can be the XOY coordinate plane in the coordinate system of the point cloud acquisition device. Considering that the projection of the point cloud of the combined boom in the XOY coordinate plane is approximately linearly distributed, in actual implementation, the target line segment can be constructed using the root A of the first boom and the end B of the second boom. Then, a target area, also called a buffer area, can be constructed in the XOY coordinate plane with the target line segment as the baseline and offset to at least one side of the target line segment. Then, the point cloud data located outside the buffer area can be removed from the first point cloud data, and the remaining first point cloud data can be determined as the third point cloud data. Then, the third point cloud data can be directly determined as the point cloud data of the combined boom. When a sling is connected to the end of the second boom, the third point cloud data can be further filtered to remove the point cloud of the sling to obtain the point cloud data of the combined boom. For specific implementation, please refer to the description in the above embodiment, which will not be repeated here.

[0057] In this way, the interference of obstacle point cloud data on the point cloud data of the combined boom can be effectively eliminated, ensuring the accuracy of the final point cloud data of the combined boom.

[0058] In some embodiments, step S3, dividing the point cloud data of the combined boom based on the hinge position to obtain the target edge point set of the target boom, includes: The first edge point set of the target boom is obtained by dividing the first edge point set of the combined boom based on the hinge position. Based on the point cloud data of the combined boom, a second edge point set of the combined boom is determined; the second edge point set is used to characterize the set of projection points of the second boundary points of the combined boom on the second projection plane; the second projection plane intersects with the first projection plane; The target edge point set is determined based on the first edge point set of the target boom and the second edge point set of the combined boom.

[0059] Please combine Figure 2 and Figure 3 The hinge position can be used to characterize the position of the hinge point between the first boom and the second boom in the XOZ coordinate plane, which can be represented by ( This can be represented by the hinge position ( ). The first edge point set of the combined boom is divided into: points whose X-axis coordinate values ​​are greater than a certain threshold. The projection points are determined as the projection points of the second boom, thus obtaining the first edge point set of the second boom; the X-axis coordinate values ​​in the first edge point set are less than... The projection point is determined as the projection point of the first boom, thus obtaining the first edge point set of the first boom. In actual implementation, for ease of division, the hinge position can also be determined first in the first edge point set of the combined boom. The nearest projection point (which can be called the nearest neighbor) is used as the dividing point. The projection points in the positive X-axis direction that are before the nearest neighbor are determined as the projection points of the second boom, and the projection points that are after the nearest neighbor are determined as the projection points of the first boom.

[0060] Optionally, the second boundary point of the combined boom can be a discrete point located on the second boundary of the combined boom; the second boundary can be the outer surface of the combined boom perpendicular to the second projection plane. Then, based on the boundary features of the combined boom, the discrete point of the combined boom located on the second boundary can be extracted from multiple discrete points of the combined boom to obtain the second boundary point; alternatively, the second boundary point of the boom can be determined based on the distance between the discrete point of the combined boom and the target line segment in the second projection plane. For specific implementation details, please refer to the relevant description below.

[0061] Optionally, the aforementioned target edge point set can be a projection point set, which may include a first edge point set of the target boom and a second edge point set of the combined boom; the target edge point set can also be a boundary point set, which may include discrete points located on the common edge of the first boundary and the second boundary of the target boom. The positions of the discrete points on the common edge of the first boundary and the second boundary can be determined based on the positions of the projection points of the first edge point set of the target boom and the positions of the projection points of the second edge point set of the combined boom.

[0062] In practical implementation, when the target edge point set is a set of boundary points, parametric 3D curve fitting methods such as B-Spline and NURBS can be used to fit the boundary points in the target edge point set to a 3D curve, obtaining the contour curve of the target boom. When the target edge point set is a set of projection points, a 2D curve fitting to the first projection plane can be performed based on the first edge point set, and a straight line fitting to the second projection plane can be performed based on the second edge point set. Then, the contour curve of the target boom can be determined based on the 2D curve fitting results and the straight line fitting results.

[0063] Thus, by segmenting the first edge point set of the target boom from the first edge point set of the combined boom based on the articulation position, the mixed interference of the point clouds of the first boom and the second boom can be eliminated; and by combining the second edge point set of the combined boom to determine the target edge point set of the target boom, the single-point extraction deviation can be effectively avoided, thereby improving the accuracy of the target edge point set.

[0064] Given the relatively long boom of the combined boom, point cloud gaps may occur in localized areas, both at the end of the second boom and at the articulated joint. While point cloud gaps exist at the end of the second boom, they do not introduce additional noise. However, the articulated joint typically has a mast; if point cloud gaps occur there, the point cloud near the articulated joint will be interfered with by the mast's point cloud, resulting in abnormally discrete points. To improve the accuracy of the first edge point set of the target boom, a pass-through filter can be applied to the first edge point set of either the combined boom or the target boom to eliminate interference from the mast's point cloud.

[0065] In some embodiments, the above-described division of the first edge point set based on the hinge position to obtain the first edge point set of the target boom includes: Based on the hinge position, projection points located in the fourth region are selected from the first edge points of the combined boom; the fourth region is used to characterize the region where the distance from the hinge position in the direction of the target common edge is less than the first distance; the target common edge is the common edge of the first projection plane and the second projection plane; For each projection point in the fourth region, in response to the distance between the projection point and the hinge position in the third direction being greater than the second distance, the projection point is removed from the first edge point of the combined boom; the third direction is the direction perpendicular to the target common edge in the first projection plane; The third edge point set of the combined boom is divided based on the hinge position to obtain the first edge point set of the target boom; the third edge point set includes the remaining projection points in the first edge point set of the combined boom.

[0066] Please combine Figure 2 and Figure 3 Using the hinge position as a reference, the first edge points of the combined boom are searched for projection points whose distance from the hinge position is less than a first distance in the X-axis (target common edge) direction. Then, for each projection point found, its distance from the hinge position in the Z-axis direction is calculated. Then, projection points whose distance in the Z-axis direction is greater than a second distance are removed from the first edge point set of the combined boom, resulting in a third edge point set including the remaining projection points in the first edge point set. Finally, the third edge point set of the combined boom can be divided based on the hinge position to obtain the first edge point set of the target boom. For specific implementation, please refer to the relevant description in the above embodiments, which will not be repeated here.

[0067] This effectively eliminates interference from the mast point cloud at the articulated position, ensuring the accuracy of the first edge point set of the target boom.

[0068] In some embodiments, determining the second edge point set of the combined boom based on the point cloud data of the combined boom includes: For each sampling point in the target line segment, the discrete point where the positional relationship between the combined boom and the sampling point in the second direction satisfies the second condition is determined as the second boundary point; the target line segment is the line connecting the root of the first boom and the end of the second boom. The second condition is that it is located on the target side of the target line segment and is the farthest from the sampling point; the second direction is the direction perpendicular to the target line segment in the second projection plane.

[0069] Please combine Figure 2 and Figure 3 The second direction can be the Y-axis. In actual implementation, the target line segment AB can be discretized first to obtain multiple sampling points; then, discrete points whose projection positions in the XOY plane are located on the same side of the target line segment AB can be selected from the multiple discrete points of the combined boom; then, for each sampling point on the target line segment AB, the component of the three-dimensional spatial distance between it and the discrete point on the target side in the Y-axis direction can be calculated to obtain the distance between the sampling point and the discrete point on the target side in the Y-axis direction; then, the distance between the sampling point and the discrete point on the target side in the Y-axis direction can be compared, and the discrete point with the farthest distance can be determined as the second boundary point; finally, the projection point of the second boundary point in the XOY plane can be determined as the projection point of the second edge point set.

[0070] Thus, by using the distance between the target side of the target line segment and the direction perpendicular to the target line segment in the second projection plane as the selection criterion for the second boundary points, and determining the corresponding second boundary points by sampling points along the target line segment, the interference of other irrelevant discrete points on the second boundary points can be effectively eliminated, thereby improving the accuracy of the second edge point set.

[0071] Considering the long boom length of the combined boom, the point cloud data of the combined boom may have some missing points, resulting in some outliers in the target edge point set of the target boom. Constraints can be added during the process of determining the contour curve using the target point set to eliminate the influence of outliers and improve the accuracy of the target boom contour curve.

[0072] In some embodiments, step S4, which involves performing three-dimensional curve fitting based on the target edge point set to obtain the contour curve of the target boom, includes: The contour curve of the target boom is obtained by 3D curve fitting based on the target edge point set and target constraints; wherein the target constraints include at least one of the following: The contour curve passes through the hinge point; The difference between the fitted value and the measured value of the first boundary point of the target boom at the third-party upward position is within the target difference range; the target edge point set includes the measured value of the first boundary point of the target boom at the third-party upward position.

[0073] In practical implementation, during the process of determining the contour curve by fitting a 3D curve based on the target edge point set, the fitted contour curve can be restricted from passing through the hinge position. In addition, the fitted contour curve can be verified by using the difference between the fitted value and the measured value within a target difference range: for each first boundary point of the target boom, the fitted value at its third-direction upward position can be determined using the fitted contour curve, and the difference between the fitted value and the measured value can be determined; if any difference for each first boundary point exceeds the target difference range, the first boundary points exceeding the target difference range can be removed, and the above verification process can be repeated until the difference for each first boundary point of the target boom is within the target difference range, and the fitted contour curve is then used as the contour curve of the target boom.

[0074] Please combine Figure 2 and Figure 3 When the target edge point set includes the first edge point set of the target boom and the second edge point set of the combined boom, the X-axis and Z-axis coordinate values ​​can be obtained by first performing quadratic curve fitting in the XOZ coordinate plane based on the first edge point set using algorithms such as least squares method and M estimation. Then, the X-axis and Y-axis coordinate values ​​obtained by performing linear fitting in the XOY coordinate plane based on the second edge point set can be combined to achieve three-dimensional curve fitting and obtain the contour curve of the target boom.

[0075] In the process of fitting a quadratic curve in the XOZ coordinate plane based on the first set of edge points, the fitting process can be constrained by formulas (1) and (2): (1) (2) in, This represents the two-dimensional curve obtained from the fitting, ( () indicates the hinge position. The fitted value representing the position of the first boundary point of the target boom in the Z-axis direction. Compared with measured values The maximum deviation between them.

[0076] In actual implementation, if the maximum difference If the difference exceeds the target range, the first boundary points where the difference exceeds the target range can be removed, and the two-dimensional curve fitting can be performed again based on the remaining first boundary points. This process continues until the maximum difference is reached. Stop fitting within the target difference range, and output the currently fitted contour curve as the contour curve of the combined boom.

[0077] In this way, even when the point cloud data of the combined boom has some missing points, the influence of outliers can be eliminated, thus improving the accuracy of the target boom profile curve.

[0078] In some embodiments, step S5, based on the contour curve and dimensional information of the target boom, constructs a structural model of the target boom, including: For each curve segment of the contour curve, a target sampling density corresponding to the target curvature is determined based on the target curvature of the curve segment; in the correspondence, the sampling density is positively correlated with the curvature; the curve segment is sampled based on the target sampling density to obtain the sampling points of the curve segment; For each sampling point of the contour curve, based on the size information of the target boom, the sampling point is offset along the third and fourth directions to obtain the envelope of the target boom and the sampling point; the fourth direction is the direction perpendicular to the common edge of the target in the second projection plane; the common edge of the target is the common edge of the first projection plane and the second projection plane. Based on the envelope corresponding to each sampling point, a structural model of the target boom is generated.

[0079] In practice, the contour curve can be divided into multiple curve segments by using a target interval; alternatively, the contour curve can be sampled at small intervals with high density, and multiple curve segments can be determined based on the high density sampling results. Then, for each curve segment, it can be sampled based on the target sampling density corresponding to the target curvature of that curve segment to obtain the sampling points of that curve segment.

[0080] Please combine Figure 2 and Figure 3 After determining the sampling points for each curve segment, for each sampling point, based on the size information of the target boom, the sampling point can be offset simultaneously along the Y-axis and Z-axis to obtain the envelope of the target boom and the sampling point. Then, the envelopes corresponding to each sampling point can be connected into a whole envelope to obtain the structural model of the target boom.

[0081] Thus, based on the correspondence between sampling density and curvature, the sampling results of the contour curve can be simplified, achieving the effect of generating low-density sampling results in small curvature areas and high-density sampling results in large curvature areas. This allows the number of sampling points of the contour curve to be controlled without affecting the accuracy of the contour curve curvature description, thereby improving the modeling efficiency of the combined boom while taking into account the modeling accuracy of the combined boom.

[0082] In some embodiments, after constructing the structural model of the target boom in step S5, the method further includes: Based on the sampling points of the contour curve, the structural model is divided into voxel elements to obtain multiple voxel elements of the target boom. For each voxel, a collision analysis is performed on the voxel to obtain the collision analysis results. Based on the collision analysis results of each volume element, the collision analysis results of the target boom are determined.

[0083] Optionally, the sampling points of the contour curve can be sampling points obtained by sampling the contour curve using the target interval; or they can be sampling points of the aforementioned curve segments. In actual implementation, if the curvature of the aforementioned curve segments is taken into account during the acquisition of sampling points, the final contour curve sampling points can be non-uniformly distributed. This allows the voxel subdivision of the structural model based on the contour curve sampling points to be non-uniform voxel subdivision. In this way, the number of voxels generated for the target boom can be controlled without affecting the accuracy of the contour curve curvature description, thereby improving the collision analysis efficiency of the target boom while maintaining the modeling accuracy of the combined boom.

[0084] The technical solution of this application, based on point cloud data collected by a point cloud acquisition device, firstly uses a region of interest segmentation algorithm to segment the point cloud data collected by the acquisition device to remove point cloud data of obstacles; then, the remaining point cloud data is segmented to remove point cloud data of sling components, obtaining the point cloud of the combined boom; next, using the density center positioning principle, the hinge position is accurately determined based on the intersection points of lines obtained by multi-line fitting, and the point cloud data of the combined boom is segmented using the hinge position to obtain the target edge point set of the target boom; then, three-dimensional curve fitting is performed based on the target edge point set of the target boom to obtain the contour curve of the target boom; finally, the structural model of the target boom is constructed based on the contour curve of the target boom. This improves the modeling accuracy of the combined boom.

[0085] Based on the same inventive concept as the foregoing embodiments, this invention provides an electronic device, such as... Figure 5 As shown, the electronic device includes: a processor 510 and a memory 511 storing a computer program; wherein, Figure 5 The processor 510 shown in the diagram does not indicate that there is only one processor 510, but only indicates the positional relationship of the processor 510 relative to other devices. In practical applications, there can be one or more processors 510; similarly, Figure 5 The memory 511 shown in the diagram has the same meaning, that is, it is only used to indicate the positional relationship of memory 511 relative to other devices. In practical applications, there can be one or more memories 511. When the processor 510 runs the computer program, it implements the boom modeling method of the operating machinery described above.

[0086] The electronic device may also include at least one network interface 512. The various components of the electronic device are coupled together via a bus system 513. It is understood that the bus system 513 is used to implement communication between these components. In addition to a data bus, the bus system 513 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 The general designated all buses as Bus System 513.

[0087] The memory 511 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 511 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0088] The memory 511 in this embodiment of the invention is used to store various types of data to support the operation of the electronic device. Examples of this data include: any computer programs used to operate on the electronic device, such as operating systems and applications; contact data; phonebook data; messages; pictures; videos, etc. The operating system includes various system programs, such as a framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications, such as media players, browsers, etc., used to implement various application services. Here, the program implementing the method of this embodiment of the invention can be included in the application.

[0089] Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a computer-readable storage medium storing a computer program. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. When the computer program stored in the computer-readable storage medium is executed by a processor, it implements the above-described boom modeling method for operating machinery. For the specific steps implemented when the computer program is executed by the processor, please refer to [link to relevant documentation]. Figure 1 The description of the illustrated embodiments will not be repeated here.

[0090] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for modeling the boom of a work machinery, the work machinery including a combined boom, the combined boom including a first boom and a second boom hinged to the first boom; the method comprising: Based on the point cloud data of the combined boom, a first edge point set of the combined boom is determined; wherein, the first edge point set is used to characterize the set of projection points of the first boundary points of the combined boom on the first projection plane; the projection of the combined boom on the first projection plane has an angle. Based on the multi-line fitting results of the first edge point set, the hinge position between the first boom and the second boom is determined; The point cloud data of the combined boom is divided based on the hinge position to obtain the target edge point set of the target boom; wherein, the target boom is used to represent at least one of the first boom and the second boom; The contour curve of the target boom is obtained by performing three-dimensional curve fitting based on the target edge point set; Based on the contour curve and dimensional information of the target boom, a structural model of the target boom is constructed.

2. The method as described in claim 1, characterized in that, The determination of the hinge position between the first boom and the second boom based on the multi-line fitting results of the first edge point set includes: Based on the first set of edge points, multiple straight line fitting is performed to obtain the first straight line of the first boom on the first projection plane and the second straight line of the second boom on the first projection plane; The hinge position is determined based on the intersection of the first straight line and the second straight line.

3. The method as described in claim 2, characterized in that, The point cloud data of the combined boom includes multiple discrete points of the combined boom; determining the hinge position based on the intersection of the first straight line and the second straight line includes: A second set of target points is obtained by filtering target projection points located in a first region from the first set of target points; wherein, the first set of target points is used to characterize the set of projection points of multiple discrete points of the combined boom on the first projection plane; the first region is the region centered on the intersection point; For each target projection point in the second target point set, a second region corresponding to the target projection point is determined, and the number of projection points in the second target point set located in the second region is counted to obtain the number of points corresponding to the target projection point; the second region is the region centered on the target projection point; The position of the target projection point corresponding to the maximum number of points is determined as the hinge position.

4. The method as described in claim 1, characterized in that, The point cloud data of the combined boom includes multiple discrete points of the combined boom; determining the first edge point set of the combined boom based on the point cloud data of the combined boom includes: Discrete sampling is performed on the target line segment to obtain multiple sampling points of the target line segment; the target line segment is the line connecting the root of the first boom and the end of the second boom; For each sampling point of the target line segment, the discrete point where the distance between the combined boom and the sampling point in the first direction satisfies the first condition is determined as the first boundary point; wherein, the first direction is the direction perpendicular to the target line segment in the first projection plane; the first condition is used to indicate the closest or farthest distance.

5. The method as described in claim 1, characterized in that, Before determining the first edge point set of the combined boom based on the point cloud data of the combined boom, the method further includes: Obtain the first point cloud data of the environment in which the combined boom is located; Point cloud data matching the third region is filtered from the first point cloud data to obtain the second point cloud data; wherein, the third region is the area away from the ground in the environmental region divided based on the target line segment; the target line segment is the line connecting the root of the first boom and the end of the second boom; Based on the second point cloud data, the point cloud data of the combined boom is determined.

6. The method as described in claim 1, characterized in that, The step of dividing the point cloud data of the combined boom based on the hinge position to obtain the target edge point set of the target boom includes: Based on the hinge position, the first edge point set of the combined boom is divided to obtain the first edge point set of the target boom; Based on the point cloud data of the combined boom, a second edge point set of the combined boom is determined; the second edge point set is used to characterize the set of projection points of the second boundary points of the combined boom on the second projection plane; the second projection plane intersects with the first projection plane; The target edge point set is determined based on the first edge point set of the target boom and the second edge point set of the combined boom.

7. The method as described in claim 6, characterized in that, The step of dividing the first edge point set of the combined boom based on the hinge position to obtain the first edge point set of the target boom includes: Based on the hinge position, projection points located in the fourth region are selected from the first edge points of the combined boom; the fourth region is used to characterize the region where the distance from the hinge position in the direction of the target common edge is less than a first distance; the target common edge is the common edge of the first projection plane and the second projection plane; For each projection point in the fourth region, in response to the distance between the projection point and the hinge position in a third direction being greater than a second distance, the projection point is removed from the first edge point of the combined boom; the third direction is the direction perpendicular to the target common edge in the first projection plane; The third edge point set of the combined boom is divided based on the hinge position to obtain the first edge point set of the target boom; the third edge point set includes the remaining projection points in the first edge point set of the combined boom.

8. The method as described in claim 6, characterized in that, The point cloud data of the combined boom includes multiple discrete points of the combined boom; determining the second edge point set of the combined boom based on the point cloud data of the combined boom includes: For each sampling point in the target line segment, the discrete point where the positional relationship between the combined boom and the sampling point in the second direction satisfies the second condition is determined as the second boundary point; the target line segment is the line connecting the root of the first boom and the end of the second boom. The second condition is that it is located on the target side of the target line segment and is the farthest from the sampling point; the second direction is the direction perpendicular to the target line segment in the second projection plane.

9. The method as described in claim 1, characterized in that, The step of performing three-dimensional curve fitting based on the target edge point set to obtain the contour curve of the target boom includes: Based on the target edge point set and target constraints, a three-dimensional curve fitting is performed to obtain the contour curve of the target boom; wherein, the target constraints include at least one of the following: The contour curve passes through the hinge position; The difference between the fitted value and the measured value of the first boundary point of the target boom at the third-party upward position is within the target difference range; the target edge point set includes the measured value of the first boundary point of the target boom at the third-party upward position.

10. The method as described in claim 1, characterized in that, The construction of the structural model of the target boom based on its contour curve and dimensional information includes: For each curve segment of the contour curve, a target sampling density corresponding to the target curvature is determined based on the target curvature of the curve segment; in the correspondence, the sampling density is positively correlated with the curvature; the curve segment is sampled based on the target sampling density to obtain the sampling points of the curve segment; For each sampling point of the contour curve, based on the size information of the target boom, the sampling point is offset along a third direction and a fourth direction to obtain the envelope of the target boom and the sampling point; the fourth direction is the direction perpendicular to the target common edge in the second projection plane; the target common edge is the common edge of the first projection plane and the second projection plane; Based on the envelope corresponding to each sampling point, a structural model of the target boom is generated.

11. The method as described in claim 1, characterized in that, After constructing the structural model of the target boom, the following is also included: Based on the sampling points of the contour curve, the structural model is divided into voxel elements to obtain multiple voxel elements of the target boom; For each of the aforementioned volume elements, a collision analysis is performed on the volume element to obtain the collision analysis result of the volume element; Based on the collision analysis results of each volume element, the collision analysis results of the target boom are determined.

12. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 11.

14. A type of operating machinery, characterized in that, It includes a combined boom, a point cloud acquisition device, and the electronic device as described in claim 12.

Images