Method for monitoring dumping state of dumping plough
By acquiring and analyzing point cloud data in the operating area of the drainage machine, the status of the drainage area is automatically monitored, and the problems of subjective judgment errors and low efficiency in traditional manual operations are solved, and the high accuracy and efficient monitoring of the drainage machine status are achieved. .
Patent Information
- Application Number
- CN202411034193.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2025-01-17
AI Technical Summary
Traditional manual operation of the drain machine has subjective errors in judgment and low efficiency when monitoring the discharge flow and stack status of the discharge arm, resulting in low accuracy and efficiency of the state analysis.
By obtaining point cloud data in the operation area of the drainage machine, extracting point cloud data in the discharge area, performing point cloud segmentation and filtering, analyzing the three-dimensional reconstruction data of the material flow and material pile, and automatically monitoring the operation status of the drainage machine, including materials The flow state and the material pile state realize automated monitoring.
It improves the accuracy and efficiency of the discharge status monitoring of the drainage machine, avoids mistakes in human subjective analysis and long-term spending, and realizes automated status monitoring.
Smart Images
Figure CN119323684A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of equipment monitoring, and in particular to a method for monitoring the discharging status of a soil dumper. Background Art
[0002] The earth spreader is a large-scale equipment used for discharging and piling up evacuated materials. Its structure includes a discharging arm, a receiving arm and a traveling mechanism. During the operation of the earth spreader, it is necessary to timely understand the discharging status, such as the discharging material flow data of the discharging arm, the height of the material pile, etc., in order to make operational adjustments.
[0003] Traditional manually operated earthmoving machines usually rely on the operator to check the on-site conditions, subjectively judge the state of the material flow and material pile of the discharging arm, and then judge whether to adjust the discharging operation based on experience. However, manual subjective judgment depends largely on experience and is prone to errors. Manual inspection and analysis takes time, and there are problems such as low accuracy and low efficiency of status analysis. Summary of the invention
[0004] The present application provides a method for monitoring the discharging status of a soil dumper, so as to solve the above-mentioned technical problems in the prior art.
[0005] According to a first aspect of the present application, a method for monitoring the discharging state of a soil discharging machine is provided, comprising:
[0006] Acquire point cloud data of the working area obtained by acquiring data of the working area of the earth discharger by the acquisition device; the working area includes the discharge arm area and the discharge area located on one side of the discharge port of the discharge arm;
[0007] Extracting point cloud data of the discharge area from the point cloud data of the operation area;
[0008] Performing point cloud segmentation on the point cloud data of the material discharging area to obtain point cloud data of a target object in the material discharging area; the target object includes at least one of a material flow and a working material pile;
[0009] Data analysis is performed on the point cloud data of the target object to obtain operating status data of the earthmoving machine, wherein the operating status data includes at least one of material flow status data and operating material pile status data.
[0010] In some embodiments, the acquisition of point cloud data of the working area obtained by the acquisition device through data acquisition of the working area of the earthmoving machine includes:
[0011] Acquire initial point cloud data obtained by collecting data from the working area of the earthmoving machine using multiple collection devices;
[0012] The initial point cloud data obtained by the multiple acquisition devices are fused to obtain point cloud data of the operation area.
[0013] In some embodiments, extracting the point cloud data of the discharge area from the point cloud data of the working area includes:
[0014] Determine the end position of one side of the discharge opening of the discharge arm;
[0015] Determining a discharge area filter parameter according to the end position and a set discharge area distance threshold;
[0016] The point cloud data of the operation area is filtered using the filter parameters of the material placement area to obtain the point cloud data of the material placement area.
[0017] In some embodiments, performing point cloud segmentation on the point cloud data of the nesting area to obtain point cloud data of the target object in the nesting area includes:
[0018] Get the material flow area filtering parameters;
[0019] Using the material flow area filtering parameters to filter the point cloud data of the discharge area to obtain the point cloud data of the material flow area;
[0020] Performing point cloud segmentation on the point cloud data of the material flow area to obtain point cloud data of the material flow; and / or
[0021] Obtain filtering parameters of the working stockpile area;
[0022] The point cloud data of the material discharging area is filtered using the filtering parameters of the working material pile area to obtain the point cloud data of the working material pile area;
[0023] Point cloud data of the working material pile area are segmented to obtain point cloud data of the working material pile.
[0024] In some implementations, the data analysis of the point cloud data of the target object to obtain the operating status data of the earthmoving machine includes at least one of the following two items:
[0025] Performing three-dimensional reconstruction on the point cloud data of the material flow to obtain a three-dimensional point cloud map of the material flow, and calculating the material flow state data based on the three-dimensional point cloud map of the material flow;
[0026] The point cloud data of the working material pile is three-dimensionally reconstructed to obtain a three-dimensional point cloud map of the working material pile, and the working material pile status data is calculated based on the three-dimensional point cloud map of the working material pile.
[0027] In some embodiments, the target object also includes surrounding stockpiles adjacent to the working stockpile;
[0028] The step of performing point cloud segmentation on the point cloud data of the nesting area to obtain point cloud data of the target object in the nesting area further includes:
[0029] Get the filter parameters of the surrounding stockpile area;
[0030] The point cloud data of the material discharging area is filtered using the filter parameters of the surrounding material pile area to obtain the point cloud data of the surrounding material pile area;
[0031] The point cloud data of the surrounding material pile area is segmented to obtain point cloud data of the surrounding material pile.
[0032] In some embodiments, the operation status data further includes surrounding material pile status data; and the step of performing data analysis on the point cloud data of the target object to obtain the operation status data of the earthmoving machine further includes:
[0033] The point cloud data of the surrounding material pile is three-dimensionally reconstructed to obtain a three-dimensional point cloud map of the surrounding material pile, and the surrounding material pile status data is calculated based on the three-dimensional point cloud map of the surrounding material pile.
[0034] In some embodiments, the working stockpile status data includes the working stockpile height, and the surrounding stockpile status data includes the left stockpile height and the right stockpile height; after performing data analysis on the point cloud data of the target object to obtain the working status data of the earthmoving machine, the method further includes:
[0035] When the working material pile height reaches a preset height, the minimum value of the left material pile height and the right material pile height is selected, the moving direction of the discharge arm corresponding to the minimum value is determined, and the discharge arm of the soil discharger is controlled to move based on the moving direction of the discharge arm.
[0036] According to a second aspect of the present application, an electronic device is provided, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, any of the above-mentioned methods for monitoring the material discharge status of a soil dumper is implemented.
[0037] In summary, the method for monitoring the discharging state of a soil dumper provided in this application has at least the following beneficial effects:
[0038] The point cloud data of the discharge area is obtained by extracting the point cloud data of the earth-discharging machine operating area, and then the point cloud data of the discharge area is segmented to obtain the point cloud data of the target object. Finally, the point cloud data of the target object is analyzed to obtain the operating status data representing the operating conditions. Automatic monitoring of the earth-discharging operating status is realized without the need for human participation, which can avoid analysis errors and long time costs caused by human subjectivity, thereby improving the accuracy and efficiency of monitoring the discharge status of the earth-discharging machine. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions in the specific implementation modes of the present application, the drawings required for use in the specific implementation modes will be briefly introduced below in conjunction with the accompanying drawings. It is obvious that the drawings described below are some implementation modes of the present application. For those skilled in the art, other drawings or solutions can be obtained based on these drawings without paying any creative work.
[0040] Figure 1 A flow chart of a method for monitoring the discharging state of a soil dumper provided according to an embodiment of the present application;
[0041] Figure 2 A specific flow chart of obtaining point cloud data of an operating area obtained by acquiring data of an operating area of a soil dumper by a collection device in one embodiment of the present application;
[0042] Figure 3 A specific flow chart of performing point cloud segmentation on the point cloud data of the nesting area to obtain point cloud data of the target object in the nesting area in one embodiment of the present application;
[0043] Figure 4 A side view of the site showing an exemplary working scene of a soil dumper
[0044] Figure 5 Point cloud image of the earthmoving machine working scene collected for exemplary demonstration;
[0045] Figure 6 The figure is a schematic diagram for exemplarily demonstrating the motion trajectory of a soil dumper during motion control. DETAILED DESCRIPTION
[0046] In order to make the above and other features and advantages of the present application more clear, the present application is further described below in conjunction with the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explaining to those skilled in the art and are only exemplary and not restrictive.
[0047] In the following description, many specific details are set forth to provide a thorough understanding of the present application. However, it is apparent to those skilled in the art that specific details need not be adopted to practice the present application. In other cases, well-known steps or operations are not described in detail to avoid blurring the present application.
[0048] refer to Figure 1 The present application provides a method for monitoring the discharging status of a soil discharging machine. Taking the application to an electronic device as an example, the method includes the following steps S110 to S170.
[0049] S110: Acquire point cloud data of the working area obtained by collecting data of the working area of the earthmoving machine through a collection device.
[0050] The operating area refers to the area involved in the discharging work of the earth discharging machine, including the discharging arm area and the discharging area located on the side of the discharging port of the discharging arm (i.e. the end of the discharging arm). Among them, the discharging arm area is the area where the discharging arm of the earth discharging machine is located; the discharging area is the area on the side of the discharging port of the discharging arm, including at least the material flow area of the flowing material discharged from the discharging port of the discharging arm and the operating material pile area at the material accumulation position under the current position of the discharging port of the discharging arm, and can also include the surrounding material pile area around the discharging arm and outside the operating material pile area.
[0051] The acquisition device is a device that collects point cloud data by scanning or photographing the target environment. For example, the acquisition device may include any one of a laser radar, a three-dimensional scanner, a binocular camera, and an RGB-D camera. The point cloud data is specifically three-dimensional point cloud data. Specifically, the acquisition device communicates with the electronic device, and the acquisition device is set at a position where the working area of the earthmoving machine can be photographed, for example, the acquisition device is installed on the discharge arm; the acquisition device collects point cloud data of the working area in real time and sends it to the electronic device.
[0052] S130: Extracting point cloud data of the nesting area from the point cloud data of the working area.
[0053] Specifically, the point cloud data of the operating area may be screened according to the position range of the material-laying area in the operating area to obtain the point cloud data of the material-laying area.
[0054] S150: performing point cloud segmentation on the point cloud data of the material arrangement area to obtain point cloud data of a target object in the material arrangement area; the target object includes at least one of a material flow and a working material pile.
[0055] Among them, the material flow refers to the flowing material discharged from the discharge port of the discharge arm, that is, the material discharged from the discharge port and has not yet reached the material pile. The working material pile refers to the material pile discharged at the current position of the discharge port of the discharge arm. The area where the material flow is located is the material flow area in the discharge area, and the area where the working material pile is located is the working material pile area in the discharge area. By performing point cloud segmentation on the point cloud data of the discharge area, at least one of the point cloud data of the material flow and the point cloud data of the working material pile is obtained, and the target object is determined according to the data that actually needs to be monitored.
[0056] S170: Analyze the point cloud data of the target object to obtain the operation status data of the earthmoving machine.
[0057] The operation status data is data that characterizes the discharging operation of the earthmoving machine, including at least one of the material flow status data and the operation material pile status data. The material flow status data characterizes the state of the material flow, for example, it may include the material flow length. The material pile status data characterizes the state of the material pile, for example, it may include the material pile height. Specifically, the point cloud data of the material flow is analyzed to obtain the material flow status data; the point cloud data of the operation material pile is analyzed to obtain the operation material pile status data.
[0058] The above-mentioned method for monitoring the discharging status of a soil discharging machine extracts the point cloud data of the operating area of the soil discharging machine to obtain the point cloud data of the discharging area, then performs point cloud segmentation on the point cloud data of the discharging area to obtain the point cloud data of the target object, and finally performs data analysis on the point cloud data of the target object to obtain the operating status data representing the operating conditions. This realizes automatic monitoring of the soil discharging operating status without the need for human intervention, and can avoid analysis errors and long time costs caused by human subjectivity, thereby improving the accuracy and efficiency of monitoring the discharging status of the soil discharging machine.
[0059] In some embodiments, reference Figure 2 , step S110 includes steps S111 to S115.
[0060] S111: Acquire initial point cloud data obtained by collecting data on the working area of the earthmoving machine using multiple collection devices.
[0061] The number of acquisition devices can be selected according to actual needs, and the relative positions of multiple acquisition devices are fixed. For example, there are two acquisition devices, the first acquisition device acquires point cloud data of the working area of the earthmoving machine to obtain the first initial point cloud data, and the second acquisition device acquires point cloud data of the working area of the earthmoving machine to obtain the second initial point cloud data. Among them, the first acquisition device and the second acquisition device are set at a position where the working area of the earthmoving machine can be photographed, and the relative positions of the first acquisition device and the second acquisition device are fixed. For example, the first acquisition device and the second acquisition device can be laser radars, and the two laser radars are respectively arranged on the left and right sides of the discharge arm of the earthmoving machine.
[0062] S115: Fusing the initial point cloud data obtained by multiple acquisition devices to obtain point cloud data of the operation area.
[0063] The multiple initial point cloud data are fused, and the fused point cloud data include the point cloud data of the discharge arm area and the discharge area located on the discharge port side of the discharge arm, which are the reflection points of all objects within the field of view of multiple acquisition devices.
[0064] For example, the first acquisition device and the second acquisition device are calibrated with external parameters to obtain external parameters that characterize the relative posture of the first acquisition device and the second acquisition device. The external parameters may be a transformation matrix, such as a transformation matrix including a rotation matrix and a translation matrix. After the external parameter calibration, the point cloud data of the two acquisition devices are in the same coordinate system. Specifically, the first initial point cloud data after the external parameter calibration is superimposed on the second initial point cloud data to perform data fusion to obtain point cloud data based on the coordinate system of any acquisition device (reference device).
[0065] By using multiple acquisition devices for data collection and fusing the data collected by the acquisition devices to obtain point cloud data of the work area, it is possible to avoid incomplete data collection due to line of sight obstruction or other failures of a single acquisition device, thereby improving the accuracy of point cloud data collection.
[0066] In some embodiments, after step S110 and before step S130, the method further includes: preprocessing the point cloud data of the operation area to obtain the preprocessed point cloud data of the operation area. Correspondingly, step S130 extracts the point cloud data of the nesting area from the preprocessed point cloud data of the operation area.
[0067] Preprocessing operations can include data cleaning, noise removal, invalid data removal, etc. Among them, invalid data removal is to remove data with abnormal values, such as data with a value of 0 or non-numeric value. By preprocessing the collected point cloud data and removing noise and invalid point cloud data, valid point cloud data can be obtained to ensure effective processing.
[0068] In some embodiments, step S130 includes steps (a1) to (a3).
[0069] Step (a1): Determine the end position of the discharge port side of the discharge arm.
[0070] Specifically, the end position of the discharge arm is close to the discharge area, and the discharge area can be conveniently positioned by determining the end position.
[0071] Step (a2): Determine the filter parameters of the discharge area according to the end position and the set discharge area distance threshold.
[0072] The discharge area distance threshold can be preset according to actual needs, and can be set specifically according to the distance between the discharge area range and the end position. According to the end position and the discharge area distance threshold, the position range of the discharge area can be determined, and further, the value of the position range of the discharge area can be set as the discharge area filter parameter for screening the discharge area. For example, the end position is (Px, Py), and the discharge area filter parameters include: x-axis range: (Px-2, Px+10). Y-axis range: (Py-2, Py+2).
[0073] Step (a3): filter the point cloud data of the working area using the filter parameters of the material placement area to obtain the point cloud data of the material placement area.
[0074] Specifically, the point cloud data of the working area can be filtered by a straight-through filtering method, and based on the filter parameters of the discharge area, the point cloud data outside the range corresponding to the filter parameters of the discharge area are filtered out, thereby screening the point cloud data of the discharge area. In this way, by filtering the point cloud data of the working area with reference to the end position of the discharge arm, the point cloud data of the discharge area can be accurately obtained.
[0075] In one embodiment, reference Figure 3 , step S150 includes steps S151 to S155.
[0076] S151: Obtain material flow area filtering parameters.
[0077] The material flow area filter parameter is used to screen the material flow area. Specifically, the material flow area filter parameter can be set according to the position and characteristics of the material flow, such as according to the distance range from the end position of the discharge arm to the material flow area.
[0078] S152: Filter the point cloud data of the material discharge area using the material flow area filtering parameters to obtain the point cloud data of the material flow area.
[0079] S153: Perform point cloud segmentation on the point cloud data of the material flow area to obtain point cloud data of the material flow.
[0080] Specifically, by performing a point cloud segmentation operation on the point cloud data of the material flow area, if point cloud data is segmented, it is considered that the material flow is being discharged, and the point cloud data of the material flow is obtained by segmentation; otherwise, it is considered that no material is being discharged. Figure 4 and Figure 5 , in the point cloud shown Figure 5 In the figure, circle S1 represents the point cloud data of the material flow.
[0081] Correspondingly, in this embodiment, step S170 includes: performing three-dimensional reconstruction on the point cloud data of the material flow to obtain a three-dimensional point cloud map of the material flow, and calculating the material flow status data based on the three-dimensional point cloud map of the material flow.
[0082] The three-dimensional point cloud map of the material flow is obtained through three-dimensional reconstruction, which can restore the real-time state of the material flow. Further, through data analysis and calculation of the three-dimensional point cloud map of the material flow, the material flow state data can be obtained to realize automatic monitoring of the material flow state. Furthermore, the material flow state data includes the length of the material flow, which can be obtained by calculating the height difference of the point cloud data at both ends of the material flow according to the position of the point cloud data at both ends of the material flow in the three-dimensional point cloud map of the material flow.
[0083] Based on step S170, in a specific embodiment, it includes:
[0084] S171, filtering out discrete points of the point cloud data of the material flow, and filtering each point in the point cloud data of the material flow based on normal vector information of the point cloud to reconstruct a three-dimensional point cloud image of the material flow;
[0085] S172, obtaining the number of point clouds in the three-dimensional point cloud map of the material flow, and performing feature extraction in the three-dimensional point cloud map of the material flow to obtain the terminal position information of the material flow;
[0086] S173, calculating the length of the material flow according to the position information of the end of the material flow and the position information of the discharge port of the discharge arm, and / or determining whether the material discharge operation is currently in progress according to the number of point clouds.
[0087] For the point cloud data of the material flow, the normal vector information of the point can be used for further filtering. The normal vector describes the local direction of the point cloud surface and can be used to distinguish the material flow surface from noise points. By calculating the normal vector of each point and setting a suitable threshold, points that do not conform to the surface characteristics of the material flow can be removed. Feature extraction is performed in the three-dimensional point cloud map of the material flow to obtain key feature information of the material flow. For example, the end position information of the material flow can be extracted. This can be achieved by analyzing features such as density changes and height changes in the point cloud data. The embodiment of the present invention preferably uses a clustering algorithm (such as DBSCAN, K-means, etc.) to divide the three-dimensional point cloud map of the material flow into different areas, and then determines the end position of the material flow by analyzing the position and attributes of these areas.
[0088] The length of the material flow can be calculated based on the extracted information on the end position of the material flow and the position information of the discharge port of the discharge arm. Since the material falls freely, it should be noted that the material length includes both the horizontal length l1 and the vertical length l2, which represents the overflow range of the material flow.
[0089] If the number of point clouds calculated is much smaller than a preset threshold, it can be judged that there is no nesting operation at present. This threshold needs to be set according to the actual application scenario and material flow characteristics. In the embodiment of the present invention, when the number of point clouds is 0, it is judged that there is no nesting operation at present.
[0090] In one embodiment, reference Figure 3 , step S150 also includes steps S154 to S156.
[0091] S154: Obtain filtering parameters of the working stockpile area.
[0092] The working stockpile area filter parameter is used to screen the working stockpile area. Specifically, the working stockpile area filter parameter can be set according to the position and characteristics of the working stockpile, such as according to the distance range from the end position of the discharge arm to the working stockpile area.
[0093] S155: Filtering the point cloud data of the material discharging area using the working material pile area filtering parameters to obtain the point cloud data of the working material pile area.
[0094] S156: performing point cloud segmentation on the point cloud data of the working material pile area to obtain point cloud data of the working material pile.
[0095] Specifically, the point cloud data of the working material pile is segmented by a point cloud segmentation algorithm. If the point cloud data is segmented, the point cloud data of the working material pile of the current operation is obtained. Figure 4 In the point cloud diagram shown, the circle S2 represents the point cloud data of the working material pile.
[0096] Correspondingly, in this embodiment, step S170 also includes: performing three-dimensional reconstruction on the point cloud data of the working material pile to obtain a three-dimensional point cloud map of the working material pile, and calculating the working material pile status data based on the three-dimensional point cloud map of the working material pile.
[0097] The automatic operation of the spoil spreader requires monitoring of the stockpile status. The 3D point cloud image of the working stockpile obtained through 3D reconstruction can restore the real-time status of the working stockpile. Further, through data analysis and calculation of the 3D point cloud image of the working stockpile, the working stockpile status data can be obtained to realize automatic monitoring of the working stockpile status.
[0098] Furthermore, the working material pile state data includes at least one of the working material pile top height and the height difference between the working material pile top and the end of the discharge arm. The working material pile top height can be obtained by calculating the height difference between the top point cloud data of the working material pile and the bottom point cloud data according to the three-dimensional point cloud map of the working material pile; the height difference between the point cloud data of the end of the discharge arm and the top point cloud data of the working material pile can be calculated according to the three-dimensional point cloud map of the working material pile to obtain the height difference between the working material pile top and the end of the discharge arm.
[0099] Based on step S170, in a specific embodiment, it includes:
[0100] S171, performing three-dimensional reconstruction on the point cloud data of the working material pile to obtain a three-dimensional point cloud map of the working material pile, and calculating the working material pile state data based on the three-dimensional point cloud map of the working material pile, including:
[0101] S172, intercepting target point cloud data from the point cloud data of the working material pile according to the material flow end position information;
[0102] S173, filtering out discrete points of the target point cloud data, and filtering each point in the target point cloud data based on normal vector information of the point cloud to reconstruct a three-dimensional point cloud image of the working stockpile;
[0103] S174, calculating the height of the working material pile according to the difference between the maximum value and the minimum value of the three-dimensional point cloud of the working material pile on the Z axis, and / or calculating the volume of the working material pile according to the volume of the three-dimensional point cloud of the working material pile.
[0104] According to the material flow end position information obtained in the previous step, the approximate range of the working material pile is determined. Specifically, in the point cloud data of the working material pile, the point cloud data including the lower side of the material flow end is intercepted as the target point cloud data.
[0105] The intercepted target point cloud data is preliminarily processed to filter out the discrete points that may be caused by noise or false detection. The target point cloud data is further filtered using the normal vector information of the point cloud. This can be done by calculating the normal vector of each point and removing points that do not conform to the surface characteristics of the pile according to the direction and distribution of the normal vector. That is, the three-dimensional reconstruction is completed to obtain the three-dimensional point cloud map of the working pile.
[0106] In one embodiment, all points in the three-dimensional point cloud of the working stockpile are traversed to find the maximum and minimum values on the Z axis (usually the Z axis), and the difference between the maximum and minimum values is calculated to obtain the height of the working stockpile.
[0107] In some embodiments, the data points of the 3D point cloud are used to convert the point cloud into a 3D grid or voxels by a suitable algorithm (such as convex hull algorithm, Delaunay triangulation, etc.) The total volume of these grids or voxels is calculated to approximate the volume of the working stockpile.
[0108] In one embodiment, continue to refer to Figure 3 , the target object also includes surrounding material piles adjacent to the working material pile; step S150 also includes steps S157 to S159.
[0109] S157: Obtain filtering parameters of the surrounding stockpile area.
[0110] Among them, the surrounding stockpile area filter parameters are used to filter the surrounding stockpile area; the surrounding stockpile area is the stockpile area around the discharge arm and outside the working stockpile, such as the area where the stockpile is located on the left and right sides of the current discharge position of the discharge arm. Specifically, the surrounding stockpile area filter parameters can be set according to the position and characteristics of the surrounding stockpile, such as according to the distance range from the end position of the discharge arm to the surrounding stockpile, and the distance range from the end position of the discharge arm to the surrounding stockpile is specifically determined by the discharge process.
[0111] S158: Filter the point cloud data of the material discharging area using the filter parameters of the surrounding material pile area to obtain the point cloud data of the surrounding material pile area.
[0112] S159: performing point cloud segmentation on the point cloud data of the surrounding material pile area to obtain point cloud data of the surrounding material pile.
[0113] Specifically, the point cloud data of the surrounding material piles are segmented by a point cloud segmentation algorithm. If the point cloud data is segmented, the point cloud data of the surrounding working material piles are obtained.
[0114] Correspondingly, in this embodiment, step S170 includes: performing three-dimensional reconstruction on the point cloud data of the surrounding material pile to obtain a three-dimensional point cloud map of the surrounding material pile, and calculating the surrounding material pile status data based on the three-dimensional point cloud map of the surrounding material pile.
[0115] The 3D point cloud image of the surrounding material pile is obtained through 3D reconstruction, and the state of the surrounding material pile can be restored. Further, the state data of the surrounding material pile can be obtained through data analysis and calculation of the 3D point cloud image of the surrounding material pile, so as to realize automatic monitoring of the state of the surrounding material pile. Furthermore, the state data of the surrounding material pile includes at least one of the height of the top of the surrounding material pile and the height difference between the top of the surrounding material pile and the end of the discharge arm.
[0116] For example, the target objects of the material discharge area include material flow, working material pile and surrounding material pile, and the point cloud data of the material discharge area is segmented to obtain the point cloud data of the material flow, the point cloud data of the working material pile and the point cloud data of the surrounding material pile respectively; the point cloud data of the material flow, the point cloud data of the working material pile and the point cloud data of the surrounding material pile are analyzed respectively to obtain the material flow status data, the working material pile status data and the surrounding material pile status data, so as to realize automatic monitoring of the status of the material flow, the working material pile and the surrounding material pile. It can be understood that in other embodiments, the point cloud data of the material discharge area is segmented to obtain any one or two of the point cloud data of the material flow, the point cloud data of the working material pile and the point cloud data of the surrounding material pile, and then the data analysis is performed to obtain any one or two of the material flow status data, the working material pile status data and the surrounding material pile status data, so as to realize the status monitoring of any one or two of the material flow, the working material pile and the surrounding material pile.
[0117] In a specific solution, step S170 includes:
[0118] S175, filtering out discrete points of the point cloud data of the surrounding material piles, and filtering each point in the point cloud data of the surrounding material piles based on normal vector information of the point cloud to reconstruct a three-dimensional point cloud map of the material flow;
[0119] S176. Calculate the top height of the surrounding material piles based on the difference between the maximum and minimum values of the three-dimensional point cloud image of the material pile on the Z axis, and / or calculate the height difference based on the difference between the top height of the surrounding material piles and the position information of the discharge port of the discharge arm on the Z axis.
[0120] Specifically, the point cloud data is further filtered according to the direction and distribution of the normal vector. This step can help us remove points that do not conform to the surface characteristics of the stockpile and further improve the accuracy of the data. Using the filtered point cloud data, a 3D reconstruction algorithm (such as surface reconstruction, voxel reconstruction, etc.) is applied to reconstruct a 3D point cloud map of the material flow. This 3D point cloud map will more accurately reflect the shape and characteristics of the surrounding stockpile.
[0121] Based on the reconstructed 3D point cloud of the material flow, traverse all points in the 3D point cloud of the material flow to find the maximum and minimum values on the Z axis (usually representing the vertical direction). Calculate the difference between the maximum and minimum values, which is the top height of the surrounding material piles.
[0122] If the position information of the discharge port of the discharge arm in three-dimensional space is known, extract its value on the Z axis (i.e. the height of the discharge port). Compare the height of the pile top with the height of the discharge port, calculate the difference between the two, and obtain the height difference between the pile top and the discharge port.
[0123] In combination with the above, the operation status data at least includes the following: whether it is in the discharging operation, the material flow length, the height of the operating material pile, the volume of the operating material pile, the height of the top of the surrounding material piles, and the height difference between the top of the surrounding material piles and the discharging port of the discharging arm.
[0124] According to the above-mentioned operation status data, in some embodiments, after step S170, the following step is further included: performing motion planning for the discharge arm of the earthmoving machine according to the operation status data.
[0125] Motion planning for the discharge arm can be to plan the moving direction of the discharge arm. By performing motion planning for the discharge arm according to the operation status data, a basis is provided for the motion control of the discharge arm, and unmanned discharge operation control under automated status monitoring is realized, thus reducing manual labor.
[0126] In a specific embodiment, the operation state is first determined: the system checks whether it is in the material arrangement operation state. If it is not in the material arrangement operation state, the motion planning is suspended and waits to enter the material arrangement operation state.
[0127] When in the discharging operation state, the material arm is controlled to start transporting and discharging materials; and the height difference between the top of the working material pile and the end of the discharging arm is read in real time; when the height difference between the top of the working material pile and the end of the discharging arm is less than the preset threshold, it is necessary to control the movement of the discharging arm.
[0128] like Figure 6 , A represents the layout position of the point cloud radar; specifically, the moving direction of the discharging arm can be selected according to the top height of the surrounding material piles, and the surrounding material pile status data includes the top height of the left material pile and the top height of the right material pile; prompting the need to move the discharging arm and determine the moving direction of the discharging arm: specifically including: selecting the minimum value of the left material pile height and the right material pile height, determining the moving direction of the discharging arm corresponding to the minimum value, and controlling the discharging arm of the earth discharging machine to move based on the moving direction of the discharging arm. In this way, the surrounding material pile corresponding to the smaller one of the left and right surrounding material pile top heights is selected, and the discharging arm is controlled to move to the selected surrounding material pile, realizing the automatic motion planning of the discharging arm.
[0129] By moving the discharging arm to the side with lower material pile height, the height of the material piles on both sides can be more effectively balanced and the accumulation of material piles can be reduced. This helps to maintain the stability of the material pile and avoid the safety risks that may be caused by excessively high material piles. Automatic motion planning: Automatically select the movement direction according to the height of the top of the left and right surrounding material piles, which can realize automatic motion planning of the discharging arm. Automated operation reduces the need for manual intervention and improves the accuracy and reliability of operation.
[0130] Furthermore, during the movement, the target position is calculated based on the length of the material flow and the current position of the end of the discharge arm, combined with the required movement angle of the discharge arm, so as to ensure that the material can be accurately and evenly discharged to the area close to the minimum value of the left material pile height or the right material pile height.
[0131] Specifically, we first need to determine the target point, that is, obtain the coordinate position of the minimum height in the left or right pile. Assume that the coordinate of this point is (Tx, Ty); then, according to the horizontal material flow length l1 and the vertical material length l2, we can reversely calculate the angle α that the discharging arm needs to rotate. This angle α minimizes the horizontal straight line distance between the new landing position (Gx, Gy) of the material and the coordinate position (Tx, Ty) where the minimum height is located.
[0132] It is understandable that due to the limitation of the length and rotation angle of the discharge arm, the landing position (Gx, Gy) of the center of the material may not completely coincide with the target point (Tx, Ty). However, a position (Gx, Gy) as close to the target point as possible can be found through calculation. The method is to simulate and calculate the landing position (G1, G2), (G3, G4), (G3, G4)... after the discharge arm rotates at angles a1, a2, and a3 in a step-by-step manner in combination with the length of the material flow; then calculate the horizontal distance from each landing position to (Tx, Ty), whichever has the shortest horizontal distance, which proves that it is closer to the lowest point; then use its corresponding rotation angle an as the required rotation angle. This allows the material to be stacked more reasonably from the ground, which helps to improve the stability and safety of the stacking and improve space utilization.
[0133] The present application provides an electronic device, which includes a processor and a memory storing computer program instructions. When the processor executes the computer program instructions, each step of the above-mentioned method for monitoring the state of material discharge of a soil dumper is implemented. The electronic device can be broadly a server, a terminal, or any other electronic device with necessary computing and / or processing capabilities.
[0134] In one embodiment, the electronic device may include a processor, a memory, a network interface, a communication interface, etc. connected through a system bus. The processor of the electronic device may be used to provide necessary computing, processing and / or control capabilities. The memory of the electronic device may include a non-volatile storage medium and an internal memory. The non-volatile storage medium may store an operating system, a computer program, etc. The internal memory may provide an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface and the communication interface of the electronic device may be used to connect and communicate with external devices through a network. When the computer program is executed by the processor, the steps of the method of the present application are executed.
[0135] The above-mentioned electronic device, because the processor it includes implements the various steps of the above-mentioned method for monitoring the discharge state of the earthmoving machine when executing computer program instructions, can also realize automatic monitoring of the earthmoving operation state, and the accuracy and efficiency of monitoring the discharge state of the earthmoving machine are high.
[0136] In addition, the present application provides a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above-mentioned method for monitoring the material discharging status of a soil discharging machine is implemented.
[0137] The above-mentioned computer-readable storage medium implements the various steps of the above-mentioned method for monitoring the discharge state of the earthmoving machine when the computer program instructions stored therein are executed. Similarly, automatic monitoring of the earthmoving operation status can be realized, and the accuracy and efficiency of monitoring the discharge state of the earthmoving machine are high.
[0138] Those skilled in the art will appreciate that the method steps of the present application can be completed by instructing related hardware such as electronic devices or processors through a computer program, and the computer program can be stored in a non-temporary computer-readable storage medium, and the computer program causes the steps of the present application to be executed when it is executed. Depending on the circumstances, any reference to memory, storage or other media herein may include non-volatile or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.
[0139] The various technical features described above can be combined arbitrarily. Although all possible combinations of these technical features are not described, any combination of these technical features should be considered to be covered by this specification as long as there is no contradiction in such combination.
[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for monitoring the discharging state of a soil discharging machine, characterized in that: include: Acquire point cloud data of the working area obtained by acquiring data of the working area of the earth discharger by the acquisition device; the working area includes the discharge arm area and the discharge area located on one side of the discharge port of the discharge arm; Extracting point cloud data of the discharge area from the point cloud data of the operation area; Performing point cloud segmentation on the point cloud data of the material discharging area to obtain point cloud data of a target object in the material discharging area; the target object includes at least one of a material flow and a working material pile; Data analysis is performed on the point cloud data of the target object to obtain operating status data of the earthmoving machine, wherein the operating status data includes at least one of material flow status data and operating material pile status data.
2. The method according to claim 1, characterized in that The point cloud data of the working area obtained by acquiring data of the working area of the earthmoving machine by the acquisition device includes: Acquire initial point cloud data obtained by collecting data from the working area of the earthmoving machine using multiple collection devices; The initial point cloud data obtained by the multiple acquisition devices are fused to obtain point cloud data of the operation area.
3. The method according to claim 1 or 2, characterized in that: The step of extracting the point cloud data of the discharge area from the point cloud data of the operation area comprises: Determine the end position of one side of the discharge opening of the discharge arm; Determining a discharge area filter parameter according to the end position and a set discharge area distance threshold; The point cloud data of the operation area is filtered using the filter parameters of the material placement area to obtain the point cloud data of the material placement area.
4. The method according to claim 1, characterized in that: The point cloud data of the nesting area is segmented to obtain the point cloud data of the target object in the nesting area, including: Acquire material flow area filtering parameters; filter the point cloud data of the material discharge area using the material flow area filtering parameters to obtain point cloud data of the material flow area; perform point cloud segmentation on the point cloud data of the material flow area to obtain point cloud data of the material flow; and / or Acquire filtering parameters of the working material pile area; use the filtering parameters of the working material pile area to filter the point cloud data of the material discharge area to obtain point cloud data of the working material pile area; perform point cloud segmentation on the point cloud data of the working material pile area to obtain point cloud data of the working material pile.
5. The method according to claim 4, characterized in that The step of performing data analysis on the point cloud data of the target object to obtain the operation status data of the earthmoving machine includes: Performing three-dimensional reconstruction on the point cloud data of the material flow to obtain a three-dimensional point cloud map of the material flow, and calculating the material flow state data based on the three-dimensional point cloud map of the material flow; The point cloud data of the working material pile is three-dimensionally reconstructed to obtain a three-dimensional point cloud map of the working material pile, and the working material pile status data is calculated based on the three-dimensional point cloud map of the working material pile.
6. The method according to claim 5, characterized in that The three-dimensional reconstruction of the point cloud data of the material flow to obtain a three-dimensional point cloud map of the material flow, and the material flow state data calculated based on the three-dimensional point cloud map of the material flow, include: Filtering out discrete points of the point cloud data of the material flow, and filtering each point in the point cloud data of the material flow based on normal vector information of the point cloud to reconstruct a three-dimensional point cloud image of the material flow; Acquiring the number of point clouds in the three-dimensional point cloud map of the material flow, and performing feature extraction in the three-dimensional point cloud map of the material flow to obtain the position information of the end of the material flow; The material flow length is calculated according to the position information of the end of the material flow and the position information of the discharge port of the discharge arm, and / or, it is determined whether the material discharge operation is currently in progress according to the number of point clouds.
7. The method according to claim 6, characterized in that Performing three-dimensional reconstruction on the point cloud data of the working material pile to obtain a three-dimensional point cloud map of the working material pile, and calculating the working material pile state data based on the three-dimensional point cloud map of the working material pile, including: intercepting target point cloud data from the point cloud data of the working material pile according to the terminal position information of the material flow; Filtering out discrete points of the target point cloud data, and filtering each point in the target point cloud data based on normal vector information of the point cloud to reconstruct a three-dimensional point cloud image of the working stockpile; The height of the working material pile is calculated according to the difference between the maximum value and the minimum value of the three-dimensional point cloud of the working material pile on the Z axis, and / or the volume of the working material pile is calculated according to the volume of the three-dimensional point cloud of the working material pile.
8. The method according to claim 4, characterized in that The target object also includes surrounding stockpiles adjacent to the working stockpile; The step of performing point cloud segmentation on the point cloud data of the nesting area to obtain point cloud data of the target object in the nesting area further includes: Get the filtering parameters of the surrounding stockpile area; The point cloud data of the material discharging area is filtered using the filter parameters of the surrounding material pile area to obtain the point cloud data of the surrounding material pile area; The point cloud data of the surrounding material pile area is segmented to obtain point cloud data of the surrounding material pile.
9. The method according to claim 7, characterized in that: The operation status data also includes surrounding material pile status data; the data analysis of the point cloud data of the target object to obtain the operation status data of the earthmoving machine also includes three-dimensional reconstruction of the point cloud data of the surrounding material pile to obtain a three-dimensional point cloud map of the surrounding material pile, and the surrounding material pile status data is calculated based on the three-dimensional point cloud map of the surrounding material pile; The three-dimensional reconstruction of the point cloud data of the surrounding material pile to obtain a three-dimensional point cloud map of the surrounding material pile, and the surrounding material pile state data calculated based on the three-dimensional point cloud map of the surrounding material pile include: Filtering out discrete points of the point cloud data of the surrounding material pile, and filtering each point in the point cloud data of the surrounding material pile based on normal vector information of the point cloud to reconstruct a three-dimensional point cloud image of the surrounding material pile; The height of the top of the surrounding material pile is calculated based on the difference between the maximum and minimum values of the three-dimensional point cloud image of the surrounding material pile on the Z axis, and / or the height difference is calculated based on the difference between the top height of the surrounding material pile and the position information of the discharge port of the discharge arm on the Z axis.
10. The method according to claim 9, characterized in that The top height of the surrounding material pile includes the height of the left material pile and the height of the right material pile; after the point cloud data of the target object is analyzed to obtain the operating status data of the earthmoving machine, it also includes: When the height difference between the top of the working material pile and the discharge port of the discharge arm is less than a preset threshold, the minimum value of the left material pile height and the right material pile height is selected, the moving direction of the discharge arm corresponding to the minimum value is determined, and the discharge arm of the soil discharger is controlled to move based on the moving direction of the discharge arm.