Excavator trajectory control method, system and excavator

Abstract

The application discloses an excavator track control method and system and an excavator in the technical field of engineering machinery control, and aims to solve the problem that the prior art cannot efficiently, timely and accurately complete the identification and detection of tooth wear, causing the deviation of excavator track control. It comprises the following steps: collecting the posture signals of the boom, dipper arm and bucket of the excavator in real time; obtaining the predicted length of each tooth, collecting the image and point cloud data of each tooth in real time, and calculating the actual coordinates of each tooth in the image; and obtaining the actual length of each tooth according to the actual coordinates of each tooth in the image and the predicted length of each tooth. Even if the camera and laser radar data fail, the tooth wear prediction model still plays a key role in identification and judgment. Through multi-source sensor fusion, comprehensive and accurate data are obtained, and in combination with innovative algorithms, the wear condition of the tooth can be grasped in real time and accurately, and high-precision control of the excavation track is realized.

Description

Technical Field

[0001] This invention relates to a method, system, and excavator trajectory control for an excavator, belonging to the field of engineering machinery control technology. Background Technology

[0002] In the field of intelligent control technology for construction machinery, excavators, as crucial earthmoving machinery, directly impact the progress and quality of engineering projects through their operational efficiency and precision. Bucket teeth inevitably wear down during long-term operation. In the intelligent control of excavators, this wear causes deviations between the actual digging trajectory and the preset trajectory, thus affecting construction accuracy. Traditionally, the wear of excavator buckets and teeth is managed primarily through regular manual inspection and maintenance. This method is not only time-consuming and labor-intensive but also fails to reflect the actual wear status of the teeth in real time. In recent years, with the development of intelligent technologies, image recognition and sensor technologies have been gradually applied to the monitoring and control of construction machinery. However, most of these existing solutions remain at the level of wear detection, failing to effectively translate the detected wear information into precise adjustments for the excavator's motion trajectory control. As the error between the bucket tooth life parameters and factory parameters gradually widens, the construction accuracy of intelligent operations decreases, thus limiting the improvement of intelligent and unmanned control levels.

[0003] In existing technologies, the wear condition of bucket teeth is usually obtained through manual inspection, sensor inspection, or image inspection. Manual inspection cannot obtain the wear condition of bucket teeth in real time and has low work efficiency. Sensor inspection requires precise installation at specific positions on the bucket teeth, which is relatively complex and can easily affect the normal digging operation of the bucket teeth. Image inspection is easily affected by the environment, such as low light environment or clay adhering to the surface of the bucket teeth, which makes it impossible to obtain accurate bucket tooth wear conditions and affects the control effect of bucket tooth trajectory.

[0004] In summary, existing technologies cannot efficiently, in real-time, and accurately identify and detect bucket tooth wear. The detection results are prone to deviation, leading to problems such as excavator trajectory control deviation and affecting the actual working performance of the excavator. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an excavator trajectory control method, system, and excavator. It achieves real-time online detection of excavator bucket wear through image perception technology, emphasizing its real-time nature and automation. The actual length of the bucket teeth is obtained through images, point clouds, and predicted lengths of the teeth. Even in the event of camera or lidar data failure, the bucket tooth wear prediction model still provides crucial identification and judgment capabilities. This allows for efficient, real-time, and accurate identification and detection of bucket tooth wear, avoiding potential deviations in detection results and ensuring the excavator's actual working performance.

[0006] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution:

[0007] In a first aspect, the present invention provides an excavator trajectory control method, comprising the following steps:

[0008] Step a: Real-time acquisition of attitude signals of the excavator boom, stick, and bucket;

[0009] Step b: Obtain the predicted length of each bucket tooth, collect images and point cloud data of each bucket tooth in real time, calculate the actual coordinates of each bucket tooth in the image, and obtain the actual length of each bucket tooth based on the actual coordinates of each bucket tooth in the image and the predicted length of each bucket tooth.

[0010] Step c: Obtain the historical length of each bucket tooth, calculate the wear amount of each bucket tooth based on the historical length and actual length, and calculate the average wear amount and wear error rate based on the wear amount of each bucket tooth.

[0011] Step d: Based on the wear error rate and the wear amount of each bucket tooth, determine whether the wear of each bucket tooth conforms to the wear pattern. If each bucket tooth conforms to the wear pattern, proceed to step e; otherwise, the excavator trajectory control process ends.

[0012] Step e: Compare the average wear amount with the preset range. If the average wear amount is greater than the preset range, then adjust the bucket tooth length parameter in the control system according to the wear amount of each bucket tooth; otherwise, do not adjust the bucket tooth length parameter in the control system.

[0013] Step f: Calculate the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space based on the attitude signals of the control system and the excavator boom, stick and bucket;

[0014] Step g: Input the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space as feedback parameters into the excavator motion control system to realize the trajectory control of the excavator bucket teeth.

[0015] Furthermore, the calculation of the actual coordinates of each bucket tooth in the image specifically includes:

[0016] Based on the images and point cloud data of each bucket tooth, identify the feature points of each bucket tooth;

[0017] The dimensions of each bucket tooth in the image are corrected by geometric transformation and projection relationship to obtain the appearance dimensions of each bucket tooth;

[0018] Based on the feature points and external dimensions of each bucket tooth, and in conjunction with the calibration parameters, the actual coordinates of each bucket tooth in the image are obtained.

[0019] Furthermore, obtaining the actual length of each bucket tooth based on its actual coordinates in the image specifically includes:

[0020] Obtain the actual coordinates of the fixing pins of each bucket tooth;

[0021] Based on the actual coordinates of each bucket tooth in the image, obtain the actual coordinates of each bucket tooth tip in the image;

[0022] The detection length of each bucket tooth is calculated based on the actual coordinates of the fixing pins of each bucket tooth and the actual coordinates of the tips of each bucket tooth in the image. The specific expression is as follows:

[0023]

[0024] In the formula: Let be the detection length of the i-th tooth inside the bucket; i is the number of the tooth. At time t; the actual coordinates of the fixed pin of the i-th tooth inside the bucket are ( , , The actual coordinates of the tip of the i-th tooth in the bucket in the image are ( , , ); Let be the coordinates of the front-rear direction of the fixed pin of the i-th tooth inside the bucket; Let be the coordinates of the left and right directions of the fixed pin of the i-th tooth inside the bucket; Let be the coordinates of the fixed pin shaft of the i-th tooth inside the bucket in the vertical direction; Let be the coordinates of the tip of the i-th tooth in the bucket in the front-back direction in the image; Let be the left-right coordinates of the tip of the i-th tooth in the bucket in the image; Let be the vertical coordinates of the tip of the i-th tooth in the bucket in the image;

[0025] The process of obtaining the predicted length of each bucket tooth specifically includes:

[0026] The detection parameters of each bucket tooth are obtained, and the detection parameters of each bucket tooth are input into the pre-constructed and trained bucket tooth wear adaptive prediction model to output the predicted length of each bucket tooth.

[0027] Based on the detected length of each bucket tooth and the corresponding predicted length of each bucket tooth, the actual length of each bucket tooth is calculated, as shown in the following expression:

[0028]

[0029]

[0030] In the formula: Let w1 be the actual length of the i-th tooth inside the bucket; w2 are the weights. Let be the predicted length of the i-th tooth inside the bucket.

[0031] Furthermore, the detection parameters for each bucket tooth include bucket tooth material, bucket tooth rotation speed, bucket tooth rotation acceleration, bucket tooth digging resistance, cumulative bucket tooth installation time, and target category attribute parameters, wherein:

[0032] The predicted length of each bucket tooth is output, and the specific expression is as follows:

[0033]

[0034] In the formula: This is a prediction function based on multiple factors; Made of bucket teeth material; Let be the rotational speed of the i-th tooth inside the bucket; Let be the rotational acceleration of the i-th tooth inside the bucket; Let be the digging resistance of the i-th tooth inside the bucket; The cumulative installation time for the i-th tooth in the bucket; To extract target category attribute parameters.

[0035] Furthermore, the calculation of the wear amount of each bucket tooth based on its historical length and actual length specifically includes:

[0036] The wear amount of the i-th tooth inside the bucket is calculated using the following expression:

[0037]

[0038] In the formula, Let represent the wear amount of the i-th tooth in the bucket, and Di represent the historical length of the i-th tooth in the bucket. is the actual length of the i-th tooth in the bucket; n is the time; i is the number of the tooth;

[0039] The calculation of average wear and wear error rate based on the wear of each bucket tooth specifically includes:

[0040] The expression for the average wear amount is as follows:

[0041]

[0042] In the formula: The average wear rate is given by i; k is the number of teeth in the bucket; i = 1, 2, ..., k.

[0043] The expression for the wear error rate is as follows:

[0044]

[0045] .

[0046] Furthermore, determining whether the wear of each bucket tooth conforms to a wear pattern specifically includes:

[0047] The wear error rate is compared with a first preset threshold. If the wear error rate is less than the first preset threshold, it is determined that the wear of each bucket tooth conforms to the wear pattern; if the wear error rate is not less than the first preset threshold, it is determined that the wear of each bucket tooth does not conform to the wear pattern.

[0048] Specifically, when the wear error rate is less than the first preset threshold, the difference between the wear amount of each bucket tooth and the average wear amount is calculated based on the wear amount of each bucket tooth and the average wear amount. The difference between the wear amount of each bucket tooth and the average wear amount is then compared with the second preset threshold. If the difference between the wear amount of each bucket tooth and the average wear amount is less than the second preset threshold, it is determined that the wear of each bucket tooth conforms to the wear pattern. If the difference between the wear amount of each bucket tooth and the average wear amount is not less than the second preset threshold, it is determined that the wear of each bucket tooth does not conform to the wear pattern.

[0049] Furthermore, the trajectory control of the excavator bucket teeth specifically includes:

[0050] The excavator motion control system continuously adjusts the control parameters based on the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space through a closed-loop feedback mechanism, and adjusts the excavator's motion in real time according to the control parameters, thereby precisely controlling the movement trajectory of the bucket teeth tip.

[0051] The control parameters include digging speed and acceleration; the excavator motion control system includes a model predictive control algorithm.

[0052] In a second aspect, the present invention provides an excavator trajectory control system, comprising:

[0053] Attitude signal detection module: used to collect attitude signals of the excavator boom, stick, and bucket in real time;

[0054] Bucket tooth recognition and detection module: used to collect images and point cloud data of each bucket tooth in real time, calculate the actual coordinates of each bucket tooth in the image, and obtain the actual length of each bucket tooth based on the actual coordinates of each bucket tooth in the image;

[0055] Error Calculation and Correction Module: This module includes a calculation module and a correction module. The calculation module obtains the historical length of each bucket tooth, calculates the wear amount of each tooth based on its historical and actual length, and calculates the average wear amount and wear error rate based on the wear amount of each tooth. Based on the wear error rate and the wear amount of each tooth, it determines whether the wear of each tooth conforms to the wear pattern. If all teeth conform to the wear pattern, the correction module is executed; otherwise, the excavator trajectory control process ends. The correction module compares the average wear amount with a preset range. If the average wear amount is greater than the preset range, the tooth length parameter in the control system is corrected based on the wear amount of each tooth; otherwise, the tooth length parameter in the control system is not corrected.

[0056] Teeth tip attitude coordinate update module: Based on the attitude signals of the control system and the excavator boom, stick and bucket, calculate the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space;

[0057] Motion trajectory control module: Used to input the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space as feedback parameters to the excavator motion control system to realize the trajectory control of the excavator bucket teeth.

[0058] Furthermore, it also includes a parameter and model storage module: used to store and manage key parameters and data related to the error calculation and correction module and the excavator motion control system;

[0059] The bucket tooth recognition and detection module is also used to identify the feature points of each bucket tooth based on the image and point cloud data of each bucket tooth; correct the size of each bucket tooth in the image through geometric transformation and projection relationship to obtain the appearance size of each bucket tooth; and obtain the actual coordinates of each bucket tooth in the image based on the feature points and appearance size of each bucket tooth, combined with the calibration parameters.

[0060] The motion trajectory control module is used to receive the three-dimensional attitude coordinate information of the bucket tooth tip in the actual working space in real time and send the three-dimensional attitude coordinate information of the bucket tooth tip in the actual working space to the excavator motion control system. The excavator motion control system is used to adjust the attitude of the excavator boom, stick and bucket according to the three-dimensional attitude coordinate of the bucket tooth tip in the actual working space so that the trajectory of the bucket tooth tip is consistent with the preset trajectory.

[0061] The excavator motion control system calculates a preset trajectory based on the posture of the excavator boom, stick, and bucket, the target position, and the constraints of the working environment.

[0062] Thirdly, the present invention provides an excavator, including the excavator trajectory control system described in the second aspect.

[0063] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0064] This invention achieves real-time online detection of excavator bucket wear using image perception technology, emphasizing its real-time nature and degree of automation. It obtains the actual length of the bucket teeth through images, point clouds, and predicted lengths. Even in the event of camera or lidar data failure, the bucket tooth wear prediction model still provides crucial identification and judgment capabilities. This allows for efficient, real-time, and accurate detection of bucket tooth wear, avoiding potential deviations in detection results and ensuring the excavator's actual working performance. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating an excavator trajectory control method according to an embodiment of the present invention;

[0066] Figure 2 This is a logical schematic diagram of an excavator trajectory control method provided according to an embodiment of the present invention;

[0067] Figure 3 This is a schematic diagram of a camera or lidar in the complete coordinate system of an excavator according to an embodiment of the present invention;

[0068] Figure 4 This is a system schematic diagram of an excavator trajectory control system according to an embodiment of the present invention;

[0069] Figure 5 This is a system schematic diagram of an excavator according to an embodiment of the present invention. Detailed Implementation

[0070] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0071] The term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0072] Example 1:

[0073] like Figures 1-2 As shown, the present invention provides a method for controlling the trajectory of an excavator, comprising the following steps:

[0074] Step a: Real-time acquisition of attitude signals of the excavator boom, stick, and bucket;

[0075] Step b: Obtain the predicted length of each bucket tooth, collect images and point cloud data of each bucket tooth in real time, calculate the actual coordinates of each bucket tooth in the image, and obtain the actual length of each bucket tooth based on the actual coordinates of each bucket tooth in the image and the predicted length of each bucket tooth.

[0076] Step c: Obtain the historical length of each bucket tooth, calculate the wear amount of each bucket tooth based on the historical length and actual length, and calculate the average wear amount and wear error rate based on the wear amount of each bucket tooth.

[0077] Step d: Based on the wear error rate and the wear amount of each bucket tooth, determine whether the wear of each bucket tooth conforms to the wear pattern. If each bucket tooth conforms to the wear pattern, proceed to step e; otherwise, the excavator trajectory control process ends.

[0078] Step e: Compare the average wear amount with the preset range. If the average wear amount is greater than the preset range, then adjust the bucket tooth length parameter in the control system according to the wear amount of each bucket tooth; otherwise, do not adjust the bucket tooth length parameter in the control system.

[0079] Step f: Calculate the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space based on the attitude signals of the control system and the excavator boom, stick and bucket;

[0080] Step g: Input the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space as feedback parameters into the excavator motion control system to realize the trajectory control of the excavator bucket teeth.

[0081] Specifically, the present invention includes the following steps:

[0082] Step SO1: Attitude signal detection:

[0083] Real-time acquisition of excavator boom, stick, and bucket attitude signals: This step mainly uses attitude sensors to collect the movement angles of the excavator boom, stick, and bucket in real time. After processing and analyzing the attitude sensor data signals, high-precision basic data is provided for subsequent image modeling, algorithm execution, and motion control. The attitude signal detection module within the system can efficiently and accurately capture and process data signals from attitude sensors on the boom, stick, and bucket. At the same time, it detects hydraulic oil pressure feedback signals from the boom, stick, and bucket. The control program uses advanced filtering and smoothing algorithms to preprocess the raw data to eliminate noise interference and improve data reliability. The data from the attitude signal detection module will be used to build a 3D model and sense dynamic data of excavation resistance, ensuring the smooth execution of a series of complex calculations such as path planning, collision detection, and motion control.

[0084] In one embodiment, calculating the actual coordinates of each bucket tooth in the image specifically includes:

[0085] Based on the images and point cloud data of each bucket tooth, identify the feature points of each bucket tooth;

[0086] The dimensions of each bucket tooth in the image are corrected by geometric transformation and projection relationship to obtain the appearance dimensions of each bucket tooth;

[0087] Based on the feature points and external dimensions of each bucket tooth, and in conjunction with the calibration parameters, the actual coordinates of each bucket tooth in the image are obtained.

[0088] In one embodiment, obtaining the actual length of each bucket tooth based on its actual coordinates in the image specifically includes:

[0089] By analyzing the data from the attitude sensor, the actual coordinates of each bucket tooth fixing pin are obtained;

[0090] Based on the actual coordinates of each bucket tooth in the image, obtain the actual coordinates of each bucket tooth tip in the image;

[0091] The detection length of each bucket tooth is calculated based on the actual coordinates of the fixing pins of each bucket tooth and the actual coordinates of the tips of each bucket tooth in the image. The specific expression is as follows:

[0092]

[0093] In the formula: Let be the detection length of the i-th tooth inside the bucket; i is the number of the tooth. At time t; the actual coordinates of the fixed pin of the i-th tooth inside the bucket are ( , , The actual coordinates of the tip of the i-th tooth in the bucket in the image are ( , , ); Let be the coordinates of the front-rear direction of the fixed pin of the i-th tooth inside the bucket; Let be the coordinates of the left and right directions of the fixed pin of the i-th tooth inside the bucket; Let be the coordinates of the fixed pin shaft of the i-th tooth inside the bucket in the vertical direction; Let be the coordinates of the tip of the i-th tooth in the bucket in the front-back direction in the image; Let be the left-right coordinates of the tip of the i-th tooth in the bucket in the image; Let be the vertical coordinates of the tip of the i-th tooth in the bucket in the image;

[0094] The detection parameters of each bucket tooth are obtained, and the detection parameters of each bucket tooth are input into the pre-constructed and trained bucket tooth wear adaptive prediction model to output the predicted length of each bucket tooth.

[0095] Based on the detected length of each bucket tooth and the corresponding predicted length of each bucket tooth, the actual length of each bucket tooth is calculated, as shown in the following expression:

[0096]

[0097]

[0098] In the formula: Let w1 be the actual length of the i-th tooth inside the bucket; w2 are the weights. Let be the predicted length of the i-th tooth inside the bucket.

[0099] In one embodiment, the calculation of the detection length of each bucket tooth is specifically expressed as follows:

[0100]

[0101] In the formula: Let be the coordinates of the front-rear direction of the fixed pin of the i-th tooth inside the bucket; Let be the coordinates of the left and right directions of the fixed pin of the i-th tooth inside the bucket; Let be the coordinates of the fixed pin shaft of the i-th tooth inside the bucket in the vertical direction; Let be the coordinates of the tip of the i-th tooth in the bucket in the front-back direction in the image; Let be the left-right coordinates of the tip of the i-th tooth in the bucket in the image; Let be the vertical coordinates of the tip of the i-th tooth in the bucket within the image.

[0102] In one embodiment, the detection parameters of each bucket tooth include bucket tooth material, bucket tooth rotation speed, bucket tooth rotation acceleration, bucket tooth digging resistance, bucket tooth cumulative installation time, and digging target category attribute parameters;

[0103] The predicted length of each bucket tooth is output, and the specific expression is as follows:

[0104]

[0105] In the formula: This is a prediction function based on multiple factors; Made of bucket teeth material; Let be the rotational speed of the i-th tooth inside the bucket; Let be the rotational acceleration of the i-th tooth inside the bucket; Let be the digging resistance of the i-th tooth inside the bucket; The cumulative installation time for the i-th tooth in the bucket; To extract target category attribute parameters.

[0106] Specifically, it includes the following steps:

[0107] Step SO2: Bucket tooth recognition and detection:

[0108] Obtaining the detection length of each bucket tooth: Based on the fusion detection of camera and LiDAR sensing signals, real-time acquisition and analysis of bucket tooth images and point cloud data are performed to accurately capture feature points of the bucket teeth in the image, such as vertices and edges, enabling dynamic assessment of the bucket tooth wear condition; automatically identifying feature points of the bucket teeth, calculating the actual distance and shooting angle between the camera or LiDAR and the bucket teeth, and correcting the bucket tooth size using geometric transformations and projection relationships to obtain accurate appearance data of the bucket teeth; in low light or harsh environments (such as smog, dust), LiDAR can compensate for the shortcomings of cameras, thereby ensuring... It maintains high recognition accuracy and robustness; real-time acquired images and point cloud data are processed by the model to automatically identify the feature points of the bucket teeth, and the size of the bucket teeth in the image is corrected through geometric transformation and projection relationship to obtain accurate appearance data of the bucket teeth; by the distribution of key feature points such as the apex of the bucket teeth in the image, combined with calibration parameters, the actual coordinate position of the bucket teeth in the X, Y, and Z directions is accurately calculated (X direction is the front-back direction, Y direction is the left-right direction, and Z direction is the up-down direction). For example, the changes in the front-back, left-right, and up-down orientations of the apex of the bucket teeth in the image reflect the actual coordinates in the X, Y, and Z directions, respectively.

[0109] like Figure 3 As shown, point M is the fixed coordinate point of the camera or lidar in the excavator's complete coordinate system. Determined by the actual installation position, it is a precise fixed value, denoted as M(Xm, Ym, Zm); point A is the fixed coordinate point of the fixed pin position of the i-th bucket tooth in the bucket in the excavator's complete coordinate system. Determined by the attitude signal detection system, it is a precise value that can be dynamically read during step S01, denoted as A( , , Point B is the actual coordinate point of the tip of the i-th tooth inside the bucket in the excavator's complete coordinate system. It is determined by image detection and recognition within the effective distance and angle recognition range using a camera or LiDAR, and is a dynamically calculated value, denoted as B( , , ), where i is the number of the bucket tooth.

[0110] The detection length parameter of the bucket teeth is based on the spatial distance between points A and B. It can be calculated using the following formula: .

[0111] Obtaining the predicted length of each bucket tooth: This invention uses a deep learning algorithm to train an adaptive prediction model for bucket tooth wear. This model takes data from multiple dimensions, such as operating time, material resistance, and digging speed, as input. By learning from a large amount of historical operating data, the model can automatically adjust its parameters to adapt to the wear of bucket teeth under different working conditions. Different materials have different hardness, resulting in varying degrees of wear on bucket teeth. The magnitude of digging force also affects bucket tooth wear. Traditional models struggle to fully consider these complex factors. The adaptive prediction model for bucket tooth wear invented in this solution can more accurately predict the wear of bucket teeth through learning, providing a strong basis for subsequent precise correction of the digging trajectory.

[0112] Optionally, the adaptive prediction model for bucket tooth wear is a neural network-based prediction model. By learning from a large amount of historical operational data (including information such as the material, speed, acceleration, digging resistance, running time, bucket tooth wear, and target type), a model is established to show the relationship between bucket tooth wear rate and various influencing factors. When data such as the current speed, acceleration, and digging resistance of the operational device are obtained, the trained model is used to predict the wear amount of the bucket teeth over a future period, aiming to accurately predict the wear rate and expected state of the bucket teeth. These feature points include, but are not limited to, the apex, edge contour, and other changes with significant geometric or morphological features of the bucket teeth. Through training with a large amount of labeled data, the model can accurately distinguish these features, maintaining high recognition accuracy and robustness even in complex and changing background environments and lighting conditions. The model can be calculated using the following formula: .

[0113] In routine equipment inspections and maintenance, the fusion detection results of cameras and lidar can be used as the primary basis to quickly determine whether there are obvious wear abnormalities in the bucket teeth and whether immediate maintenance or replacement is necessary. In complex and variable working conditions, such as when operating in harsh environments, the fusion detection of cameras and lidar may be affected. In this case, it is necessary to combine the prediction results of deep learning models to comprehensively judge the wear of the bucket teeth and avoid misjudgment due to the limitations of a single detection method. Based on the fusion detection results of the camera and lidar sensing signal system and the bucket tooth wear prediction model trained by deep learning algorithms, the adaptability of the two methods under different working conditions should be analyzed. For example, if the fusion detection may have significantly distorted detection results under certain specific working conditions, while the prediction accuracy of the deep learning model is higher under such conditions, the weight of the model prediction results can be appropriately increased under such conditions.

[0114] The confirmation of weights w1 and w2 specifically includes: assigning different weights to the camera and LiDAR fusion detection results and the deep learning wear prediction model results based on factors such as data reliability, time dimension, and application scenario; the weights can be determined based on factors such as the accuracy of historical data, model performance, and the complexity of the current working environment; generally, if the camera and LiDAR communication data is abnormal, and the bucket teeth are not covered with deposits during normal application scenarios, the data can be considered reliable, and the corresponding fusion detection weight w1 can be used normally; conversely, if either the camera or LiDAR malfunctions, or the bucket teeth are easily covered with deposits, or the working conditions are particularly harsh (including but not limited to extremely low visibility, heavy rain, heavy fog, high dust, etc.), the corresponding fusion detection weight can be directly defined as w1=0, thus ensuring that even if the camera and LiDAR data fail, the bucket tooth wear adaptive prediction model still plays a key role in identification and judgment.

[0115] Optionally, under normal operating conditions, w1 = 0.6 and w2 = 0.4 can be defined; in complex operating conditions or long-term prediction scenarios, the weight ratio can be adjusted appropriately.

[0116] Based on the above, the weights of the real-time assessment results and model prediction results are comprehensively considered, and a weighted sum or weighted average is performed to obtain the final optimal calculation result. This result can more accurately reflect the actual wear condition and future trend of the bucket teeth. The two results are then weighted according to their respective weights to obtain a comprehensive wear assessment result, as shown in the formula:

[0117]

[0118]

[0119] In the formula: is the actual length of the i-th tooth inside the bucket; i is the number of the tooth. w1 and w2 are time intervals; w1 and w2 are weights. Let be the detection length of the i-th tooth inside the bucket; Let be the predicted length of the i-th tooth inside the bucket.

[0120] The optimized calculation results are compared with the actual situation, and feedback data is collected to optimize the data processing algorithms of cameras and LiDAR, the training parameters of deep learning models, etc., in order to improve the accuracy and reliability of future calculations.

[0121] The present invention also includes step so3: retrieving data from the storage area.

[0122] The system automatically retrieves various mechanical dimensional parameters of a single excavator tooth tip that are pre-stored in the storage area, including key historical parameters such as initial length, width, height, and shape characteristics. The storage area ensures that the mechanical dimensional parameters of the tooth tip and the correction rule model required by the control system are provided, thus guaranteeing the smoothness and timeliness of data interaction between modules.

[0123] In one embodiment, the present invention includes step so4: calculating the bucket tooth error.

[0124] The calculation of the wear amount of each bucket tooth based on its historical length and actual length specifically includes:

[0125] The wear amount of the i-th tooth inside the bucket is calculated using the following expression:

[0126]

[0127] In the formula, Let represent the wear amount of the i-th tooth in the bucket, and Di represent the historical length of the i-th tooth in the bucket. is the actual length of the i-th tooth inside the bucket; n is the time point;

[0128] The calculation of average wear and wear error rate based on the wear of each bucket tooth specifically includes:

[0129] The expression for the average wear amount is as follows:

[0130]

[0131] In the formula: The average wear rate is given by ; k is the number of bucket teeth; i = 1, 2, ..., k; optionally, the number of bucket teeth is five.

[0132] The expression for the wear error rate is as follows:

[0133]

[0134] This formula is used to measure the dispersion of wear on each bucket tooth; the smaller the standard deviation, the more uniform the wear.

[0135]

[0136] The smaller the wear error rate calculated by this formula, the more in line with the normal wear pattern of the entire set of bucket teeth.

[0137] In one embodiment, the present invention includes step so5: wear pattern verification:

[0138] The determination of whether the wear of each bucket tooth conforms to the wear pattern specifically includes:

[0139] The wear error rate is compared with a first preset threshold. If the wear error rate is less than the first preset threshold, it is determined that the wear of each bucket tooth conforms to the wear pattern; if the wear error rate is not less than the first preset threshold, it is determined that the wear of each bucket tooth does not conform to the wear pattern.

[0140] Specifically, when the wear error rate is less than the first preset threshold, the difference between the wear amount of each bucket tooth and the average wear amount is calculated based on the wear amount of each bucket tooth and the average wear amount. The difference between the wear amount of each bucket tooth and the average wear amount is then compared with the second preset threshold. If the difference between the wear amount of each bucket tooth and the average wear amount is less than the second preset threshold, it is determined that the wear of each bucket tooth conforms to the wear pattern. If the difference between the wear amount of each bucket tooth and the average wear amount is not less than the second preset threshold, it is determined that the wear of each bucket tooth does not conform to the wear pattern.

[0141] Specifically, if the wear of one or more bucket teeth is significantly greater or less than the average wear, it may be due to improper installation location, impact from special external forces, or quality problems. This indicates abnormal wear and requires further inspection and analysis. The excavator trajectory control process ends, and an alarm is triggered to alert the operator of the abnormal information.

[0142] In one embodiment, the present invention includes SO6: comparison of bucket tooth errors:

[0143] When the wear of the entire set of bucket teeth conforms to the normal pattern, the bucket tooth error can be compared; by analyzing the bucket tooth model data from the factory or historical storage and the changes in the bucket tooth size in the actual physical coordinate system, the magnitude of the excavation trajectory error can be determined.

[0144] For example, using cm as a uniform unit of measurement, the average wear amount is compared with a preset range. If the average wear amount is greater than the preset range, the length parameter of the bucket teeth in the control system is corrected according to the wear amount of each bucket tooth; otherwise, the length parameter of the bucket teeth in the control system is not corrected.

[0145] In one embodiment, the present invention includes step so7: tooth parameter correction:

[0146] Once the calculated wear error is confirmed, the stored mechanical dimension parameters of the bucket teeth are automatically corrected based on the error value caused by the wear of the bucket teeth, and the stored codes of the inherent tooth tip mechanical dimensions are also corrected.

[0147] For example, if the length of the bucket teeth wears down by 5cm, the stored length parameter of the bucket teeth will be automatically reduced by 5cm according to the preset correction rules to avoid a decrease in operational accuracy due to the accumulation of errors.

[0148] In one embodiment, the present invention includes step so8: storing bucket tooth parameters and model;

[0149] After the bucket tooth parameters are corrected, the necessary parameters, including the effective length D of the bucket teeth, are updated. The updated parameters and models are backed up in a timely manner to prevent damage to important data due to hardware failures, data loss or other unexpected situations, thereby ensuring that the system always uses the most accurate and effective data and models for operation.

[0150] In one embodiment, the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space is calculated based on the attitude signals of the control system and the excavator boom, stick and bucket.

[0151] Specifically, the present invention includes step so0: tooth tip attitude coordinate update:

[0152] Using the corrected tooth tip mechanical dimension code stored in step S08 and the current pose information parsed in step S01, the three-dimensional attitude coordinate calculation module will apply complex mathematical models and algorithms to perform calculations, specifically including: automatically updating and generating a three-dimensional attitude coordinate calculation model for the tooth tip; combining the excavator kinematic model to replan the tooth tip motion trajectory; including constructing corresponding rotation matrices and displacement vectors based on the angles of the boom, stick, and bucket, and calculating the three-dimensional attitude coordinate information of the tooth tip in the actual working space.

[0153] In one embodiment, the trajectory control of the excavator bucket teeth specifically includes:

[0154] The excavator motion control system continuously adjusts the control parameters and adjusts the excavator's motion in real time based on the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space through a closed-loop feedback mechanism, thereby precisely controlling the movement trajectory of the bucket teeth tip.

[0155] The control parameters include digging speed and acceleration; the excavator motion control system includes a model predictive control algorithm.

[0156] Specifically, the present invention includes step so10: excavation trajectory control:

[0157] The calculated three-dimensional attitude coordinates of the actual space of the tooth tip are used as feedback parameters and input into the excavator's motion control system. Based on these precise coordinate information, the motion control system continuously adjusts the control parameters through a closed-loop feedback mechanism, and adjusts the movement of the excavator's working device in real time, thereby precisely controlling the movement trajectory of the tooth tip. This ensures that the excavator can perform digging operations according to the preset high-precision trajectory, regardless of the wear level of the bucket teeth, thus ensuring the stability and efficiency of the operation accuracy.

[0158] Specific control methods include:

[0159] Model Predictive Trajectory Planning and Control: Model predictive control algorithms are used to plan the motion trajectory of the excavator's working device. The model predictive control algorithm can predict the future output of the system based on the current system state and future constraints, and obtain the optimal control input sequence through optimization calculation.

[0160] In this embodiment of the invention, the model predictive control algorithm predicts the position of the tooth tip at multiple future moments based on the real-time acquired information on the location distribution of the object to be excavated, the wear amount of the tooth tip, the position of the tooth tip, the current state of the excavator's working device (such as the angle and speed of each component), and the preset excavation trajectory. Then, it calculates the optimal control quantity of the working device at each moment, such as the extension length and extension speed of the hydraulic cylinder, thereby achieving precise control of the excavation trajectory and ensuring that the excavator operates efficiently according to the predetermined trajectory.

[0161] The core parameters of the model predictive control algorithm are implemented through the following: real-time adjustment of fuzzy logic control parameters: The motion control parameters are adjusted in real time using the fuzzy logic control algorithm. Fuzzy logic control can handle imprecise and fuzzy information, especially during excavation operations, where factors such as excavation resistance and the conditions of the excavated object are often difficult to measure and quantify precisely. In this embodiment, the fuzzy logic control stage combines the acquisition of image recognition data from cameras or LiDAR to identify the excavated object, such as confirming its specific type (sand, clay, rock, etc.). Based on these actual conditions, the speed, acceleration, and other control parameters of the excavator's working device are adjusted in real time. When the excavated object is a hard material such as rock, the excavation resistance increases, and the excavation speed is automatically reduced to prevent damage to the working device due to overload, while ensuring the stability and efficiency of the excavation process. When the excavated object is a loose material such as sand, the excavation speed is appropriately increased to improve operational efficiency.

[0162] This invention achieves real-time online detection of excavator bucket wear using image perception technology, emphasizing its real-time nature and degree of automation. It obtains the actual length of the bucket teeth through images, point clouds, and predicted lengths. Even in the event of camera or lidar data failure, the bucket tooth wear prediction model still provides crucial identification and judgment capabilities. This allows for efficient, real-time, and accurate detection of bucket tooth wear, avoiding potential deviations in detection results and ensuring the excavator's actual working performance.

[0163] Compared with the prior art, the present invention not only provides qualitative analysis of wear, but also quantitative analysis, and automatically corrects the motion model and control trajectory of the bucket based on the calculated error.

[0164] This invention acquires comprehensive and accurate data through multi-source sensor fusion, and combines it with innovative algorithms to monitor the status of the excavator's working device and the wear of the bucket teeth in real time and with precision. This enables high-precision control of the excavation trajectory, ensuring the stability and efficiency of operational accuracy, and improving the quality and efficiency of excavation operations.

[0165] The adaptive wear model and fuzzy logic control algorithm of this invention can automatically and flexibly adjust the control strategy according to different working conditions and operating conditions, so that the excavator can operate stably and efficiently in various complex excavation environments, whether it is hard rock, soft soil or harsh weather conditions.

[0166] Example 2:

[0167] like Figure 4 As shown, the present invention provides an excavator trajectory control system, comprising:

[0168] Attitude signal detection module: used to collect attitude signals of the excavator boom, stick, and bucket in real time;

[0169] Bucket tooth recognition and detection module: used to collect images and point cloud data of each bucket tooth in real time, calculate the actual coordinates of each bucket tooth in the image, and obtain the actual length of each bucket tooth based on the actual coordinates of each bucket tooth in the image;

[0170] Error Calculation and Correction Module: This module includes a calculation module and a correction module. The calculation module obtains the historical length of each bucket tooth, calculates the wear amount of each tooth based on its historical and actual length, and calculates the average wear amount and wear error rate based on the wear amount of each tooth. Based on the wear error rate and the wear amount of each tooth, it determines whether the wear of each tooth conforms to the wear pattern. If all teeth conform to the wear pattern, the correction module is executed; otherwise, the excavator trajectory control process ends. The correction module compares the average wear amount with a preset range. If the average wear amount is greater than the preset range, the tooth length parameter in the control system is corrected based on the wear amount of each tooth; otherwise, the tooth length parameter in the control system is not corrected.

[0171] Teeth tip attitude coordinate update module: Based on the attitude signals of the control system and the excavator boom, stick and bucket, calculate the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space;

[0172] Motion trajectory control module: Used to input the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space as feedback parameters to the excavator motion control system to realize the trajectory control of the excavator bucket teeth.

[0173] Specifically, the attitude signal detection module is a key component of the excavator's intelligent control system. It is mainly responsible for real-time acquisition, processing, and analysis of data signals from attitude sensors, providing high-precision basic data for subsequent image modeling, algorithm execution, and motion control.

[0174] The attitude signal detection module efficiently and accurately captures and processes data signals from attitude sensors on the boom, stick, and bucket. At the software level, the module first decodes and verifies the angle change and angular velocity signals from these sensors to ensure data integrity and accuracy. Subsequently, the module uses advanced filtering and smoothing algorithms to preprocess the raw angle change and angular velocity signal data to eliminate noise interference and improve data reliability. Based on this, the module further analyzes the processed data to extract the precise angle values ​​of the boom, stick, and bucket, as well as their relative positional relationships.

[0175] In addition, the digging resistance signal detection is provided by pressure sensors installed on the hydraulic oil lines of the boom, stick, and bucket. At the software level, the pressure sensor data first decodes and verifies the hydraulic oil pressure signals from the boom, stick, and bucket to ensure the integrity and accuracy of the data. Subsequently, the module uses advanced filtering algorithms to eliminate noise interference and improve the reliability of the pressure data. Based on this, the module further analyzes the processed data to extract the comprehensive digging resistance data of the boom, stick, and bucket.

[0176] This processed data not only provides accurate pose parameters for subsequent image modeling, but also becomes an indispensable foundation for algorithm execution. In image modeling, the data from the pose signal detection module is used to construct 3D models, enabling real-time rendering and visualization of the work scene. At the algorithm execution level, this data serves as input parameters, supporting the smooth execution of a series of complex calculations such as path planning, collision detection, and motion control. The software functions of the pose signal detection module not only ensure accurate data capture and processing, but also lay a solid foundation for the stable operation and efficient operation of the entire system.

[0177] Bucket tooth recognition and detection module: Based on the fusion of data from camera and lidar sensing signal system, it can dynamically identify feature points of bucket teeth in images, such as the apex and edge points of bucket teeth, and detect and analyze the wear data of each bucket tooth;

[0178] The system collects real-time images and point cloud data of the bucket teeth and compares them with a standard model to calculate the wear amount of the bucket teeth in various dimensions, and then analyzes the error caused by the wear of the bucket teeth. For example, by comparing the position change data of the bucket tooth apex in the image and combining the image calibration parameters, the wear error of the bucket teeth in the X, Y and Z directions can be calculated.

[0179] The system acquires images and point cloud data of the bucket teeth in real time. Through image processing, it automatically identifies various feature points of the bucket teeth. By calculating the actual distance and shooting angle between the camera or lidar and the bucket teeth, and using geometric transformations and projection relationships, it corrects and confirms the size of the bucket teeth in the image, obtaining accurate appearance data. By analyzing stored bucket tooth models with factory dimensions, it establishes a mapping relationship between the image coordinate system and the actual physical coordinate system to complete image calibration. Based on pixel changes in the image, the system calculates the actual size changes and positional offsets of the bucket teeth. Furthermore, by comparing the positional changes of key feature points such as the apex of the bucket teeth in the image, combined with parameters obtained from image calibration, the system can accurately calculate the wear error of the bucket teeth in the three orthogonal directions of X, Y, and Z. For example, if the position of the apex of the bucket teeth in the image is shifted to the left by a certain number of pixels relative to the standard image, and considering the actual distance represented by each pixel in the image calibration, the system can calculate the wear amount of the bucket teeth in the Y direction. Similarly, changes in the position of the apex in the front-back or up-down directions in the image reflect the wear in the Y and Z directions, respectively.

[0180] Error Calculation and Correction Module: In the intelligent control system of an excavator, the wear of the tooth tips directly affects the accuracy and efficiency of excavation operations. In order to ensure that the excavator maintains high-precision control during long-term operation, the system needs to monitor the wear of the tooth tips in real time and correct the mechanical dimension code of the tooth tips based on the wear error.

[0181] In the excavator's control system, the memory is used to store the mechanical dimensional parameters of the tooth tips; these parameters include the initial length, width, height, and shape characteristics of the tooth tips; the data in the memory is the basis for the system to correct wear errors; the control system will compare the stored tooth tip mechanical dimensional data, and correct the tooth tip mechanical dimensional code according to the calculated wear error and preset correction rules.

[0182] After receiving the wear error data, the module first calls and reads the corresponding parameters in the memory; then, based on the real-time monitoring of the bucket tooth wear, it precisely adjusts the tooth tip mechanical dimension code in the system according to the modification rules, and outputs the calculation results to the actual attitude coordinate update module accordingly.

[0183] When the error calculation and coordinate correction module receives the wear error data from the image recognition and error calculation module, it will first call and read the corresponding parameters in the memory. The correction rules can be customized according to actual needs. For example, if the bucket teeth wear by 5cm in the X direction, the module will automatically reduce the size code of the tooth tip in the X direction in the system by 5cm according to the preset correction rules to avoid the decrease in operating accuracy due to error accumulation.

[0184] Tooth Tip Attitude Coordinate Update Module: After the error data correction is completed, the control system will call the tooth tip spatial three-dimensional attitude coordinate calculation module within the system. This module uses the corrected tooth tip mechanical dimension code, combined with the excavator's current posture information (such as the angles of the boom, stick, and bucket), to calculate the three-dimensional attitude coordinate information of the tooth tip in the actual working space through complex mathematical models and algorithms. This coordinate information is crucial for the precise control and operation of the excavator. It can not only analyze the real-time position and attitude of the bucket teeth, but also provide data support for advanced functions such as automatic digging and obstacle avoidance.

[0185] The 3D attitude coordinate calculation module first acquires the excavator's current pose information, including key parameters such as the angles of the boom, stick, and bucket. Then, using the corrected tooth tip mechanical dimension code and the current pose information, the 3D attitude coordinate calculation module applies complex mathematical models and algorithms to perform calculations. This includes constructing corresponding rotation matrices and displacement vectors based on the angles of the boom, stick, and bucket. The system will calculate the 3D attitude coordinate information of the tooth tip in the actual working space. This coordinate information is usually represented in the form of (X,Y,Z), which reflects the position and orientation of the tooth tip in 3D space.

[0186] The system analyzes the current position and attitude of the bucket teeth in real time, ensuring the stability and accuracy of the excavator during operation. It also provides data support for advanced functions such as automatic digging and obstacle avoidance. This not only improves the excavator's operating accuracy and efficiency but also reduces operating errors and failure rates caused by wear, providing a strong guarantee for the excavator's efficient and safe operation in complex environments.

[0187] Motion trajectory control module: The calculated three-dimensional attitude coordinates of the tooth tip are sent to the excavator's motion trajectory control module. Based on this coordinate information, the motion control module adjusts the extension and retraction of the boom, stick, and bucket cylinders by controlling the solenoid valves of the hydraulic system, thereby changing the attitude of the excavator's working device to ensure that the tooth tip's motion trajectory remains consistent with the preset trajectory. For example, when the tooth tip is detected to deviate from the preset trajectory by 10cm in the Y direction, the motion controller controls the corresponding cylinder to move the tooth tip 10cm in the Y direction, returning it to the preset trajectory.

[0188] The motion trajectory control module is the core module in the automated and intelligent operation of excavators. It is responsible for accurately transmitting the actual three-dimensional attitude coordinates of the tooth tip output by the error calculation and coordinate correction module to the excavator's motion controller. This transmission process not only requires the accuracy and timeliness of the data, but also needs to ensure that the motion controller can respond to these coordinate information quickly and accurately in order to achieve precise operation of the excavator.

[0189] After receiving the actual three-dimensional spatial posture coordinates of the tooth tip, the motion controller will immediately begin to analyze and process this information. It will first calculate the motion path and action sequence required to reach the target position based on the excavator's current posture, the configuration of the working device, and the constraints of the working environment. This process involves complex kinematic and dynamic calculations to ensure that the excavator can complete the task with the optimal posture and path.

[0190] After calculating the required motion path, the motion controller further adjusts the extension and retraction of the boom, stick, and bucket cylinders by controlling the solenoid valves of the hydraulic system. The extension and retraction of these cylinders directly determine the posture and position of the excavator's working device, which is the key to the excavator's precise operation. The motion controller will precisely control the opening and closing time of the solenoid valves, the current magnitude, and other parameters according to the preset control strategy and algorithm, thereby achieving precise control of the cylinder extension and retraction.

[0191] For example, when the motion trajectory control module detects that the tooth tip deviates from the preset trajectory by 10cm in the Y direction, it will immediately calculate the amount and direction of motion adjustment that need to be adjusted, and control the corresponding hydraulic cylinder action through the motion controller; if it is necessary to move the tooth tip 10cm in the Y direction to return to the preset trajectory, the motion controller will increase or decrease the extension and retraction of the hydraulic cylinder in the Y direction accordingly until the tooth tip is aligned with the preset trajectory again.

[0192] Throughout the motion control process, the motion trajectory control module continuously receives real-time coordinate information from the error calculation and coordinate correction module and performs closed-loop control. The system continuously adjusts the control strategy based on the deviation between the actual position and posture of the tooth tip and the preset trajectory to ensure the excavator's operating accuracy and stability. This closed-loop control mechanism greatly improves the excavator's operating efficiency and quality, enabling the excavator to maintain high precision and stability in complex and ever-changing operating environments.

[0193] In summary, the motion trajectory control module achieves precise control of the excavator's working device by accurately receiving and processing the actual three-dimensional attitude coordinates of the tooth tip, combined with complex kinematic and dynamic calculations, and a precise hydraulic control system.

[0194] This invention also includes a parameter and model storage module: a core component of the excavator's intelligent control system, primarily responsible for storing and managing key parameters and model data related to tooth tip mechanical dimensions, wear error correction rules, and system operation. This module provides fundamental data support for functions such as error calculation and correction, and attitude coordinate updates, ensuring that the system can adjust the tooth tip mechanical dimension code in real time and accurately, thereby maintaining high precision and efficiency in excavation operations.

[0195] The system is primarily responsible for storing various mechanical dimensional parameters of the excavator's tooth tips, including key parameters such as initial length, width, height, and shape characteristics. These raw parameters form the basis for subsequent wear error calculations and corrections, and ensuring the accuracy and completeness of the data is crucial for the precision control of the entire system.

[0196] In addition to parameters, it can also store relevant models for error calculation and correction; for example, preset wear error correction rule models. These models are built based on actual operation requirements and experience, guiding the system on how to accurately adjust the mechanical dimension code according to the wear of the tooth tip, providing strong support for the intelligent operation of the system.

[0197] It also possesses efficient data management capabilities, enabling the classification, indexing, and updating of stored parameters and models. When tooth tip wear occurs, it updates the corresponding parameters based on real-time monitoring data to ensure data timeliness. Simultaneously, when optimizing models or adjusting correction rules, it promptly updates the stored models, ensuring that the system always uses the most accurate and effective data and models for operation. It can conveniently and quickly provide data reading and calling services to other modules. When the error calculation and correction module receives wear error data, it can respond quickly and accurately provide the required tooth tip mechanical dimension parameters and correction rule models, ensuring the smoothness and timeliness of data interaction between modules. This is a key link in achieving system collaborative work.

[0198] It can also regularly back up stored parameters and models to prevent damage to important data due to hardware failures, data loss, or other unexpected situations. In the event of system anomalies, it can quickly restore data, enabling the excavator's intelligent control system to rapidly return to normal operation, reducing downtime and ensuring the continuity and stability of excavation operations.

[0199] Example 3:

[0200] like Figure 5 As shown, the present invention provides an excavator, characterized in that it includes the excavator trajectory control system described in Embodiment 2.

[0201] Specifically, to ensure the real-time performance and accuracy of image acquisition, processing, storage, and control, the excavator of this invention is equipped with a controller, sensors, a camera or lidar, and a memory.

[0202] Sensors: Attitude sensors are fixedly mounted on the boom, stick, and bucket assembly, and connected to the controller via data cables. They transmit received angle change data signals to the controller. When the boom, stick, or bucket undergoes any angle change, the sensors immediately capture these changes and convert them into corresponding data signals. In addition, digging resistance signals are detected by pressure sensors installed on the hydraulic lines of the boom, stick, and bucket assembly. These data signals are transmitted to the controller via data cables. The controller, as the core of data processing, can receive and process these data signals from the attitude sensors in real time, thus providing basic data for subsequent image modeling, algorithm execution, etc.

[0203] Cameras or LiDAR: Cameras and / or LiDAR are installed on the top of the excavator's cab, boom, or stick to clearly capture the position and movement of the bucket teeth from different angles. The installation positions of the cameras or LiDAR need to be precisely adjusted to ensure that the captured images of the bucket teeth accurately reflect their actual state. An appropriate image acquisition frequency is set, such as acquiring images every N seconds, to meet the needs of real-time detection. By acquiring image recognition data from cameras or LiDAR, the excavated objects can also be identified, such as different types of sand, clay, and rock.

[0204] The controller, installed inside the excavator system, is responsible for processing image data and point cloud modeling, training deep learning algorithms and models, analyzing and calculating bucket tooth wear, solving the spatial attitude coordinates of the tooth tips, and planning and controlling the tooth tip motion trajectory. It executes core modules such as these: 1) Processing image data and point cloud modeling; 2) Analyzing and calculating bucket tooth wear; 3) Determining the wear level of the excavator's bucket teeth; 4) Using advanced algorithms to accurately calculate the spatial attitude coordinates of the tooth tips, thus determining their specific position in three-dimensional space. Based on this information, the controller plans the tooth tip's motion trajectory and, through precise control algorithms, achieves coordinated control of the excavator's boom, stick, and bucket. The excavator can then operate according to the preset motion trajectory and force, ensuring the accuracy and efficiency of the construction work.

[0205] Memory: As an important component of the controller, it is integrated inside the controller. It is mainly responsible for key tasks such as storing the excavator's factory mechanical dimensions and the overall machine modeling data. When the excavator leaves the factory, its various mechanical dimensions and parameters are accurately measured, confirmed, and stored in the memory. This stored information is crucial for image modeling and algorithm execution, providing the basic data for the digital representation of the excavator. The memory also stores the overall machine modeling data, including the excavator's 3D model, the structural parameters of each part, and their relative positional relationships. When the controller performs image modeling and algorithm execution, it frequently accesses and calls this data to ensure that the excavator's intelligent target precision control tasks can be carried out smoothly.

[0206] In summary, cameras or lidar, attitude sensors, controllers, and memory together constitute the core system of intelligent excavator operation. They work together in coordination to achieve high-precision and high-efficiency operation of the excavator.

[0207] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0208] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0209] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0210] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0211] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for controlling the trajectory of an excavator, characterized in that, Includes the following steps: Step a: Real-time acquisition of attitude signals of the excavator boom, stick, and bucket; Step b: Obtain the predicted length of each bucket tooth, collect images and point cloud data of each bucket tooth in real time, calculate the actual coordinates of each bucket tooth in the image, and obtain the actual length of each bucket tooth based on the actual coordinates of each bucket tooth in the image and the predicted length of each bucket tooth. Step c: Obtain the historical length of each bucket tooth, calculate the wear amount of each bucket tooth based on the historical length and actual length, and calculate the average wear amount and wear error rate based on the wear amount of each bucket tooth. Step d: Based on the wear error rate and the wear amount of each bucket tooth, determine whether the wear of each bucket tooth conforms to the wear pattern. If each bucket tooth conforms to the wear pattern, proceed to step e. Otherwise, the excavator trajectory control process ends; Step e: Compare the average wear amount with the preset range. If the average wear amount is greater than the preset range, then adjust the bucket tooth length parameter in the control system according to the wear amount of each bucket tooth; otherwise, do not adjust the bucket tooth length parameter in the control system. Step f: Calculate the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space based on the attitude signals of the control system and the excavator boom, stick and bucket; Step g: Input the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space as feedback parameters into the excavator motion control system to realize the trajectory control of the excavator bucket teeth.

2. The excavator trajectory control method according to claim 1, characterized in that, The calculation of the actual coordinates of each bucket tooth in the image specifically includes: Based on the images and point cloud data of each bucket tooth, identify the feature points of each bucket tooth; The dimensions of each bucket tooth in the image are corrected by geometric transformation and projection relationship to obtain the appearance dimensions of each bucket tooth; Based on the feature points and external dimensions of each bucket tooth, and in conjunction with the calibration parameters, the actual coordinates of each bucket tooth in the image are obtained.

3. The excavator trajectory control method according to claim 1, characterized in that, The step of obtaining the actual length of each bucket tooth based on its actual coordinates in the image specifically includes: Obtain the actual coordinates of the fixing pins of each bucket tooth; Based on the actual coordinates of each bucket tooth in the image, obtain the actual coordinates of each bucket tooth tip in the image; The detection length of each bucket tooth is calculated based on the actual coordinates of the fixing pins of each bucket tooth and the actual coordinates of the tips of each bucket tooth in the image. The specific expression is as follows: ， In the formula: Let be the detection length of the i-th tooth inside the bucket; i is the number of the tooth. At time t; the actual coordinates of the fixed pin of the i-th tooth inside the bucket are ( , , The actual coordinates of the tip of the i-th tooth in the bucket in the image are ( , , ); Let be the coordinates of the front-rear direction of the fixed pin of the i-th tooth inside the bucket; Let be the coordinates of the left and right directions of the fixed pin of the i-th tooth inside the bucket; Let be the coordinates of the fixed pin shaft of the i-th tooth inside the bucket in the vertical direction; Let be the coordinates of the tip of the i-th tooth in the bucket in the front-back direction in the image; Let be the coordinates of the tip of the i-th tooth in the bucket in the left-right direction in the image; Let be the vertical coordinates of the tip of the i-th tooth in the bucket in the image; The process of obtaining the predicted length of each bucket tooth specifically includes: The detection parameters of each bucket tooth are obtained, and the detection parameters of each bucket tooth are input into the pre-built and trained bucket tooth wear adaptive prediction model to output the predicted length of each bucket tooth. Based on the detected length of each bucket tooth and the corresponding predicted length of each bucket tooth, the actual length of each bucket tooth is calculated, as shown in the following expression: ， ， In the formula: Let w1 be the actual length of the i-th tooth inside the bucket; w2 are the weights. Let be the predicted length of the i-th tooth inside the bucket.

4. The excavator trajectory control method according to claim 3, characterized in that, The detection parameters for each bucket tooth include the bucket tooth material, bucket tooth rotation speed, bucket tooth rotation acceleration, bucket tooth digging resistance, cumulative bucket tooth installation time, and target category attribute parameters, among which: The predicted length of each bucket tooth is output, and the specific expression is as follows: ， In the formula: This is a prediction function based on multiple factors; Made of bucket teeth material; Let be the rotational speed of the i-th tooth inside the bucket; Let be the rotational acceleration of the i-th tooth inside the bucket; Let be the digging resistance of the i-th tooth inside the bucket; The cumulative installation time for the i-th tooth in the bucket; To extract target category attribute parameters.

5. The excavator trajectory control method according to claim 1, characterized in that, The calculation of the wear amount of each bucket tooth based on its historical length and actual length specifically includes: The wear amount of the i-th tooth inside the bucket is calculated using the following expression: ， In the formula, Let represent the wear amount of the i-th tooth in the bucket, and Di represent the historical length of the i-th tooth in the bucket. is the actual length of the i-th tooth in the bucket; n is the time; i is the number of the tooth. The calculation of average wear and wear error rate based on the wear of each bucket tooth specifically includes: The expression for the average wear amount is as follows: ， In the formula: The average wear rate is given by i; k is the number of teeth in the bucket; i = 1, 2, ..., k. The expression for the wear error rate is as follows: ， 。 6. The excavator trajectory control method according to claim 1, characterized in that, The determination of whether the wear of each bucket tooth conforms to the wear pattern specifically includes: The wear error rate is compared with a first preset threshold. If the wear error rate is less than the first preset threshold, it is determined that the wear of each bucket tooth conforms to the wear pattern; if the wear error rate is not less than the first preset threshold, it is determined that the wear of each bucket tooth does not conform to the wear pattern. Specifically, when the wear error rate is less than the first preset threshold, the difference between the wear amount of each bucket tooth and the average wear amount is calculated based on the wear amount of each bucket tooth and the average wear amount. The difference between the wear amount of each bucket tooth and the average wear amount is then compared with the second preset threshold. If the difference between the wear amount of each bucket tooth and the average wear amount is less than the second preset threshold, it is determined that the wear of each bucket tooth conforms to the wear pattern. If the difference between the wear amount of each bucket tooth and the average wear amount is not less than the second preset threshold, it is determined that the wear of each bucket tooth does not conform to the wear pattern.

7. The excavator trajectory control method according to claim 1, characterized in that, The trajectory control of the excavator bucket teeth specifically includes: The excavator motion control system continuously adjusts the control parameters based on the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space through a closed-loop feedback mechanism, and adjusts the excavator's motion in real time according to the control parameters, thereby precisely controlling the movement trajectory of the bucket teeth tip. The control parameters include digging speed and acceleration; the excavator motion control system includes a model predictive control algorithm.

8. An excavator trajectory control system, characterized in that, include: Attitude signal detection module: used to collect attitude signals of the excavator boom, stick, and bucket in real time; Bucket tooth recognition and detection module: used to collect images and point cloud data of each bucket tooth in real time, calculate the actual coordinates of each bucket tooth in the image, and obtain the actual length of each bucket tooth based on the actual coordinates of each bucket tooth in the image; Error Calculation and Correction Module: This module includes a calculation module and a correction module. The calculation module is used to obtain the historical length of each bucket tooth, calculate the wear amount of each bucket tooth based on the historical length and the actual length, calculate the average wear amount and wear error rate based on the wear amount of each bucket tooth, and determine whether the wear of each bucket tooth conforms to the wear pattern based on the wear error rate and the wear amount of each bucket tooth. If all bucket teeth conform to the wear pattern, the correction module is executed. Otherwise, the excavator trajectory control process ends: The correction module is used to compare the average wear amount with the preset range. If the average wear amount is greater than the preset range, the bucket tooth length parameter in the control system is corrected according to the wear amount of each bucket tooth; otherwise, the bucket tooth length parameter in the control system is not corrected. Teeth tip attitude coordinate update module: Based on the attitude signals of the control system and the excavator boom, stick and bucket, calculate the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space; Motion trajectory control module: Used to input the three-dimensional attitude coordinate information of the bucket teeth tip in the actual working space as feedback parameters to the excavator motion control system to realize the trajectory control of the excavator bucket teeth.

9. The excavator trajectory control system according to claim 8, characterized in that, It also includes a parameter and model storage module: used to store and manage key parameters and data related to the error calculation and correction module and the excavator motion control system; The bucket tooth recognition and detection module is also used to identify the feature points of each bucket tooth based on the image and point cloud data of each bucket tooth; correct the size of each bucket tooth in the image through geometric transformation and projection relationship to obtain the appearance size of each bucket tooth; and obtain the actual coordinates of each bucket tooth in the image based on the feature points and appearance size of each bucket tooth, combined with the calibration parameters. The motion trajectory control module is used to receive the three-dimensional attitude coordinate information of the bucket tooth tip in the actual working space in real time and send the three-dimensional attitude coordinate information of the bucket tooth tip in the actual working space to the excavator motion control system. The excavator motion control system is used to adjust the attitude of the excavator boom, stick and bucket according to the three-dimensional attitude coordinate of the bucket tooth tip in the actual working space so that the trajectory of the bucket tooth tip is consistent with the preset trajectory. The excavator motion control system calculates a preset trajectory based on the posture of the excavator boom, stick, and bucket, the target position, and the constraints of the working environment.

10. An excavator, characterized in that, Includes the excavator trajectory control system as described in any one of claims 8 to 9.

Images