An automatic calibration system and calibration method for an excavator

Abstract

The present invention discloses an automatic calibration system and a calibration method for an excavator. The system is applied to a sensor platform including a camera and a lidar, and includes: a bucket installation position detection module for detecting and judging whether there is a deviation in the bucket installation position; a bucket state recognition module for recognizing the state of the bucket; an inclination sensor angle data inspection module for inspecting the angle data of the inclination sensor; and a boom and arm state detection module for judging whether the boom and the arm are deformed through inspection. The present invention can inspect the working device of the excavator, judge whether deformation occurs, and at the same time can also judge whether the information obtained by the inclination sensor during work is reliable.

Description

Technical Field

[0001] The present invention relates to an automatic calibration system and a calibration method for an excavator, belonging to the technical field of environmental perception. Background Art

[0002] An excavator is one of the most typical, complex and widely used construction machinery. It plays an extremely important role in construction projects such as industrial and civil buildings, transportation, water conservancy and electric power projects, mining, and military projects. Generally, an excavator measures the attitude angles of the bucket, arm, boom and vehicle body through inclination sensors to achieve three-dimensional attitude control of the excavator. This requires very high working precision of the excavator. If the working device of the excavator deforms during operation, or the data obtained by the inclination sensor is deviated, it will lead to a decrease in the working precision of the excavator, which is not allowed in a working environment with certain precision requirements. Therefore, it is necessary to detect the working device of the excavator and the inclination sensors used to ensure its relatively reliable operation. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies in the prior art and provide an automatic calibration system and a calibration method for an excavator, which can check the working device of the excavator to determine whether deformation occurs, and can also determine whether the information obtained by the inclination sensor during operation is reliable.

[0004] To achieve the above purpose, the present invention is implemented by the following technical solutions:

[0005] In a first aspect, the present invention provides an automatic calibration system for an excavator, which is applied to a sensor platform including a camera and a lidar, and includes:

[0006] A bucket installation position detection module, configured to detect and determine whether there is a deviation in the bucket installation position;

[0007] A bucket state recognition module, configured to recognize the state of the bucket;

[0008] An inclination sensor angle data inspection module, configured to inspect the angle data of the inclination sensor;

[0009] An arm and boom state detection module, configured to determine whether the arm and the boom are deformed through inspection.

[0010] Further, it includes:

[0011] A human-machine interaction system, which is provided with a human-machine interaction module and a display module, wherein:

[0012] A human-computer interaction module, which is used to implement human-computer interaction and control the bucket installation position detection module, the bucket state recognition module, the inclination sensor angle data verification module, and the arm and boom state detection module;

[0013] A display module, which is used to display image information or the point cloud information of the radar.

[0014] In a second aspect, the present invention provides a calibration method for an excavator automatic calibration system according to any one of the foregoing, including:

[0015] The detection method of the bucket installation position detection module includes:

[0016] Obtain the pre-collected image information between the bucket and the arm of the excavator;

[0017] Perform edge detection on the image information between the bucket and the arm, and judge whether there is a deviation in the bucket installation position according to the detection result;

[0018] The recognition method of the bucket state recognition module includes:

[0019] Obtain the pre-collected bucket image information, input it into a pre-constructed and trained neural network model, and obtain the bucket state recognition result;

[0020] The detection method of the inclination sensor angle data verification module includes:

[0021] Detect and obtain the point cloud information of the arm and the boom of the excavator through lidar, and calculate the first included angle value between the arm and the boom;

[0022] Obtain the inclination angles of the arm and the boom through the inclination sensor, and calculate the second included angle value between the arm and the boom;

[0023] Compare the first included angle value and the second included angle value to judge whether it is necessary to check the inclination sensor;

[0024] The detection method of the arm and boom state detection module includes:

[0025] Detect and obtain the point cloud information of the arm and the boom of the excavator through lidar, fit the contours of the arm and the boom into two parallel straight lines, and judge whether the arm and the boom are deformed by judging the slopes of the two straight lines.

[0026] Further, after preprocessing the image information between the bucket and the arm, edge detection is performed. Among them, the preprocessing includes grayscale processing, blurring processing, obtaining the region of interest, and extracting features through Hough transform.

[0027] Further, perform edge detection on the image information between the bucket and the dipper arm, and judge whether there is a deviation in the installation position of the bucket according to the detection result, including:

[0028] Perform edge detection on the image information between the bucket and the dipper arm, and respectively detect the edge contours on one side of the bucket and the dipper arm;

[0029] According to the edge contours on one side of the bucket and the dipper arm, respectively fit two straight lines to obtain the slopes of the two straight lines;

[0030] Calculate the slope difference between the two straight lines, compare it with the set threshold, and if it is within the threshold range, it can be considered that there is no deviation in the installation position of the bucket, otherwise give a prompt message for adjustment.

[0031] Further, the training method of the neural network model includes:

[0032] Obtain the bucket image information, perform recognition through deep learning methods, and output two states: normal and abnormal;

[0033] Obtain the image data of these two states as the training data set, and input it into the neural network model built based on the deep learning framework for training.

[0034] Further, perform augmentation processing on the training data set using the image augmentation method.

[0035] Further, use an optimization algorithm to quickly train the neural network model to obtain appropriate model parameters.

[0036] Further, obtain the point cloud information of the dipper arm and the boom of the excavator through lidar detection, and calculate the first included angle value between the dipper arm and the boom; obtain the inclination angles of the dipper arm and the boom through an inclination sensor, and calculate and obtain the second included angle value between the dipper arm and the boom; compare the first included angle value and the second included angle value to judge whether it is necessary to check the inclination sensor, including:

[0037] Obtain the point cloud information of the dipper arm and the boom of the excavator through lidar detection, and obtain the three-dimensional coordinate information of the special points by tracking three special points on the dipper arm and the boom;

[0038] Calculate the first included angle value between the dipper arm and the boom according to the included angle formula of two spatial vectors;

[0039] Obtain the inclination angles of the dipper arm and the boom through an inclination sensor, and obtain the second included angle value between the dipper arm and the boom according to the triangle interior angle sum theorem;

[0040] Compare the first included angle value and the second included angle value, and judge whether it is necessary to check the inclination sensor according to the magnitude of the numerical deviation of the comparison.

[0041] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0042] The present invention provides an automatic calibration system and a calibration method for an excavator. By detecting the installation position of the bucket, identifying the state of the bucket, verifying the angle data of the inclination sensor, and detecting the states of the boom and the arm, it is possible to inspect the working device of the excavator, determine whether deformation has occurred, and also determine whether the information obtained by the inclination sensor during operation is reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a diagram of the human-machine interaction system interface provided by an embodiment of the present invention;

[0044] Figure 2 is a camera perception flow chart provided by an embodiment of the present invention;

[0045] Figure 3 is a bucket state identification flow chart provided by an embodiment of the present invention;

[0046] Figure 4 is a radar perception flow chart provided by an embodiment of the present invention;

[0047] Figure 5 is a schematic diagram of finding the included angle of spatial vectors provided by an embodiment of the present invention;

[0048] Figure 6 is a flow chart for detecting the states of the boom and the arm provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be used to limit the protection scope of the present invention.

[0050] Embodiment 1

[0051] This embodiment introduces an automatic calibration system for an excavator, including: a sensor platform applied to include a camera and a lidar, including:

[0052] A bucket installation position detection module for detecting and determining whether the installation position of the bucket is deviated;

[0053] A bucket state identification module for identifying the state of the bucket;

[0054] An inclination sensor angle data verification module for verifying the angle data of the inclination sensor;

[0055] A boom and arm state detection module for detecting and determining whether the boom and the arm are deformed through inspection.

[0056] A human - machine interaction system, in which a human - machine interaction module and a display module are provided, where:

[0057] The human - machine interaction module is used to implement human - machine interaction and control the bucket installation position detection module, bucket state recognition module, inclination sensor angle data verification module, and arm and boom state detection module;

[0058] The display module is used to display image information or the point cloud information of the radar.

[0059] Embodiment 2

[0060] This embodiment provides a calibration method for the excavator automatic calibration system according to any one of Embodiment 1, including:

[0061] The detection method of the bucket installation position detection module includes:

[0062] Obtain the pre - collected image information between the excavator bucket and the arm;

[0063] Perform edge detection on the image information between the bucket and the arm, and judge whether there is a deviation in the bucket installation position according to the detection result;

[0064] The recognition method of the bucket state recognition module includes:

[0065] Obtain the pre - collected bucket image information, input it into the pre - constructed and trained neural network model, and obtain the bucket state recognition result;

[0066] The detection method of the inclination sensor angle data verification module includes:

[0067] Detect and obtain the point cloud information of the excavator arm and boom through lidar, and calculate the first included - angle value between the arm and the boom;

[0068] Obtain the inclination angles of the arm and the boom through the inclination sensor, and calculate the second included - angle value between the arm and the boom;

[0069] Compare the first included - angle value and the second included - angle value to judge whether it is necessary to check the inclination sensor;

[0070] The detection method of the arm and boom state detection module includes:

[0071] Detect and obtain the point cloud information of the excavator arm and boom through lidar, fit the contours of the arm and the boom into two parallel straight lines, and judge whether the arm and the boom are deformed by judging the slopes of the two straight lines.

[0072] The calibration method of the excavator automatic calibration system provided by this embodiment specifically involves the following steps in its application process:

[0073] (1) First, build a sensor platform including a camera and a lidar on the excavator.

[0074] (2) Create a human-computer interaction system, as Figure 1 shown. When the excavator starts working, first open this interaction interface to check the working device and the inclination sensor. The process is as follows: Run the human-computer interaction program, the usage interface appears, click the welcome button to enter the operation interface, where the display area shows image information or the point cloud information of the radar. The camera static / real-time function is used for the recognition of the bucket. When the camera senses that there is a deviation in the installation of the bucket or deformation occurs during work, the sensed information will be displayed at the warning information; the radar static / real-time is used to detect whether deformation occurs to the stick and the boom during work, and at the same time can also detect whether there is an error in the data of the inclination sensor, and the sensed information will also be displayed at the warning information.

[0075] (3) Bucket installation detection technical route (camera sensing): The camera can capture the image information in front of the excavator. Before the excavator works or after replacing the bucket, according to the image information between the bucket and the stick captured by the camera, use relevant algorithms to perform a series of preprocessing operations on it, and then perform edge detection on it, respectively detect the edge contours on one side of the bucket and the stick, then fit the corresponding straight lines, so as to obtain the slopes of the two straight lines, calculate the slope difference, and compare it with the set threshold. If it meets the threshold range, it can be considered that the installation position of the bucket has no deviation, otherwise, a prompt message is given for adjustment. The specific implementation method is shown in the appendix Figure 2 ;

[0076] (4) Bucket state recognition: Use the method of deep learning to recognize the bucket, and output two states: normal and abnormal. First, it is necessary to obtain the image data of these two states as the training data set, and then use the deep learning framework to build a neural network model and train it. The specific implementation method is shown in the appendix Figure 3 ;

[0077] (5) Inclination Sensor Angle Data Inspection Technical Route (Radar Sensing): The lidar can detect the point cloud information of the excavator's stick and boom. By tracking three special points on the stick and boom and obtaining the three-dimensional coordinate information of the special points, the angle between the stick and the boom can be calculated according to the angle formula of two vectors in space. The inclination sensor can directly obtain the inclination angles of the stick and the boom, and the angle between the stick and the boom can be directly obtained according to the triangle interior angle sum theorem. Compare the angles obtained by these two methods. If there is a large deviation between the two values, the inclination sensor needs to be inspected. If the difference between the two is very small, it means that the data accuracy of the inclination sensor is relatively high, and it can also be used in combination with the lidar. The specific implementation method is shown in the appendix Figure 4 , where the method for calculating the angle is shown in the appendix Figure 5 .

[0078] (6) Stick and Boom State Detection: (5) mentioned that the lidar can detect the point cloud information of the excavator's stick and boom. Then, the contours of the stick and the boom can be fitted into two parallel lines because the stick and the boom are in the same plane. By judging the slopes of the two lines, it can be determined whether the stick and the boom have deformed. The specific implementation method is shown in the appendix Figure 6 .

[0079] The present invention can inspect the working device of the excavator, determine whether deformation has occurred, and at the same time can also determine whether the information obtained by the inclination sensor during operation is reliable.

[0080] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can still be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.

Claims

1. A calibration method for an automatic calibration system of an excavator, characterized in that, it includes: Obtain the pre - collected image information between the excavator bucket and the arm; Perform edge detection on the image information between the bucket and the arm, and judge whether there is a deviation in the installation position of the bucket according to the detection result; Obtain the pre - collected bucket image information, input it into the pre - constructed and trained neural network model, and obtain the bucket state recognition result; Detect and obtain the point cloud information of the excavator arm and the boom through lidar, and calculate the first included angle value between the arm and the boom; Obtain the inclination angles of the arm and the boom through an inclination sensor, and calculate the second included angle value between the arm and the boom; Compare the first included angle value and the second included angle value to judge whether it is necessary to check the inclination sensor; Detect and obtain the point cloud information of the excavator arm and the boom through lidar, fit the contours of the arm and the boom into two parallel straight lines, and judge whether the arm and the boom have deformed by judging the slopes of the two straight lines.

2. The calibration method for the automatic calibration system of an excavator according to claim 1, characterized in that, After pre - processing the image information between the bucket and the arm, perform edge detection, wherein the pre - processing includes grayscale processing, blurring processing, obtaining the region of interest, and extracting features through Hough transform.

3. The calibration method for the automatic calibration system of an excavator according to claim 2, characterized in that, Performing edge detection on the image information between the bucket and the arm, and judging whether there is a deviation in the installation position of the bucket according to the detection result includes: Perform edge detection on the image information between the bucket and the arm, and respectively detect the edge contours on one side of the bucket and the arm; According to the edge contours on one side of the bucket and the arm, respectively fit two straight lines to obtain the slopes of the two straight lines; Calculate the slope difference between the two straight lines, compare it with the set threshold, and if it meets the threshold range, it can be considered that there is no deviation in the installation position of the bucket, otherwise give a prompt message for adjustment.

4. The calibration method for the automatic calibration system of an excavator according to claim 1, characterized in that, The training method of the neural network model includes: Obtain the bucket image information, perform recognition through deep learning methods, and output two states: normal and abnormal; Obtain the image data of these two states as the training data set, and input it into the neural network model built based on the deep learning framework for training.

5. The calibration method for the automatic calibration system of an excavator according to claim 4, characterized in that, Perform augmentation processing on the training data set using the image augmentation method.

6. The calibration method for the automatic calibration system of an excavator according to claim 4, characterized in that, Use an optimization algorithm to quickly train the neural network model to obtain appropriate model parameters.

7. The calibration method for the automatic calibration system of an excavator according to claim 1, characterized in that, The point cloud information of the dipper arm and the boom of the excavator is obtained through lidar detection, and the first included angle value between the dipper arm and the boom is calculated; the inclination angles of the dipper arm and the boom are obtained through an inclination sensor, and the second included angle value between the dipper arm and the boom is calculated and obtained; The first included angle value and the second included angle value are compared to determine whether it is necessary to check the inclination sensor, including: The point cloud information of the dipper arm and the boom of the excavator is obtained through lidar detection, and the three-dimensional coordinate information of the special points is obtained by tracking three special points on the dipper arm and the boom; The first included angle value between the dipper arm and the boom is calculated according to the included angle formula of two vectors in space; The inclination angles of the dipper arm and the boom are obtained through an inclination sensor, and the second included angle value between the dipper arm and the boom is obtained according to the triangle interior angle sum theorem; The first included angle value and the second included angle value are compared, and whether it is necessary to check the inclination sensor is judged according to the magnitude of the numerical deviation of the comparison.

Images