Drilling edge burr detection method and system based on TOF camera

Abstract

The invention discloses a method and system for detecting burrs on the edge of a drilling hole based on a TOF camera, and the invention relates to the technical field of burr detection. The method comprises the following steps: using a calibrated TOF camera and a thermal infrared camera to collect images of the edge of the drilling hole at the same angle, and preprocessing the depth image and the thermal infrared image respectively; converting the pixel points in the depth image into point cloud data, and calculating the curvature of each point; setting a curvature threshold, identifying points whose curvature exceeds the curvature threshold, and marking the area where the points with high curvature values ​​are located, and recording it as a high curvature value area; marking the area where the temperature exceeds the set temperature threshold through the infrared image, and recording it as a temperature abnormality area; establishing an image recognition model, marking the burr contour, inputting the high curvature value area and the temperature abnormality area in the depth image into the recognition model, marking the potential burrs, and obtaining the edge features of the potential burrs. If the burr feature requirements are met, the potential burrs are marked as burrs, and their locations are output.

Description

Technical Field

[0001] The present invention relates to the technical field of burr detection, and in particular to a method and system for detecting burrs on the edge of a drill hole based on a TOF camera. Background Art

[0002] In modern industrial production, drilling is a widely used manufacturing process, especially in the fields of aerospace, automobile manufacturing, electronic equipment, etc. The quality of precision drilling directly affects the final performance of the product. However, during the drilling process, burrs often appear on the edges of the holes due to tool wear, improper process parameter settings, etc. Burrs not only affect the quality of subsequent assembly, but may also cause a decrease in the fatigue strength of parts or cause serious safety problems. Therefore, timely and effective detection and removal of burrs has become a key link in industrial production.

[0003] Traditional burr detection methods mainly rely on manual inspection or automatic detection systems based on optical imaging. Manual inspection is time-consuming and labor-intensive, and is highly subjective and easily affected by the experience and status of the inspector. Although the optical imaging detection system improves the detection efficiency, it is difficult to accurately identify the three-dimensional shape of the drilled edge because it only relies on two-dimensional images, especially in the detection of complex curved surfaces and fine burrs. Therefore, how to comprehensively and accurately detect burrs on the edge of the drilled hole has become a technical problem that needs to be solved urgently.

[0004] In the prior art, publication number CN110987959A discloses an online burr detection method, which includes the following steps: a camera distance adjustment step: obtaining data from a distance sensor and calculating the data to obtain distance data between the edge of a sheet material and a camera. According to the data, a correction actuator adjusts the camera to be close to or away from the edge of a sheet material, thereby adjusting the distance from the camera to the edge of the sheet material, and keeping the distance from the camera to the edge of the sheet material unchanged, thereby obtaining a clear image while maintaining the depth of field of the lens; an image acquisition step: controlling a light source so that the light of the light source is irradiated on the edge of the sheet material, and obtaining a clear image of the edge of the sheet material by controlling the camera; a coordinate conversion step: processing the obtained image to convert the pixel coordinates of the image into coordinates of a world coordinate system; an image analysis step: analyzing the image to determine what kind of image in the image has burrs, and determining the size of the burrs, and determining whether the size of the burrs is greater than a preset value. When it is greater than the preset value, it is determined that the sheet material has a burr defect. This method adjusts the distance between the camera and the object to ensure clear imaging. However, it is only based on the optical properties of the two-dimensional image and cannot fully display the morphological characteristics of the burrs. This limitation may lead to insufficient accuracy in burr detection, especially in complex geometric shapes or edge cases. It may face recognition errors or missed detections due to lack of feature information.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The object of the present invention is to provide a method and system for detecting burrs on the edge of a drill hole based on a TOF camera, so as to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A method for detecting burrs on the edge of a drilling hole based on a TOF camera, the specific steps comprising:

[0009] Calibrate the parameters of the TOF camera and the thermal infrared camera, and obtain the camera parameters, wherein the camera parameters include the focal length and the coordinates of the principal point position, use the calibrated TOF camera and the thermal infrared camera to collect images of the edge of the borehole at the same angle, obtain a depth image and a thermal infrared image, respectively, and pre-process the depth image and the thermal infrared image, respectively;

[0010] In the preprocessed depth image, the depth value is extracted according to the pixel value corresponding to each pixel, and the pixel points in the depth image are converted into point cloud data using the depth values ​​of each pixel point obtained. The normal vector of each point in the point cloud data within its neighborhood is calculated, and the curvature of each point is calculated based on the normal vector of each point.

[0011] Set the curvature threshold, identify the points whose curvature exceeds the curvature threshold, mark them as high curvature points, determine their positions in the point cloud data, mark the positions of the high curvature points in the depth image according to the position mapping in the point cloud data, and define a window to identify the area where the points with high curvature values ​​are located, which is recorded as the high curvature value area;

[0012] From the preprocessed thermal infrared image, calculate the temperature mean and standard deviation in the infrared image, set the temperature threshold based on the temperature mean and standard deviation, mark the area where the temperature exceeds the set temperature threshold, record it as the temperature abnormal area, and mark the area where the temperature abnormal area is located in the depth image;

[0013] An image recognition model was established, and several types of burr images of drilling edges were collected. The burr contours in the images were marked manually, and the model was trained using the collected burr images of drilling edges as input and the burr contours as labels.

[0014] Regions with high curvature values ​​and regions with abnormal temperature are extracted from the depth image as regional images, and the regional images are input into the trained image recognition model to identify potential burrs in the regional images and obtain edge features of the potential burrs. If the edge features meet the burr feature requirements, the potential burrs are marked as burrs and their locations are output.

[0015] Furthermore, the depth image and the thermal infrared image are preprocessed respectively, wherein the depth image is preprocessed by Gaussian filtering, and the thermal infrared image is preprocessed by histogram equalization.

[0016] Furthermore, the Zhang Zhengyou calibration method is used to calibrate the TOF and thermal infrared camera parameters to obtain the camera parameters, which include focal length and principal point position coordinates, wherein the focal length includes the focal length fu on the u-axis and the focal length fv in the v-axis direction in the image coordinate system, and the principal point position coordinates are (u0, v0).

[0017] Furthermore, from the preprocessed depth image, the formula for extracting the corresponding depth value according to the pixel value corresponding to each pixel point is:

[0018] z p =h[i][j]

[0019] In the formula, z p represents the depth value of the p-th pixel in the depth image, h[i][j] represents the pixel value corresponding to the i-th row and j-th column, where p is the index of the pixel in the depth image, i represents the row index of the pixel, j represents the row index of the pixel, i=1,2,…,n,j=1,2,…,m, n and m represent the total number of rows and columns of pixels in the depth image respectively, and p=1,2,…,n*m;

[0020] Based on the depth value of the pixel point, the two-dimensional pixel coordinates are converted to three-dimensional world coordinates; the specific formula is as follows:

[0021]

[0022] In the formula, x p and p They represent the x-axis coordinate and y-axis coordinate of the p-th pixel in the world coordinate system, u i and v j They represent the u-axis coordinates of the i-th row of pixels in the image coordinate system and the v-axis coordinates of the j-th row of pixels in the image coordinate system respectively; according to the converted three-dimensional world coordinates, the world coordinates corresponding to each pixel are stored to form point cloud data K p =

[0023] (x p ,y p ,zp ).

[0024] Furthermore, the formula for calculating the normal vector of each point in its neighborhood in the point cloud data is:

[0025]

[0026] Where N p Represents the normal vector of the pth data point in the point cloud data, K p+1 and K p-1 K p The adjacent points of ||(K p+1 -K p )×(K p-1 -K p )|| means (K p+1 -K p )×(K p-1 -K p )The magnitude of the vector;

[0027] The formula for calculating the curvature of each point based on the normal vector of each point is:

[0028]

[0029] In the formula, G p represents the curvature of the pth data point in the point cloud data, ||N p || and ||K p || respectively represent the normal vector modulus corresponding to the p-th data point in the point cloud data and the modulus value of the p-th data point.

[0030] Furthermore, a curvature threshold is set to identify points whose curvature exceeds the curvature threshold and mark them as high curvature points. The logic is as follows: the curvature threshold is calibrated as G th , the curvature G of the data points in the calculated point cloud data p With G th In contrast, when G p Greater than or equal to G th When , the data point is marked as a high curvature point. Based on the position of the high curvature point in the point cloud data, the position of the high curvature point in the depth image is mapped and marked. A 5*5 window is defined with the high curvature point as the window center, and the area where the point with high curvature value is located is marked as the high curvature value area.

[0031] Furthermore, the formula for calculating the mean and standard deviation of temperature in the infrared image is:

[0032]

[0033]

[0034] Where, T mean and σ T represent the mean and standard deviation of temperature respectively, Q is the total number of pixels in the infrared image, T d is the temperature value of the dth pixel, d is the index of the infrared image pixel, d = 1, 2, ..., Q; the temperature threshold is set to T mean +2σ T , mark the area where the temperature exceeds the set temperature threshold pixel point, and record it as the temperature abnormal area.

[0035] Furthermore, an image recognition model is established based on the YOLOv5s network, and the high curvature value area and the temperature abnormality area in the depth image are input into the image recognition model. The model identifies the potential burrs in the regional image extracted from the depth image, and uses the Sobel operator to extract edge features for the identified potential burrs. The edge features include the length and width of the potential burrs, and thresholds for the burr length and width are set. When the identified potential burr edge feature is greater than the set burr length and width thresholds, the potential burr is determined to be a burr, and its location is output.

[0036] The present invention also provides a drilling edge burr detection system based on a TOF camera, wherein the drilling edge burr detection system based on a TOF camera is used to execute the above-mentioned drilling edge burr detection method based on a TOF camera, comprising:

[0037] An image acquisition preprocessing module is used to calibrate the parameters of the TOF camera and the thermal infrared camera and obtain the camera parameters, wherein the camera parameters include the focal length and the coordinates of the principal point position, and use the calibrated TOF camera and the thermal infrared camera to acquire images of the edge of the borehole at the same angle to obtain a depth image and a thermal infrared image, respectively, and preprocess the depth image and the thermal infrared image, respectively;

[0038] The curvature calculation and analysis module is used to extract the depth value according to the pixel value corresponding to each pixel in the preprocessed depth image, convert the pixel points in the depth image into point cloud data using the depth value of each pixel point obtained, calculate the normal vector of each point in the point cloud data within its neighborhood, and calculate the curvature of each point based on the normal vector of each point;

[0039] The high curvature value area definition module is used to set the curvature threshold, identify the points whose curvature exceeds the curvature threshold, mark them as high curvature points, determine their positions in the point cloud data, mark the positions of the high curvature points in the depth image according to the position mapping in the point cloud data, and define a window to identify the area where the points with high curvature values ​​are located, which is recorded as the high curvature value area;

[0040] The temperature abnormal area marking module is used to calculate the temperature mean and standard deviation in the infrared image from the preprocessed thermal infrared image, set the temperature threshold based on the temperature mean and standard deviation, mark the area where the temperature exceeds the set temperature threshold as the temperature abnormal area, and mark the area where the temperature abnormal area is located in the depth image;

[0041] The recognition model training module is used to establish an image recognition model, collect several types of drilling edge burr images, and mark the burr contours in the images by manual marking. The collected drilling edge burr images are used as input and the burr contours are used as labels to train the model;

[0042] The burr recognition and analysis module is used to extract high curvature value areas and temperature abnormality areas from the depth image as regional images, input the regional images into the trained image recognition model, identify potential burrs in the regional images, obtain edge features of potential burrs, and if the edge features meet the burr feature requirements, mark the potential burr as a burr and output its location.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] By combining the multimodal information of TOF cameras and thermal infrared cameras, accurate detection of burrs on the edge of drill holes is achieved. First, the depth information provided by the TOF camera can effectively reflect the three-dimensional depth information of the edge of the drill hole. By converting the depth image into point cloud data and further calculating the normal vector and curvature of each point, the high curvature areas of the edge of the drill hole can be accurately identified. These areas are usually the key areas for burr formation. At the same time, the temperature distribution image captured by the thermal infrared camera can reflect the subtle burrs that may be caused by frictional heat effects during the processing process. By calculating the mean and standard deviation of the temperature, the temperature abnormality area is further marked, which enhances the sensitivity of burr detection. By establishing an image recognition model, only the high curvature value area in the depth image and the temperature abnormality area in the thermal infrared image need to be identified, which effectively improves the accuracy and reliability of burr detection, while reducing the amount of model calculation and improving the effectiveness of the system. Finally, accurate marking is performed according to the edge features. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0046] Figure 2 It is a schematic diagram of the overall system structure of the present invention. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0048] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0049] Example:

[0050] See also Figure 1 , the present invention provides a technical solution:

[0051] A method for detecting burrs on the edge of a drilling hole based on a TOF camera, the specific steps comprising:

[0052] Step 1: Calibrate the parameters of the TOF camera and the thermal infrared camera, and obtain the camera parameters, wherein the camera parameters include the focal length and the coordinates of the principal point position, use the calibrated TOF camera and the thermal infrared camera to collect images of the edge of the borehole at the same angle, obtain a depth image and a thermal infrared image respectively, and pre-process the depth image and the thermal infrared image respectively;

[0053] The parameters of the TOF and thermal infrared cameras are calibrated using the Zhang Zhengyou calibration method to obtain the camera parameters, which include focal length and principal point position coordinates. The specific steps include:

[0054] First, you need to prepare a calibration plate with a known geometric shape. The most commonly used one is a checkerboard calibration plate. There are several black squares or circles on the calibration plate, and their geometric shapes and sizes are known. Place the calibration plate in the shooting field of view of the digital camera. Keep the camera position fixed during the entire calibration process, and move the calibration plate to different positions to ensure that the image captured by the camera each time completely covers the calibration plate and the image quality is clear, until about 20 sets of calibration plate images with different positions are collected.

[0055] Use the Camera Calibrator toolbox in Matlab software to import all calibration plate images and input the actual size of the checkerboard. Filter all imported images, remove images with inconsistent coordinate systems, and extract the corner points of each image;

[0056] After extracting corner points from all calibration plate images, use the detected corner point data and the geometric parameters of the calibration plate to run the calibration program using the Calibration toolbox. The program will output calibration-related parameters, including the camera's internal parameters: focal length and principal point position.

[0057] The focal length includes the focal length fu on the u-axis and the focal length fv on the v-axis in the image coordinate system, and the coordinates of the principal point position are (u0, v0).

[0058] The depth image and the thermal infrared image are preprocessed respectively, wherein the depth image is preprocessed by Gaussian filtering, and the thermal infrared image is preprocessed by histogram equalization. Gaussian filtering is a commonly used image processing technology used to remove noise in an image, especially when processing a depth image, it can effectively smooth out the depth discontinuous area, thereby improving the image quality.

[0059] Histogram equalization is performed on infrared images to improve contrast, based on the following formula:

[0060]

[0061] In the formula, h s is the pixel value after equalization, L represents the maximum pixel value in the infrared image, Q represents the total number of pixels in the infrared image, F represents the specific pixel value, which represents the pixel value that needs to be equalized, E(b) represents the frequency of the pixel value, and b represents the current accumulated pixel value level, which usually starts from 0 and gradually accumulates to a certain pixel value.

[0062] Step 2: In the preprocessed depth image, extract the depth value according to the pixel value corresponding to each pixel, convert the pixel points in the depth image into point cloud data using the depth value of each pixel point, calculate the normal vector of each point in the point cloud data within its neighborhood, and calculate the curvature of each point based on the normal vector of each point;

[0063] From the preprocessed depth image, the formula for extracting the corresponding depth value according to the pixel value corresponding to each pixel is:

[0064] z p =h[i][j]

[0065] In the formula, z p represents the depth value of the p-th pixel in the depth image, h[i][j] represents the pixel value corresponding to the i-th row and j-th column, where p is the index of the pixel in the depth image, i represents the row index of the pixel, j represents the row index of the pixel, i=1,2,…,n,j=1,2,…,m, n and m represent the total number of rows and columns of pixels in the depth image respectively, and p=1,2,…,n*m;

[0066] Based on the depth value of the pixel point, the two-dimensional pixel coordinates are converted to three-dimensional world coordinates. The specific formula is as follows:

[0067]

[0068] In the formula, x p and p They represent the x-axis coordinate and y-axis coordinate of the p-th pixel in the world coordinate system, u i and v j They represent the u-axis coordinates of the i-th row of pixels in the image coordinate system and the v-axis coordinates of the j-th row of pixels in the image coordinate system respectively; according to the converted three-dimensional world coordinates, the world coordinates corresponding to each pixel are stored to form point cloud data K p =

[0069] (x p ,y p ,z p ).

[0070] The formula for calculating the normal vector of each point in its neighborhood in the point cloud data is:

[0071]

[0072] Where N p Represents the normal vector of the pth data point in the point cloud data, K p+1 and K p-1 K p The adjacent points of ||(K p+1 -K p )×(K p-1 -K p )|| means (K p+1 -K p )×(K p-1 -K p )The magnitude of the vector;

[0073] The formula for calculating the curvature of each point based on the normal vector of each point is:

[0074]

[0075] In the formula, G p represents the curvature of the pth data point in the point cloud data, ||N p || and ||K p || respectively represent the normal vector modulus corresponding to the p-th data point in the point cloud data and the modulus value of the p-th data point.

[0076] High curvature points usually refer to points on the surface of an object where the geometry changes significantly, such as edges, sharp corners, or other places where the shape changes suddenly. When processing large point clouds or depth images, it may be inefficient to process all the data directly. By identifying and processing only high curvature points, the amount of data can be greatly reduced while retaining information that is critical to shape understanding, improving processing speed and efficiency.

[0077] Step 3: Set the curvature threshold, identify the points whose curvature exceeds the curvature threshold, mark them as high curvature points, determine their positions in the point cloud data, mark the positions of the high curvature points in the depth image according to the position mapping in the point cloud data, and define a window to identify the area where the points with high curvature values ​​are located, which is recorded as the high curvature value area;

[0078] Set the curvature threshold, identify the points whose curvature exceeds the curvature threshold, and mark them as high curvature points. The logic is as follows: The curvature threshold is G th , the curvature G of the data points in the calculated point cloud data p With G th In contrast, when G p Greater than or equal to G th When , the data point is marked as a high curvature point. Based on the position of the high curvature point in the point cloud data, the position of the high curvature point in the depth image is mapped and marked. A 5*5 window is defined with the high curvature point as the window center, and the area where the point with high curvature value is located is marked as the high curvature value area.

[0079] Step 4: Calculate the temperature mean and standard deviation in the infrared image from the preprocessed thermal infrared image, set the temperature threshold based on the temperature mean and standard deviation, mark the area where the temperature exceeds the set temperature threshold as the abnormal temperature area, and mark the area where the abnormal temperature area is located in the depth image;

[0080] The formula for calculating the mean and standard deviation of temperature in infrared images is:

[0081]

[0082] Where, T mean and σ T represent the mean and standard deviation of temperature respectively, Q is the total number of pixels in the infrared image, T d is the temperature value of the dth pixel, d is the index of the infrared image pixel, d = 1, 2, ..., Q; the temperature threshold is set to T mean +2σ T , mark the area where the temperature exceeds the set temperature threshold pixel point, and record it as the temperature abnormal area.

[0083] Step 5: Establish an image recognition model, collect several types of drilling edge burr images, and mark the burr contours in the images by manual marking. The collected drilling edge burr images are used as input and the burr contours are used as labels to train the model.

[0084] An image recognition model is established based on the YOLOv5s network, and the high curvature value areas and temperature abnormality areas in the depth image are input into the image recognition model. Among them, several types of drilling edge burr images can be selected, such as sharp edge burrs, impurity burrs and tearing burrs. According to specific actual needs, different types of drilling edge burr images can be used as samples for model training. The model identifies potential burrs in the input image, annotates all potential burrs in the input image, and annotates the outer contour of a single potential burr through the LabelImg tool by manual marking.

[0085] Step 6: Extract high curvature value areas and temperature abnormality areas from the depth image as regional images, input the regional images into the trained image recognition model, identify potential burrs in the regional images, obtain edge features of potential burrs, and if the edge features meet the burr feature requirements, mark the potential burr as a burr and output its location.

[0086] The high curvature value area and temperature abnormality area images in the depth image are input into the image recognition model. The model identifies the potential burrs in the extracted area image. For the identified potential burrs, the Sobel operator is used to extract edge features. The edge features include the length and width of the potential burrs. The specific steps include: using the Sobel operator to calculate the gradient of the image in the horizontal direction and the vertical direction, wherein the formula for calculating the gradient of the image in the horizontal direction and the vertical direction is:

[0087]

[0088]

[0089] In the formula, G x is the horizontal gradient of the image, G y is the vertical gradient of the image, represents the convolution operation, I is the input image; the formula for calculating the gradient magnitude and direction is:

[0090]

[0091] In the formula, G is the gradient amplitude and θ is the direction;

[0092] Non-maximum suppression is performed along the gradient direction, local maxima are retained, non-edge points are suppressed, high thresholds and low thresholds are used to distinguish strong edges from weak edges, and edge detection is completed by connecting weak edges and strong edges. The length and width of potential burrs are obtained as l and w respectively, and the thresholds of burr length and width are set. The thresholds of burr length and width are calibrated as l respectively. th and w th When the identified potential burr edge feature is greater than the set burr length and width threshold, that is, when the potential burr length l ≥ l th and width w ≥ w th When the potential glitch is detected, the potential glitch is determined to be a glitch and its location is output.

[0093] See also Figure 2 The present invention also provides a drilling edge burr detection system based on a TOF camera, and the drilling edge burr detection system based on a TOF camera is used to perform the above-mentioned drilling edge burr detection method based on a TOF camera, including:

[0094] An image acquisition preprocessing module is used to calibrate the parameters of the TOF camera and the thermal infrared camera and obtain the camera parameters, wherein the camera parameters include the focal length and the coordinates of the principal point position, and use the calibrated TOF camera and the thermal infrared camera to acquire images of the edge of the borehole at the same angle to obtain a depth image and a thermal infrared image, respectively, and preprocess the depth image and the thermal infrared image, respectively;

[0095] The curvature calculation and analysis module is used to extract the depth value according to the pixel value corresponding to each pixel in the preprocessed depth image, convert the pixel points in the depth image into point cloud data using the depth value of each pixel point obtained, calculate the normal vector of each point in the point cloud data within its neighborhood, and calculate the curvature of each point based on the normal vector of each point;

[0096] The high curvature value area definition module is used to set the curvature threshold, identify the points whose curvature exceeds the curvature threshold, mark them as high curvature points, determine their positions in the point cloud data, mark the positions of the high curvature points in the depth image according to the position mapping in the point cloud data, and define a window to identify the area where the points with high curvature values ​​are located, which is recorded as the high curvature value area;

[0097] The temperature abnormal area marking module is used to calculate the temperature mean and standard deviation in the infrared image from the preprocessed thermal infrared image, set the temperature threshold based on the temperature mean and standard deviation, mark the area where the temperature exceeds the set temperature threshold as the temperature abnormal area, and mark the area where the temperature abnormal area is located in the depth image;

[0098] The recognition model training module is used to establish an image recognition model, collect several types of drilling edge burr images, and mark the burr contours in the images by manual marking. The collected drilling edge burr images are used as input and the burr contours are used as labels to train the model;

[0099] The burr recognition and analysis module is used to extract high curvature value areas and temperature abnormality areas from the depth image as regional images, input the regional images into the trained image recognition model, identify potential burrs in the regional images, obtain edge features of potential burrs, and if the edge features meet the burr feature requirements, mark the potential burr as a burr and output its location.

[0100] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0101] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0103] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for detecting burrs on the edge of a drilling hole based on a TOF camera, characterized in that: The specific steps include: Calibrate the parameters of the TOF camera and the thermal infrared camera, and obtain the camera parameters, wherein the camera parameters include the focal length and the coordinates of the principal point position, use the calibrated TOF camera and the thermal infrared camera to collect images of the edge of the borehole at the same angle, obtain a depth image and a thermal infrared image, respectively, and pre-process the depth image and the thermal infrared image, respectively; In the preprocessed depth image, the depth value is extracted according to the pixel value corresponding to each pixel, and the pixel points in the depth image are converted into point cloud data using the depth values ​​of each pixel point obtained. The normal vector of each point in the point cloud data within its neighborhood is calculated, and the curvature of each point is calculated based on the normal vector of each point. Set the curvature threshold, identify the points whose curvature exceeds the curvature threshold, mark them as high curvature points, determine their positions in the point cloud data, mark the positions of the high curvature points in the depth image according to the position mapping in the point cloud data, and define a window to identify the area where the points with high curvature values ​​are located, which is recorded as the high curvature value area; From the preprocessed thermal infrared image, calculate the temperature mean and standard deviation in the infrared image, set the temperature threshold based on the temperature mean and standard deviation, mark the area where the temperature exceeds the set temperature threshold, record it as the temperature abnormal area, and mark the area where the temperature abnormal area is located in the depth image; An image recognition model was established, and several types of burr images of drilling edges were collected. The burr contours in the images were marked manually, and the model was trained using the collected burr images of drilling edges as input and the burr contours as labels. Regions with high curvature values ​​and regions with abnormal temperature are extracted from the depth image as regional images, and the regional images are input into the trained image recognition model to identify potential burrs in the regional images and obtain edge features of the potential burrs. If the edge features meet the burr feature requirements, the potential burrs are marked as burrs and their locations are output.

2. The method for detecting burrs on the edge of a drilling hole based on a TOF camera according to claim 1, characterized in that: The depth image and the thermal infrared image are preprocessed respectively, wherein the depth image is preprocessed by Gaussian filtering, and the thermal infrared image is preprocessed by histogram equalization.

3. The method for detecting burrs on the edge of a drilling hole based on a TOF camera according to claim 1, characterized in that: The TOF and thermal infrared camera parameters are calibrated using Zhang Zhengyou calibration method to obtain camera parameters, which include focal length and principal point position coordinates, where the focal length includes the focal length fu on the u-axis and the focal length fv in the v-axis direction in the image coordinate system, and the principal point position coordinates are (u0, v0).

4. The method for detecting burrs on the edge of a drilling hole based on a TOF camera according to claim 3, characterized in that: From the preprocessed depth image, the formula for extracting the corresponding depth value according to the pixel value corresponding to each pixel is: z p =h[i][j] In the formula, z p represents the depth value of the pth pixel in the depth image, h[i][j] represents the pixel value of the corresponding pixel in the i-th row and j-th column, where p is the index of the pixel in the depth image, i represents the row index of the pixel, and j represents the row index of the pixel. i=1,2,…,n,j=1,2,…,m, n and m represent the total number of rows and columns of depth image pixels respectively, and p=1,2,…,n*m; Based on the depth value of the pixel point, the two-dimensional pixel coordinates are converted to three-dimensional world coordinates; the specific formula is as follows: In the formula, x p and p They represent the x-axis coordinate and y-axis coordinate of the p-th pixel in the world coordinate system, u i and v j They represent the u-axis coordinates of the i-th row of pixels in the image coordinate system and the v-axis coordinates of the j-th row of pixels in the image coordinate system respectively; according to the converted three-dimensional world coordinates, the world coordinates corresponding to each pixel are stored to form point cloud data K p = (x p ,y p ,z p )。 5. The method for detecting burrs on the edge of a drilling hole based on a TOF camera according to claim 4, characterized in that: The formula for calculating the normal vector of each point in its neighborhood in the point cloud data is: Where N p Represents the normal vector of the pth data point in the point cloud data, K p+1 and K p-1 K p The adjacent points of ||(K p+1 -K p )×(K p-1 -K p )|| means (K p+1 -K p )×(K p-1 -K p )The magnitude of the vector; The formula for calculating the curvature of each point based on the normal vector of each point is: In the formula, G p represents the curvature of the pth data point in the point cloud data, ||N p || and ||K p || respectively represent the normal vector modulus corresponding to the p-th data point in the point cloud data and the modulus value of the p-th data point.

6. The method for detecting burrs on the edge of a drilling hole based on a TOF camera according to claim 1, characterized in that: Set the curvature threshold, identify points whose curvature exceeds the curvature threshold, and mark them as high curvature points based on the following logic: The calibration curvature threshold is G th , the curvature G of the data points in the calculated point cloud data p With G th In contrast, when G p Greater than or equal to G th When , the data point is marked as a high curvature point. Based on the position of the high curvature point in the point cloud data, the position of the high curvature point in the depth image is mapped and marked. A 5*5 window is defined with the high curvature point as the window center, and the area where the point with high curvature value is located is marked as the high curvature value area.

7. The method for detecting burrs on the edge of a drilling hole based on a TOF camera according to claim 1, characterized in that: The formula for calculating the mean and standard deviation of temperature in infrared images is: Where, T mean and σ T represent the mean and standard deviation of temperature respectively, Q is the total number of pixels in the infrared image, T d is the temperature value of the dth pixel, d is the index of the infrared image pixel, d = 1, 2, ..., Q; the temperature threshold is set to T mean +2σ T , mark the area where the temperature exceeds the set temperature threshold pixel point, and record it as the temperature abnormal area.

8. The method for detecting burrs on the edge of a drilling hole based on a TOF camera according to claim 1, characterized in that: An image recognition model is established based on the YOLOv5s network, and the high curvature value area and the temperature abnormality area in the depth image are input into the image recognition model. The model identifies the potential burrs in the regional image extracted from the depth image. For the identified potential burrs, the Sobel operator is used to extract edge features. The edge features include the length and width of the potential burrs. Thresholds for the burr length and width are set. When the identified potential burr edge feature is greater than the set burr length and width thresholds, the potential burr is determined to be a burr, and its location is output.

9. A drilling edge burr detection system based on TOF camera, characterized in that: The drilling edge burr detection system is used to execute the drilling edge burr detection method based on a TOF camera according to any one of claims 1 to 8, comprising: An image acquisition preprocessing module is used to calibrate the parameters of the TOF camera and the thermal infrared camera and obtain the camera parameters, wherein the camera parameters include the focal length and the coordinates of the principal point position, and use the calibrated TOF camera and the thermal infrared camera to acquire images of the edge of the borehole at the same angle to obtain a depth image and a thermal infrared image, respectively, and preprocess the depth image and the thermal infrared image, respectively; The curvature calculation and analysis module is used to extract the depth value according to the pixel value corresponding to each pixel in the preprocessed depth image, convert the pixel points in the depth image into point cloud data using the depth value of each pixel point obtained, calculate the normal vector of each point in the point cloud data within its neighborhood, and calculate the curvature of each point based on the normal vector of each point; The high curvature value area definition module is used to set the curvature threshold, identify the points whose curvature exceeds the curvature threshold, mark them as high curvature points, determine their positions in the point cloud data, mark the positions of the high curvature points in the depth image according to the position mapping in the point cloud data, and define a window to identify the area where the points with high curvature values ​​are located, which is recorded as the high curvature value area; The temperature abnormal area marking module is used to calculate the temperature mean and standard deviation in the infrared image from the preprocessed thermal infrared image, set the temperature threshold based on the temperature mean and standard deviation, mark the area where the temperature exceeds the set temperature threshold as the temperature abnormal area, and mark the area where the temperature abnormal area is located in the depth image; The recognition model training module is used to establish an image recognition model, collect several types of drilling edge burr images, and mark the burr contours in the images by manual marking. The collected drilling edge burr images are used as input and the burr contours are used as labels to train the model; The burr recognition and analysis module is used to extract high curvature value areas and temperature abnormality areas from the depth image as regional images, input the regional images into the trained image recognition model, identify potential burrs in the regional images, obtain edge features of potential burrs, and if the edge features meet the burr feature requirements, mark the potential burr as a burr and output its location.

Images