Engine blade defect detection system

Abstract

The utility model belongs to the technical field of defect detection, and discloses an engine blade defect detection system. The engine blade defect detection system comprises a material disc used for storing blades; a blade storage groove is formed in the material disc; the robot is used for clamping and moving the blade; the robot is provided with a clamping jaw and a mechanical arm; the two-dimensional camera is used for photographing the blade clamped by the clamping jaw; the three-dimensional camera is used for photographing the blade clamped by the clamping jaw; and the light supplementing light source is used for supplementing light when the two-dimensional camera and the three-dimensional camera take pictures. The engine blade defect detection system provided by the utility model can detect blade surface defects digitally, automatically and intelligently, and realizes high-efficiency intelligent detection with stable quality, high efficiency, high traceability and quantitative detection results.

Description

TECHNICAL FIELD

[0001] The utility model relates to the technical field of defect detection, and concretely relates to engine blade defect detection system. BACKGROUND

[0002] As the key power device of the aircraft, aerospace engine is called the "heart" of the aircraft and rocket, and its independent development is of great significance to the national defense construction, energy security and sustainable development of the environment. Among them, the blade, as the core component of guiding airflow and generating power in the aerospace engine, the detection of its key quality characteristics directly affects the service safety and reliability of the aircraft.

[0003] Among the numerous components of the engine, the number of blades is large, and its manufacturing amount accounts for about one third of the whole machine manufacturing amount. The blade has the characteristics of large number, complex shape, high precision requirement, great processing difficulty and frequent failure, and is a key component of the engine. The engine relies on numerous blades to complete the compression and expansion of gas to generate powerful power to push the aircraft forward. Therefore, the quality of the blade is a key factor affecting the energy conversion efficiency and working performance of the engine.

[0004] At present, the detection means for the surface defects of the blade is mainly artificial visual inspection. From the detection quality, the detection result is greatly affected by human factors. From the detection efficiency, the blade appearance is complex, and the artificial detection efficiency is low. From the detection data, the artificial detection result is only qualitative determination, lacks quantitative data, and the result is difficult to be digitally stored, which is not conducive to the later quality problem tracing. From the detection quantity, at present, there is still a lack of intelligent detection ability for batch blade surface defects on site, and the artificial visual detection method has reached the production capacity bottleneck, which is difficult to meet the needs of production and delivery. UTILITY MODEL CONTENT

[0005] The main purpose of the utility model is to provide an engine blade defect automatic detection method based on images and an engine blade defect detection system, which detects the surface defects of the blade in a digital, automatic and intelligent manner, realizes efficient intelligent detection with stable quality, high efficiency, high traceability and quantitative detection results.

[0006] In order to realize the above purpose, the technical scheme of the engine blade defect automatic detection method based on images and the engine blade defect detection system provided by the utility model is as follows:

[0007] The engine blade defect automatic detection method based on images comprises the steps that the robot gripper clamps the blade and moves to the detection site, and the two-dimensional camera and the three-dimensional camera take pictures of the blade at the detection site, and further comprises the following steps:

[0008] Step (1), a two-dimensional camera is used to take pictures of the blade held by the robot gripper to obtain a plurality of two-dimensional images covering the surface of the blade;

[0009] Step (2), a target detection neural network model is used to detect defects in the plurality of two-dimensional images;

[0010] Step (3), for the two-dimensional images in which defects are detected, three-dimensional coordinates of the defect center in the robot base coordinate system are calculated, and then the gripper is caused to move the defect center to the imaging center of the three-dimensional camera;

[0011] Step (4), a three-dimensional camera is used to collect a grayscale image and point cloud data of the defect;

[0012] Step (5), a defect quantification method based on the fusion of a three-dimensional camera and a neural network is used to process the grayscale image and point cloud data of the defect to obtain geometric information of the defect.

[0013] An engine blade defect detection system comprises a tray for storing blades, wherein a blade storage groove is arranged on the tray; a robot for holding and moving the blades, wherein the robot has a gripper and a mechanical arm; a two-dimensional camera for taking pictures of the blades held by the gripper; a three-dimensional camera for taking pictures of the blades held by the gripper; and a light supplementing light source for supplementing light when the two-dimensional camera and the three-dimensional camera take pictures.

[0014] In the above image-based engine blade defect automatic detection method, when steps (3) and (5) adopt the following calculation method for obtaining defect center coordinates in a two-dimensional image and the defect quantification method based on the fusion of a three-dimensional camera and a neural network, the calculation method is simple, and the calculation efficiency and accuracy are high.

[0015] Therefore, the utility model provides a calculation method for obtaining defect center coordinates in a two-dimensional image and a defect quantification method based on the fusion of a three-dimensional camera and a neural network, and the technical scheme is as follows:

[0016] The calculation method for obtaining defect center coordinates in a two-dimensional image is used to make the gripper of the robot hold a test object and move the defect center of the test object to the imaging center of a three-dimensional camera, and comprises the following steps:

[0017] Step 100, multiplying the two-dimensional pixel coordinate matrix of the defect in the two-dimensional image and the intrinsic matrix of the two-dimensional camera to calculate the normalized plane coordinates of the defect in the two-dimensional camera coordinate system;

[0018] Step 200, calculating the actual three-dimensional camera coordinates of the defect in the two-dimensional camera coordinate system according to the normalized plane coordinates and the distance between the test object and the two-dimensional camera;

[0019] Step 300, using the transformation matrix of the two-dimensional camera coordinate system to the robot base coordinate system, the actual three-dimensional coordinates of the defect in the two-dimensional camera coordinate system are converted into three-dimensional coordinates in the robot base coordinate system;

[0020] Step 400, using the transformation matrix of the robot base coordinate system to the robot gripper coordinate system, the three-dimensional coordinates of the defect in the robot base coordinate system are converted into three-dimensional coordinates in the robot gripper coordinate system;

[0021] Step 500, the three-dimensional coordinates of the defect in the robot gripper coordinate system are subtracted from the origin of the robot gripper coordinate system, and a vector representing the relative position of the defect in the robot gripper coordinate system is calculated;

[0022] Step 600, using the transformation matrix of the robot gripper coordinate system to the robot base coordinate system, the vector is converted into three-dimensional coordinates of the defect in the robot base coordinate system, and then the three-dimensional coordinates of the defect center in the robot base coordinate system are obtained, and the robot gripper can move the defect center to the imaging center of the three-dimensional camera.

[0023] The defect quantification method based on the fusion of three-dimensional cameras and neural networks includes the following steps:

[0024] Step A, the gray-scale image and point cloud data of a larger area collected by the three-dimensional camera are cropped to a smaller area related to the defect, obtaining the gray-scale image and point cloud data of the smaller area;

[0025] Step B, the point cloud data of the cropped smaller area is subjected to surface fitting to obtain a smooth surface model;

[0026] Step C, the instance segmentation neural network model is used to process the gray-scale image of the smaller area to obtain the corresponding mask image;

[0027] Step D, according to the defect edge on the mask image, the sub-point cloud data of the defect edge is extracted from the point cloud data of the smaller area;

[0028] Step E, difference calculation is performed between the sub-point cloud data and the smooth surface model, and the geometric information of the defect is obtained.

[0029] The above-mentioned calculation method for obtaining the defect center coordinates in the two-dimensional image and the action object of the defect quantification method based on the fusion of three-dimensional cameras and neural networks can not be limited to engine blades and their defects, but can also be target areas of other test objects. The target area is not necessarily a defect, but can also be a functional structure intentionally processed, and the defect is not necessarily a pit, but can also be a crack, a bump, etc.

[0030] The utility model will be further explained in connection with the drawings and specific embodiments. The additional aspects and advantages of the utility model will be partly given in the following description, some will become obvious from the following description, or be known by the practice of the utility model. BRIEF DESCRIPTION OF DRAWINGS

[0031] The drawings which form part of the utility model serve to assist the understanding of the utility model, the contents provided in the drawings and the related description thereof in the utility model can be used to explain the utility model, but do not constitute improper limitation on the utility model.

[0032] Figure 1 It is the flow chart of the embodiment of the calculation method of the utility model for obtaining the defect center coordinate from the two-dimensional image.

[0033] Figure 2 It is the flow chart of the embodiment of the defect quantification method based on the fusion of three-dimensional camera and neural network of the utility model.

[0034] Figure 3 It is the flow chart of the embodiment of the engine blade defect automatic detection method based on the image of the utility model.

[0035] Figure 4 It is the perspective view of the embodiment of the engine blade defect detection system of the utility model.

[0036] Figure 5 It is the top view of the embodiment of the engine blade defect detection system of the utility model.

[0037] Figure 6 It is the point cloud data image of the utility model that is obtained by using three-dimensional camera to collect the larger area.

[0038] Figure 7 It is the smaller area gray scale image obtained by cutting the larger area gray scale image of the utility model.

[0039] Figure 8 It is the smaller area point cloud data image obtained by cutting the larger area point cloud data image of the utility model.

[0040] Figure 9 It is the smooth surface model image obtained by carrying out surface fitting to the smaller area point cloud data of the utility model after cutting.

[0041] Figure 10 It is the mask chart obtained by using instance segmentation neural network model to process the smaller area gray scale image of the utility model.

[0042] Figure 11 It is the sub point cloud data image of the defect edge extracted from the smaller area point cloud data image of the utility model.

[0043] The relevant labels in the above figures are: 100 - tray, 200 - robot, 210 - gripper, 220 - mechanical arm, 230 - base, 300 - two-dimensional camera, 400 - three-dimensional camera, 510 - bar light source, 520 - ring light source, 600 - test table, 710 - U-shaped base, 720 - column, 730 - hinged seat, 740 - crossbar. DETAILED DESCRIPTION

[0044] The present application will be described in detail below with reference to the drawings. Those skilled in the art will be able to implement the present application based on these descriptions. Before the present application is described in detail with reference to the drawings, it is important to note that:

[0045] The technical solutions and technical features provided in each part including the following description can be combined with each other without conflict.

[0046] In addition, the embodiments of the present application involved in the following description are usually only a part of the embodiments of the present application, not all. Therefore, based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0047] Regarding the terms and units in the present application. The terms "include", "have" and any variations thereof in the description and claims of the present application and related parts are intended to cover non-exclusive inclusion.

[0048] Figure 1 Flowchart of an embodiment of the calculation method of the present application for obtaining defect center coordinates from a two-dimensional image.

[0049] As shown in Figure 1 The embodiment of the calculation method of the present application for obtaining defect center coordinates from a two-dimensional image is used to make the gripper of the robot hold the test object and move its defect center to the imaging center of the three-dimensional camera. The calculation method includes steps 100-600, as follows:

[0050] Step 100, multiply the two-dimensional pixel coordinate matrix of the defect in the two-dimensional image with the intrinsic matrix of the two-dimensional camera to calculate the normalized plane coordinates of the defect in the two-dimensional camera coordinate system.

[0051] Step 200, according to the normalized plane coordinates and the distance between the test object and the two-dimensional camera, the actual three-dimensional camera coordinates of the defect in the two-dimensional camera coordinate system are calculated.

[0052] Step 300: Using the transformation matrix from the 2D camera coordinate system to the robot base coordinate system, the actual 3D coordinates of the defect in the 2D camera coordinate system are transformed into 3D coordinates in the robot base coordinate system. The process of obtaining the transformation matrix from the 2D camera coordinate system to the robot base coordinate system includes steps 310 to 370, as follows:

[0053] Step 310: Use a 2D camera to capture images of the chessboard pattern grasped by the robot gripper in different poses;

[0054] Step 320: Obtain the pixel coordinates of the corner points of the chessboard.

[0055] Use OpenCV's cv2.find Chess board Corners function to extract values ​​from each 2D image I gray,k Extract the corner pixel coordinates P of the chessboard grid. uv,k =(x uv,k ,y uv,k ), where u and v represent the positions of the corner points on the chessboard, and k represents the index of the image;

[0056] Step 330: Establish the mapping relationship between corner pixel coordinates and robot base coordinate system;

[0057] The Zhang Dingyou calibration method is used to establish the mapping relationship between the corner pixel coordinates and the robot base coordinate system. Here, it is assumed that the three-dimensional coordinates of the chessboard grid in the robot base coordinate system are P. uv =(X uv Y uv Z uv ), where Z uv =0;

[0058] Step 340: Solve for the intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients of the 2D camera;

[0059] The intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients of a 2D camera are calculated using OpenCV's `calibrate Camera` function, and are expressed as follows:

[0060] H k =K·[R k |T k ];d=[k1,k2,p1,p2,k3];

[0061] In the formula, K is the intrinsic parameter matrix, f x and f y It's the focal length, c x and c y These are the principal point coordinates; H k R is the homography matrix; k Let T be the rotation matrix.k is a 3x4 matrix, representing the combination of rotation matrix and translation vector, i.e. the extrinsic matrix; d is the distortion coefficient, k1, k2, k3, k4 are radial distortion coefficients, p1, p2 are tangential distortion coefficients; k k is a 3x4 matrix, representing the combination of rotation matrix and translation vector, i.e. the extrinsic matrix; d is the distortion coefficient, k1, k2, k3, k4 are radial distortion coefficients, p1, p2 are tangential distortion coefficients;

[0062] Step 350, minimizing the re-projection error between the actual detected corner position and the theoretical corner position, denoted as:

[0063] Minimize∑ k ∑ u,v ||p uv,k -K(R k X u x+T k )| 2 ;

[0064] Step 360, using the maximum likelihood estimation method to optimize the intrinsic matrix, distortion coefficient and extrinsic matrix of the two-dimensional camera;

[0065] Step 370, applying the intrinsic matrix and extrinsic matrix to the conversion of the corner pixel coordinates, obtaining the transformation matrix from the two-dimensional camera coordinate system to the robot base coordinate system, denoted as:

[0066] The actual three-dimensional coordinates of the defect in the two-dimensional camera coordinate system are denoted as:

[0067]

[0068] The three-dimensional coordinates of the defect in the robot base coordinate system are denoted as:

[0069]

[0070] wherein, Z 2D is the distance between the test object and the two-dimensional camera, (u, v) is the two-dimensional pixel coordinate matrix of the defect.

[0071] Step 400, using the transformation matrix from the robot base coordinate system to the robot gripper coordinate system to convert the three-dimensional coordinates of the defect in the robot base coordinate system to the three-dimensional coordinates in the robot gripper coordinate system;

[0072] The three-dimensional coordinates of the defect in the robot gripper coordinate system are denoted as:

[0073]

[0074] wherein, T base-to-gripper is the transformation matrix from the robot base coordinate system to the robot gripper coordinate system. ​

[0075] Step 500, the three-dimensional coordinates of the defect in the robot gripper coordinate system are subtracted from the origin of the robot gripper coordinate system, and a vector representing the relative position of the defect in the robot gripper coordinate system is calculated;

[0076] The relative position of the defect in the robot gripper coordinate system is a vector is represented as:

[0077]

[0078] In the formula, are unit vectors along the x, y, and z axes, respectively.

[0079] Step 600, using the transformation matrix from the robot gripper coordinate system to the robot base coordinate system, the vector is converted into three-dimensional coordinates of the defect in the robot base coordinate system, and then the three-dimensional coordinates of the defect center in the robot base coordinate system are obtained, and the robot gripper can move the defect center to the imaging center of the three-dimensional camera;

[0080] The three-dimensional coordinates of the defect center in the robot base coordinate system (x base , y base , z base ) are represented as:

[0081]

[0082] In the formula, T gripper-to-base The method for obtaining T gripper-to-base is to move the robot gripper to the imaging center of the three-dimensional camera without holding the test object, and obtain the coordinates of the robot gripper through three-dimensional camera imaging, that is, the transformation matrix T camera-to-gripper from the imaging center of the three-dimensional camera to the gripper coordinate system and the robot base coordinate system is established.

[0083] In the above calculation method for obtaining the defect center coordinates from the two-dimensional image, a target detection neural network model is used to detect the defect in the two-dimensional image and output a rectangular coordinate frame of the defect. The two-dimensional pixel coordinate matrix includes the coordinates of the four corner points and the center point of the rectangular coordinate frame. There are two paths to obtain the three-dimensional coordinates of the defect center in the robot base coordinate system: (1) first convert the four corner point coordinates into four three-dimensional coordinates in the robot base coordinate system through T base-to-gripper , T gripper-to-base , T camera-to-gripper , and then calculate the three-dimensional coordinates of the defect center from the four three-dimensional coordinates; (2) directly convert the center point coordinates to obtain the three-dimensional coordinates of the defect center through T base-to-gripper , T gripper-to-base It is found through verification that the second path has smaller error and higher efficiency.

[0084] Figure 2 A flow chart of an embodiment of the defect quantification method based on fusion of a three-dimensional camera and a neural network of the utility model.

[0085] As Figure 2 shown, the embodiment of the defect quantification method based on fusion of a three-dimensional camera and a neural network of the utility model comprises steps A-E, and the details are as follows:

[0086] Step A, according to the three-dimensional camera imaging center and the defect size calculated from the two-dimensional image, the gray scale image and the point cloud data of a larger area collected by the three-dimensional camera are cropped into a smaller area related to the defect, and the gray scale image and the point cloud data of the smaller area are obtained; the three-dimensional camera adopts a monocular structured light camera;

[0087] The defect size is calculated from the four three-dimensional coordinates of the rectangular coordinate frame of the defect output by the target detection neural network model processing the two-dimensional image in the robot base coordinate system, and the details are as follows:

[0088] Let the four three-dimensional coordinates of the four corner points in the robot base coordinate system be and

[0089] Calculate the maximum and minimum values of the four three-dimensional coordinates on the x-axis:

[0090]

[0091]

[0092] Calculate the maximum and minimum values of the four three-dimensional coordinates on the y-axis:

[0093]

[0094]

[0095] Calculate the maximum and minimum values of the four three-dimensional coordinates on the z-axis:

[0096]

[0097]

[0098] According to the maximum and minimum values of the coordinate axes, the length, width and height of the defect are calculated:

[0099] Length: length = max x -min x ;

[0100] Width: width = max y-min y ;

[0101] high: height = max z -min z ;

[0102] wherein, the four three-dimensional coordinates also have two paths, which are: (1) sequentially passing the four corner point coordinates through T camera-to-gripper , T base-to-gripper , T gripper-to-base to obtain; (2) passing the four corner point coordinates through T camera-to-gripper to obtain. It has been verified that the errors of the two paths are almost the same, but the efficiency of the second path is obviously higher.

[0103] Step B, surface fitting is performed on the point cloud data of the cut smaller area to obtain a smooth surface model; specifically comprising the following steps B1-B4:

[0104] Step B1, effective point cloud data is screened out, and the expression is calculated as: P valid = Mask·P raw ; in the formula, P valid is the screened effective point cloud data; P raw is the point cloud data of the smaller area; Mask is a mask obtained from the point cloud data collected by the three-dimensional camera;

[0105] Step B2, surface fitting is performed on the screened effective point cloud data to obtain a reference surface;

[0106] Step B3, the BFGS algorithm in the minimization method is used to minimize the target function of the fitting error to obtain the optimal fitting parameter: wherein, the expression of the target function f(c) of the fitting error is:

[0107]

[0108] in the formula, c = [A, B, C, D, E, F] is the fitting parameter; (x n , y n , z n ) is the coordinate of the nth point cloud data.

[0109] Step B4, the depth of the fitting surface is calculated using the optimal fitting parameter to obtain a smooth surface model;

[0110] The calculation expression of the depth z of the fitting surface using the optimal fitting parameter is:

[0111] fit z = A*·x 2 +B * ·y 2 +C* x y + D * x + E * y + F * ;

[0112] where c* = [A*, B*, C*, D*, E*, F*] are the optimal fitting parameters; (x, y, z) are the coordinates at the depth of the fitted surface.

[0113] Step C, processing the gray image of the smaller area using the instance segmentation neural network model to obtain a corresponding mask image.

[0114] Step D, extracting sub-point cloud data of the defect edge from the point cloud data of the smaller area according to the defect edge on the mask image; the perimeter, diameter and volume of the defect can be calculated according to the sub-point cloud data.

[0115] Step E, calculating the depth deviation of the defect by difference calculation of the sub-point cloud data and the smooth surface model, determining the actual depth of the defect, and finally obtaining the geometric information of the defect;

[0116] The calculation expression of the depth deviation △D of the defect is: △D = P valid -fit z .

[0117] Figure 3 The flowchart of the embodiment of the image-based engine blade defect automatic detection method of the utility model.

[0118] As Figure 3 shown, the embodiment of the image-based engine blade defect automatic detection method of the utility model comprises adopting a robot gripper to clamp a blade and move it to a detection site, and adopting a two-dimensional camera and a three-dimensional camera to take a photograph of the blade at the detection site, and further comprises steps (1) to (5), which are specifically as follows:

[0119] Step (1), taking a photograph of the blade clamped by the robot gripper through the two-dimensional camera to obtain a plurality of two-dimensional images covering the surface of the blade.

[0120] Preferably, the surface of the blade is divided into a plurality of detection points, and the gripper accurately moves these detection points to the imaging center of the two-dimensional camera for photographing. The robot gripper clamps the same blade twice, and the positions of the two clamping are different, so that the surface covered by the gripper during the first clamping can also be detected, so that the obtained two-dimensional image covers all the key areas and avoids blind areas.

[0121] Preferably, the plurality of two-dimensional images comprises a furnace batch number image and a local position image; wherein, for the furnace batch number image, the recognition of the furnace batch number is carried out using an OCR algorithm processing; for the local position image, the defect detection is carried out using a target detection neural network model, whereby the recognized furnace batch number is bound with subsequent detection data (including defect detection results, quantitative data, etc.), ensuring that all detection information can be traced back to a specific production batch. Wherein, the OCR algorithm adopts DBM (Deep Bidirectional Model), uses bidirectional modeling technology for feature extraction and sequence modeling, captures the context relationship between characters, can deep feature learning to improve recognition accuracy and robustness, has strong generalization ability to adapt to various fonts and backgrounds, uses efficient training and reasoning mechanism to ensure fast processing of image data, so as to realize high-precision recognition under complex conditions.

[0122] Step (2), using a target detection neural network model to detect defects in the plurality of two-dimensional images.

[0123] The architecture of the target detection neural network model includes a backbone network, a neck network and a detection head, the backbone network is responsible for extracting image features, the neck network further processes and fuses these features, and the detection head generates the final detection results. The target detection neural network model adopts a continuous double assignment strategy without NMS (non-maximum suppression) training, which significantly improves the detection accuracy of small size and overlapping defects, can accurately identify small size defects, and effectively handles overlapping target problems, enhances the adaptability to complex background and high density targets. In addition, the NMS-free strategy reduces the possibility of false positives and false negatives, ensures the accuracy and stability of the detection results, making it more suitable for efficient and accurate detection of blade defects.

[0124] Step (3), using the above-mentioned calculation method of obtaining the center coordinates of the defects from the two-dimensional images, for the two-dimensional images detected with defects, calculating the three-dimensional coordinates of the defect center in the robot base coordinate system, and then moving the defect center to the imaging center of the three-dimensional camera by the gripper.

[0125] Step (4), acquiring the gray-scale image and point cloud data of the defect by the three-dimensional camera.

[0126] Step (5), using the above-mentioned defect quantification method based on the fusion of three-dimensional camera and neural network to process the gray-scale image and point cloud data of the defect, and obtaining the geometric information of the defect.

[0127] Figure 4 A perspective view of an embodiment of the engine blade defect detection system of the utility model. Figure 5 A top view of an embodiment of the engine blade defect detection system of the utility model.

[0128] As shown in Figures 4-5 the engine blade defect detection system includes a tray 100, a robot 200, a two-dimensional camera 300, a three-dimensional camera 400, a light supplement source, a terminal device and a test bench 600. The surface of the test bench 600 is assembled by T-shaped grooves. The tray 100 is used to store blades, and the tray 100 is provided with blade storage grooves.

[0129] The robot 200 is used to clamp and move the blades. The robot 200 has a clamping jaw 210, a mechanical arm 220 and a base 230. The mechanical arm 220 can drive the blades to rotate and lift. The base 230 is detachably connected with the T-shaped grooves and can slide horizontally along the T-shaped grooves.

[0130] The two-dimensional camera 300 and the three-dimensional camera 400 are used to take pictures of the blades clamped by the clamping jaw 210. The three-dimensional camera 400 is a monocular structured light camera. The two-dimensional camera 300 is arranged in front of the robot 200, the tray 100 is arranged beside the robot 200, and the three-dimensional camera 400 is arranged between the tray 100 and the two-dimensional camera 300 and is arranged obliquely towards the robot 200.

[0131] The light supplement source is used to supplement light when the two-dimensional camera 300 and the three-dimensional camera 400 take pictures. The light supplement source includes a strip light source 510 and a ring light source 520. The ring light source 520 is arranged in front of the two-dimensional camera 300, and the strip light source 510 is arranged between the two-dimensional camera 300 and the robot 200. The strip light source 510 is at least two.

[0132] The two-dimensional camera 300, the three-dimensional camera 400 and the light supplement source are fixed on the test bench 600 through adjustable supports. The adjustable support includes a U-shaped base 710, a stand column 720 and a hinged seat 730. The U-shaped base 710 is detachably connected with the T-shaped grooves and can slide horizontally along the T-shaped grooves. The two-dimensional camera 300, the three-dimensional camera 400 and the light supplement source are detachably connected with the stand column 720 through the vertical through holes of the hinged seat 730 and can slide vertically and rotate horizontally along the stand column 720. Further, the hinged seat 730 connected with the two-dimensional camera 300 or the three-dimensional camera 400 also includes a horizontal through hole. The adjustable support also includes a crossbar 740, which is detachably connected with the stand column 720 through the horizontal through hole of the hinged seat 730 and can move along the horizontal through hole. The end of the crossbar 740 is connected with the two-dimensional camera 300 or the three-dimensional camera 400.

[0133] The terminal device communicates with the two-dimensional camera 300, the three-dimensional camera 400 and the robot 200 through a wireless network and controls the clamping and movement of the blades by the clamping jaw 210.

[0134] The beneficial effects of the present application are illustrated by the following specific test data. The preferred technical solution with small error and higher efficiency is obtained by directly converting the center point coordinates of the rectangular coordinate frame through T camera-to-gripper , T base-to-gripper , T gripper-to-base to obtain the three-dimensional coordinates of the defect center, and converting the four corner point coordinates of the rectangular coordinate frame through T camera-to-gripper to obtain the three-dimensional coordinates for calculating the defect size, and the specific calculation process and data are as follows:

[0135] 1. The parameters of the two-dimensional camera are calculated as follows:

[0136]

[0137] The distortion coefficient d = [0.20334415671740425, 4.904385112163188, -0.0022254112636877267, 1.4572813224837689e-05, -1037.0344399454984];

[0138] The re-projection error = 0.047132498496030244;

[0139]

[0140] 2. The actual three-dimensional camera coordinates of the two-dimensional pixel coordinate matrix in the two-dimensional camera coordinate system are calculated as follows:

[0141] The target detection neural network model is used to detect defects in the two-dimensional image and output the rectangular coordinate frame of the defects, and the four corner point coordinates of the rectangular coordinate frame are [1136.0, 1818.0], [1186.0, 1818.0], [1136.0, 1872.0], [1186.0, 1872.0], and the center point coordinates are [1161.0, 1845.0];

[0142] The actual three-dimensional camera coordinates of the four corner point coordinates in the two-dimensional camera coordinate system are calculated as follows:

[0143] The three-dimensional coordinates corresponding to [1136.0, 1818.0] are [-14.37580796, -5.92899294, 270.5];

[0144] The three-dimensional coordinates corresponding to [1186.0, 1818.0] are [-13.83966873, -5.92899294, 270.5];

[0145] [1136.0, 1872.0] corresponding three-dimensional coordinates: [-14.37580796, -5.34946993, 270.5];

[0146] [1186.0, 1872.0] corresponding three-dimensional coordinates: [-13.83966873, -5.34946993, 270.5];

[0147] The actual three-dimensional camera coordinates of the center point coordinates in the two-dimensional camera coordinate system are calculated as follows:

[0148] [-14.10773834, -5.63923143, 270.5].

[0149] 3. The three-dimensional coordinates of the center point coordinates under the robot base coordinate system are calculated as follows:

[0150] Through T camera-to-base The center point coordinates are converted into three-dimensional coordinates under the robot base coordinate system: [-631.83898003,

[0151] -128.41387102, 254.0063209].

[0152] 4. The three-dimensional coordinates of the center point coordinates under the robot gripper coordinate system are calculated as follows:

[0153]

[0154] Through T base-to-gripper The three-dimensional coordinates of the center point coordinates under the robot base coordinate system are converted into three-dimensional coordinates under the robot gripper coordinate system: [-3.98719122, 37.2480781, 36.18581824].

[0155] 5. The three-dimensional coordinates of the center point coordinates under the robot base coordinate system are calculated as follows:

[0156]

[0157] Through T gripper-to-base The three-dimensional coordinates of the center point coordinates under the robot gripper coordinate system are converted into three-dimensional coordinates under the robot base coordinate system: [-651.80876029, 255.67971206, 319.5136791].

[0158] 6. The gripper holds the blade to [-651.80876029, 255.67971206, 319.5136791], and then a three-dimensional camera is used to collect a large area of gray scale image and point cloud data image (as shown in Figure 6 ).

[0159] 7. Calculate the defect size as follows:

[0160] By T camera-to-base Convert the four corner point coordinates into three-dimensional coordinates in the robot base coordinate system, respectively:

[0161] [1136.0, 1818.0] corresponding three-dimensional coordinates: [-631.83906977, -128.14745721, 254.29644636];

[0162] [1186.0, 1818.0] corresponding three-dimensional coordinates: [-631.84835501, -128.68251816, 254.2949805];

[0163] [1136.0, 1872.0] corresponding three-dimensional coordinates: [-631.82960527, -128.14523899, 253.71765873];

[0164] [1186.0, 1872.0] corresponding three-dimensional coordinates: [-631.83889034, -128.68031196, 253.71616966];

[0165] Due to slight errors in the conversion data, the three-dimensional coordinates of the four corner points after compensation and lengthening and widening by 1mm are:

[0166] [1136.0, 1818.0] corresponding three-dimensional coordinates: [-631.83906977, -127.14745721, 255.29644636];

[0167] [1186.0, 1818.0] corresponding three-dimensional coordinates: [-631.84835501, -129.68251816, 255.2949805];

[0168] [1136.0, 1872.0] corresponding three-dimensional coordinates: [-631.82960527, -127.14523899, 252.71765873];

[0169] [1186.0, 1872.0] corresponding three-dimensional coordinates: [-631.83889034, -129.68031196, 252.71616966];

[0170] The length, width and height of the defect are calculated as follows: length ≈ 0.0187mm, width ≈ 2.5372mm, height ≈ 2.5802mm.

[0171] 8. According to the length, width and height of the defect, the gray scale image and the point cloud data image of the larger area are cut to obtain the gray scale image and the point cloud data image of the smaller area, respectively as shown in Figure 7 and Figure 8 .

[0172] 9. The point cloud data of the cut smaller area is subjected to surface fitting to obtain a smooth surface model as shown in Figure 9 .

[0173] 10. The gray scale image of the smaller area is processed using an instance segmentation neural network model to obtain a mask map as shown in Figure 10 .

[0174] 11. According to the defect edge on the mask map, the sub point cloud data image of the defect edge is extracted from the point cloud data image of the smaller area as shown in Figure 11 .

[0175] 12. The sub point cloud data is subjected to difference calculation with the smooth surface model, and finally the type of the defect is obtained as a pit with a pit diameter of 0.549 mm and a pit depth of -0.158 mm, and the judgment result is unqualified.

[0176] The related content of the utility model has been described above. The ordinary skilled in the art will be able to realize the utility model based on these descriptions. Based on the above content of the utility model, all other embodiments obtained by the ordinary skilled in the art without creative labor shall belong to the protection scope of the utility model.

Claims

1. An engine blade defect detection system, characterized by: The application relates to a leaf picking and moving device. The device comprises: a tray (100) for storing leaves; the tray (100) is provided with a leaf storage groove; a robot (200) for picking and moving leaves; the robot (200) is provided with a gripper (210) and a mechanical arm (220); a two-dimensional camera (300) for taking pictures of the leaves held by the gripper (210); a three-dimensional camera (400) for taking pictures of the leaves held by the gripper (210); 2. The engine blade defect detection system of claim 1, wherein: a light supplementing light source for supplementing light when the two-dimensional camera (300) and the three-dimensional camera (400) take pictures.

3. The engine blade defect detection system of claim 1, wherein: The mechanical arm (220) can rotate and lift the leaves.

4. The engine blade defect detection system of claim 1, wherein: The three-dimensional camera (400) is a monocular structured light camera.

5. The engine blade defect detection system of claim 1, wherein: The two-dimensional camera (300) is arranged in front of the robot (200), the tray (100) is arranged beside the robot (200), and the three-dimensional camera (400) is arranged between the tray (100) and the two-dimensional camera (300) and is arranged obliquely towards the robot (200).

6. The engine blade defect detection system of claim 5, wherein: The light supplementing light source comprises a strip-shaped light source (510) and a ring-shaped light source (520); the ring-shaped light source (520) is arranged in front of the two-dimensional camera (300), and the strip-shaped light source (510) is arranged between the two-dimensional camera (300) and the robot (200).

7. The engine blade defect detection system of claim 1, wherein: The strip-shaped light source (510) is at least two.

8. The engine blade defect detection system of claim 7, wherein: The device further comprises a test table (600); the two-dimensional camera (300), the three-dimensional camera (400) and the light supplementing light source are fixed on the test table (600) through adjustable supports, and the robot (200) is fixed on the test table (600) through a base (230).

9. The engine blade defect detection system of claim 8, wherein: The surface of the test table (600) is assembled by T-shaped grooves; the adjustable support comprises a U-shaped base (710), a stand (720) and a hinged seat (730); the U-shaped base (710) is detachably connected with the T-shaped groove and can slide horizontally along the T-shaped groove; the two-dimensional camera (300), the three-dimensional camera (400) and the light supplementing light source are detachably connected with the stand (720) through the vertical through holes of the hinged seat (730) and can slide vertically and rotate horizontally along the stand (720); and the base (230) is detachably connected with the T-shaped groove and can slide horizontally along the T-shaped groove.

10. The engine blade defect detection system of claim 1, wherein: The hinged seat (730) connected with the two-dimensional camera (300) or the three-dimensional camera (400) further comprises a horizontal through hole; the adjustable support further comprises a cross rod (740); the cross rod (740) is detachably connected with the stand (720) through the horizontal through hole of the hinged seat (730) and can move along the horizontal through hole; and the end of the cross rod (740) is connected with the two-dimensional camera (300) or the three-dimensional camera (400). The device further comprises a terminal device; the terminal device communicates with the two-dimensional camera (300), the three-dimensional camera (400) and the robot (200) through a wireless network and controls the gripper (210) to pick and move leaves.

Images