An in-machine calibration method for vision units in a robotic hole-making system

Abstract

This invention discloses an on-machine calibration method for vision units in a robotic hole-making system, relating to the field of digital assembly manufacturing of aircraft. The method includes the following steps: S10: When a unit in the end effector is working, other units are moved from the working position, and this unit is moved to the working position; S20: The pose of the end effector is adjusted so that the hole-making direction coincides with the normal vector of the test cutting plate, and the end effector processes a calibration hole on the test cutting plate; S30: The calibration hole in S20 is measured using an industrial camera based on the vision unit, and a reference hole feature segmentation and detection method is used to obtain the pixel diameter and pixel position coordinates of the calibration hole in the image coordinate system; S40: The actual diameter of the calibration hole in S30 is measured using a hole-detecting unit, and the intrinsic parameters of the industrial camera are calculated to achieve on-machine calibration of the vision unit of the hole-making system. This method ensures measurement and positioning accuracy while reducing costs, improving the efficiency and positioning accuracy of the robotic hole-making system, eliminating the need for manual intervention, and improving the efficiency and convenience of the calibration process.

Description

Technical Field

[0001] This invention relates to the field of digital assembly and manufacturing of aircraft, and in particular to an in-flight calibration method for vision units in a robotic hole-making system. Background Technology

[0002] When using a robotic drilling system to drill holes in aircraft panels, the theoretical model of the system and the panel forms the basis for the drilling process. However, discrepancies between the theoretical model and the actual drilling conditions lead to positional errors. Considering low cost, non-contact operation, and satisfactory measurement accuracy, a vision unit can be integrated into the robotic drilling system for workpiece positioning. Effective calibration is fundamental to the accurate measurement by the vision unit.

[0003] In existing technologies, visual unit calibration methods often require high-precision dedicated calibration boards. The installation of these boards and multi-view imaging typically lead to prolonged interruptions in the robotic hole-making system's operation, reducing production efficiency and usability, and increasing production costs. Furthermore, online calibration of the visual units is difficult to achieve, failing to meet the on-machine calibration requirements of visual units during robotic hole-making. Therefore, this invention proposes an on-machine calibration method for visual units in robotic hole-making systems, enabling on-machine calibration of visual units within the system. Summary of the Invention

[0004] To overcome the technical problems of existing technologies that require the use of dedicated calibration plates, have cumbersome calibration processes, and require manual intervention, resulting in long-term interruptions in the processing of robotic hole-making systems, this invention provides an on-machine calibration method for vision units in robotic hole-making systems. The actual size of the calibration hole is obtained through a hole probe unit, and the camera's intrinsic parameters and the hand-eye relationship of the robotic hole-making system are calculated in conjunction with the reference hole image obtained by the vision unit. Combined with the robot's forward kinematics model and the tool coordinate system, the on-machine calibration of the vision unit during robotic hole-making is finally achieved.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0006] A method for on-machine calibration of vision units for a robotic hole-making system includes the following steps:

[0007] S10: When a unit in the end effector of the robot hole-making system is working, it moves away from other units from its working position and moves the unit to a working position located on the geometric symmetry plane of the end effector.

[0008] S20: Adjust the position of the end effector so that the hole-making direction coincides with the normal vector of the test cutting plate, and the end effector processes the calibration hole on the test cutting plate;

[0009] S30: The calibration hole in S20 is measured by an industrial camera based on vision units. The pixel diameter and pixel position coordinates of the calibration hole in the image coordinate system are obtained by using the reference hole feature segmentation and detection method.

[0010] S40: The actual diameter of the calibration hole in S30 is measured by the borescope unit, the internal parameters of the industrial camera in the vision unit are calculated, the position coordinates of the calibration hole in the camera coordinate system are solved, and the hand-eye relationship of the robot hole-making system is solved to realize the on-machine calibration of the vision unit of the hole-making system.

[0011] Preferably, in S10, when a certain unit of the end effector of the robot hole-making system is working, other units in the end effector are moved away from the working position, and the unit is moved to the working position located on the geometric symmetry plane of the end effector for performing hole-making, visual measurement and hole exploration tasks.

[0012] Preferably, in step S20, the robot hole-making system moves the end effector to the calibration hole processing position on the test cutting plate according to the NC machining file for calibration hole making, adjusts the pose of the end effector so that the hole-making direction coincides with the normal vector of the test cutting plate, and moves the hole-making unit of the end effector to the working position, and the end effector processes the calibration hole on the test cutting plate.

[0013] Preferably, in step S30, after the robot hole-making system processes the calibration hole on the test cutting plate according to the program logic of the NC machining file, it keeps the robot and end effector pose unchanged, moves the vision unit of the end effector to the working position, and measures the calibration hole processed in step S20 by using an industrial camera based on the vision unit. Then, it uses a deep learning-based reference hole feature segmentation and detection method to obtain the pixel diameter and pixel position coordinates of the calibration hole in the image coordinate system, thereby realizing the geometric feature detection of the calibration hole based on deep learning.

[0014] Preferably, in step S40, after completing the geometric feature detection of the calibration hole, the robot and end effector poses remain unchanged. The robot hole-making system moves the hole probe unit of the end effector to the working position according to the program logic of the NC machining file. The actual diameter of the calibration hole measured by the vision unit in step S30 is measured by the hole probe unit. Based on the actual diameter of the calibration hole measured by the hole probe unit and the pixel diameter of the calibration hole obtained in step S30, the internal parameters of the industrial camera in the vision unit are calculated. The position coordinates of the calibration hole in the camera coordinate system are obtained by using the pixel position coordinates of the calibration hole. Thus, the hand-eye relationship of the robot hole-making system, that is, the pose relationship between the robot hole-making system TCP and the vision unit, is solved, and the on-machine calibration of the vision unit is realized.

[0015] Preferably, in step S30, the reference hole feature segmentation and detection method based on deep learning includes the following steps:

[0016] S31: An industrial camera with a vision unit captures images of the reference hole and calculates the original image of the reference hole. Visual saliency images Normalize its significance value to an interval The integer within the range, then set an appropriate fixed threshold. For visually saliency images Thresholding is performed to obtain its binary image;

[0017] S32: Visually saliency images The thresholded binary image is used as a semi-supervised semantic label for the reference hole image. A small reference hole feature segmentation U-Net deep neural network is built. Exponential linear units are used instead of modified linear units. Pooling indexing is used and dropblocks are added at the end of each convolutional layer. Deformable convolution is used. The reference hole feature segmentation U-Net network is trained based on the semi-supervised semantic labels of the reference hole image. After each training, the reference hole feature regions predicted by the reference hole feature segmentation U-Net deep neural network model are further optimized using morphological methods to obtain semantic labels, thus realizing iterative training of the reference hole feature segmentation U-Net deep neural network.

[0018] S33: Using a contour tracking algorithm, the edge contours in the binary segmentation image of the reference hole features output by the U-Net deep neural network are retrieved to obtain the reference hole contours. Then, a random selection is made from the retrieved contours. For each point, fit a candidate ellipse, calculate the distance between the reference hole contour point and each candidate ellipse, count the number of contour points within the specified distance tolerance range of each candidate ellipse, and take the candidate ellipse with the most points as the ellipse fitting result of the reference hole feature.

[0019] Preferably, in S31, the visual saliency image The calculation formula is as follows:

[0020] ;

[0021] in, This is the original reference hole image. The average pixel value, This is the original reference hole image. Gaussian filtered image.

[0022] Preferably, in S32, pooling indexing is used and dropblocks are added at the end of each convolutional layer, and deformable convolution is used to reduce training parameters and prevent overfitting so that it works under the training image conditions.

[0023] Preferably, in step S33, the maximum number of candidate ellipses is set to N, and N candidate ellipses are fitted. Calculate the distance between the reference hole profile point and each candidate ellipse, count the number of profile points within the specified distance tolerance range of each candidate ellipse, and select the candidate ellipse with the most points as the ellipse fitting result for the reference hole feature. The ellipse fitting equation is as follows:

[0024] ;

[0025] in, These are the coordinates of the reference hole profile point. These are the coordinates of the center of the ellipse fitted by the reference hole. These are the semi-major and semi-minor axis lengths of the ellipse fitted by the reference hole, respectively. It is the rotation angle of the ellipse fitted by the reference hole.

[0026] Preferably, in step S40, the on-machine calibration calculation steps of the vision unit of the hole-making system are as follows:

[0027] S41: The diameter of the calibration hole machined in S20 is measured using the borehole probing unit of the end effector, and denoted as... ;

[0028] S42: Let the pixel diameter of the calibration hole obtained by deep learning-based geometric feature detection in S30 be . ;

[0029] S43: Note Here are the camera's intrinsic parameters on the X and Y axes of the camera coordinate system. The camera's intrinsic parameters are calculated using the following formulas: ;

[0030] S44: Let the pixel coordinates of the calibration hole obtained from the deep learning-based geometric feature detection of the calibration hole in S30 be... ;

[0031] S45: Adjust the camera coordinate system Set at the center of the image, camera coordinate system Each axis and robot drilling system Parallel axes, hand-eye relationship matrix of robotic hole-making system The specific calculation formula is as follows:

[0032] ;

[0033] in,

[0034] .

[0035] Compared with the prior art, the advantages of the present invention are:

[0036] This invention provides an on-machine calibration method for vision units by combining vision units and borehole detection units. This method ensures measurement and positioning accuracy while reducing costs, improving the efficiency and positioning accuracy of the robotic borehole system. Furthermore, it eliminates the need for a dedicated calibration plate and manual intervention, simplifying the calibration process and enhancing its efficiency and convenience. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0038] Figure 1 This invention provides a schematic flowchart of an on-machine calibration method for a vision unit in a robot hole-making system.

[0039] Figure 2 This is a schematic diagram of the layout of the main unit of the end effector of the robot hole-making system in this invention;

[0040] Figure 3 This is a schematic diagram of the on-machine calibration of the vision unit in this invention.

[0041] In the diagram: 1. Base, 2. Hole probing unit, 3. Hole making unit, 4. Vision unit, 5. Calibration hole, 6. Image center, 7. Geometric symmetry plane. Detailed Implementation

[0042] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0043] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0044] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0045] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0046] See Figure 1-3 This is an embodiment of an on-machine calibration method for a vision unit 4 in a robot hole-making system according to the present invention, comprising the following steps:

[0047] S10: When a unit in the end effector of the robot hole-making system is working, it moves away from other units from its working position and moves the unit to the working position located on the geometric symmetry plane 7 of the end effector.

[0048] S20: Adjust the position of the end effector so that the hole-making direction coincides with the normal vector of the test cutting plate, and the end effector processes the calibration hole 5 on the test cutting plate;

[0049] S30: The calibration hole 5 in S20 is measured by an industrial camera based on vision unit 4. The pixel diameter and pixel position coordinates of the calibration hole 5 in the image coordinate system are obtained by using the reference hole feature segmentation and detection method.

[0050] S40: The actual diameter of the calibration hole 5 in S30 is measured by the borehole probe unit 2, the internal parameters of the industrial camera in the vision unit 4 are calculated, the position coordinates of the calibration hole 5 in the camera coordinate system are solved, and the hand-eye relationship of the robot hole-making system is solved to realize the on-machine calibration of the vision unit 4 of the hole-making system.

[0051] In this embodiment, during S10, when a certain unit of the end effector of the robot hole-making system is working, other units in the end effector are moved away from their working positions, and the unit is moved to a working position located on the geometric symmetry plane 7 of the end effector to perform hole-making, visual measurement, and hole exploration tasks.

[0052] In this embodiment, in step S20, an NC machining file for drilling the calibration hole 5 is prepared. The NC machining file includes program logic, drilling position coordinate parameters, and machining process parameters. According to the NC machining file for drilling the calibration hole 5, the robot drilling system moves the end effector to the drilling position of the calibration hole 5 on the test cutting plate, adjusts the pose of the end effector so that the drilling direction coincides with the normal vector of the test cutting plate, so as to ensure the perpendicularity of the axis of the drilled calibration hole 5 and the surface of the test cutting plate. At the same time, the drilling unit 3 of the end effector is moved to the working position, and the end effector uses a drill bit to drill the circular calibration hole 5 on the test cutting plate.

[0053] In this embodiment, in step S30, the robot hole-making system processes the calibration hole 5 on the test cutting plate according to the program logic of the NC machining file. After keeping the robot and end effector pose unchanged, the vision unit 4 of the end effector is moved to the working position. The calibration hole 5 processed in step S20 is measured by the industrial camera based on the vision unit 4. The reference hole feature segmentation and detection method based on deep learning is used to obtain the pixel diameter and pixel position coordinates of the calibration hole 5 in the image coordinate system, thereby realizing the geometric feature detection of the calibration hole based on deep learning.

[0054] In this embodiment, in step S40, after completing the geometric feature detection of the calibration hole, the robot and end effector poses remain unchanged. The robot hole-making system, according to program logic, moves the hole-detecting unit 2 of the multi-functional end effector to the working position. Based on the hole-detecting unit 2, the actual diameter of the calibration hole 5, processed in step S2 and measured by the vision unit 4 in step S3, is measured. Based on the actual diameter of the calibration hole 5 measured by the hole-detecting unit 2 and the pixel diameter of the calibration hole 5 obtained in step S3, the intra-camera parameters in the vision unit 4 are calculated. Based on these parameters, the position coordinates of the calibration hole 5 in the camera coordinate system are obtained, thereby solving the hand-eye relationship of the robot hole-making system, i.e., the robot hole-making system... The pose relationship between the visual system and the system enables on-machine calibration of the visual system.

[0055] In this embodiment, step S30, the deep learning-based reference hole feature segmentation and detection method includes the following steps:

[0056] S31: Use an industrial camera with vision unit 4 to capture images of the reference hole and calculate the original image of the reference hole. Visual saliency images Normalize its significance value to an interval The integer within the range, then set an appropriate fixed threshold. For visually saliency images Thresholding is performed to obtain its binary image;

[0057] S32: Visually saliency images The thresholded binary image is used as a semi-supervised semantic label for the reference hole image. A small reference hole feature segmentation U-Net deep neural network is built. Exponential linear units are used instead of modified linear units. Pooling indexing is used and dropblocks are added at the end of each convolutional layer. Deformable convolution is used. The reference hole feature segmentation U-Net network is trained based on the semi-supervised semantic labels of the reference hole image. After each training, the reference hole feature regions predicted by the reference hole feature segmentation U-Net deep neural network model are further optimized using morphological methods to obtain semantic labels, thus realizing iterative training of the reference hole feature segmentation U-Net deep neural network.

[0058] S33: Using a contour tracking algorithm, the edge contours in the binary segmentation image of the reference hole features output by the U-Net deep neural network are retrieved to obtain the reference hole contours. Then, a random selection is made from the retrieved contours. For each point, fit a candidate ellipse, calculate the distance between the reference hole contour point and each candidate ellipse, count the number of contour points within the specified distance tolerance range of each candidate ellipse, and take the candidate ellipse with the most points as the ellipse fitting result of the reference hole feature.

[0059] like Figure 2 The diagram shows the layout of the main units of the end effector of the robot drilling system in this invention. The end effector includes a drilling unit 3, a vision unit 4, and a hole-exploring unit 2. These units can move laterally along the slide of the end effector. When a unit of the end effector is working, other units are moved from the working position to the working position. The working position is located on the geometric symmetry plane 7 of the actuator. Drilling, vision measurement, and hole exploration tasks can only be performed after the corresponding unit has moved to the working position. The drilling unit 3, vision unit 4, and hole-exploring unit 2 are mounted on the base 1 of the end effector, enabling lateral movement along the slide of the end effector.

[0060] In this embodiment, in S31, the visual saliency image The calculation formula is as follows:

[0061] ;

[0062] in, This is the original reference hole image. The average pixel value, This is the original reference hole image. Gaussian filtered image.

[0063] In this embodiment, in step S32, pooling indexing is used and dropblocks are added at the end of each convolutional layer. Deformable convolution is also used to reduce training parameters and prevent overfitting so that it works under the training image conditions.

[0064] In this embodiment, in step S33, candidate ellipses are fitted, and the maximum number of candidate ellipses is set to N, fitting N candidate ellipses. Calculate the distance between the reference hole profile point and each candidate ellipse, count the number of profile points within the specified distance tolerance range of each candidate ellipse, and select the candidate ellipse with the most points as the ellipse fitting result for the reference hole feature. The ellipse fitting equation is as follows:

[0065] ;

[0066] in, These are the coordinates of the reference hole profile point. These are the coordinates of the center of the ellipse fitted by the reference hole. These are the semi-major and semi-minor axis lengths of the ellipse fitted by the reference hole, respectively. It is the rotation angle of the ellipse fitted by the reference hole.

[0067] In this embodiment, the on-machine calibration calculation steps of the vision unit 4 of the hole-making system in S40 are as follows:

[0068] S41: The diameter of the calibration hole 5 machined in S20 is measured using the borehole probing unit 2 of the end effector, and denoted as... ;

[0069] S42: Let the 5-pixel diameter of the calibration hole obtained from the deep learning-based geometric feature detection in S30 be... ;

[0070] S43: Note Here are the camera's intrinsic parameters on the X and Y axes of the camera coordinate system. The camera's intrinsic parameters are calculated using the following formulas: ;

[0071] S44: Let the 5-pixel coordinates of the calibration hole obtained from the deep learning-based geometric feature detection in S30 be... ;

[0072] S45: Adjust the camera coordinate system Set at image center 6, camera coordinate system Each axis and robot drilling system Parallel axes, hand-eye relationship matrix of robotic hole-making system The specific calculation formula is as follows:

[0073] ;

[0074] in,

[0075] .

[0076] like Figure 3 The diagram shown is a schematic of the on-machine calibration of vision unit 4. Figure 3 In The diameter of the calibration hole 5, which is machined and measured by the borehole probe unit 2 of the end effector, is obtained in mm. The calibration hole diameter is 5 pixels, obtained from deep learning-based geometric feature detection. The coordinates of the 5-pixel position of the calibration hole are obtained from the geometric feature detection of the calibration hole based on deep learning.

[0077] remember The camera's intrinsic parameters on the X and Y axes are the camera coordinate system parameters. For simplicity, the camera's intrinsic parameters are calculated using the following formulas:

[0078] ;

[0079] camera coordinate system Set the camera coordinate system to center 6 in the image. Each axis and robot drilling system Parallel axes, hand-eye relationship matrix of robotic hole-making system The specific calculation formula is as follows:

[0080] ;

[0081] in,

[0082] .

[0083] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0084] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for on-machine calibration of vision units in a robotic hole-making system, characterized in that, Includes the following steps: S10: When a unit in the end effector of the robot hole-making system is working, it moves away from other units from its working position and moves the unit to a working position located on the geometric symmetry plane of the end effector. S20: Adjust the position of the end effector so that the hole-making direction coincides with the normal vector of the test cutting plate, and the end effector processes the calibration hole on the test cutting plate; S30: The calibration hole in S20 is measured by an industrial camera based on vision units. The pixel diameter and pixel position coordinates of the calibration hole in the image coordinate system are obtained by using the reference hole feature segmentation and detection method. S40: The actual diameter of the calibration hole in S30 is measured by the borescope unit, the internal parameters of the industrial camera in the vision unit are calculated, the position coordinates of the calibration hole in the camera coordinate system are solved, and the hand-eye relationship of the robot hole-making system is solved to realize the on-machine calibration of the vision unit of the hole-making system. In step S30, the deep learning-based benchmark hole feature segmentation and detection method includes the following steps: S31: An industrial camera with a vision unit captures images of the reference hole and calculates the original image of the reference hole. Visual saliency images Normalize its significance value to an interval The integer within the range, then set an appropriate fixed threshold. For visually saliency images Thresholding is performed to obtain its binary image; S32: Visually saliency images The thresholded binary image is used as a semi-supervised semantic label for the reference hole image. A U-Net deep neural network for small reference hole feature segmentation is established. Exponential linear units are used instead of modified linear units. Pooling indexing is used and dropblocks are added at the end of each convolutional layer. Deformable convolution is used. The reference hole feature segmentation U-Net network is trained based on the semi-supervised semantic labels of the reference hole image. After each training, the reference hole feature regions predicted by the reference hole feature segmentation U-Net deep neural network model are further optimized using morphological methods to obtain semantic labels, thus realizing iterative training of the reference hole feature segmentation U-Net deep neural network. S33: Using a contour tracking algorithm, the edge contours in the binary segmentation image of the reference hole features output by the U-Net deep neural network are retrieved to obtain the reference hole contours. Then, a random selection is made from the retrieved contours. For each point, fit a candidate ellipse, calculate the distance between the reference hole contour point and each candidate ellipse, count the number of contour points within the specified distance tolerance range of each candidate ellipse, and take the candidate ellipse with the most points as the ellipse fitting result of the reference hole feature.

2. The on-machine calibration method for a vision unit in a robot hole-making system according to claim 1, characterized in that, In S10, when a certain unit of the end effector of the robot hole-making system is working, other units in the end effector are moved away from the working position and the unit is moved to the working position located on the geometric symmetry plane of the end effector to perform hole-making, vision measurement and hole exploration tasks.

3. The on-machine calibration method for a vision unit in a robot hole-making system according to claim 1, characterized in that, In step S20, the robot hole-making system moves the end effector to the calibration hole processing position on the test cutting plate according to the NC machining file used for calibration hole making, adjusts the pose of the end effector so that the hole-making direction coincides with the normal vector of the test cutting plate, and moves the hole-making unit of the end effector to the working position, and the end effector processes the calibration hole on the test cutting plate.

4. The method for on-machine calibration of a vision unit for a robot hole-making system according to claim 1, characterized in that, In step S30, the robot hole-making system processes calibration holes on the test cutting plate according to the program logic of the NC machining file. After keeping the robot and end effector pose unchanged, the vision unit of the end effector is moved to the working position. The calibration holes processed in step S20 are measured by an industrial camera based on the vision unit. The pixel diameter and pixel position coordinates of the calibration holes in the image coordinate system are obtained by using a reference hole feature segmentation and detection method based on deep learning, thereby realizing the geometric feature detection of calibration holes based on deep learning.

5. The method for on-machine calibration of a vision unit for a robot hole-making system according to claim 1, characterized in that, In step S40, after completing the geometric feature detection of the calibration hole, the robot and end effector poses remain unchanged. The robot hole-making system moves the hole probe unit of the end effector to the working position according to the program logic of the NC machining file. The actual diameter of the calibration hole measured by the vision unit in step S30 is measured by the hole probe unit. Based on the actual diameter of the calibration hole measured by the hole probe unit and the pixel diameter of the calibration hole obtained in step S30, the internal parameters of the industrial camera in the vision unit are calculated. The position coordinates of the calibration hole in the camera coordinate system are obtained by using the pixel position coordinates of the calibration hole. Thus, the hand-eye relationship of the robot hole-making system, that is, the pose relationship between the robot hole-making system TCP and the vision unit, is solved, and the on-machine calibration of the vision unit is realized.

6. The method for on-machine calibration of a vision unit for a robot hole-making system according to claim 1, characterized in that, In S31, the visual saliency image The calculation formula is as follows: ； in, This is the original reference hole image. The average pixel value, This is the original reference hole image. Gaussian filtered image; The x and y values ​​are the coordinates of the reference hole profile point.

7. The method for on-machine calibration of a vision unit for a robot hole-making system according to claim 1, characterized in that, In S32, pooling indexing is used and dropblocks are added at the end of each convolutional layer. Deformable convolution is also used to reduce training parameters and prevent overfitting so that it works under the training image conditions.

8. The method for on-machine calibration of a vision unit for a robot hole-making system according to claim 1, characterized in that, In step S33, candidate ellipses are fitted, and the maximum number of candidate ellipses is set to [value missing]. , fitting One alternative ellipse Calculate the distance between the reference hole profile point and each candidate ellipse, count the number of profile points within the specified distance tolerance range of each candidate ellipse, and select the candidate ellipse with the most points as the ellipse fitting result for the reference hole feature. The ellipse fitting equation is as follows: ； in, These are the coordinates of the reference hole profile point. These are the coordinates of the center of the ellipse fitted by the reference hole. These are the semi-major and semi-minor axis lengths of the ellipse fitted by the reference hole, respectively. It is the rotation angle of the ellipse fitted by the reference hole.

9. The method for on-machine calibration of a vision unit for a robot hole-making system according to claim 5, characterized in that, In step S40, the on-machine calibration calculation steps of the vision unit of the hole-making system are as follows: S41: The diameter of the calibration hole machined in S20 is measured using the borehole probing unit of the end effector, and denoted as... ; S42: Let the pixel diameter of the calibration hole obtained by deep learning-based geometric feature detection in S30 be . ; S43: Note For the camera coordinate system in and The camera's intrinsic parameters on the axis are calculated using the following formula: ； S44: Let the pixel coordinates of the calibration hole obtained from the deep learning-based geometric feature detection of the calibration hole in S30 be... ; S45: Adjust the camera coordinate system Set at the center of the image, camera coordinate system Each axis is parallel to the TCP axis of the robot hole-making system; the hand-eye relationship matrix of the robot hole-making system. The specific calculation formula is as follows: ； in, 。

Images