Multi-sensor integrated drilling and milling quality detection end effector and method

Abstract

The invention discloses a multi-sensor integrated drilling and milling quality detection end effector which comprises a connecting piece used for being connected with a flange at the tail end of an industrial robot and further comprises a main bearing piece, the tail end of the main bearing piece is connected with the connecting piece, the main bearing piece is of a cuboid structure, and the four rectangular surfaces of the main bearing piece are each provided with a binocular vision distance measuring subsystem, a binocular vision distance measuring subsystem and a binocular vision distance measuring subsystem. The sensor is used for acquiring relative distance and angle attitude between the tail end of the actuator and the surface of a to-be-measured workpiece; the drilling size visual detection subsystem is used for classifying drilling holes and detecting drilling size precision; the milling surface roughness visual detection subsystem is used for detecting the surface roughness distribution condition of the workpiece to be detected and forming a visual image; the ultrasonic thickness measurement subsystem is used for visually displaying the wall thickness distribution state of the to-be-detected workpiece in the detection area so as to identify potential defects; the four rectangular surfaces of the main bearing part are each provided with two parallel grooves in the length direction, and the grooves are used for being detachably and movably connected with the subsystems.

Description

Technical Field

[0001] This invention belongs to the field of intelligent drilling and milling technology, specifically relating to an end effector and method for drilling and milling quality inspection that integrates multiple functions such as drilling size detection, milling surface roughness detection, ultrasonic thickness measurement, illumination and distance measurement. Background Technology

[0002] Currently, industrial robots are increasingly being applied to drilling and milling composite machining, particularly in aerospace, automotive manufacturing, and high-end equipment industries. Leveraging the flexible, multi-degree-of-freedom characteristics of robots, efficient machining of complex surfaces can be achieved. However, existing end effectors primarily focus on performing machining tasks. While they possess a certain degree of automated drilling and milling capabilities, their bodies often lack integrated, high-precision post-machining inspection capabilities. This results in machining quality assessment still relying on manual disassembly of the workpiece and transfer to a dedicated measurement platform for inspection. This method is not only inefficient but also susceptible to human error and secondary clamping errors, limiting its application in high-precision manufacturing scenarios.

[0003] Therefore, integrating the end effector with a multimodal detection system to build an intelligent detection platform capable of drilling size detection, milling surface quality assessment, and material internal defect detection can significantly improve detection efficiency and accuracy. It also enables real-time data upload and closed-loop feedback control, providing a basis for adjusting subsequent processing states. This truly achieves in-situ detection and intelligent decision-making for processing quality, greatly enhancing the flexibility and intelligence of the manufacturing system. Such integrated platforms have significant engineering value and promising prospects for realizing an adaptive processing closed-loop system from "perception" to "judgment" to "control." Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multi-sensor integrated end effector and method for drilling and milling quality inspection. This device can perform in-situ detection of key parameters such as drilling size, milling surface roughness and structural thickness, thereby effectively improving detection efficiency and measurement accuracy.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A multi-sensor integrated end effector for drilling and milling quality inspection includes a connector for connecting to the end flange of an industrial robot, and a main support member whose end is connected to the connector. The main support member has a cuboid structure and four rectangular surfaces are respectively provided with:

[0007] A binocular vision ranging subsystem is used to obtain the relative distance and angular orientation between the actuator end effector and the surface of the workpiece to be measured;

[0008] A borehole size visual inspection subsystem is used to classify boreholes and inspect the accuracy of borehole dimensions.

[0009] The visual inspection subsystem for milled surface roughness is used to detect the surface roughness distribution of the workpiece and generate a visual image.

[0010] And an ultrasonic thickness measurement subsystem, used to visualize the wall thickness distribution of the workpiece under test in the detection area to identify potential defects;

[0011] The main support component has two parallel grooves along the length of each of its four rectangular surfaces for detachable and movable connection of various subsystems.

[0012] Furthermore, the binocular vision ranging subsystem includes a binocular camera and a camera support block. The camera support block is connected to the main support component via a connecting block. The binocular camera is fixed on the camera support block, and the lens orientation of the binocular camera is consistent with the front end orientation of the main support component.

[0013] Furthermore, during operation, the binocular vision ranging subsystem acquires real-time three-dimensional spatial images of the end effector and the surface of the workpiece to be measured, and generates depth information through a stereo matching algorithm. Based on the obtained depth information, it calculates the relative distance and angular attitude between the end effector and the surface of the workpiece to be measured, and transmits the ranging data to the robot control system in real time. Combined with the robot's kinematic model, it performs posture error compensation and pose correction operations to ensure that when the end effector reaches the detection position, the principal directions of its various detection subsystems are aligned with the normal direction of the target detection surface.

[0014] Furthermore, the drilling size visual inspection subsystem includes a ring light source, a light source bracket, a light source extension column, a corner bracket profile, a first industrial camera, and a secondary support component. The secondary support component has a cuboid structure and a volume smaller than the main support component. Each rectangular surface of the secondary support component has a groove along its length. The secondary support component is located on one side of the main support component. The first industrial camera is located on the side of the secondary support component away from the main support component, and the lens of the first industrial camera extends into the central through hole of the ring light source. The light source extension column is installed on one of the rectangular sides of the secondary support component adjacent to the main support component via the corner bracket profile. The front end of the light source extension column has a light source bracket with a circular through hole. The periphery of the circular through hole of the light source bracket is fastened to the bottom periphery of the ring light source by fasteners.

[0015] Furthermore, during operation, the borehole size visual inspection subsystem acquires a sequence of images with different exposure times using a first industrial camera through multiple exposure imaging. A high dynamic range (HDR) image is generated from these image sequences using multi-exposure image fusion technology. The HDR image is then subjected to polar coordinate transformation, mapping the circularly distributed borehole edges in the two-dimensional image space to a one-dimensional boundary space, transforming the edge detection problem from a two-dimensional contour to a one-dimensional signal change. A sub-pixel-level edge detection algorithm is used to process the transformed image, extracting the boundary coordinates of the borehole edges to obtain a sub-pixel edge point set. Based on the distribution and hierarchical structure of the edge stripes, the borehole size visual inspection subsystem automatically determines the borehole type. Finally, a circular geometric model is fitted based on the sub-pixel edge point set to obtain the key dimensional parameters of the borehole, including the center, diameter, borehole position deviation, stepped borehole diameter, and concentricity.

[0016] Furthermore, the visual inspection subsystem for milled surface roughness includes a dome light source, a dome light source bracket, a microscope head, a support, and a second industrial camera. The support is installed on one side of the main support component, and the microscope head is fixed to the support via a clamping device. The front end of the microscope head extends into a through hole in the middle of the dome light source bracket, and the dome light source is fixed to the dome light source bracket. The dome light source has a frustum-shaped structure that is larger at the front and smaller at the back, which is used to form diffuse reflection and uniform illumination when photographing surface textures. The through hole in the middle of the dome light source is connected to the through hole in the middle of the dome light source bracket.

[0017] Furthermore, when the milling surface roughness visual inspection subsystem is working, the end effector equipped with a second industrial camera scans along the workpiece surface along a preset S-shaped trajectory to cover the inspection area, acquiring surface images point by point; the acquired images are subjected to contrast-limited adaptive histogram equalization (CLAHE) preprocessing; the preprocessed images are input into a trained surface roughness regression model for feature extraction and numerical regression, outputting the roughness index value corresponding to each image and recording the spatial location number; based on the location number, the discrete roughness data is reconstructed into a two-dimensional heat map, and the surface roughness distribution is represented by color gradient.

[0018] Furthermore, the ultrasonic thickness measurement subsystem includes a probe bracket, an ultrasonic probe, a probe slide rod, a spring, a sliding sleeve, and a connecting block. The sliding sleeve is connected to one side of the main support component via the connecting block. The sliding sleeve has a hollow cavity structure, and one side of the sliding sleeve has a track hole communicating with the cavity. The end of the probe slide rod is movable back and forth and connected to the sliding sleeve, with its movement direction and distance restricted by the track hole. The spring is installed inside the cavity of the sliding sleeve, with one end of the spring contacting the bottom of the cavity inside the sliding sleeve and the other end contacting the end of the probe slide rod. The ultrasonic probe is fixed to the front end of the probe slide rod via the probe bracket. The central axis of the ultrasonic probe is parallel to the central axis of the probe slide rod, and a stop block is provided at the front of the probe slide rod.

[0019] Furthermore, during operation, the ultrasonic thickness measurement subsystem places the ultrasonic probe against the surface of the workpiece to be measured and scans the target area along a preset two-dimensional grid path, acquiring ultrasonic echo signals at the scanning nodes; the local thickness value corresponding to the scanning node is calculated through the sound velocity-time inversion relationship; the obtained local thickness value data is registered and calibrated in real time according to spatial coordinates and stored as three-dimensional point cloud data; all local thickness value data are visualized and reconstructed in the form of three-dimensional point cloud to generate a three-dimensional point cloud map or isosurface map of the thickness distribution.

[0020] This invention also provides a multi-sensor integrated method for detecting the quality of drilling and milling processes, comprising:

[0021] After the workpiece has been drilled and milled, the industrial robot is controlled by the robot control system to move the end effector to the target inspection area.

[0022] Start the binocular vision ranging subsystem to obtain the relative distance and angular orientation between the surfaces of the workpieces to be measured;

[0023] Perform attitude error compensation and pose correction based on attitude deviation to align the main direction of each subsystem with the normal of the detection surface;

[0024] Activate the borehole size visual inspection subsystem at an object distance of 400mm or more:

[0025] (101) Acquire multiple exposure images and fuse them to generate a high dynamic range (HDR) image;

[0026] (102) Perform polar coordinate transformation on high dynamic range (HDR) images to optimize edge detection;

[0027] (103) The hole edge coordinates are extracted using a sub-pixel level edge detection algorithm and the drilling type is automatically determined based on the edge morphology features;

[0028] (104) The key dimension parameters of the borehole were obtained by fitting calculation;

[0029] Start the visual inspection subsystem for milled surface roughness at an object distance of 5-150mm:

[0030] (201) Scan the detection area along the surface of the workpiece to be tested in a preset S-shaped trajectory and collect surface images point by point;

[0031] (202) Perform contrast-limited adaptive histogram equalization (CLAHE) preprocessing on the acquired images;

[0032] (203) Input the preprocessed image into the pre-trained surface roughness regression model, output the roughness index value corresponding to each image and record the spatial location number;

[0033] (204) Based on the location number, the discrete roughness data is reconstructed into a two-dimensional heat map, and the surface roughness distribution is characterized by color gradient.

[0034] The ultrasonic thickness measurement subsystem is activated after the ultrasonic probe makes contact with the surface of the workpiece.

[0035] (301) Control the ultrasonic probe to contact the surface of the workpiece with a contact force of 3-5N;

[0036] (302) Scan the target area according to the preset two-dimensional grid path and collect ultrasonic echo signals at the scanning nodes;

[0037] (303) Calculate the local thickness value of each scanning node based on the sound wave propagation time and associate it with the spatial coordinates;

[0038] (304) Generate a thickness distribution map from the three-dimensional point cloud to identify workpiece defects.

[0039] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0040] 1. This invention integrates a binocular vision ranging subsystem, a drilling size vision inspection subsystem, a milling surface roughness vision inspection subsystem, and an ultrasonic thickness measurement subsystem at the robot end effector, enabling in-situ quality inspection after drilling and milling. This avoids the inefficient method of disassembling and transferring workpieces for offline inspection after traditional processing, significantly improving inspection accuracy and efficiency, while reducing operational risks and enhancing the safety and automation level of on-site inspection.

[0041] Furthermore, each detection subsystem is integrated into the same end effector body, resulting in a compact overall structure with a small footprint, making it easy to deploy on various types of industrial robots or composite processing platforms. This integrated structure not only achieves the organic integration of multiple functions on a single platform but also reduces equipment deployment and maintenance costs, thereby enhancing the system's engineering practicality and functional integrity.

[0042] 2. The data acquisition actions of each detection subsystem in this invention are highly consistent, exhibiting excellent consistency in detection location and repeatability, effectively ensuring the comparability and stability of detection data. This solves the problems of time-consuming "disassembly-transfer-clamping" processes and the inability to provide online feedback in traditional offline detection workflows. Simultaneously, the detection results can be correlated one-to-one with specific processing tasks, facilitating quality control, error tracing, and data traceability management during production, providing a data foundation for building a closed-loop quality control system for intelligent manufacturing. This integrated approach enables in-situ multimodal detection, with detection data uploaded to the robot control system in real time. Closed-loop feedback allows for rapid adjustment of processing parameters, significantly improving production efficiency and quality control levels.

[0043] 3. This invention solves the problem of inaccurate positioning before drilling inspection, which easily introduces edge detection and geometric fitting errors, by performing binocular visual ranging first and then visually inspecting the borehole size. By aligning the image in real time through the binocular visual ranging subsystem before performing HDR-subpixel detection, the borehole image is always at the optimal focal length and viewing angle, significantly improving the accuracy of key borehole diameter parameters and concentricity measurement.

[0044] 4. This invention solves the problem of large focusing distance errors of the microscope head affecting the quantification results of roughness measurement by first performing binocular visual ranging and then performing milling surface roughness detection. First, a stereo matching algorithm is used for positioning, and then an S-shaped trajectory is used for scanning, ensuring accurate focusing of each frame of microscopic image acquisition and improving the predictive reliability of the roughness regression model.

[0045] 5. The borehole size visual inspection subsystem addresses issues such as strong reflections from metal borehole walls, blurred edges, and the difficulty of extracting accurate contours using traditional binarization algorithms by combining HDR image fusion, polar coordinate transformation, and subpixel edge detection. Multi-exposure fusion eliminates highlight interference and improves the contrast and detail quality of the borehole edge boundary region; polar coordinate transformation converts the borehole edge into a one-dimensional signal, improving the robustness and computational efficiency of subsequent edge extraction; and subpixel extraction further refines the boundary, achieving high-precision borehole diameter measurement with an accuracy of 0.01 mm.

[0046] 6. The visual inspection subsystem for milling surface roughness combines CLAHE preprocessing with a surface roughness regression model to address issues such as loss of image texture information and impact on regression accuracy caused by uneven lighting and shadow interference. Specifically, an "S"-shaped scanning strategy is used to acquire images point-by-point, ensuring complete coverage and uniform sampling of the entire inspection area, avoiding the omission of minute surface defects or local roughness fluctuations. Then, a contrast-limited adaptive histogram equalization CLAHE preprocessing method enhances the contrast of local image details, reducing detection interference caused by uneven lighting or workpiece surface shadows, thereby improving overall image quality and edge clarity. The preprocessed image undergoes feature extraction and numerical regression using the surface roughness regression model, resulting in more robust and comparable roughness indices.

[0047] 7. The ultrasonic thickness measurement subsystem employs a combination of spring preload and sliding guide structure to address issues such as unstable ultrasonic probe contact force and the susceptibility of thickness measurement data to vibration and poor contact. Specifically, the spring provides constant preload, while the sliding guide limits the stroke. Together, they ensure continuous and stable contact between the probe and the workpiece surface, improving the repeatability and reliability of ultrasonic thickness measurement.

[0048] 8. The present invention can reconstruct the detection results of all image regions into a two-dimensional thermal map-like visualization image according to their relative positions on the surface of the workpiece to be tested. Different colors represent roughness value ranges, clearly showing the processing uniformity, roughness gradient changes and possible abnormal areas of the entire workpiece surface.

[0049] It can visualize and reconstruct the collected thickness data in the form of a 3D point cloud, generating a 3D point cloud map or isosurface map of the thickness distribution. The 3D point cloud map or isosurface map uses spatial coordinates as the horizontal axis and thickness as the vertical axis, intuitively showing the wall thickness distribution of the workpiece throughout the inspection area. It can be used to identify potential defects such as localized excessive thinning, missing internal layers, or structural depressions.

[0050] The aforementioned technical methods solve the problems of difficulty in grasping the overall situation of scattered roughness and thickness data and lack of intuitive display methods; they reconstruct discrete data into two-dimensional heat maps and three-dimensional isosurface maps, which intuitively present the surface / internal quality distribution, make it easy to find abnormal areas, and assist in judgment and decision-making. Attached Figure Description

[0051] Figure 1 and Figure 2 These are all schematic diagrams of the end effector for drilling and milling quality inspection of the present invention from different perspectives.

[0052] Figure 3 This is a side view of the structural diagram showing the connection status between each subsystem and the main load-bearing component;

[0053] Figure 4 This is a flowchart of the detection process.

[0054] Reference numerals: 1-Dome light source, 2-Dome light source bracket, 3-Microscope head, 4-Subsystem support, 5-Industrial camera, 6-Actuator end connection flange, 7-Main support component, 8-Probe bracket, 9-Ultrasonic probe, 10-Probe slide bar, 11-Spring, 12-Sliding sleeve, 13-Profile groove connecting block, 14-Light source extension column, 15-Annular light source bracket, 16-Annular light source, 17-Angle bracket profile, 18-Industrial camera, 19-Secondary support component, 20-Binocular camera, 21-Camera support block, 22-Profile groove connecting block, 23-Profile groove connecting block, 24-Threaded fastener. Detailed Implementation

[0055] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0056] Example 1

[0057] See Figures 1 to 3 This embodiment provides a multi-sensor integrated drilling and milling machining quality inspection end effector, including a flange connector 6 for connecting to the end flange of an industrial robot, and a main support member 7. The end of the main support member 7 is connected to the bolt connector 6. The main support member 7 has a cuboid structure and four rectangular surfaces are respectively provided with a binocular vision ranging subsystem, a drilling size vision inspection subsystem, a milling surface roughness vision inspection subsystem, and an ultrasonic thickness measurement subsystem.

[0058] The end flange connector 6 is bolted to the end flange of the industrial robot, achieving quick, simple, and easy installation. The main support component 7 serves as the main skeleton structure of the end effector, providing unified support and connection for each functional subsystem, achieving modularity and functional integration of the structural layout, which is beneficial for the independent deployment and flexible combination of subsystems. Furthermore, in this embodiment, each of the four rectangular surfaces of the main support component 7 has two parallel grooves along its length, allowing for detachable and movable connection of the aforementioned subsystems.

[0059] Preferably, in this embodiment, the connector 6 is located at the end, i.e. the tail, of the main support member 7, the milling surface roughness visual inspection subsystem is installed on the top of the main support member 7, the ultrasonic thickness measurement subsystem is installed on the right side of the main support member 7, the drilling size visual inspection subsystem is installed at the bottom of the main support member 7, and the binocular visual distance measurement subsystem is installed on the left side of the main support member 7.

[0060] The visual inspection subsystem for milled surface roughness is used to acquire images of the microscopic surface after milling, thereby enabling visual analysis and evaluation of surface roughness. It includes a dome light source 1, a dome light source support 2, a microscope head 3, a bracket 4, and an industrial camera 5. The bracket 4 is bolted to the main support 7. The microscope head 3 is fixed to the bracket 4 via a clamping device. The front end of the microscope head 3 is connected to the dome light source support 2, and the dome light source 1 is fixed to the dome light source support 2. The dome light source 1 has a frustum-shaped structure, wider at the front and narrower at the back. This design provides uniform diffused illumination when photographing surface textures, ensuring inspection accuracy and precision, as shadows and reflections may occur during its use. Both the dome light source support 2 and the dome light source 1 have interconnected through holes in their middle sections. The microscope head 3 extends into and is fixed within the through holes of the dome light source support 2.

[0061] Preferably, in this embodiment, the upper part of the support 4 may be provided with an openable and symmetrically arranged clamping member, and the microscope tube of the microscope head 3 is fixed by locking the opposite clamping member with bolts; or it may be a clamping device with a circular through hole, and the purpose of fastening the microscope tube is achieved by screwing a bolt into the top of the clamping device. In this embodiment, the second method is adopted, which is more convenient and simple to operate.

[0062] The ultrasonic thickness measurement subsystem includes a probe support 8, an ultrasonic probe 9, a probe slide rod 10, a spring 11, a sliding sleeve 12, and a connecting block 13. In this embodiment, the sliding sleeve 12 is connected to the right side of the main support component 7 via the connecting block 13. The sliding sleeve 12 has a cuboid structure with a hollow interior. The top of the sliding sleeve 12 has a track hole communicating with the cavity. The end of the probe slide rod 10 has a stop perpendicular to its axial direction. The stop is movable and connected to the sliding sleeve 12, and its movement direction and distance are limited by the track hole. The spring 11 is installed inside the cavity of the sliding sleeve 12. One end of probe 11 contacts the bottom of the cavity inside the sliding sleeve 12, and the other end contacts the end stop of probe slide rod 10. Probe slide rod 10 can move back and forth inside the sliding sleeve 12. The front end of probe slide rod 10 is fixed with ultrasonic probe 9 through probe bracket 8. The central axis of ultrasonic probe 9 is parallel to the central axis of probe slide rod 10. A stop is provided at the front of probe slide rod 10 to limit the movement distance of probe slide rod 10. Together with the end stop of probe slide rod 10 and track hole, it provides multiple protections for ultrasonic probe 9 to avoid damage to the device or increase detection error caused by excessive stroke of ultrasonic probe 9 due to other factors.

[0063] The system utilizes spring 11 to provide flexible preload, ensuring stable contact between the ultrasonic probe 9 and the surface of the workpiece being measured. This structure can adaptively conform to the machined surface, improving the reliability and accuracy of thickness measurement.

[0064] The drilling size visual inspection subsystem includes a ring light source 16, a light source bracket 15, a light source extension column 14, a corner bracket profile 17, an industrial camera 18, and a secondary support component 19. In this embodiment, the secondary support component 19 uses the same profile as the main support component 7, has a cuboid structure, and is smaller in volume than the main support component 7. Each rectangular surface of the secondary support component 19 has a groove along its length. The secondary support component 19 is installed on the side of the main support component 7 via a profile groove connecting block 23. An industrial camera 18 is located on the side of the secondary support component 19 away from the main support component 7. Machine 18 is mounted on sub-support member 19 via threaded fastener 24; the lens of industrial camera 18 extends into the central through hole of ring light source 16; on one of the rectangular sides of sub-support member 19 adjacent to main support member 7, a light source extension column 14 is mounted via corner bracket profile 17, the corner bracket profile 17 is located at the end of the light source extension column 14, and a light source bracket 15 is provided at the front end of the light source extension column 14. The light source bracket 15 is a plate-shaped structure, one side is a circular structure that fits the bottom shape of ring light source 16 and has a circular through hole, and the other side is a rectangular structure. The periphery of the circular through hole of the light source bracket 15 is threaded to the bottom periphery of the ring light source (16) via screws, and the ring light source (16) provides illumination for drilling inspection; the overall layout of this subsystem is reasonable and easy to adjust.

[0065] The binocular vision ranging subsystem includes a binocular camera 20 and a camera support block 21. The camera support block 21 is connected to the main support member 7 via a profile groove connecting block 22. The binocular camera 20 is fixed on the camera support block 21, and the lens orientation of the binocular camera 20 is consistent with the front end orientation of the main support member 7. The binocular camera 20 is used to acquire the spatial distance between the end effector and the workpiece to be measured. This system can assist in adjusting the end effector position before detection to ensure that each vision / ultrasonic subsystem is at the optimal working distance, thereby improving detection accuracy.

[0066] Figure 3 This is a schematic diagram of the connection structure between the various detection subsystems and the main support component 7 in this invention. While maintaining a compact overall layout, this structure achieves efficient integration and modular installation of multiple functional subsystems, exhibiting good structural versatility and scalability. The main support component 7, as the main frame structure, is made of aluminum alloy industrial profiles, combining high strength and light weight. Its multi-faceted, multi-groove design allows for the simultaneous arrangement of functional modules in multiple directions, significantly improving structural utilization and system expandability. In the structure shown in the figure, each module uses a "slot + fastener" combination connection method, ensuring structural rigidity while providing good assembly flexibility. For example, the visual inspection subsystem uses a nested positioning method combined with multi-point threaded fastening, effectively avoiding displacement errors during processing vibration or posture changes; the ultrasonic inspection module adopts a design that emphasizes both sliding guidance and rigid connection, ensuring stable contact between the probe and the workpiece surface under the synergistic action of springs.

[0067] Furthermore, this modular connection structure possesses excellent anti-interference and vibration suppression capabilities. The connecting blocks are pre-tightened through threaded fastening, effectively distributing the load transmission path while maintaining the installation accuracy of the subsystem. This reduces the adverse effects of concentrated stress at a single point on measurement accuracy, thereby improving the stability of the entire system and the consistency of the test results. Overall, Figure 3 The structure shown not only excels in assembly efficiency, ease of maintenance, and functional integration, but also provides a solid structural foundation for the subsequent functional expansion and technological upgrade of the end effector, possessing extremely high engineering practical value and promotion potential.

[0068] In this embodiment, the industrial camera 5 in the milling surface roughness visual inspection subsystem, the ultrasonic probe 9 in the ultrasonic thickness measurement subsystem, the industrial camera 18 in the drilling size visual inspection subsystem, and the binocular camera 20 in the binocular vision ranging subsystem are all connected to the robot control system of the industrial robot via cables to achieve precise control.

[0069] Example 2

[0070] This embodiment provides a drilling and milling quality inspection method based on the aforementioned end effector for drilling and milling quality inspection. (See attached image) Figure 4 Specifically, it includes:

[0071] After drilling and milling are completed, the robot precisely moves the multi-sensor integrated end effector to the target inspection area according to a preset trajectory program, entering the automated quality inspection stage. During this process, the binocular vision ranging subsystem is first activated to acquire real-time 3D spatial images between the workpiece and the end effector. Depth information is obtained through a block matching algorithm (BM), and the relative distance and angular orientation between the end effector and the workpiece surface are calculated. The ranging data is transmitted to the robot control system in real time and, combined with the kinematic model of the industrial robot body, performs posture error compensation and pose correction operations. This process ensures that when the end effector reaches the inspection position, the principal directions of each subsystem are highly aligned with the normal direction of the target inspection surface, effectively avoiding inspection errors or ultrasonic reflection interference caused by posture deviations or angular skew. Through the above spatial alignment and posture adjustment operations, each inspection subsystem can operate under optimal geometric perspective and contact conditions, thereby significantly improving the accuracy and stability of the inspection data and providing reliable initial conditions for subsequent inspection tasks. After completing positioning and attitude adjustment, the robot control system sequentially initiates various quality inspection processes to achieve comprehensive, multimodal, in-situ inspection of the workpiece's processing quality. This binocular vision ranging subsystem achieves an attitude alignment accuracy of ±0.05° ~ ±0.1°; data transmission latency of 3–15 ms; and a correction response time of 100–300 ms.

[0072] At a relatively long detection distance (approximately 500mm in this embodiment, depending on lens parameters), the drilling size visual inspection subsystem is activated to detect the hole size accuracy. The process is as follows: Figure 4 As shown, the industrial camera 18 is controlled to perform multiple exposure image acquisition operations to address the problem of strong reflection on metal surfaces in complex lighting environments. The borehole size visual inspection subsystem acquires image sequences at different exposure times, and then uses multi-exposure image fusion technology to generate high dynamic range (HDR) images, significantly suppressing overexposure in highlight areas and improving the contrast and detail quality of the borehole edge boundary area. Next, the acquired borehole area images are processed by polar coordinate transformation. This operation maps the circularly distributed borehole edges in the two-dimensional image space to a one-dimensional boundary space, transforming the edge detection problem from a two-dimensional contour to a one-dimensional signal change, thereby improving the robustness and computational efficiency of subsequent edge extraction. A sub-pixel-level edge detection algorithm (Zernike moment sub-pixel edge detection) is used to process the transformed image, extracting high-precision boundary coordinates of the borehole edges to obtain a sub-pixel edge point set. The borehole size visual inspection subsystem can automatically determine the type of hole based on the distribution and hierarchical structure of edge stripes, such as whether it has a chamfered structure or whether there is a stepped hole, thereby achieving intelligent classification of borehole categories. Finally, based on the extracted sub-pixel edge point set, a circular geometric model is fitted and calculated to obtain the key size parameters of the hole, including the center coordinates, diameter value, and hole position deviation. Simultaneously, in the case of multiple edge layers, the diameters of each level of the stepped hole and their concentricity index are calculated. σ represents the standard deviation.

[0073] Under conditions of relatively close detection distance (approximately 10mm in this embodiment, depending on the focal length of the selected microscope head), the milling surface roughness visual inspection subsystem is activated to perform high-precision detection of the workpiece surface roughness distribution. The detection process is as follows: Figure 4As shown, the robot control system first works in conjunction with the industrial camera 5 to move along a preset path along the milled surface of the workpiece. An "S"-shaped scanning strategy is used to acquire images point by point, ensuring complete coverage and uniform sampling of the entire detection area, avoiding the omission of minute surface defects or local roughness fluctuations. After acquisition, each raw image undergoes preprocessing, including Limit Contrast Adaptive Histogram Equalization (CLAHE), to enhance local detail contrast and reduce detection interference caused by uneven lighting or workpiece surface shadows, thereby improving overall image quality and edge clarity. The preprocessed images are then input into the training... In a good surface roughness regression model, feature extraction and numerical regression are performed. The system can output the roughness index value (such as Ra, Rz, etc.) corresponding to each image and calibrate and number its position to ensure that the detection results have spatial distribution consistency and traceability. Finally, the detection results of all image regions are reconstructed into a two-dimensional heat map visualization image according to their relative positions on the surface of the workpiece. Specifically, the detection results are mapped to a unified workpiece coordinate grid according to their positions, and different colors represent the roughness value range, clearly showing the processing uniformity, roughness gradient changes and possible abnormal areas of the entire workpiece surface.

[0074] With the ultrasonic probe 9 in contact with the workpiece surface (contact force approximately 3–5 N, adjustable according to material surface hardness and ultrasonic coupling requirements), the ultrasonic thickness measurement subsystem, driven by the robot control system, performs a large-area (2000-800000 mm²) scanning operation within the target detection area. During scanning, the probe slide rod 10 within the sliding sleeve 12 slowly brings the ultrasonic probe 9 into contact with the workpiece surface in the vertical direction. The built-in spring 11 provides a flexible constant force, ensuring stable contact force and effectively guaranteeing the coupling effect of ultrasonic waves between the probe and the workpiece surface, avoiding measurement errors caused by local floating or poor contact. The ultrasonic thickness measurement subsystem acquires the ultrasonic echo signal at each scanning node according to a preset two-dimensional grid path, and calculates the local thickness value corresponding to each scanning point through the sound velocity-time inversion relationship. All measured local thickness data are registered and calibrated in real time according to spatial coordinates and stored as a structured point dataset. After scanning the entire area, the system visualizes and reconstructs the structured point dataset as a 3D point cloud, generating a 3D point cloud map or isosurface map of the thickness distribution. This 3D point cloud map or isosurface map, with spatial coordinates on the horizontal axis and thickness on the vertical axis, intuitively displays the wall thickness distribution of the workpiece throughout the entire inspection area. It can be used to identify potential defects such as excessively thin local machining, missing internal layers, or structural depressions.

[0075] In summary, the multi-sensor integrated drilling and milling machining quality inspection end effector provided by this invention can sequentially complete in-situ detection and multi-modal data acquisition of key quality indicators such as drilling size, surface roughness, and remaining structural thickness after drilling and milling. This significantly improves inspection efficiency, accuracy, and data consistency, and reduces the cumbersome "disassembly-transfer-clamping" process in traditional inspection. It has good engineering application value and promotion prospects in scenarios such as complex structural parts and high-precision components.

[0076] This invention is not limited to the embodiments described above. The above description of specific embodiments is intended to illustrate and explain the technical solutions of this invention. The specific embodiments described above are merely illustrative and not restrictive. Without departing from the spirit and scope of the claims, those skilled in the art can make many specific modifications based on the teachings of this invention, and these modifications all fall within the scope of protection of this invention.

Claims

1. A multi-sensor integrated end effector for drilling and milling quality inspection, comprising a connector (6) for connection to an end flange of an industrial robot, characterized in that, It also includes a main support member (7), the end of which is connected to the connector (6). The main support member (7) has a cuboid structure and its four rectangular surfaces are respectively provided with: A binocular vision ranging subsystem is used to obtain the relative distance and angular orientation between the actuator end effector and the surface of the workpiece to be measured; A borehole size visual inspection subsystem is used to classify boreholes and inspect the accuracy of borehole dimensions. The visual inspection subsystem for milled surface roughness is used to detect the surface roughness distribution of the workpiece and generate a visual image. And an ultrasonic thickness measurement subsystem, used to visualize the wall thickness distribution of the workpiece under test in the detection area to identify potential defects; The main support member (7) has two parallel grooves on each of its four rectangular surfaces along its length, which are used to detachably and movablely connect the various subsystems.

2. The end effector for drilling and milling quality inspection according to claim 1, characterized in that, The binocular vision ranging subsystem includes a binocular camera (20) and a camera support block (21). The camera support block (21) is connected to the main support member (7) through a connecting block (22). The binocular camera (20) is fixed on the camera support block (21), and the lens orientation of the binocular camera (20) is consistent with the front end orientation of the main support member (7).

3. The end effector for drilling and milling quality inspection according to claim 1 or 2, characterized in that, When the binocular vision ranging subsystem is working, it acquires three-dimensional spatial images of the end effector and the surface of the workpiece in real time, and generates depth information through a stereo matching algorithm. Based on the obtained depth information, the relative distance and angular attitude between the end effector and the surface of the workpiece to be measured are calculated, and the distance measurement data is transmitted to the robot control system in real time. Combined with the robot kinematic model, posture error compensation and pose correction operations are performed to ensure that when the end effector reaches the detection position, the principal directions of its various detection subsystems are aligned with the normal direction of the target detection surface.

4. The end effector for drilling and milling quality inspection according to claim 1, characterized in that, The drilling size visual inspection subsystem includes a ring light source (16), a light source bracket (15), a light source extension column (14), a corner bracket profile (17), a first industrial camera (18), and a secondary support component (19). The secondary support component (19) has a cuboid structure and a volume smaller than that of the main support component (7). Each rectangular surface of the secondary support component (19) has a groove along its length. The secondary support component (19) is located on one side of the main support component (7), and the side of the secondary support component (19) away from the main support component (7) is... The first industrial camera (18) is provided on the upper part, and the lens of the first industrial camera (18) extends into the central through hole of the ring light source (16); the light source extension column (14) is installed on one of the rectangular sides of the sub-support member (19) adjacent to the main support member (7) through the corner bracket profile (17), and the front end of the light source extension column (14) is provided with a light source bracket (15) with a circular through hole, and the periphery of the circular through hole of the light source bracket (15) is fastened to the bottom periphery of the ring light source (16) by fasteners.

5. The end effector for drilling and milling quality inspection according to claim 1 or 4, characterized in that, When the drilling size visual inspection subsystem is working, it acquires a series of images with different exposure times using the first industrial camera (18) through multiple exposure image method; it generates a high dynamic range image (HDR) by using the multi-exposure image fusion technology on the above image sequence; it performs polar coordinate transformation on the HDR image to map the circularly distributed drilling edges in the two-dimensional image space to a one-dimensional boundary space, so that the edge detection problem is transformed from two-dimensional contour to one-dimensional signal change; it processes the transformed image using a sub-pixel level edge detection algorithm to extract the boundary coordinates of the hole edge to obtain a sub-pixel edge point set; the drilling size visual inspection subsystem automatically determines the drilling type based on the distribution pattern and hierarchical structure of the edge stripes; finally, it performs circular geometric model fitting based on the sub-pixel edge point set to obtain the key size parameters of the drilling, including the center, diameter, hole position deviation, stepped hole diameter and concentricity.

6. The end effector for drilling and milling quality inspection according to claim 1, characterized in that, The visual inspection subsystem for milled surface roughness includes a dome light source (1), a dome light source bracket (2), a microscope head (3), a support (4), and a second industrial camera (5). The support (4) is installed on one side of the main support (7). The support (4) is fixed to the microscope head (3) via a clamping device. The front end of the microscope head (3) extends into the through hole in the middle of the dome light source bracket (2). The dome light source (1) is fixed in the dome light source bracket (2). The dome light source (1) has a frustum-shaped structure that is larger in the front and smaller in the back. It is used to form diffuse reflection and uniform lighting when photographing surface texture. The through hole in the middle of the dome light source (1) is connected to the through hole in the middle of the dome light source bracket (2).

7. The end effector for drilling and milling quality inspection according to claim 1 or 6, characterized in that, When the milling surface roughness visual inspection subsystem is working, the end effector equipped with the second industrial camera (5) scans and covers the inspection area along the workpiece surface in a preset S-shaped trajectory, and collects surface images point by point; The acquired images are preprocessed using contrast-limited adaptive histogram equalization (CLAHE). The preprocessed images are then input into a trained surface roughness regression model for feature extraction and numerical regression. The roughness index values ​​corresponding to each image are output and the spatial location numbers are recorded. Based on the location numbers, the discrete roughness data are reconstructed into a two-dimensional heat map, and the surface roughness distribution is represented by color gradients.

8. The end effector for drilling and milling quality inspection according to claim 1, characterized in that, The ultrasonic thickness measurement subsystem includes a probe bracket (8), an ultrasonic probe (9), a probe slide rod (10), a spring (11), a sliding sleeve (12), and a connecting block (13). The sliding sleeve (12) is connected to one side of the main support member (7) through the connecting block (13). The sliding sleeve (12) has a hollow structure inside. One side of the sliding sleeve (12) is provided with a track hole communicating with the cavity. The end of the probe slide rod (10) can move back and forth and is connected to the sliding sleeve (12), and the movement direction and distance are limited by the track hole. The spring (11) is provided in the cavity of the sliding sleeve (12). One end of the spring (11) contacts the bottom of the cavity inside the sliding sleeve (12), and the other end contacts the end of the probe slide rod (10). The front end of the probe slide rod (10) is fixed with the ultrasonic probe (9) through the probe bracket (8). The central axis of the ultrasonic probe (9) is parallel to the central axis of the probe slide rod (10), and a stop is provided at the front of the probe slide rod (10).

9. The end effector for drilling and milling quality inspection according to claim 1 or 8, characterized in that, When the ultrasonic thickness measurement subsystem is working, the ultrasonic probe (9) is attached to the surface of the workpiece to be measured and scans the target area according to the preset two-dimensional grid path. Ultrasonic echo signals are collected at the scanning nodes. The local thickness value corresponding to the scanning node is calculated through the sound velocity-time inversion relationship. The obtained local thickness value data is registered and calibrated in real time according to the spatial coordinates and stored as three-dimensional point cloud data. All local thickness value data are visualized and reconstructed in the form of three-dimensional point cloud to generate a three-dimensional point cloud map or isosurface map of the thickness distribution.

10. A multi-sensor integrated drilling and milling machining quality inspection method, based on the drilling and milling machining quality inspection end effector according to any one of claims 1-9, characterized in that, After the workpiece has been drilled and milled, the industrial robot is controlled by the robot control system to move the end effector to the target inspection area. Start the binocular vision ranging subsystem to obtain the relative distance and angular orientation between the surfaces of the workpieces to be measured; Perform attitude error compensation and pose correction based on attitude deviation to align the main direction of each subsystem with the normal of the detection surface; Activate the borehole size visual inspection subsystem at an object distance of 400mm or more: (101) Acquire multiple exposure images and fuse them to generate a high dynamic range (HDR) image; (102) Perform polar coordinate transformation on high dynamic range (HDR) images to optimize edge detection; (103) The hole edge coordinates are extracted using a sub-pixel level edge detection algorithm and the drilling type is automatically determined based on the edge morphology features; (104) The key dimension parameters of the borehole were obtained by fitting calculation; Start the visual inspection subsystem for milled surface roughness at an object distance of 5-150mm: (201) Scan the detection area along the surface of the workpiece to be tested in a preset S-shaped trajectory and collect surface images point by point; (202) Perform contrast-limited adaptive histogram equalization (CLAHE) preprocessing on the acquired images; (203) Input the preprocessed image into the pre-trained surface roughness regression model, output the roughness index value corresponding to each image and record the spatial location number; (204) Based on the location number, the discrete roughness data is reconstructed into a two-dimensional heat map, and the surface roughness distribution is characterized by color gradient; The ultrasonic thickness measurement subsystem is activated after the ultrasonic probe makes contact with the surface of the workpiece. (301) Control the ultrasonic probe to contact the surface of the workpiece with a contact force of 3-5N; (302) Scan the target area according to the preset two-dimensional grid path and collect ultrasonic echo signals at the scanning nodes; (303) Calculate the local thickness value of each scanning node based on the sound wave propagation time and associate it with the spatial coordinates; (304) Generate a thickness distribution map from the three-dimensional point cloud to identify workpiece defects.

Images