A human-robot collaborative precision drilling system

Abstract

The present application relates to a kind of man-machine collaborative precision drilling system, belong to the field of aviation manufacturing engineering / aircraft assembly.The present application solves the problem of expensive and poor accessibility of automatic drilling, unstable precision and low efficiency of traditional manual drilling, combines machine vision and operator to form a kind of man-machine collaborative precision drilling system, avoids manual marking link, and greatly improves the efficiency of manual drilling.Using digital measurement technology ensures the accuracy of drilling position, improves the quality of aircraft assembly, makes the whole operation process precision controllable and efficient, and improves the intelligent level of aircraft manufacturing.

Description

Technical Field

[0001] This invention relates to a human-machine collaborative precision drilling system, belonging to the field of aerospace manufacturing engineering / aircraft assembly. Background Technology

[0002] Human-computer collaborative systems, also known as human-computer associative systems, are generally considered to be systems jointly composed of humans and computers. The computer primarily handles large amounts of data computation and some reasoning tasks, while humans are responsible for tasks where computing or execution power is insufficient, such as selection, decision-making, execution, and evaluation. This fully leverages human flexibility and creativity. Humans and computers collaborate closely, enabling more efficient handling of various complex problems. Aircraft assembly can be divided into frame assembly and panel assembly, involving the positioning and connection of numerous parts, groups, and components. These connections are almost entirely fastener connections, thus requiring the creation of numerous fastener mounting holes at different stages of assembly. Hole-making methods are divided into automatic and manual hole-making. Automatic hole-making offers high efficiency and precision, but the equipment is expensive and only suitable for hole-making in open areas. Manual hole making is the most traditional method, generally using pneumatic or electric drills. Quality is ensured by the operator's skill level. Its advantage is less restriction of operating space, but its disadvantages include inconsistent quality and low efficiency. This is because the manual hole making process involves marking to determine the hole position and then drilling with hand tools. Both the marking and drilling processes are susceptible to human error and misjudgment. Furthermore, the marking process is time-consuming, and the limited resolution of steel rulers leads to significant marking errors, which are even more pronounced on curved parts. In mass production, over 70% of fastener holes require repeated marking and drilling, inevitably leading to quality problems. Moreover, hole position deviations are difficult to inspect, and excessive quantities can reduce aircraft fatigue life and potentially cause flight accidents.

[0003] To address the aforementioned issues, a human-machine collaborative precision hole-making system needs to be designed, which combines digital measurement technology with the operator's execution to form a digital interaction link between reality and virtuality, thereby realizing the digital transmission of information throughout the hole-making process. Summary of the Invention

[0004] This project has invented a human-machine collaborative precision drilling system, including its operation process and a trackable drilling tool.

[0005] The technical solution of the present invention is as follows:

[0006] A human-machine collaborative precision hole-making system combines the advantages of automated hole-making equipment and traditional manual hole-making, integrating machine vision, augmented reality, and flexible assembly concepts to create a novel human-machine collaborative processing system. The system mainly consists of eight parts: assembly fixture, reference target, product, binocular tracking camera, computer, trackable hole-making tool, operator, AR glasses, etc. Figure 1 As shown.

[0007] The assembly fixture is used for product positioning and clamping, ensuring that the assembled product is in the correct position in the aircraft coordinate system. The assembly fixture is welded from steel plates, and the product locator is screwed to the surface of the plate. The target is installed at an open area on the surface of the jig.

[0008] The aforementioned reference target is used for coordinate system transformation in the binocular tracking camera measurement system. The target needs to be securely mounted on the assembly fixture, typically using a magnetic attachment method. The reference target is a hemisphere with a diameter of 0.5 inches, and a highly reflective photogrammetric sticker with an outer diameter of 10 mm and an inner diameter of 6 mm is affixed to its center.

[0009] The product is composed of various parts and is installed on an assembly fixture, where it is positioned and clamped by a locator on the fixture.

[0010] The binocular tracking camera is used to track the real-time spatial position of a trackable drilling tool in the aircraft coordinate system.

[0011] The computer is equipped with data analysis software for processing data collected by the binocular tracking camera, analyzing the deviation between the position of the trackable drilling tool and the theoretical drilling position, and transmitting the deviation information to the AR glasses.

[0012] The trackable drilling tool can be tracked by a binocular tracking camera to determine the coordinates of the drill bit tip in the aircraft coordinate system in real time, and then used for drilling the product. Figure 3 As shown.

[0013] The operator holds a trackable hole-making tool, wears AR glasses, and adjusts the hole-making position according to the positional deviation information provided by the computer to complete the hole-making work.

[0014] The AR glasses receive information from the computer and are used to display the positional deviation information of the trackable drilling tool in real time. The AR glasses are worn on the operator's head.

[0015] Operation process of human-machine collaborative precision hole making system

[0016] The human-machine collaborative precision drilling system represents a novel process implementation route. It enables virtual-to-real mapping of tool positions and point models, visualized in a visual manner, and allows for low-latency hand-eye coordination between the operator, scientifically integrating humans and equipment within the system. This process consists of 16 steps, as follows: Figure 2 As shown.

[0017] Step 1: Product placement. Position the product on the assembly fixture, using the fixture to position the product at its theoretical location in the aircraft coordinate system.

[0018] Step 2: Set up the binocular tracking camera. Set up the binocular tracking camera according to the position of the assembly fixture and the drilling area of ​​the product, ensuring that the camera's measurement field of view covers the assembly fixture and the drilling area of ​​the product.

[0019] Step 3: Connect to the computer and open the software. Install data processing and analysis software on the computer. The software can communicate with the stereo tracking camera, obtain the coordinates of the target points captured by the camera in real time, and highlight the target points in the camera image.

[0020] Step 4: Import the theoretical coordinates of the assembly tooling reference target point. The theoretical coordinates of the target point are the coordinate values ​​of the tooling in the aircraft coordinate system. These are used for subsequent coordinate system transformations in measurements.

[0021] Step 5: Import the hole location model data. Extract the center coordinates of all holes from the aircraft theoretical model, number them, and import them into the software in .txt format as theoretical target points. These coordinates are in the aircraft coordinate system.

[0022] Step 6: Import the coordinates of the hole-making tool frame points. For example... Figure 9 As shown, the frame points include the frame points of the reamer / spot drill body and the drill bit frame points. The frame points are obtained by capturing images of the target points attached to the surface of the target frame using a binocular tracking camera.

[0023] Step 7: Binocular tracking camera coordinate system transformation. The camera captures images of four reference targets on the assembly fixture, obtaining their coordinate values ​​in the camera's own measurement coordinate system. Using the coordinate values ​​of the reference targets imported in Step 4 as a reference, a coordinate system transformation is performed, converting the measurement coordinate system to the aircraft coordinate system. After the transformation, the target point coordinate values ​​captured by the camera are all in the aircraft coordinate system.

[0024] Step 8: The operator wears AR glasses. The operator wears AR glasses and communicates with the computer via the network, ensuring that the software analysis results displayed on the computer screen can be seen in real time through the glasses.

[0025] Step 9: The operator holds the trackable drilling tool. The operator attaches the drill bit to the drilling tool, connects the tool's power source, forming a complete trackable drilling tool. The operator moves the tool within the camera's field of view and checks whether the software can display the tool's real-time pose information.

[0026] Step 10: The operator locates the drilling point on the product. The operator holds the trackable drilling tool and moves it to the area of ​​the product to be drilled, bringing the drill tip to the product surface to check the pose deviation information of the AR glasses.

[0027] Step 11: Tracking the spatial pose of the drilling tool. When the operator holds the drilling tool and it enters the camera's field of view, the camera can track the spatial pose of the drill bit tip based on the drilling tool frame points imported in Step 6.

[0028] Step 12: Analyze the positional deviation of the drilling tool tip. Based on the drilling point model data imported in Step 5, analyze the deviation of the current tip position. The analysis strategy is to calculate the distance between the current tip coordinates and the coordinates of all point models in the software. Find the value with the smallest distance and display it on the software interface. The interface will display the ID, X, Y, and Z axis deviations, and 3D distance of that theoretical point in real time.

[0029] Step 13: Transmit the information to the AR glasses. Transmit the deviation information displayed in real time by the software in Step 12 to the AR glasses, allowing the operator to view the current deviation of the drill bit tip from the theoretical hole position in real time.

[0030] Step 14: The operator adjusts the tool's pose based on the information. The operator adjusts the tool's pose according to the deviation information displayed in the AR glasses until the deviation is within the acceptable range required by the process specifications, at which point the adjustment is stopped.

[0031] Step 15: Drilling. Activate the trackable drilling tool to drill the hole.

[0032] Step 16: Locate the next hole. Repeat the above steps until all holes have been machined.

[0033] Trackable hole-making tool

[0034] A hole-making tool has been invented that allows the position of the installed drill bit tip to be tracked in real time by a binocular tracking camera. This eliminates the need for manual marking, and the operator can hold the tool and determine the hole position according to the instructions in the software to make the hole.

[0035] The principle behind the tracking of drilling tools is as follows: After assembling the pneumatic drill / electric drill, the target frame, the target, and the process marker, a binocular tracking camera is used to photograph the target on the tool, obtaining data such as... Figure 9The frame points in the right-hand view include the body frame points and the drill bit frame points. This set of points is output as a TXT file and imported into the binocular tracking camera software. The software defines this set of points as the frame points of a rigid component. The process benchmark is then replaced with the drill bit, with the drill bit tip coinciding with the center of the process benchmark target. When a trackable drilling tool enters the binocular tracking camera's field of view, and at least four target point coordinates correspond to the four points in the TXT file, the binocular tracking camera recognizes the target as the rigid component corresponding to the TXT file. At this point, the software interface displays the data for the entire frame points, such as... Figure 9 Regardless of whether other target points are currently visible, because these target points are a whole, obtaining the positions of a portion of the target points allows us to obtain a 4×4 coordinate transformation matrix using the coordinates of the corresponding points in the txt file. Since every point in rigid body motion has the same coordinate transformation matrix, the software can obtain the spatial pose of the entire rigid body from the currently visible points, and thus obtain the coordinates of the drill bit tip. Because the current measurement is in the aircraft coordinate system, the drilling position can be determined by calculating the deviation between the drill bit tip coordinates and the theoretical drilling point.

[0036] The tool consists of five parts: pneumatic drill / electric drill, main body target frame, target, process guide, and drill bit. Figure 3 .

[0037] Pneumatic drills / electric drills: These are standard hole-making tools, bolted together with the main target frame to form a single unit. Process markers and drill bits can be mounted separately on the chuck of the pneumatic drill / electric drill. Figure 4 .

[0038] The target frame is used for target placement and maintaining the relative positions of the target points. It is bolted to the pneumatic drill / electric drill. The frame consists of multiple aluminum target mounts and carbon fiber connecting rods, forming a rigid polyhedron. Its internal dimensions are large enough to enclose most of the pneumatic drill / electric drill body. Figure 5 .

[0039] Targets: High-reflectivity photogrammetry targets with an outer diameter of 20mm and an inner diameter of 10mm are selected. The targets are attached to the target holders on both the main target frame and the process marker. The center of the target attached to the process marker should coincide with the center of the target holder. Targets attached to the main target frame only need to be secure and not overlap. Figure 6 .

[0040] Process marker: A process marker is used to replace the drill bit and serves to mark the position of the drill bit tip. It is mounted on the chuck of a pneumatic / electric drill and is made of aluminum alloy, consisting of a shank and a target seat. The shank diameter is 6mm, and the target seat diameter is 20mm with a thickness of 6mm. The overall length of the process marker should not exceed 30mm and should match the length of the drill bit. Figure 7 .

[0041] The effects of the invention

[0042] This invention solves the problems of high cost and poor accessibility of automated hole making, and unstable accuracy and low efficiency of traditional manual hole making. It combines machine vision with operator skills to form a human-machine collaborative precision hole making system, avoiding the manual marking process and significantly improving the efficiency of manual hole making. Utilizing digital measurement technology ensures the accuracy of hole positioning, improving aircraft assembly quality, making the entire operation process precise, controllable, and efficient, and enhancing the level of intelligence in aircraft manufacturing. Attached Figure Description

[0043] Figure 1 A schematic diagram of the components of a human-machine collaborative precision hole-making system;

[0044] Figure 2 A flowchart illustrating the operation of a human-machine collaborative precision hole-making system;

[0045] Figure 3 This is a schematic diagram of a traceable hole-making tool.

[0046] Figure 4 This is a schematic diagram of a pneumatic drill / electric drill;

[0047] Figure 5 This is a schematic diagram of the ontological target framework;

[0048] Figure 6 This is a schematic diagram of the target.

[0049] Figure 7 This is a schematic diagram of the process benchmark.

[0050] Figure 8 This is a schematic diagram of a drill bit;

[0051] Figure 9 This is a schematic diagram of the frame points for the hole-making tool.

[0052] In the diagram: 1-Assembly fixture, 2-Benchmark target, 3-Product, 4-Binocular tracking camera, 5-Computer, 6-Trackable hole-making tool, 7-Operator, 8-AR glasses, 51-Pneumatic drill / electric drill, 52-Body target frame, 53-Target, 54-Process benchmark, 55-Drill bit, 11-Body frame point, 12-Drill bit frame point. Detailed Implementation

[0053] A method for using a human-machine collaborative precision hole-making system, comprising the following steps:

[0054] Step 1: Product placement. Position the product on the assembly fixture, using the fixture to position the product at its theoretical location in the aircraft coordinate system.

[0055] Step 2: Set up the binocular tracking camera. Set up the binocular tracking camera according to the position of the assembly fixture and the drilling area of ​​the product, ensuring that the camera's measurement field of view covers the assembly fixture and the drilling area of ​​the product.

[0056] Step 3: Connect to the computer and open the software. Install data processing and analysis software on the computer. The software can communicate with the stereo tracking camera to obtain the coordinates of the target points captured by the camera in real time (the coordinates at this time are the coordinate values ​​in the camera's own measurement coordinate system), and highlight the target points in the camera image.

[0057] Step 4: Import the theoretical coordinates of the assembly tooling reference target point. The theoretical coordinates of the target point are the coordinate values ​​of the tooling in the aircraft coordinate system. These are used for subsequent coordinate system transformations in measurements.

[0058] Step 5: Import the hole location model data. Extract the center coordinates of all holes from the aircraft theoretical model, number them, and import them into the software in .txt format as theoretical target points. These coordinates are in the aircraft coordinate system.

[0059] Step 6: Import the coordinates of the hole-making tool frame points. For example... Figure 9 As shown, the frame points include the frame points of the drilling tool body and the drill bit frame points. The frame points are obtained by using a binocular tracking camera to photograph the target points bonded to the surface of the trackable drilling tool.

[0060] Step 7: Binocular tracking camera coordinate system transformation. The camera captures images of four reference targets on the assembly fixture, obtaining the coordinate values ​​in the camera's own measurement coordinate system. Using the coordinate values ​​of the reference targets imported in Step 4 as a reference, a coordinate system transformation is performed to convert the measurement coordinate system to the aircraft coordinate system. After the transformation, the target point coordinate values ​​captured by the camera are all coordinate values ​​in the aircraft coordinate system.

[0061] Step 8: The operator wears AR glasses. The operator wears AR glasses and communicates with the computer via the network, ensuring that the software analysis results displayed on the computer screen can be seen in real time through the glasses.

[0062] Step 9: The operator holds the trackable drilling tool. The operator attaches the drill bit to the drilling tool, connects the tool's power source, forming a complete trackable drilling tool. The operator moves the tool within the camera's field of view and checks whether the software can display the tool's real-time position information.

[0063] Step 10: The operator locates the drilling point on the product. The operator holds the drilling tool and moves it to the area of ​​the product to be drilled, bringing the drill bit tip into contact with the product surface, and checks the pose deviation information of the AR glasses.

[0064] Step 11: Tracking the spatial pose of the drilling tool. When the operator holds the drilling tool and it enters the camera's field of view, the camera can track the spatial pose of the drill bit tip based on the drilling tool frame points imported in Step 6.

[0065] Step 12: Analyze the positional deviation of the drilling tool tip. Based on the drilling point model data imported in Step 5, analyze the deviation of the current tip position. The strategy is to calculate the distance between the current tip coordinates and the coordinates of all point models in the software. Find the value with the smallest distance and display it on the software interface. The interface will display the ID of the theoretical point, the deviations in the X, Y, and Z directions, and the 3D distance (the 3D distance is the length of the line connecting two points in space) in real time.

[0066] Step 13: Transmit the information to the AR glasses. Transmit the deviation information displayed in real time by the software in Step 12 to the AR glasses, allowing the operator to view the current deviation of the drill bit tip from the theoretical hole position in real time.

[0067] Step 14: The operator adjusts the tool's pose based on the information. The operator adjusts the tool's pose according to the deviation information displayed in the AR glasses until the deviation is within the acceptable range required by the process specifications, at which point the adjustment is stopped.

[0068] Step 15: Drilling. Activate the trackable drilling tool to drill the hole.

[0069] Step 16: Locate the next hole. Repeat the above steps until all holes have been machined.

Claims

1. A human-machine collaborative precision hole-making system, characterized in that, Includes assembly fixture (1), reference target (2), product (3), binocular tracking camera (4), computer (5), trackable hole-making tool (6), AR glasses (8); The assembly fixture (1) is used for positioning and clamping the product (3) to ensure that the assembled product is in the correct position in the aircraft coordinate system; the assembly fixture is welded from steel plates, and the product locator is screwed to the surface of the plate; the reference target is installed on the open part of the surface of the frame. The product (3) is composed of various parts. The product is installed on the assembly fixture (1) and is positioned and clamped by the positioner on the fixture. The binocular tracking camera (4) is used to track the real-time spatial position of the trackable drilling tool (6) in the aircraft coordinate system; The computer (5) is equipped with data analysis software for data processing of the binocular tracking camera (4), analyzing the deviation between the position of the trackable hole-making tool (6) and the theoretical hole-making position, and transmitting the deviation information to the AR glasses (8). The trackable hole-making tool (6) can be tracked by a binocular tracking camera (4) to determine the coordinate value of the drill bit tip of the hole-making tool in the aircraft coordinate system in real time, and is used for hole-making processing of the product (3). The operator (7) holds the trackable hole-making tool (6), wears AR glasses (8), and adjusts the hole-making position according to the position deviation information provided by the computer (5) to complete the hole-making work; The AR glasses (8) receive information from the computer (5) and are used to display the position deviation information of the trackable hole-making tool (6) in real time. The AR glasses are worn on the head by the operator (7). The reference target (2) is used for the transformation of the coordinate system of the binocular tracking camera (4). The target needs to be firmly installed on the assembly fixture (1) by magnetic attraction. The reference target (2) is a hemisphere with a diameter of 0.5 inches. A high reflective photographic measurement sticker with an outer diameter of 10 mm and an inner diameter of 6 mm is pasted at the center of the sphere. It also includes trackable drilling tools, including pneumatic / electric drills (51), body target frame (52), target (53), process benchmark (54), and drill bit (55). The pneumatic drill / electric drill (51) is bolted to the main body target frame (52) to form a whole. The process marker (54) and drill bit (55) can be installed on the chuck of the pneumatic drill / electric drill respectively. The target frame (52) is used for target point arrangement and maintaining the relative positions between target points. It is connected to the air drill / electric drill (51) by bolts. The frame is composed of multiple aluminum target seats and carbon fiber connecting rods to form a polyhedron with good rigidity. The internal space size can ensure that it can cover most of the air drill / electric drill (51) body.

2. The human-machine collaborative precision hole-making system as described in claim 1, characterized in that, The target (53) is a high reflective photogrammetric target with an outer diameter of 20 mm and an inner diameter of 10 mm. The target is pasted on the target base of the main target frame (52) and the target base of the process rod (54). The center of the target pasted on the target base of the process rod (54) should coincide with the center of the target base.

3. A human-machine collaborative precision hole-making system as described in claim 1 or 2, characterized in that, The process marker (54) is used to replace the drill bit (55) and serves to mark the position of the drill bit tip. The process marker is installed on the chuck of the pneumatic drill / electric drill (51) and consists of a shank and a target seat.

4. The human-machine collaborative precision hole-making system as described in claim 3, characterized in that, The aforementioned process benchmark (54) is made of aluminum alloy.

5. The human-machine collaborative precision hole-making system as described in claim 3, characterized in that, The process benchmark (54) has a handle diameter of 6mm, a target base diameter of 20mm, a thickness of 6mm, and an overall length of no more than 30mm, and its length is consistent with that of the drill bit (55).

6. A method for operating a human-machine collaborative precision drilling system, used in the human-machine collaborative precision drilling system of claim 1, characterized in that, The steps are as follows: Step 1: Product placement; Position the product on the assembly fixture, using the fixture to position the product at its theoretical position in the aircraft coordinate system; Step 2: Set up a binocular tracking camera; according to the position of the assembly fixture and the drilling area of ​​the product, set up a binocular tracking camera to ensure that the camera's measurement field of view covers the assembly fixture and the drilling area of ​​the product. Step 3: Connect to the computer and open the software; Install data processing and analysis software on the computer. The software can communicate with the binocular tracking camera to obtain the coordinates of the target points that the camera can capture in real time, and highlight the target points in the camera image. Step 4: Import the theoretical coordinates of the assembly tooling reference target point; the theoretical coordinates of the target point are the coordinate values ​​of the tooling in the aircraft coordinate system; used for subsequent measurement coordinate system transformation; Step 5: Import the hole-making point model data; extract the center coordinates of all holes from the aircraft theoretical model, number them, and import them into the software in txt format as theoretical target points; these coordinates are coordinates in the aircraft coordinate system. Step 6: Import the coordinates of the drilling tool frame points; the frame points include the frame points of the pneumatic drill / spot drill body and the drill bit frame points; The frame points are obtained by using a binocular tracking camera to capture target points that can be attached to the surface of the target frame. Step 7: Binocular tracking camera coordinate system transformation; The camera captures images of four reference targets on the assembly fixture, obtaining the coordinate values ​​in the camera's own measurement coordinate system. Using the coordinate values ​​of the reference targets imported in Step 4 as a reference, the coordinate system is transformed to the aircraft coordinate system. After the transformation, the target point coordinate values ​​captured by the camera are all coordinate values ​​in the aircraft coordinate system. Step 8: The operator wears AR glasses; the operator wears AR glasses and communicates with the computer via the network to ensure that the software analysis results displayed on the computer screen can be displayed in real time in the glasses; Step 9: The operator holds the trackable hole-making tool; the operator installs the drill bit onto the hole-making tool, connects the tool's power source, forming a complete trackable hole-making tool, moves the tool in the camera's field of view, and checks whether the software can display the tool's real-time pose information; Step 10: The operator locates the drilling point on the product; the operator holds the trackable drilling tool and moves it to the area of ​​the product to be drilled, bringing the drill tip to the product surface to check the pose deviation information of the AR glasses; Step 11: Track the spatial pose of the drilling tool; When the operator holds the drilling tool and it enters the camera's field of view, the camera can track the spatial pose of the drill bit tip based on the drilling tool frame points imported in Step 6. Step 12: Analyze the positional deviation of the drilling tool tip; Based on the drilling point model data imported in Step 5, perform a deviation analysis on the current tip position; The analysis strategy is to calculate the distance between the current tip coordinates and the coordinates of all point models in the software; Find the value with the smallest distance and display it on the software interface, which will display the theoretical point ID, X, Y, Z direction deviations and 3D distances in real time. Step 13: Transmit the information to the AR glasses; transmit the deviation information displayed in real time by the software in step 12 to the AR glasses, so that the operator can view the deviation of the current drill bit tip from the theoretical hole position in real time; Step 14: The operator adjusts the tool's position based on the information; the operator adjusts the tool's position based on the deviation information in the AR glasses until the deviation is within the acceptable range of the process requirements before stopping the adjustment. Step 15: Drilling; Activate the trackable drilling tool to drill the hole; Step 16: Locate the next hole; repeat the above steps until all holes have been machined.

Images