A mechanical arm drilling trajectory correction method and system based on visual detection

Abstract

This invention relates to the field of robotic arm drilling trajectory correction technology, specifically to a method and system for robotic arm drilling trajectory correction based on vision detection. The method includes controlling a six-axis robotic arm to move along a photographed trajectory; visually acquiring images of pits and holes on the workpiece and comparing them with a template to identify the actual hole positions; comparing the actual hole positions with a theoretical digital model to obtain a spatial deviation as a compensation value; generating a drilling trajectory based on the theoretical digital model; correcting the trajectory with the compensation value; and controlling the robotic arm to perform drilling according to the compensated trajectory. This method introduces a vision detection mechanism to identify the actual positions of pits and holes on the workpiece before drilling and to compensate and correct the drilling trajectory based on the identification results. Even if the six-axis robotic arm itself has problems such as repeated clamping errors or structural flexibility, its end drill bit can still accurately align with the center of the pits and holes processed by a high-precision machine tool, thereby achieving high-precision drilling and significantly improving processing consistency and stability.

Description

Technical Field

[0001] This invention relates to the field of robotic arm drilling trajectory correction technology, and more specifically, to a method and system for robotic arm drilling trajectory correction based on vision detection. Background Technology

[0002] With the rapid development of aerospace, rail transportation, and large equipment manufacturing, structural components are increasingly characterized by large dimensions, numerous holes, and complex hole orientations. Drilling, as a key process in the manufacturing of these structural components, directly affects the assembly quality and service performance of the parts due to its machining accuracy.

[0003] Currently, robotic arm drilling involves utilizing the multiple degrees of freedom of a six-axis industrial robotic arm, with a drilling actuator mounted at the end effector to achieve multi-angle drilling. The robotic arm trajectory is typically generated by manual teaching or CAM software based on a digital model. This approach offers high flexibility, but the rigidity and absolute positioning accuracy of the robotic arm itself are lower than those of CNC machine tools. Furthermore, the trajectory generation relies on a theoretical digital model, which cannot be synchronized with the actual workpiece state, resulting in poor actual machining accuracy (typically exceeding 2mm).

[0004] In summary, existing technologies have significant shortcomings in terms of processing accuracy, equipment utilization, and automation level, and there is an urgent need for a drilling method that can balance high precision, high efficiency, and low cost. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for correcting the drilling trajectory of a robotic arm based on vision detection, so as to solve the problem that the existing robotic arm solutions are highly efficient but lack a precision guarantee mechanism.

[0006] To achieve the above objectives, the embodiments of this application provide the following technical solutions: On one hand, this application provides a vision-based robotic arm drilling trajectory correction method. The method includes: controlling a six-axis robotic arm to move according to a photographic trajectory; acquiring actual images of each point hole on the workpiece using a vision camera; comparing the acquired images with the vision template to identify the actual spatial position of each point hole in the current clamping state; comparing the identified actual point hole positions with the theoretical hole positions in the theoretical digital model to calculate the spatial deviation of each hole position, and using the spatial deviation as a hole position compensation value; generating a theoretical drilling trajectory based on the theoretical digital model, and correcting the theoretical drilling trajectory using the hole position compensation value to generate a compensated drilling trajectory; and controlling the six-axis robotic arm to perform drilling processing according to the compensated drilling trajectory.

[0007] Optionally, before controlling the six-axis robotic arm to move according to the photographed trajectory, the following steps are also included: Based on the theoretical digital model of the workpiece, a photographic trajectory is generated to guide the movement of the robotic arm. The photographic trajectory is used to move the vision camera installed at the end of the robotic arm to a preset photographic position and posture near the holes at each point. The coordinate transformation relationship between the vision camera and the drilling tool is calibrated, and a template mapping between the shooting posture and the drilling posture is constructed based on the coordinate transformation relationship, so that when the drill bit is aligned with the center of the hole, the camera can acquire image information reflecting the true position of the hole in the corresponding shooting posture. Feature extraction is performed on the pitted images, and the extracted features are stored as visual templates to provide a reference standard for subsequent image comparison.

[0008] Optionally, the extracted features include one or more of the contour features, center position features, and depth features of the pit.

[0009] Optionally, after comparing the identified actual pit positions with the theoretical hole positions in the theoretical digital model, the method further includes: determining whether the spatial deviation exceeds a preset threshold; if it does, triggering an anomaly handling mechanism.

[0010] Optionally, after comparing the identified actual pit locations with the theoretical pit positions in the theoretical model, the method further includes: determining whether the spatial deviation exceeds a preset threshold; if it does, triggering an exception handling mechanism. Specifically, the implementation method can be: For each identified pit, its actual spatial location is calculated, and its confidence factor is determined based on image sharpness, illumination uniformity, and multi-view consistency. Based on the magnitude of the credibility factor, the identification results are divided into different credibility levels, and corresponding processing strategies are adopted for different levels. If the confidence factor is higher than the first preset threshold, the recognition result is directly used as the actual spatial location of the pit. If the confidence factor is between the first preset threshold and the second preset threshold, it is used after spatial interpolation smoothing based on the recognition results of adjacent holes; If the confidence factor is lower than the second preset threshold, an anomaly handling mechanism is triggered. The anomaly handling mechanism includes one or more of local re-identification, manual intervention, or hole skipping.

[0011] Step S510: For cases where there are multiple identification results for the same hole location (e.g., multiple angle photos or multiple photos), perform weighted fusion based on the credibility factor of each identification result to obtain the fused actual spatial location. Where n is the number of times the hole position is identified. Let be the coordinates (three-dimensional vector) of the hole position identified in the kth instance. Its corresponding credibility factor; Step S530: If the confidence factor is lower than the second preset threshold, an anomaly handling mechanism is triggered, including: The system immediately pauses subsequent processing of the current hole position and enters the local re-identification sub-process: Since preliminary analysis suggests that the low confidence level may be related to local reflections, the system reduces the brightness of the ring light source from the default 80% to 60% and enables the polarization filter function. The robotic arm is offset by ±5mm along the normal direction of the dot hole, and one image is acquired at each of the three different positions (near point, far point, and side point). Then the above confidence calculation process is repeated to determine the actual spatial position after fusion. If the number of repeated judgments exceeds the preset threshold, and the confidence factor is still lower than the second preset threshold, an alarm command is sent so that the operator can intervene to adjust or skip the processing of the current hole and continue to process the subsequent holes.

[0012] Secondly, embodiments of this application provide a vision-based robotic arm drilling trajectory correction system, the system comprising: A robotic arm with a drilling tool and a vision camera mounted at its end; The trajectory generation module is used to generate photographic trajectories and theoretical drilling trajectories based on theoretical numerical models; The vision processing module is used to create visual templates, acquire actual images, and identify the actual spatial location of the pits and holes. The deviation calculation module is used to calculate the spatial deviation between the actual hole position and the theoretical hole position, and generate the hole position compensation value; The trajectory compensation module is used to correct the theoretical drilling trajectory using the hole position compensation value and generate the compensated drilling trajectory. The control module is used to control the robotic arm to move according to the photographing trajectory to acquire images, and to perform drilling according to the compensated drilling trajectory.

[0013] Optionally, the features extracted by the vision processing module include one or more of the contour features, center position features, and depth features of the pit holes.

[0014] Optionally, the deviation calculation module is also used to perform threshold verification on the hole position compensation value to determine whether it exceeds a preset threshold.

[0015] Thirdly, embodiments of this application provide a vision-based robotic arm drilling trajectory correction device, the device including a memory and a processor.

[0016] The memory is used to store computer programs; the processor is used to execute the computer programs to implement the steps of the above-described vision-based robotic arm drilling trajectory correction method.

[0017] Fourthly, embodiments of this application provide a medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-described vision-based robotic arm drilling trajectory correction method.

[0018] The beneficial effects of this invention are as follows: This invention introduces a visual inspection mechanism to identify the actual position of the pits and holes on the workpiece before drilling, and compensates and corrects the drilling trajectory based on the identification results. Even if the six-axis robotic arm has problems such as limited absolute positioning accuracy, repeated clamping errors, or structural flexibility, its end drill bit can still accurately align with the center of the pits and holes processed by a high-precision machine tool, thereby achieving high-precision drilling and significantly improving processing consistency and stability.

[0019] Secondly, by performing the piercing process on a high-precision three-axis or five-axis CNC machine tool, while transferring the drilling process to a six-axis robotic arm, this invention fully leverages the advantages of the machine tool's high-precision positioning and the robotic arm's high flexibility and efficiency. This processing method can significantly reduce the time spent on high-value CNC machine tools during the drilling process, thereby improving the overall equipment utilization rate of the production line and reducing manufacturing costs.

[0020] By introducing a design that separates the imaging trajectory from the drilling trajectory, the reliability of visual inspection is ensured without affecting the execution efficiency of drilling. Therefore, this invention can meet both the technical requirements of high-precision drilling and the requirements of processing cycle time and efficiency in actual production, and has high engineering application value.

[0021] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a vision-based robotic arm drilling trajectory correction method as described in an embodiment of the present invention. Figure 2 This is a schematic diagram of a vision-based robotic arm drilling device as described in an embodiment of the present invention. Figure 3 This is a schematic diagram of a vision-based robotic arm drilling trajectory correction device as described in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0025] It should be noted that similar reference numerals or letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] Example 1: like Figure 1 and Figure 2 As shown, this embodiment provides a vision-based method for correcting the drilling trajectory of a robotic arm, the method comprising: Step S100: Based on the theoretical digital model of the workpiece, generate a photographic trajectory to guide the movement of the robotic arm. The photographic trajectory is used to move the vision camera installed at the end of the robotic arm to a preset photographic position and posture near the hole at each point. Step S200: Calibrate the coordinate transformation relationship between the vision camera and the drilling tool, and construct a template mapping between the shooting posture and the drilling posture based on the coordinate transformation relationship, so that when the drill bit is aligned with the center of the hole, the camera can acquire image information reflecting the true position of the hole in the corresponding shooting posture. Step S300: Extract features from the pit image and store the extracted features as a visual template to provide a reference standard for subsequent photo comparison. Step S400: Control the six-axis robotic arm to move according to the photographing trajectory, collect actual images of each point of the workpiece through the vision camera, and compare the collected images with the vision template to identify the actual spatial position of each point of the hole in the current clamping state. Step S500: Compare the actual hole positions identified with the theoretical hole positions in the theoretical digital model, calculate the spatial deviation of each hole position, and use the spatial deviation as the hole position compensation value. Step S600: Generate a theoretical drilling trajectory based on the theoretical digital model, and correct the theoretical drilling trajectory using the hole position compensation value to generate a compensated drilling trajectory. Control the six-axis robotic arm to perform drilling processing according to the compensated drilling trajectory.

[0027] The features extracted in step S300 include one or more of the following: the contour features, center position features, and depth features of the pit.

[0028] Secondly, in step S500 of this embodiment, after comparing the identified actual pit positions with the theoretical pit positions in the theoretical digital model, the method further includes: determining whether the spatial deviation exceeds a preset threshold; if it does, triggering an exception handling mechanism. Specifically, the implementation method can be as follows: Step S510: For the actual spatial location of each identified pit, calculate its confidence factor. The confidence factor is determined based on a comprehensive assessment of image clarity, illumination uniformity, and multi-view consistency. Step S520: Based on the magnitude of the confidence factor, the identification results are divided into different confidence levels, and corresponding processing strategies are adopted for different levels. If the confidence factor is higher than the first preset threshold, the recognition result is directly used as the actual spatial location of the pit. If the confidence factor is between the first preset threshold and the second preset threshold, it is used after spatial interpolation smoothing based on the recognition results of adjacent holes; If the confidence factor falls below a second preset threshold, an anomaly handling mechanism is triggered. This mechanism includes one or more of the following: local re-identification, manual intervention, or hole skipping. Step S510: For cases where there are multiple identification results for the same hole location (e.g., multiple angle photos or multiple photos), perform weighted fusion based on the credibility factor of each identification result to obtain the fused actual spatial location. Where n is the number of times the hole position is identified. Let be the coordinates (three-dimensional vector) of the hole position identified in the kth instance. Its corresponding credibility factor; Step S530: If the confidence factor is lower than the second preset threshold, an anomaly handling mechanism is triggered, including: The system immediately pauses subsequent processing of the current hole position and enters the local re-identification sub-process: Since preliminary analysis suggests that the low confidence level may be related to local reflections, the system reduces the brightness of the ring light source from the default 80% to 60% and enables the polarization filter function. The robotic arm is offset by ±5mm along the normal direction of the dot hole, and one image is acquired at each of the three different positions (near point, far point, and side point). Then the above confidence calculation process is repeated to determine the actual spatial position after fusion. If the number of repeated judgments exceeds the preset threshold, and the confidence factor is still lower than the second preset threshold, an alarm command is sent so that the operator can intervene to adjust or skip the processing of the current hole and continue to process the subsequent holes.

[0029] The vision-based robotic arm drilling trajectory correction method described in this embodiment introduces a vision detection mechanism to identify the actual positions of the pits and holes on the workpiece before drilling, and then compensates and corrects the drilling trajectory based on the identification results. Even if the six-axis robotic arm body has problems such as limited absolute positioning accuracy, repeated clamping errors, or structural flexibility, its end drill bit can still accurately align with the center of the pits and holes processed by a high-precision machine tool, thereby achieving high-precision drilling and significantly improving processing consistency and stability.

[0030] Secondly, by performing the piercing process on a high-precision three-axis or five-axis CNC machine tool, while transferring the drilling process to a six-axis robotic arm, this invention fully leverages the advantages of the machine tool's high-precision positioning and the robotic arm's high flexibility and efficiency. This processing method can significantly reduce the time spent on high-value CNC machine tools during the drilling process, thereby improving the overall equipment utilization rate of the production line and reducing manufacturing costs.

[0031] By introducing a design that separates the imaging trajectory from the drilling trajectory, the reliability of visual inspection is ensured without affecting the execution efficiency of drilling. Therefore, this invention can meet both the technical requirements of high-precision drilling and the requirements of processing cycle time and efficiency in actual production, and has high engineering application value.

[0032] Example 2: This embodiment, based on Embodiment 1, provides a vision-based robotic arm drilling trajectory correction system, the system comprising: A robotic arm with a drilling tool and a vision camera mounted at its end; The trajectory generation module is used to generate photographic trajectories and theoretical drilling trajectories based on theoretical numerical models; The vision processing module is used to create visual templates, acquire actual images, and identify the actual spatial location of the pits and holes. The deviation calculation module is used to calculate the spatial deviation between the actual hole position and the theoretical hole position, and generate the hole position compensation value; The trajectory compensation module is used to correct the theoretical drilling trajectory using the hole position compensation value and generate the compensated drilling trajectory. The control module is used to control the robotic arm to move according to the photographing trajectory to acquire images, and to perform drilling according to the compensated drilling trajectory.

[0033] Secondly, in this embodiment, the features extracted by the visual processing module include one or more of the contour features, center position features, and depth features of the pit holes.

[0034] Secondly, in this embodiment, the deviation calculation module is also used to perform threshold verification on the hole position compensation value to determine whether it exceeds the preset threshold.

[0035] Example 3: Corresponding to the above method embodiments, this disclosure also provides a vision-based robotic arm drilling trajectory correction device. The vision-based robotic arm drilling trajectory correction device described below and the vision-based robotic arm drilling trajectory correction method described above can be referred to in correspondence.

[0036] Figure 3 This is a block diagram illustrating a vision-based robotic arm drilling trajectory correction electronic device according to an exemplary embodiment. Figure 3 As shown, the electronic device 800 may include a processor 801 and a memory 802. The electronic device 800 may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0037] The processor 801 controls the overall operation of the electronic device 800 to complete all or part of the steps in the vision-based robotic arm drilling trajectory correction method described above. The memory 802 stores various types of data to support the operation of the electronic device 800. This data may include, for example, instructions for any application or method operating on the electronic device 800, and application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 802 or transmitted via communication component 805. The audio component also includes at least one speaker for outputting audio signals. I / O interface 804 provides an interface between processor 801 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 805 is used for wired or wireless communication between the electronic device 800 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 805 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0038] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the vision-based robotic arm drilling trajectory correction method described above.

[0039] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the vision-based robotic arm drilling trajectory correction method described above. For example, the computer-readable storage medium may be the memory 802 including the program instructions described above, which may be executed by the processor 801 of the electronic device 800 to complete the vision-based robotic arm drilling trajectory correction method described above.

[0040] Example 4: Corresponding to the above method embodiments, this disclosure also provides a readable storage medium. The readable storage medium described below can be referred to in conjunction with the vision detection-based robotic arm drilling trajectory correction method described above.

[0041] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the vision-based robotic arm drilling trajectory correction method described in the above method embodiments.

[0042] Specifically, the readable storage medium can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other readable storage medium capable of storing program code.

[0043] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for correcting the drilling trajectory of a robotic arm based on vision detection, characterized in that, The method includes: The six-axis robotic arm is controlled to move according to the photographing trajectory. The actual images of each point of the workpiece are collected by the vision camera, and the collected images are compared with the vision template to identify the actual spatial position of each point of the hole in the current clamping state. The actual hole positions identified are compared with the theoretical hole positions in the theoretical model to calculate the spatial deviation of each hole position, and the spatial deviation is used as the hole position compensation value. The theoretical drilling trajectory is generated based on the theoretical numerical model, and the theoretical drilling trajectory is corrected by the hole position compensation value to generate a compensated drilling trajectory. The six-axis robotic arm is then controlled to perform drilling processing according to the compensated drilling trajectory.

2. The method for correcting the drilling trajectory of a robotic arm based on vision detection according to claim 1, characterized in that, Before controlling the six-axis robotic arm to move according to the photographed trajectory, the following steps are also included: Based on the theoretical digital model of the workpiece, a photographic trajectory is generated to guide the movement of the robotic arm. The photographic trajectory is used to move the vision camera installed at the end of the robotic arm to a preset photographic position and posture near the holes at each point. The coordinate transformation relationship between the vision camera and the drilling tool is calibrated, and a template mapping between the shooting posture and the drilling posture is constructed based on the coordinate transformation relationship, so that when the drill bit is aligned with the center of the hole, the camera can acquire image information reflecting the true position of the hole in the corresponding shooting posture. Feature extraction is performed on the pitted images, and the extracted features are stored as visual templates to provide a reference standard for subsequent image comparison.

3. The method for correcting the drilling trajectory of a robotic arm based on vision detection according to claim 2, characterized in that, The extracted features include one or more of the following: the contour features, the center position features, and the depth features of the pits.

4. The method for correcting the drilling trajectory of a robotic arm based on vision detection according to claim 2, characterized in that, After comparing the actual location of the identified pits with the theoretical location of the pits in the theoretical model, the process also includes: determining whether the spatial deviation exceeds a preset threshold; if it does, an anomaly handling mechanism is triggered.

5. A vision-based robotic arm drilling trajectory correction system, characterized in that, The system includes: A robotic arm with a drilling tool and a vision camera mounted at its end; The trajectory generation module is used to generate photographic trajectories and theoretical drilling trajectories based on theoretical numerical models; The vision processing module is used to create visual templates, acquire actual images, and identify the actual spatial location of the pits and holes. The deviation calculation module is used to calculate the spatial deviation between the actual hole position and the theoretical hole position, and generate the hole position compensation value; The trajectory compensation module is used to correct the theoretical drilling trajectory using the hole position compensation value and generate the compensated drilling trajectory. The control module is used to control the robotic arm to move according to the photographing trajectory to acquire images, and to perform drilling according to the compensated drilling trajectory.

6. The vision-based robotic arm drilling trajectory correction system according to claim 5, characterized in that, The features extracted by the visual processing module include one or more of the following: the contour features, center position features, and depth features of the pit holes.

7. The vision-based robotic arm drilling trajectory correction system according to claim 5, characterized in that, The deviation calculation module is also used to perform threshold verification on the hole position compensation value to determine whether it exceeds the preset threshold.

Images