A fast screw locking system and method based on machine vision

Abstract

The present invention relates to the technical field of screw locking, and in particular to a fast screw locking system and method based on machine vision, the system comprising: a data acquisition unit, used to perform camera calibration on a to-be-locked area, and obtain a screw hole position image of the to-be-locked area; a data processing unit, used to obtain the pixel coordinate position of the center of the screw hole in the image coordinate system according to the screw hole position, and convert the pixel coordinate position into a mechanical arm coordinate system; a path determination unit, used to determine the motion path between the mechanical arm and the center of the screw hole in the mechanical arm coordinate system; a locking execution unit, used to control the mechanical arm to the screw hole position according to the motion path to perform a locking operation; a locking warning unit, used to determine whether there is an abnormality in the locking operation, and when it is determined that there is an abnormality in the locking operation, the locking warning unit performs a warning process. The present invention has significant advantages such as high automation, high production efficiency, strong quality monitoring capability, strong adaptability, and high intelligence.

Description

Technical Field

[0001] The present invention relates to the technical field of screw locking, and in particular to a system and method for fast screw locking based on machine vision. Background Art

[0002] As the most commonly used assembly connection method in manufacturing, screw locking is widely used in many industries, such as electronic products, plastics, automotive parts, communication equipment, home appliances, etc. Its quality directly affects the overall performance and reliability of the product. However, the traditional screw locking method mainly relies on manual or traditional screw locking machines, which requires operators to have certain skills and experience, and the operation process is cumbersome. It is easy for human factors to cause the screws to loosen or be damaged, thus affecting product quality and consistency.

[0003] Although machine vision technology has made some progress in the field of industrial automation, the existing machine vision-based screw locking systems still have some shortcomings in the field of screw locking. Some systems are not accurate enough in identifying screw holes in complex environments, which can easily lead to inaccurate screw locking. When processing a large number of screw locking tasks, the processing speed is slow and it is difficult to meet the needs of efficient production. Summary of the invention

[0004] In view of this, the present invention proposes a machine vision-based screw fastening system and method, which is mainly to solve the problems of low screw hole position recognition accuracy, inaccurate locking and low locking efficiency in the existing machine vision-based locking system.

[0005] In one aspect, the present invention provides a screw fastening system based on machine vision, the system comprising:

[0006] A data acquisition unit, used to perform camera calibration on the area to be locked, obtain a screw hole image of the area to be locked, and determine whether it needs to be re-acquired according to image quality data of the screw hole image;

[0007] A data processing unit, configured to identify the screw hole position by an edge detection algorithm after the data acquisition unit determines that it is not necessary to reacquire the screw hole position image, and acquire the pixel coordinate position of the screw hole center in the image coordinate system according to the screw hole position, and convert the pixel coordinate position into the robot arm coordinate system;

[0008] a path determination unit, configured to determine a motion path between the robot arm and the center of the screw hole in the robot arm coordinate system after the data processing unit converts the pixel coordinate position into the robot arm coordinate system;

[0009] A locking execution unit, used to pre-set an initial screw locking torque and a screw locking depth, and control the mechanical arm to the screw hole position to perform a locking operation according to the motion path of the path determination unit;

[0010] The lock payment warning unit is used to obtain the lock payment process data of the lock payment execution unit during the lock payment process, and determine whether the lock payment operation is abnormal according to the lock payment process data. When it is determined that the lock payment operation is abnormal, the lock payment warning unit performs warning processing.

[0011] In some embodiments of the present application, the image quality data includes image resolution, image contrast and image illumination intensity.

[0012] In some embodiments of the present application, the data acquisition unit is used to perform camera calibration on the area to be locked, obtain the screw hole position image of the area to be locked, and determine whether it needs to be re-acquired according to the image quality data of the screw hole position image, including:

[0013] The data acquisition unit pre-sets a minimum contrast threshold and a minimum resolution threshold;

[0014] When the image resolution is higher than the minimum resolution threshold and the image contrast is higher than the minimum contrast threshold, the data acquisition unit performs auxiliary judgment according to the image illumination intensity;

[0015] When the image resolution is not higher than the minimum resolution threshold and the image contrast is not higher than the minimum contrast threshold, the data acquisition unit reacquires the screw hole image.

[0016] In some embodiments of the present application, when the image resolution is higher than the minimum resolution threshold and the image contrast is higher than the minimum contrast threshold, the data acquisition unit performs auxiliary judgment according to the image illumination intensity, including:

[0017] Obtaining a pixel brightness mean and a pixel brightness standard deviation of the screw hole image according to the image illumination intensity data;

[0018] The data acquisition unit pre-sets a brightness mean threshold range and a brightness standard deviation threshold range;

[0019] When the pixel brightness mean is within the brightness mean threshold range and the pixel brightness standard deviation is within the brightness standard deviation threshold range, the data acquisition unit determines that it is not necessary to reacquire the screw hole position image;

[0020] When the condition that the pixel brightness mean is within the brightness mean threshold range and the pixel brightness standard deviation is within the brightness standard deviation threshold range is not satisfied, the data acquisition unit determines to re-acquire the screw hole position image.

[0021] In some embodiments of the present application, when the path determination unit determines the motion path between the robotic arm and the center of the screw hole in the robotic arm coordinate system, it includes:

[0022] The path determination unit obtains the coordinates of the starting point of the robot arm and the center point of the screw hole as control points in the robot arm coordinate system;

[0023] After determining the control points, a B-spline curve is fitted, and path points are generated according to the B-spline curve;

[0024] The generated path points are smoothed by a Gaussian filtering method to obtain the motion path.

[0025] In some embodiments of the present application, the locking process data includes: screw locking depth, locking completion time, real-time vibration intensity, and real-time screw contact force.

[0026] In some embodiments of the present application, the lock payment warning unit is used to obtain the lock payment process data of the lock payment execution unit during the lock payment process, and dynamically determine whether the lock payment operation is abnormal according to the lock payment process data, including:

[0027] The locking warning unit pre-sets the standard screw locking depth;

[0028] When the screw locking depth is greater than or equal to the screw locking standard depth, the locking warning unit determines that the locking operation is completed, and obtains the real-time vibration intensity and the real-time contact force of the screw during the locking process to perform abnormal warning judgment;

[0029] When the screw locking depth is less than the screw locking standard depth, the locking warning unit determines that the locking operation is not completed, and obtains the corresponding locking completion time to perform abnormal warning judgment.

[0030] In some embodiments of the present application, when the locking warning unit determines that the locking operation is completed and obtains the real-time vibration intensity and the real-time contact force of the screw during the locking process for abnormal warning judgment, it includes:

[0031] The real-time vibration intensity and the real-time contact force of the screw are standardized in advance so that the values ​​of the real-time vibration intensity and the real-time contact force of the screw fall within the same range;

[0032] The average value of the real-time vibration intensity after the normalization process is calculated and recorded as X, and the average value of the real-time contact force of the screw after the normalization process is calculated and recorded as Y;

[0033] Calculate the real-time lock payment abnormality index TI based on X and Y, TI = X × a + Y × b;

[0034] Among them, a is the weight corresponding to the real-time vibration intensity; b is the weight corresponding to the real-time contact force of the screw; and a+b=1;

[0035] The lock payment warning unit pre-sets a minimum threshold value of the lock payment abnormality index;

[0036] When TI is greater than the lowest threshold of the locking abnormality index, the locking warning unit determines that the locking operation is abnormal and performs warning processing;

[0037] When TI is less than or equal to the lowest threshold of the locking abnormality index, the locking warning unit determines that there is no abnormality in the locking operation and does not perform warning processing.

[0038] In some embodiments of the present application, when the screw locking depth is less than the screw locking standard depth, the locking warning unit determines that the locking operation is not completed, and obtains the corresponding locking completion time to perform abnormal warning judgment, including:

[0039] The lock payment warning unit pre-sets a maximum lock payment completion time threshold;

[0040] When the lock completion time is longer than the lock completion maximum time threshold, the lock early warning unit determines that the lock operation is abnormal and directly performs early warning processing;

[0041] When the locking completion time is less than or equal to the locking completion maximum time threshold, the locking warning unit obtains the real-time vibration intensity and the real-time contact force of the screw during the locking process to perform abnormal warning judgment.

[0042] In another aspect, the present invention provides a method for fast screw fastening based on machine vision, the method comprising:

[0043] Performing camera calibration on the area to be locked, acquiring a screw hole image of the area to be locked, and determining whether to re-acquire based on image quality data of the screw hole image;

[0044] When it is determined that there is no need to reacquire the screw hole position image, the screw hole position is identified by an edge detection algorithm, and the pixel coordinate position of the center of the screw hole in the image coordinate system is acquired according to the screw hole position, and the pixel coordinate position is converted into the robot arm coordinate system;

[0045] After converting the pixel coordinate position into a robot arm coordinate system, determining a motion path between the robot arm and the center of the screw hole in the robot arm coordinate system;

[0046] Presetting the initial screw locking torque and screw locking depth, and controlling the robot arm to the screw hole position according to the motion path to perform the locking operation;

[0047] The locking process data is acquired during the locking process, and it is determined whether the locking operation is abnormal according to the locking process data. When it is determined that the locking operation is abnormal, an early warning process is performed.

[0048] Compared with the prior art, the present invention has the following beneficial effects: the present invention automatically identifies and locates the screw hole position through machine vision technology, thus avoiding the errors that may be caused by manual positioning and significantly improving the automation and accuracy of screw locking. At the same time, combined with the precise control of the robotic arm, it can ensure that the screws are accurately and quickly locked to the specified position. The automated locking process reduces manual intervention and improves production efficiency. In addition, since human errors and repetitive work are reduced, it helps to reduce production costs and human resource investment. During the locking process, the locking process data, such as the screw locking depth, the locking completion time, the real-time vibration intensity, and the real-time contact force of the screw, are collected in real time. Through data analysis, abnormal conditions in the locking process can be discovered in time and early warning processing can be carried out. This helps to improve product quality and reduce the defective rate.

[0049] In summary, the machine vision-based screw fastening method and system proposed in the present invention have significant advantages such as high degree of automation, high production efficiency, strong quality monitoring capability, strong adaptability and high degree of intelligence, which are of great significance for improving the level of industrial automation and product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only used for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. In the accompanying drawings:

[0051] Figure 1 A structural block diagram of a machine vision-based screw fastening system provided in an embodiment of the present invention;

[0052] Figure 2 A flowchart of a method for fast screw fastening based on machine vision is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with the implementation regulations.

[0054] See also Figure 1 As shown, the present invention proposes a screw fast locking system based on machine vision, the system comprising:

[0055] A data acquisition unit, used to perform camera calibration on the area to be locked, obtain a screw hole image of the area to be locked, and determine whether it needs to be re-acquired according to image quality data of the screw hole image;

[0056] A data processing unit, configured to identify the screw hole position by an edge detection algorithm after the data acquisition unit determines that it is not necessary to reacquire the screw hole position image, and acquire the pixel coordinate position of the screw hole center in the image coordinate system according to the screw hole position, and convert the pixel coordinate position into the robot arm coordinate system;

[0057] a path determination unit, configured to determine a motion path between the robot arm and the center of the screw hole in the robot arm coordinate system after the data processing unit converts the pixel coordinate position into the robot arm coordinate system;

[0058] A locking execution unit, used to pre-set an initial screw locking torque and a screw locking depth, and control the mechanical arm to the screw hole position to perform a locking operation according to the motion path of the path determination unit;

[0059] The lock payment warning unit is used to obtain the lock payment process data of the lock payment execution unit during the lock payment process, and determine whether the lock payment operation is abnormal according to the lock payment process data. When it is determined that the lock payment operation is abnormal, the lock payment warning unit performs warning processing.

[0060] It can be understood that in this embodiment, the camera calibration and image quality evaluation of the area to be locked are performed by the data acquisition unit, which ensures the accuracy and reliability of the screw hole position image, effectively avoids recognition errors or repeated operations caused by image quality problems, and improves the overall efficiency and stability of the system. The data processing unit uses an edge detection algorithm to accurately identify the screw hole position and quickly converts the pixel coordinate position to the robot arm coordinate system. This conversion process realizes the accurate mapping from image space to physical space, and provides a solid foundation for the subsequent robot arm motion control. The path determination unit plans the optimal motion path between the robot arm and the center of the screw hole in the robot arm coordinate system, ensuring that the robot arm can safely and accurately reach the designated position for locking operation. The locking execution unit controls the robot arm to perform the locking operation according to the planned motion path through the pre-set initial screw locking torque and screw locking depth, realizes the automation and standardization of the locking process, and improves the accuracy and consistency of the locking. The locking warning unit monitors the various data in the locking process in real time, and immediately issues a warning once an abnormal situation is found. This function effectively avoids equipment damage or safety accidents caused by improper locking operation, and improves the safety and reliability of the system.

[0061] In some embodiments of the present application, the image quality data includes image resolution, image contrast and image illumination intensity.

[0062] In some embodiments of the present application, the data acquisition unit is used to perform camera calibration on the area to be locked, obtain the screw hole position image of the area to be locked, and determine whether it needs to be re-acquired according to the image quality data of the screw hole position image, including:

[0063] The data acquisition unit pre-sets a minimum contrast threshold and a minimum resolution threshold;

[0064] When the image resolution is higher than the minimum resolution threshold and the image contrast is higher than the minimum contrast threshold, the data acquisition unit performs auxiliary judgment according to the image illumination intensity;

[0065] When the image resolution is not higher than the minimum resolution threshold and the image contrast is not higher than the minimum contrast threshold, the data acquisition unit reacquires the screw hole image.

[0066] It can be understood that in this embodiment, by introducing image resolution, image contrast and image illumination intensity as key indicators of image quality, it is ensured that the data acquisition unit can obtain high-quality screw hole images. This multi-dimensional quality assessment method is more comprehensive and accurate than a single indicator, and can more effectively screen out low-quality images, thereby avoiding errors and delays in subsequent processing. By presetting the minimum contrast threshold and the minimum resolution threshold, the data acquisition unit can automatically determine whether the image meets the basic requirements. This automated judgment mechanism not only improves work efficiency, but also reduces the subjectivity and uncertainty caused by human intervention. When the image meets these basic requirements, auxiliary judgment is performed based on the image illumination intensity to further ensure the quality of the image. This instant feedback and correction mechanism avoids the risk of using low-quality images for subsequent processing, thereby ensuring the accuracy and reliability of the entire processing flow. This embodiment significantly improves the accuracy and reliability of data acquisition by introducing a multi-dimensional image quality assessment mechanism and an automated judgment and correction process, and provides strong support for subsequent locking operations. This improvement not only improves work efficiency, but also reduces the risk of human error, and has significant technical advancement and practical application value.

[0067] In some embodiments of the present application, when the image resolution is higher than the minimum resolution threshold and the image contrast is higher than the minimum contrast threshold, the data acquisition unit performs auxiliary judgment according to the image illumination intensity, including:

[0068] Obtaining a pixel brightness mean and a pixel brightness standard deviation of the screw hole image according to the image illumination intensity data;

[0069] The data acquisition unit pre-sets a brightness mean threshold range and a brightness standard deviation threshold range;

[0070] When the pixel brightness mean is within the brightness mean threshold range and the pixel brightness standard deviation is within the brightness standard deviation threshold range, the data acquisition unit determines that it is not necessary to reacquire the screw hole position image;

[0071] When the condition that the pixel brightness mean is within the brightness mean threshold range and the pixel brightness standard deviation is within the brightness standard deviation threshold range is not satisfied, the data acquisition unit determines to re-acquire the screw hole position image.

[0072] It can be understood that in this embodiment, by setting the minimum threshold of image resolution and contrast, images that cannot be clearly identified due to low resolution or insufficient contrast are first excluded, thereby avoiding the acquisition and processing of invalid images and improving the overall efficiency. Further, by introducing the auxiliary judgment of image illumination intensity, combined with the analysis of pixel brightness mean and standard deviation, it is possible to more accurately judge whether the image quality meets the requirements. This multi-dimensional, multi-index judgment mechanism greatly improves the accuracy of image acquisition. When the image meets all preset conditions (i.e., resolution, contrast, and illumination intensity are all up to standard), the data acquisition unit can directly determine that the image does not need to be re-acquired, thereby reducing unnecessary image processing steps, such as image enhancement, denoising, etc., and further improving processing efficiency. On the contrary, when the image does not meet the conditions, timely reminders are given to re-acquire, avoiding possible errors or deviations in subsequent processing, and ensuring the smoothness of the entire processing flow and the reliability of the results. This embodiment optimizes and improves the acquisition and processing of screw hole images by introducing a comprehensive judgment mechanism of image resolution, contrast, and illumination intensity, as well as auxiliary analysis of pixel brightness mean and standard deviation. These beneficial effects not only improve the accuracy and efficiency of image acquisition, but also optimize the image processing process, improve the level of automation and intelligence, and enhance adaptability and flexibility.

[0073] In some embodiments of the present application, when the path determination unit determines the motion path between the robotic arm and the center of the screw hole in the robotic arm coordinate system, it includes:

[0074] The path determination unit obtains the coordinates of the starting point of the robot arm and the center point of the screw hole as control points in the robot arm coordinate system;

[0075] After determining the control points, a B-spline curve is fitted, and path points are generated according to the B-spline curve;

[0076] The generated path points are smoothed by a Gaussian filtering method to obtain the motion path.

[0077] It can be understood that in this embodiment, by obtaining the coordinates of the starting point of the robot arm and the coordinates of the center point of the screw hole as control points, and fitting the B-spline curve based on these control points, the motion path of the robot arm can be accurately generated. Due to its good continuity and controllability, the B-spline curve ensures high-precision positioning of the robot arm during movement, which is particularly important for operations that require precise alignment of the screw holes. The generated path points are smoothed by the Gaussian filtering method. This step effectively removes noise and unnecessary fluctuations in the path, making the movement of the robot arm more stable and smooth. The smooth path not only reduces the vibration and wear of the robot arm during movement, but also improves the stability and reliability of the operation. By automatically determining the motion path and smoothing it, this embodiment significantly improves the work efficiency of the robot arm.

[0078] In some embodiments of the present application, the locking process data includes: screw locking depth, locking completion time, real-time vibration intensity, and real-time screw contact force.

[0079] In some embodiments of the present application, the lock payment warning unit is used to obtain the lock payment process data of the lock payment execution unit during the lock payment process, and dynamically determine whether the lock payment operation is abnormal according to the lock payment process data, including:

[0080] The locking warning unit pre-sets the standard screw locking depth;

[0081] When the screw locking depth is greater than or equal to the screw locking standard depth, the locking warning unit determines that the locking operation is completed, and obtains the real-time vibration intensity and the real-time contact force of the screw during the locking process to perform abnormal warning judgment;

[0082] When the screw locking depth is less than the screw locking standard depth, the locking warning unit determines that the locking operation is not completed, and obtains the corresponding locking completion time to perform abnormal warning judgment.

[0083] It is understandable that in this embodiment, by real-time monitoring and recording of multiple key data in the screw locking process, such as screw locking depth, locking completion time, real-time vibration intensity, and real-time screw contact force, the system can fully and accurately grasp the actual situation of the locking operation. These data not only provide a solid basis for subsequent abnormal warnings, but also make the locking process more transparent and traceable, thereby improving production efficiency and product quality. Specifically, when the screw locking depth reaches or exceeds the preset standard depth, the system considers that the locking operation has been completed, and turns to parameters such as real-time vibration intensity and real-time screw contact force to determine whether there are abnormal conditions such as looseness, overtightening or damage. This judgment method helps to timely discover and solve potential quality problems and avoid safety hazards during product use. When the screw locking depth does not reach the preset standard, the system considers that the locking operation is not completed and focuses on the locking completion time. By comparing the difference between the actual completion time and the preset standard time, the system can determine whether there is a decrease in locking efficiency due to equipment failure, improper operation or material problems. This judgment method helps to timely discover and solve bottleneck problems in the production process and improve production efficiency and capacity.

[0084] In some embodiments of the present application, when the locking warning unit determines that the locking operation is completed and obtains the real-time vibration intensity and the real-time contact force of the screw during the locking process for abnormal warning judgment, it includes:

[0085] The real-time vibration intensity and the real-time contact force of the screw are standardized in advance so that the values ​​of the real-time vibration intensity and the real-time contact force of the screw fall within the same range;

[0086] The average value of the real-time vibration intensity after the normalization process is calculated and recorded as X, and the average value of the real-time contact force of the screw after the normalization process is calculated and recorded as Y;

[0087] Calculate the real-time lock payment abnormality index TI based on X and Y, TI = X × a + Y × b;

[0088] Among them, a is the weight corresponding to the real-time vibration intensity; b is the weight corresponding to the real-time contact force of the screw; and a+b=1;

[0089] The lock payment warning unit pre-sets a minimum threshold value of the lock payment abnormality index;

[0090] When TI is greater than the lowest threshold of the locking abnormality index, the locking warning unit determines that the locking operation is abnormal and performs warning processing;

[0091] When TI is less than or equal to the lowest threshold of the locking abnormality index, the locking warning unit determines that there is no abnormality in the locking operation and does not perform warning processing.

[0092] It can be understood that in this embodiment, the vibration intensity and screw contact force during the locking process are obtained in real time and standardized so that data of different dimensions can be compared and analyzed on the same scale. This step ensures the comparability of the data and the accuracy of the analysis, and provides a solid foundation for subsequent abnormal warnings. Using the standardized vibration intensity and contact force data, the locking abnormality index TI is calculated through a specific algorithm, and abnormal judgment is made based on a preset threshold. This method not only realizes automated abnormality detection, but also significantly improves the timeliness and accuracy of the early warning, and effectively avoids potential problems caused by human negligence or misjudgment. Through timely abnormal warnings, operators can respond to and deal with potential problems quickly, avoiding rework and safety accidents caused by poor locking. This not only helps to improve production efficiency and product quality, but also significantly reduces production costs and safety risks.

[0093] In some embodiments of the present application, when the screw locking depth is less than the screw locking standard depth, the locking warning unit determines that the locking operation is not completed, and obtains the corresponding locking completion time to perform abnormal warning judgment, including:

[0094] The lock payment warning unit pre-sets a maximum lock payment completion time threshold;

[0095] When the lock completion time is longer than the lock completion maximum time threshold, the lock early warning unit determines that the lock operation is abnormal and directly performs early warning processing;

[0096] When the locking completion time is less than or equal to the locking completion maximum time threshold, the locking warning unit obtains the real-time vibration intensity and the real-time contact force of the screw during the locking process to perform abnormal warning judgment.

[0097] It is understandable that, by introducing a detection mechanism for the screw locking depth in this embodiment, the present application can accurately determine whether the locking operation has reached the standard depth. When it is detected that the locking depth is less than the standard depth, the system will immediately start the early warning process, thereby avoiding product quality problems and safety hazards caused by insufficient locking. This real-time and accurate monitoring greatly improves the accuracy and reliability of the locking operation. This embodiment not only sets a threshold for the length of time for the locking to be completed to determine whether there is an abnormality in the locking operation, but also further combines parameters such as the real-time vibration intensity and the real-time contact force of the screw during the locking process for comprehensive judgment. This multi-parameter early warning mechanism enables the system to more comprehensively evaluate the state of the locking operation, thereby making more flexible and accurate early warning decisions in different situations. Through automated early warning processing, this embodiment can promptly detect and correct errors and abnormalities in the locking operation, thereby avoiding production delays and quality problems caused by human negligence or misjudgment. This not only helps to improve production efficiency, but also significantly improves the overall quality and stability of the product. In summary, this embodiment has demonstrated significant beneficial effects in improving the accuracy of locking operations, enhancing the flexibility of abnormal warnings, improving production efficiency and product quality, reducing production costs and maintenance costs, and enhancing the traceability of the production process.

[0098] See also Figure 2 As shown, in another aspect, this embodiment provides a method for fast screw fastening based on machine vision, the method comprising:

[0099] S101: performing camera calibration on the area to be locked, acquiring a screw hole image of the area to be locked, and determining whether to re-acquire according to image quality data of the screw hole image;

[0100] S102: When it is determined that there is no need to reacquire the screw hole position image, identify the screw hole position by an edge detection algorithm, obtain the pixel coordinate position of the screw hole center in the image coordinate system according to the screw hole position, and convert the pixel coordinate position into the robot arm coordinate system;

[0101] S103: after converting the pixel coordinate position into the robot arm coordinate system, determining a motion path between the robot arm and the center of the screw hole in the robot arm coordinate system;

[0102] S104: presetting an initial screw locking torque and a screw locking depth, and controlling the robot arm to the screw hole position according to the motion path to perform a locking operation;

[0103] S105: acquiring locking process data during the locking process, determining whether the locking operation is abnormal according to the locking process data, and performing early warning processing when it is determined that the locking operation is abnormal.

[0104] It can be understood that in this embodiment, the position of the screw hole is automatically identified and located by machine vision technology, and the motion path of the robot arm is accurately controlled, thereby realizing the rapid locking of the screw. Compared with the traditional manual locking method, it greatly reduces manual intervention and waiting time, and significantly improves production efficiency and automation level. Through camera calibration and edge detection algorithm, the position of the screw hole in the image coordinate system can be accurately obtained and converted into precise coordinates in the robot arm coordinate system. This high-precision positioning and conversion mechanism ensures that the screw can be locked into the hole accurately and accurately, and reduces the locking failure rate caused by inaccurate positioning. During the locking process, the method can obtain the locking process data in real time, and judge whether there is an abnormality in the locking operation based on these data. Once an abnormality is found, an early warning process is immediately carried out, thereby effectively avoiding equipment damage or product quality problems caused by improper locking. This real-time monitoring and early warning mechanism improves the stability and reliability of the production process. The method realizes automatic identification and positioning of the screw hole position through machine vision technology, and is not affected by factors such as ambient light and workpiece surface state. At the same time, by adjusting the camera calibration parameters and edge detection algorithm, it can adapt to the screw locking requirements of different specifications and models, and has strong adaptability and flexibility. The traditional manual locking method requires workers to maintain high concentration for a long time and perform repetitive locking operations, which is labor-intensive and easy to fatigue. The method provided in this embodiment completes the locking task through automated means, greatly reducing the labor intensity of workers and improving the comfort of the working environment.

[0105] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0106] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A fast screw locking system based on machine vision, characterized in that: include: A data acquisition unit, used to perform camera calibration on the area to be locked, obtain a screw hole image of the area to be locked, and determine whether it needs to be re-acquired according to image quality data of the screw hole image; A data processing unit, configured to identify the screw hole position by an edge detection algorithm after the data acquisition unit determines that it is not necessary to reacquire the screw hole position image, and acquire the pixel coordinate position of the screw hole center in the image coordinate system according to the screw hole position, and convert the pixel coordinate position into the robot arm coordinate system; a path determination unit, configured to determine a motion path between the robot arm and the center of the screw hole in the robot arm coordinate system after the data processing unit converts the pixel coordinate position into the robot arm coordinate system; A locking execution unit, used to pre-set an initial screw locking torque and a screw locking depth, and control the mechanical arm to the screw hole position to perform a locking operation according to the motion path of the path determination unit; a lock payment warning unit, used for obtaining lock payment process data of the lock payment execution unit during the lock payment process, and determining whether the lock payment operation is abnormal according to the lock payment process data; when it is determined that the lock payment operation is abnormal, the lock payment warning unit performs a warning process; The image quality data includes image resolution, image contrast and image illumination intensity; The data acquisition unit is used to perform camera calibration on the area to be locked, obtain the screw hole position image of the area to be locked, and determine whether it needs to be re-acquired according to the image quality data of the screw hole position image, including: The data acquisition unit pre-sets a minimum contrast threshold and a minimum resolution threshold; When the image resolution is higher than the minimum resolution threshold and the image contrast is higher than the minimum contrast threshold, the data acquisition unit performs auxiliary judgment according to the image illumination intensity; When the image resolution is not higher than the minimum resolution threshold and the image contrast is not higher than the minimum contrast threshold, the data acquisition unit reacquires the screw hole position image; When the image resolution is higher than the minimum resolution threshold and the image contrast is higher than the minimum contrast threshold, the data acquisition unit performs auxiliary judgment according to the image illumination intensity, including: Obtaining a pixel brightness mean and a pixel brightness standard deviation of the screw hole image according to the image illumination intensity data; The data acquisition unit pre-sets a brightness mean threshold range and a brightness standard deviation threshold range; When the pixel brightness mean is within the brightness mean threshold range and the pixel brightness standard deviation is within the brightness standard deviation threshold range, the data acquisition unit determines that it is not necessary to reacquire the screw hole position image; When the condition that the pixel brightness mean is within the brightness mean threshold range and the pixel brightness standard deviation is within the brightness standard deviation threshold range is not satisfied, the data acquisition unit determines to re-acquire the screw hole position image; When the path determination unit determines the motion path between the robot arm and the center of the screw hole in the robot arm coordinate system, it includes: The path determination unit obtains the coordinates of the starting point of the robot arm and the center point of the screw hole as control points in the robot arm coordinate system; After determining the control points, a B-spline curve is fitted, and path points are generated according to the B-spline curve; The generated path points are smoothed by a Gaussian filtering method to obtain the motion path; The locking process data includes: screw locking depth, locking completion time, real-time vibration intensity, and real-time screw contact force; The locking early warning unit is used to obtain the locking process data of the locking execution unit during the locking process, and dynamically determine whether the locking operation is abnormal according to the locking process data, including: The locking warning unit pre-sets the standard screw locking depth; When the screw locking depth is greater than or equal to the screw locking standard depth, the locking warning unit determines that the locking operation is completed, and obtains the real-time vibration intensity and the real-time contact force of the screw during the locking process to perform abnormal warning judgment; When the screw locking depth is less than the screw locking standard depth, the locking warning unit determines that the locking operation is not completed, and obtains the corresponding locking completion time to perform abnormal warning judgment; When the locking warning unit determines that the locking operation is completed and obtains the real-time vibration intensity and the real-time contact force of the screw during the locking process for abnormal warning judgment, it includes: The real-time vibration intensity and the real-time contact force of the screw are standardized in advance so that the values ​​of the real-time vibration intensity and the real-time contact force of the screw fall within the same range; The average value of the real-time vibration intensity after the normalization process is calculated and recorded as X, and the average value of the real-time contact force of the screw after the normalization process is calculated and recorded as Y; Calculate the real-time lock payment abnormality index TI based on X and Y, TI = X × a + Y × b; Among them, a is the weight corresponding to the real-time vibration intensity; b is the weight corresponding to the real-time contact force of the screw; and a+b=1; The lock payment warning unit pre-sets a minimum threshold value of the lock payment abnormality index; When TI is greater than the lowest threshold of the locking abnormality index, the locking warning unit determines that the locking operation is abnormal and performs warning processing; When TI is less than or equal to the lowest threshold of the locking abnormality index, the locking warning unit determines that there is no abnormality in the locking operation and does not perform warning processing.

2. The machine vision-based screw fastening system according to claim 1 is characterized in that: When the screw locking depth is less than the screw locking standard depth, the locking warning unit determines that the locking operation is not completed, and obtains the corresponding locking completion time to perform abnormal warning judgment, including: The lock payment warning unit pre-sets a maximum lock payment completion time threshold; When the lock completion time is longer than the lock completion maximum time threshold, the lock early warning unit determines that the lock operation is abnormal and directly performs early warning processing; When the locking completion time is less than or equal to the locking completion maximum time threshold, the locking warning unit obtains the real-time vibration intensity and the real-time contact force of the screw during the locking process to perform abnormal warning judgment.

3. A method for fast screw fastening based on machine vision, characterized in that: The machine vision-based fast screw locking system as described in any one of claims 1 to 2 comprises: Performing camera calibration on the area to be locked, acquiring a screw hole image of the area to be locked, and determining whether to re-acquire based on image quality data of the screw hole image; When it is determined that there is no need to reacquire the screw hole position image, the screw hole position is identified by an edge detection algorithm, and the pixel coordinate position of the center of the screw hole in the image coordinate system is acquired according to the screw hole position, and the pixel coordinate position is converted into the robot arm coordinate system; After converting the pixel coordinate position into a robot arm coordinate system, determining a motion path between the robot arm and the center of the screw hole in the robot arm coordinate system; Presetting the initial screw locking torque and screw locking depth, and controlling the robot arm to the screw hole position according to the motion path to perform the locking operation; The locking process data is acquired during the locking process, and it is determined whether the locking operation is abnormal according to the locking process data. When it is determined that the locking operation is abnormal, an early warning process is performed.

Images