Cutter reference intelligent control system and method based on vision measurement

Abstract

The invention relates to the technical field of control systems, in particular to a tool reference intelligent control system and method based on vision measurement. The control system comprises a high-definition camera, a standard ball and a controller, the standard balls are respectively mounted at a preset position easy to photograph and measure in a machine tool and on a cutter bar; the high-definition camera is in signal connection with the controller; the high-definition camera comprises a structured light camera, a visual camera and a light time-of-flight camera; wherein the structured light camera obtains position information of an object by emitting invisible infrared laser with a specific wavelength and calculating distortion of a returned coding pattern through a Harris corner detection algorithm, and is used for determining key feature points in an image; and the visual camera is used for simulating human eyes to obtain 3D data through two common cameras with a certain distance. According to the invention, unstable factors caused by manual participation in the machining process are removed, so that the production changing time is shortened, the product quality is stabilized, manual operation errors are avoided, and the working efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of tool adjustment and control system technology, and specifically to an intelligent control system and method for tool reference based on vision measurement. Background Technology

[0002] In the machining industry, after the workpiece and cutting tool are installed, the tool's position within the machine tool is determined manually through trial cutting to find the reference position before the machining program is called for production. This step determines the precision of the machining, which varies depending on the skill level of the operator. Furthermore, product scrap and machine tool collisions due to improper operation are frequent occurrences. Meanwhile, some high-end machine tools are equipped with tool setting devices to measure the tool reference point. However, the measured data requires secondary parameter input, making the process complex and difficult to operate, and also carries the risk of collisions during tool setting. Therefore, we need an intelligent control system and method for tool reference based on vision measurement. Summary of the Invention

[0003] To address the shortcomings and deficiencies of existing technologies where manual measurement of tool reference points is inaccurate and prone to errors, this invention provides a vision-based intelligent control system and method for tool reference points that eliminates the unstable factors of manual intervention in machining processes, shortens changeover time, stabilizes product quality, avoids human error, and improves work efficiency.

[0004] The present invention achieves the above objectives by adopting the following technical solution: A vision-based intelligent control system for tool references, characterized in that: the control system includes a high-definition camera, a standard sphere, and a controller; multiple standard spheres are present, wherein each standard sphere is installed inside the machine tool at a preset first position easily accessible for photographing and measurement; one standard sphere is fixedly installed at a preset second position, which can be any position visible to the camera, such as on the tool holder or the turret of the machine tool; the high-definition camera maintains a signal connection with the controller; wherein the high-definition camera includes a structured light camera, a vision camera, and a time-of-flight optical camera; wherein the structured light camera emits invisible infrared laser of a specific wavelength and transmits the laser signal via Harris... Corner detection algorithms calculate the distortion of the returned encoded pattern to obtain the object's position information, which is used to determine key feature points in the image. These corner points are stable under different viewpoints and can be used for object localization and pose estimation. A vision camera is used to simulate the human eye to acquire 3D data by using two ordinary cameras at a certain distance. A time-of-flight camera is used to calculate the distance information of the object by measuring the time of flight of light. The known target template is compared with the image to find the most similar region, thereby determining the position of the target. This completes the identification of areas with drastic brightness changes in the image, determines the reference contour of the reference object, and realizes the operation of measuring the position of the cutting tool tip. The controller includes a signal receiving module, a preprocessing module, a vision algorithm module, a noise reduction module, and a fusion security module. The signal receiving module receives image data streams from a structured light camera, a vision camera, and a time-of-flight optical camera. The preprocessing module uses mean filtering to smooth the image, median filtering to effectively remove salt-and-pepper noise, Gaussian filtering to process Gaussian noise, and histogram equalization to enhance contrast and highlight noise. The vision algorithm module receives data from the preprocessing module and uses edge learning technology to accurately identify the shape, position, and movement trajectory of the cutting tool during tool changing. The noise reduction module filters and eliminates complex noise from the workshop. The system mitigates the effects of lighting conditions and electromagnetic interference to ensure accurate environmental perception for the robot. The integrated safety module receives visual anomaly information captured by the vision algorithm module and responds rapidly. Visual anomalies include any of the following: tool cracks, tool wear exceeding a threshold, incorrect tool gripping position in the tool-changing robot, excessively long tool changing time, or robot arm movement stagnation. The response includes: first, triggering the machine tool's emergency braking system to pause the tool changing process to prevent the problem from worsening and causing more serious mechanical failures or safety accidents; second, clearly displaying the anomaly details on the external human-machine interface, providing operators with intuitive troubleshooting information.

[0005] As a preferred technical solution, the vision camera can be either a binocular 3D camera or a quad-camera.

[0006] A further optimized technical solution: During the tool offset adjustment process of the vision algorithm module, the acquisition marker remains unchanged, and the relative position of the tool does not change. Only the camera posture is adjusted. During testing, when the robot moves to different positions, images are taken of the tool and marker whose relative positions do not change. The camera obtains the corresponding data through the algorithm, and the precise position of the tool edge on the machine tool is obtained through the corresponding tool offset calculation formula. The tool offset calculation formula is as follows: (1) Formula for calculating the position of linear interpolation points; The formula for calculating the location is as follows: Xn = X0 + (n / N) (X1 X0); Yn = Y0 + (n / N) (Y1 Y0); Zn = Z0 + (n / N) (Z1 Z0); Where Xn, Yn, and Zn are the coordinates of the interpolation points, X0, Y0, and Z0 are the starting point coordinates, X1, Y1, and Z1 are the ending point coordinates, n is the current point position, and N is the total number of points; (2) Formula for calculating the position of circular interpolation points; The formula for calculating the location is as follows: Xn = I + R cos(θn); Yn = J + R sin(θn); Zn = Z0 + (n / N) (Z1 Z0); Where Xn, Yn, and Zn are the coordinates of the interpolation point, I and J are the coordinates of the arc center, R is the arc radius, θn is the angle corresponding to the current point, Z0 and Z1 are the Z coordinates of the start and end points, n is the current point position, and N is the total number of points. (3) Formula for calculating the position of the spiral interpolation point; The formula for calculating the location is as follows: Xn = R cos(θn); Yn = R sin(θn); Zn = Z0 + (n / N) (Z1 Z0); Where Xn, Yn, and Zn are the coordinates of the interpolation points, R is the radius of the helix, θn is the angle corresponding to the current point, Z0 and Z1 are the Z coordinates of the starting and ending points, n is the current point position, and N is the total number of points.

[0007] A further preferred technical solution: The vision algorithm module also includes an anti-collision unit and a robot control unit. The anti-collision unit is used to monitor the dynamics around the machine tool in real time, accurately identify the position and movement trend of tools, workpieces, and other obstacles, and quickly feed back to the robot control unit once a collision risk is predicted. The robot control unit is used to adjust the robot's motion trajectory in a timely manner to ensure the tool changing process is safe and error-free. The anti-collision algorithm of the anti-collision unit is as follows: During its movement, the robot interacts with other objects captured by visual imaging to adjust its own speed and avoid collisions. Assume there is an agent A and an obstacle B, both represented by circles: the center represents position, and the radius represents size; A and B each have a speed... , The system runs and, after a period of time, arrives at new positions A′ and B′ respectively; the two circular surfaces intersect here, indicating a collision; to solve this problem, it is necessary to predict the new positions a certain time later and adjust the current speed in response to possible collisions. Obstacle avoidance trajectory calculation: using This represents the time interval between refreshing the environment state and adjusting the speed; the calculated time series is expressed as... ,in ; The shorter the time, the more timely the feedback on changes in environmental information, and the more likely the decision will be correct, but the greater the computational cost. For any time The position of the agent can be represented as Based on the current environmental information, the achievable obstacle avoidance speed zone can be calculated, denoted as: ; In plane geometry, it is a polyhedron, and its specific shape is related to the dynamics of the agent and the VO region; to simplify the calculation process, it can be... Meshization uses discrete, finite points in a grid to represent the set of possible velocities for an agent; When making decisions, the intelligent agent needs to take measures to adjust its speed; the adjusted speed must belong to... Assuming the operation is Then there is ; The choice of different speeds will cause the agent to... After arriving at different locations, then in operation Then the agent is based on speed The new location reached can be represented as Note that both position and velocity are vectors. Using the agent's position at a certain moment as a node, different speeds are selected to obtain the agent's new position at the next moment, and a directed edge is established from the current position to the new position at the next moment; this will form an acyclic directed graph with the current position as the root node. Collision avoidance proof: Assume that the current velocities of agents A and B are respectively... and Both chose new speeds. and All are outside the corresponding RVO region, then and They must also be outside their respective VO zones. This ensures that collisions are avoided. ; The proof is as follows: Based on the definition of RVO, we can obtain: ; Based on the symmetry of the VO region, we can conclude that: ; Combining (1) and (2), we can obtain: ; Using translation invariance, we get: ; (5) Based on the characteristics of convex sets, let =0.5, weighting the two velocity expressions in equation (4) yields: ; Applying translation invariance to equation (5), we get: ; Applying translation invariance to equation (5), we get: ; By coordinating the work of robots and vision, using motion trajectory planning, obstacle avoidance trajectory calculation, and proof of obstacle avoidance algorithms, and through continuous optimization, a collision avoidance behavior planning algorithm is finally realized.

[0008] A further preferred technical solution: The robot control unit also includes a force sensor, which is used to stop the robot when its movement trajectory is abnormal.

[0009] A further preferred technical solution: The vision algorithm module also includes a drag-and-drop teaching and feedback unit, which is used to identify and learn the tool changing process. The operator only needs to manually drag the robot arm to complete a tool changing demonstration. During this process, the robot control unit tracks the subtle movements of the arm throughout and converts the changes in spatial position and posture into precise control codes.

[0010] A further preferred technical solution: the integrated security module also includes a self-test unit and an alert unit; the self-test unit is used to initiate automatic restart and troubleshooting procedures in the event of an abnormal interruption; the alert unit is used to notify maintenance personnel to come for repair when necessary.

[0011] A further preferred technical solution: In the feature extraction stage, the preprocessing module adopts the robust feature extraction algorithms SIFT and SURF. In model training, noisy data is added for training. Adversarial training and model fusion are used to enhance the model's resistance to noise, so that the images or data acquired by the vision system can remain clear and remove noise caused by factors such as light changes and electromagnetic interference, so as to ensure that the positioning of the target object is accurate enough.

[0012] A control method for a vision-based intelligent control system for tool references, employing the aforementioned control system, includes the following steps: Step S1: Initialization settings: Install some standard balls inside the machine tool in a preset first position that is easy to photograph and measure; fix one of the standard balls in a preset second position, wherein the preset second position can be any one of the tool holder position point of the machine tool or the position point on the turret that can be photographed; Step S2: Camera capture: A structured light camera emits invisible infrared laser of a specific wavelength, and the distortion of the returned coded pattern is calculated using the Harris corner detection algorithm to obtain the object's position information, which is used to determine key feature points in the image. These corner points are stable under different viewpoints and can be used for object localization and pose estimation. A vision camera is used to simulate the human eye to acquire 3D data through two ordinary cameras at a certain distance. A time-of-flight camera is used to calculate the distance information of the object by measuring the time of flight of light. The known target template is compared with the image to find the most similar area, thereby determining the position of the target. This completes the identification of areas with drastic brightness changes in the image, determines the reference contour of the reference object, and realizes the operation of measuring the position of the tool tip. Step S3: Image reception and preprocessing: The signal receiving module is used to receive image data stream information from the structured light camera, vision camera and optical time-of-flight camera; the preprocessing module is used to preprocess the image by using mean filtering to smooth it, using median filtering to effectively remove salt-and-pepper noise, using Gaussian filtering to process Gaussian noise, and using histogram equalization to enhance contrast and highlight noise. Step S4: Visual Learning and Tool Compensation Adjustment: The visual algorithm module receives data from the preprocessing module and uses edge learning technology to accurately identify the shape, position, and movement trajectory of the tool during tool changing. During the tool compensation adjustment process, the acquisition marker remains unchanged, the relative position of the tool does not change, and only the camera posture is adjusted. When the robot moves to different positions, the tool and marker that do not change relative position are photographed. The camera obtains the corresponding data through the algorithm, and the precise position of the tool edge on the machine tool is obtained through the corresponding tool compensation calculation formula. Step S5: Tool Collision Avoidance Safety Handling: During the robot's movement, the collision avoidance unit, based on information such as the position and speed of other objects captured by visual imaging, uses a tool collision avoidance algorithm to interact with the robot and adjust the robot's own speed to avoid collisions. Assume there is an agent A and an obstacle B, both represented by circles: the center represents the position, and the radius represents the size; A and B each have a speed... , The system runs and, after a period of time, arrives at new positions A′ and B′ respectively; the two circular surfaces intersect here, indicating a collision; to solve this problem, it is necessary to predict the new positions a certain time later and adjust the current speed in response to possible collisions. Step S6: Tool Safety Warning: Utilizing the fusion safety module, receive visual anomaly information captured by the vision algorithm module and quickly respond. Visual anomalies include any of the following: tool cracks, tool wear exceeding a threshold, incorrect tool clamping position in the tool changer, excessively long tool change time, or robotic arm movement stagnation. The response includes: First, triggering the machine tool's emergency braking system to pause the tool change process, preventing the problem from worsening and causing more serious mechanical failures or safety accidents; Second, clearly displaying the anomaly details on the external human-machine interface of the controller, providing operators with intuitive troubleshooting information. Step S7: Product processing: After completing the automatic calculation of tool compensation parameters in steps S1-S6 above, the machine tool program is called to start the automated operation of processing the product.

[0013] The advantages of this invention compared to existing technologies are as follows: This invention uses multiple standard balls fixed at appropriate positions on the machine tool and cutting tool to provide a reference benchmark for precise camera measurement. Through a high-precision vision measurement technology solution combined with a self-developed control algorithm, it fundamentally solves the problem of inaccurate and error-prone manual measurement of tool reference points, avoiding the impact of human factors on machining accuracy and improving changeover speed and precision. Using this patented product eliminates the need for tool setting actions; the system automatically corrects the equipment's feed data based on measurement results, producing qualified products. This invention removes the unstable factors of manual intervention in machining processes, shortens changeover time, stabilizes product quality, avoids human error, prevents collisions, and improves work efficiency. This invention uses a 3D camera to first photograph a reference sphere fixed inside the machine tool and a reference sphere fixed on the tool holder, thereby calculating the coordinate values ​​of the cutting edge in the machine tool space. Then, it calculates the tool compensation parameters, calls the machine tool program, and performs product processing. The process avoids manual intervention and no longer relies on human skills. It automatically completes the calculation of tool compensation parameters and calls the machine tool program to process the product, eliminating the need for manual machine adjustment in this process, simplifying the complexity of the work, and improving the stability of the workpiece. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating the connection principle of the overall control system of the present invention; Figure 2 This is a schematic diagram of the control system of the present invention using a binocular camera to take pictures of a standard sphere; Figure 3 This is a schematic diagram of the control system of the present invention using a quad-camera to take pictures of a standard sphere; Figure 4 A process diagram for product processing using the control system of the robot of the present invention for tool compensation; Figure 5 This is a schematic diagram of the electrical function of the control system of the present invention.

[0016] In the diagram: 1. High-definition camera; 11. Structured light camera; 12. Vision camera; 13. Time-of-flight optical camera; 2. Standard sphere; 3. Controller; 31. Signal receiving module; 32. Preprocessing module; 33. Vision algorithm module; 331. Anti-collision unit; 332. Robot control unit; 333. Force sensor; 334. Drag-and-drop teaching and feedback unit; 34. Noise reduction module; 35. Fusion safety module; 351. Self-test unit; 352. Reminder unit. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] In the description of this invention, unless otherwise explicitly specified and limited, the terms "installed," "connected," "linked," and "equipped" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; or they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0019] Example: Figures 1 to 5The diagram illustrates an intelligent control system for a tool reference based on vision measurement. This control system includes a high-definition camera 1, a standard sphere 2, and a controller 3. Multiple standard spheres 1 are used, each installed inside the machine tool at a preset first position easily accessible for photographing and measurement; that is, the preset first position simply needs to facilitate photographing and measurement. Specifically, there can be two or more standard spheres. One standard sphere 1 is fixedly installed at a preset second position, which can be any of the tool holder position or a position point on the turret that can be photographed. The high-definition camera 1 maintains a signal connection with the controller 3 to achieve image data acquisition, processing, and transmission. The high-definition camera 1 includes a structured light camera 11, a vision camera 12, and a time-of-flight optical camera 13. The structured light camera 11 obtains the object's position information by emitting invisible infrared laser of a specific wavelength and calculating the distortion of the returned coded pattern using the Harris corner detection algorithm. This information is used to determine key feature points in the image; these corner points are stable under different viewing angles and can be used for object positioning and attitude estimation. The vision camera 12 is used to simulate the human eye to acquire 3D data through two ordinary cameras at a certain distance. The vision camera can be either a binocular 3D camera or a quad-camera. The time-of-flight camera 13 is used to calculate the distance information of an object by measuring the flight time of light, compare the known target template with the image, find the most similar region, and thus determine the position of the target. This completes the task of identifying areas with drastic brightness changes in the image, determining the reference outline of the reference object, and realizing the operation of measuring the position of the cutting tool tip.

[0020] In a preferred embodiment, the controller 3 includes a signal receiving module 31, a preprocessing module 32, a vision algorithm module 33, a noise reduction module 34, and a fusion security module 35. The signal receiving module 31 receives image data stream information from the structured light camera 11, the vision camera 12, and the optical time-of-flight camera 13. The preprocessing module 32 performs image smoothing using mean filtering, effectively removes salt-and-pepper noise using median filtering, processes Gaussian noise using Gaussian filtering, and enhances contrast using histogram equalization to highlight noise. In the feature extraction stage, the preprocessing module 32 employs robust feature extraction algorithms such as SIFT and SURF. During algorithm model training, noisy data is added for training, and adversarial training and model fusion are used to enhance the model's resistance to noise, thereby ensuring that the images or data acquired by the vision system remain clear and removing noise caused by factors such as light changes and electromagnetic interference, thus ensuring sufficient accuracy in locating the target object.

[0021] The vision algorithm module 33 receives data from the preprocessing module and uses edge learning technology to accurately identify the shape, position, and movement trajectory of the cutting tool during tool changing. The noise reduction module 34 filters and eliminates the influence of complex lighting conditions and electromagnetic interference in the workshop to ensure accurate environmental perception for the robot. During tool compensation adjustment, the vision algorithm module 33 keeps the acquisition marker constant and the relative position of the cutting tool unchanged. Only the camera posture is adjusted for testing. Each time the robot moves to a different position, images of the cutting tool and marker without relative position changes are captured. The camera obtains the corresponding data through an algorithm, and the precise position of the cutting edge on the machine tool is obtained through a corresponding tool compensation calculation formula. The tool compensation calculation formula is as follows: (1) Formula for calculating the position of linear interpolation points; The formula for calculating the location is as follows: Xn = X0 + (n / N) (X1 X0); Yn = Y0 + (n / N) (Y1 Y0); Zn = Z0 + (n / N) (Z1 Z0); Where Xn, Yn, and Zn are the coordinates of the interpolation points, X0, Y0, and Z0 are the starting point coordinates, X1, Y1, and Z1 are the ending point coordinates, n is the current point position, and N is the total number of points.

[0022] (2) Formula for calculating the position of circular interpolation points; The formula for calculating the location is as follows: Xn = I + R cos(θn); Yn = J + R sin(θn); Zn = Z0 + (n / N) (Z1 Z0); Where Xn, Yn, and Zn are the coordinates of the interpolation points, I and J are the coordinates of the arc center, R is the arc radius, θn is the angle corresponding to the current point, Z0 and Z1 are the Z coordinates of the starting and ending points, n is the current point position, and N is the total number of points.

[0023] (3) Formula for calculating the position of the spiral interpolation point; The formula for calculating the location is as follows: Xn = R cos(θn); Yn = R sin(θn); Zn = Z0 + (n / N) (Z1 Z0); Where Xn, Yn, and Zn are the coordinates of the interpolation points, R is the radius of the helix, θn is the angle corresponding to the current point, Z0 and Z1 are the Z coordinates of the starting and ending points, n is the current point position, and N is the total number of points.

[0024] In this embodiment, the vision algorithm module 33 further includes an anti-collision unit 331 and a robot control unit 332. The anti-collision unit 331 is used to monitor the dynamics around the machine tool in real time, accurately identify the position and movement trend of the tool, workpiece, and other obstacles, and quickly feed back to the robot control unit 332 once a collision risk is predicted. The robot control unit 332 is used to adjust the robot's motion trajectory in a timely manner to ensure the tool changing process is safe and error-free. The robot control unit 332 also includes a force sensor 333, which will stop the robot when its motion trajectory is abnormal. The vision algorithm module 33 also includes a drag-and-drop teaching and feedback unit 334, which is used to identify and learn the tool changing process. The operator only needs to manually drag the robot arm to complete a tool changing demonstration. During this process, the robot control unit tracks the subtle movements of the arm throughout, converting spatial position and posture changes into precise control codes.

[0025] The anti-collision algorithm of the anti-collision unit 331 is as follows: During its movement, the robot interacts with other objects captured by visual imaging to adjust its own speed and avoid collisions. Assume there is an agent A and an obstacle B, both represented by circles: the center represents position, and the radius represents size; A and B each have a speed... , The system runs and, after a period of time, arrives at new positions A′ and B′ respectively; the two circular surfaces intersect here, indicating a collision; to solve this problem, it is necessary to predict the new positions a certain time later and adjust the current speed in response to possible collisions. Obstacle avoidance trajectory calculation: using This represents the time interval between refreshing the environment state and adjusting the speed; the calculated time series is expressed as... ,in ; The shorter the time, the more timely the feedback on changes in environmental information, and the more likely the decision will be correct, but the greater the computational cost. For any time The position of the agent can be represented as Based on the current environmental information, the achievable obstacle avoidance speed zone can be calculated, denoted as: ; In plane geometry, it is a polyhedron, and its specific shape is related to the dynamics of the agent and the VO region; to simplify the calculation process, it can be... Meshization uses discrete, finite points in a grid to represent the set of possible velocities for an agent; When making decisions, the agent needs to take measures to adjust its speed; the adjusted speed must belong to the range where, assuming the operation is , then we have: The choice of different speeds will cause the agent to... After arriving at different locations, then in operation Then the agent is based on speed The new location reached can be represented as Note that both position and velocity are vectors. Using the agent's position at a certain moment as a node, different speeds are selected to obtain the agent's new position at the next moment, and a directed edge is established from the current position to the new position at the next moment; this will form an acyclic directed graph with the current position as the root node. Collision avoidance proof: Assume that the current velocities of agents A and B are respectively... And, both choose a new speed. and All are outside the corresponding RVO region, then and They must also be outside their respective VO zones. This ensures that collisions are avoided.

[0026] The proof is as follows: (1) Based on the definition of RVO, we can obtain: ; (2) Based on the symmetry of the VO region, we can obtain: ; (3) Combining (1) and (2), we can obtain: ; (4) Using translation invariance, we get: ; (5) Based on the characteristics of convex sets, let =0.5, weighting the two velocity expressions in equation (4) yields:

[0027] (6) Applying translation invariance to equation (5) yields: ; (7) Applying translation invariance to equation (5), we get: .

[0028] In this embodiment, the fusion safety module 35 is used to receive visual anomaly information captured by the vision algorithm module and quickly respond. The visual anomaly information includes any of the following: tool cracks, tool wear exceeding a threshold, incorrect tool clamping position in the tool-changing robot arm, excessively long tool changing time, or robot arm movement jamming. The response includes: first, triggering the machine tool's emergency braking system to suspend the tool changing process to prevent the problem from worsening and causing more serious mechanical failures or safety accidents; second, clearly displaying the anomaly details on the human-machine interface connected to the controller, providing operators with intuitive troubleshooting information. The fusion safety module 35 also includes a self-test unit 351 and an alert unit 352. The self-test unit 351 is used to initiate automatic restart and troubleshooting procedures in the event of an abnormal interruption; the alert unit 352 is used to notify maintenance personnel to perform repairs when necessary.

[0029] A control method for a vision-based intelligent control system for tool references, employing the aforementioned control system, includes the following steps: Step S1: Initialization settings: Install some standard balls inside the machine tool in a preset first position that is easy to photograph and measure; fix one of the standard balls in a preset second position, wherein the preset second position can be any one of the tool holder position point of the machine tool or the position point on the turret that can be photographed; Step S2: Camera capture: A structured light camera emits invisible infrared laser of a specific wavelength, and the distortion of the returned coded pattern is calculated using the Harris corner detection algorithm to obtain the object's position information, which is used to determine key feature points in the image. These corner points are stable under different viewpoints and can be used for object localization and pose estimation. A vision camera is used to simulate the human eye to acquire 3D data through two ordinary cameras at a certain distance. A time-of-flight camera is used to calculate the distance information of the object by measuring the time of flight of light. The known target template is compared with the image to find the most similar area, thereby determining the position of the target. This completes the identification of areas with drastic brightness changes in the image, determines the reference contour of the reference object, and realizes the operation of measuring the position of the tool tip. Step S3: Image reception and preprocessing: The signal receiving module is used to receive image data stream information from the structured light camera, vision camera and optical time-of-flight camera; the preprocessing module is used to preprocess the image by using mean filtering to smooth it, using median filtering to effectively remove salt-and-pepper noise, using Gaussian filtering to process Gaussian noise, and using histogram equalization to enhance contrast and highlight noise. Step S4: Visual Learning and Tool Compensation Adjustment: The visual algorithm module receives data from the preprocessing module and uses edge learning technology to accurately identify the shape, position, and movement trajectory of the tool during tool changing. During the tool compensation adjustment process, the acquisition marker remains unchanged, the relative position of the tool does not change, and only the camera posture is adjusted. When the robot moves to different positions, the tool and marker that do not change relative position are photographed. The camera obtains the corresponding data through the algorithm, and the precise position of the tool edge on the machine tool is obtained through the corresponding tool compensation calculation formula. Step S5: Tool Collision Avoidance Safety Handling: During the robot's movement, the collision avoidance unit, based on information such as the position and speed of other objects captured by visual imaging, uses a tool collision avoidance algorithm to interact with the robot and adjust the robot's own speed to avoid collisions. Assume there is an agent A and an obstacle B, both represented by circles: the center represents the position, and the radius represents the size. A and B each move at a speed and, after a period of time, arrive at new positions A′ and B′ respectively. The two circles intersect at this point, indicating a collision. To solve this problem, it is necessary to predict the new position a certain time in advance and adjust the current speed accordingly for potential collisions. Step S6: Tool Safety Warning: Utilizing the fusion safety module, receive visual anomaly information captured by the vision algorithm module and quickly respond. Visual anomalies include any of the following: tool cracks, tool wear exceeding a threshold, incorrect tool clamping position in the tool changer, excessively long tool change time, or robotic arm movement stagnation. The response includes: First, triggering the machine tool's emergency braking system to pause the tool change process, preventing the problem from worsening and causing more serious mechanical failures or safety accidents; Second, clearly displaying the anomaly details on the external human-machine interface of the controller, providing operators with intuitive troubleshooting information. Step S7: Product processing: After completing the automatic calculation of tool compensation parameters in steps S1-S6 above, the machine tool program is called to start the automated operation of processing the product.

[0030] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A vision-based intelligent control system for tool references, characterized in that: The control system includes a high-definition camera, a standard sphere, and a controller. Multiple standard spheres are installed inside the machine tool at a preset first position for easy photographing and measurement. One standard sphere is fixedly installed at a preset second position, which can be any position visible from the machine tool shank or turret. The high-definition camera maintains a signal connection with the controller. The high-definition camera includes a structured light camera, a vision camera, and a time-of-flight (TOF) camera. The structured light camera obtains the object's position information by emitting invisible infrared laser of a specific wavelength and calculating the distortion of the returned coded pattern using the Harris corner detection algorithm. This information is used to determine key feature points in the image; these corner points are stable under different viewing angles and can be used for object positioning and attitude estimation. The vision camera uses two ordinary cameras at a certain distance to simulate human vision and acquire 3D data. The TOF camera calculates the object's distance information by measuring the time of flight of light, compares a known target template with the image, finds the most similar region, and thus determines the target's position. This allows for the identification of areas with drastic brightness changes in the image, the determination of the reference object's baseline contour, and the measurement of the tool tip position. The controller includes a signal receiving module, a preprocessing module, a vision algorithm module, a noise reduction module, and a fusion security module. The signal receiving module receives image data streams from a structured light camera, a vision camera, and a time-of-flight optical camera. The preprocessing module uses mean filtering to smooth the image, median filtering to effectively remove salt-and-pepper noise, Gaussian filtering to process Gaussian noise, and histogram equalization to enhance contrast and highlight noise. The vision algorithm module receives data from the preprocessing module and uses edge learning technology to accurately identify the shape, position, and movement trajectory of the cutting tool during tool changing. The noise reduction module filters and eliminates complex noise from the workshop. The system mitigates the effects of lighting conditions and electromagnetic interference to ensure accurate environmental perception for the robot. The integrated safety module receives visual anomaly information captured by the vision algorithm module and responds rapidly. Visual anomalies include any of the following: tool cracks, tool wear exceeding a threshold, incorrect tool gripping position in the tool-changing robot, excessively long tool changing time, or robot arm movement stagnation. The response includes: first, triggering the machine tool's emergency braking system to pause the tool changing process to prevent the problem from worsening and causing more serious mechanical failures or safety accidents; second, clearly displaying the anomaly details on the external human-machine interface, providing operators with intuitive troubleshooting information.

2. The intelligent control system for tool reference based on vision measurement as described in claim 1, characterized in that: The vision camera can be either a binocular 3D camera or a quad-camera.

3. The intelligent control system for tool reference based on vision measurement as described in claim 2, characterized in that: in, During the tool offset adjustment process of the vision algorithm module, the acquisition marker remains unchanged, and the relative position of the tool does not change. Only the camera posture is adjusted for testing. Each time the robot moves to a different position, images are taken of the tool and marker whose relative positions do not change. The camera uses the algorithm to obtain the corresponding data, and the precise position of the tool edge on the machine tool is obtained through the corresponding tool offset calculation formula. The tool offset calculation formula is as follows: (1) Formula for calculating the position of linear interpolation points; The formula for calculating the location is as follows: Xn = X0 + (n / N) (X1 X0); Yn = Y0 + (n / N) (Y1 Y0); Zn = Z0 + (n / N) (Z1 Z0); Where Xn, Yn, and Zn are the coordinates of the interpolation points, X0, Y0, and Z0 are the starting point coordinates, X1, Y1, and Z1 are the ending point coordinates, n is the current point position, and N is the total number of points; Formula for calculating the position of circular interpolation points; The formula for calculating the location is as follows: Xn = I + R cos(θn); Yn = J + R sin(θn); Zn = Z0 + (n / N) (Z1 Z0); Where Xn, Yn, and Zn are the coordinates of the interpolation point, I and J are the coordinates of the arc center, R is the arc radius, θn is the angle corresponding to the current point, Z0 and Z1 are the Z coordinates of the start and end points, n is the current point position, and N is the total number of points. Formula for calculating the position of helical interpolation points; The formula for calculating the location is as follows: Xn = R cos(θn); Yn = R sin(θn); Zn = Z0 + (n / N) (Z1 Z0); Where Xn, Yn, and Zn are the coordinates of the interpolation points, R is the radius of the helix, θn is the angle corresponding to the current point, Z0 and Z1 are the Z coordinates of the starting and ending points, n is the current point position, and N is the total number of points.

4. The intelligent control system for tool reference based on vision measurement as described in claim 3, characterized in that: The vision algorithm module also includes an anti-collision unit and a robot control unit. The anti-collision unit is used to monitor the dynamics around the machine tool in real time, accurately identify the position and movement trend of tools, workpieces, and other obstacles, and quickly feed back to the robot control unit once a collision risk is predicted. The robot control unit is used to adjust the robot's motion trajectory in a timely manner to ensure the tool changing process is safe and error-free. The anti-collision algorithm of the anti-collision unit is as follows: During its movement, the robot interacts with other objects captured by visual imaging to adjust its own speed and avoid collisions. Assume there is an agent A and an obstacle B, both represented by circles: the center represents position, and the radius represents size; A and B each have a speed... , The system runs and, after a period of time, arrives at new positions A′ and B′ respectively; the two circular surfaces intersect here, indicating a collision; to solve this problem, it is necessary to predict the new positions a certain time later and adjust the current speed in response to possible collisions. Obstacle avoidance trajectory calculation: Let represent the time interval between refreshing the environment state and adjusting the speed; the calculated time series is expressed as , where The shorter the timeframe, the more timely the feedback on changes in environmental information, and the more likely the decision-making will be correct, but the greater the computational cost. For any time The position of the agent can be represented as Based on the current environmental information, the achievable obstacle avoidance speed zone can be calculated, denoted as: ; In plane geometry, it is a polyhedron, the specific shape of which is related to the dynamics of the agent and the VO region; to simplify the calculation process, it can be meshed, and the set of optional velocities of the agent can be represented by discrete, finite points in the mesh. When making decisions, the agent needs to take measures to adjust its speed; the adjusted speed must belong to the range where, assuming the operation is , then we have: Choosing different speeds will cause the agent to arrive at different locations later, thus affecting the operation. Then the agent is based on speed The new position reached can be represented as follows; note that both position and velocity are vectors. Using the agent's position at a certain moment as a node, different speeds are selected to obtain the agent's new position at the next moment, and a directed edge is established from the current position to the new position at the next moment; this will form an acyclic directed graph with the current position as the root node. Collision avoidance proof: Assume that the current velocities of agents A and B are respectively... and Both chose new speeds. and All are outside the corresponding RVO region, then Both must be outside their respective VO zones. This ensures that collisions are avoided. ； The proof is as follows: (1) Based on the definition of RVO, we can obtain: ； Based on the symmetry of the VO region, we can conclude that: ; Combining (1) and (2), we can obtain: ； Using translation invariance, we get: ； Based on the characteristics of convex sets, let =0.5, weighting the two velocity expressions in equation (4) yields: ； Applying translation invariance to equation (5), we get: ; Applying translation invariance to equation (5), we get: .

5. The intelligent control system for tool reference based on vision measurement as described in claim 4, characterized in that: The robot control unit also includes a force sensor, which will cause the robot to stop immediately when its movement trajectory is abnormal.

6. The intelligent control system for tool reference based on vision measurement as described in claim 5, characterized in that: The vision algorithm module also includes a drag-and-drop teaching and feedback unit, which is used to identify and learn the tool changing process. The operator only needs to manually drag the robot arm to complete a tool changing demonstration. During this process, the robot control unit tracks the subtle movements of the arm throughout and converts the changes in spatial position and posture into precise control codes.

7. The intelligent control system for tool reference based on vision measurement as described in claim 6, characterized in that: The integrated security module also includes a self-test unit and an alert unit; the self-test unit is used to initiate automatic restart and troubleshooting procedures in the event of an abnormal interruption; the alert unit is used to notify maintenance personnel to come for repairs when necessary.

8. The intelligent control system for tool reference based on vision measurement as described in claim 7, characterized in that: In the feature extraction stage, the preprocessing module employs robust feature extraction algorithms such as SIFT and SURF. During algorithm model training, noisy data is added for training, and adversarial training and model fusion are used to enhance the model's resistance to noise. This ensures that the images or data acquired by the vision system remain clear and removes noise caused by factors such as changes in lighting and electromagnetic interference, thereby ensuring sufficient accuracy in locating the target object.

9. A control method for a vision-based intelligent control system for tool references, employing the control system described in any one of claims 1-8, characterized in that: Includes the following steps: Step S1: Initialization settings: Install some standard balls inside the machine tool in a preset first position that is easy to photograph and measure; fix one of the standard balls in a preset second position, wherein the preset second position can be any position that can be photographed on the machine tool shank or the turret. Step S2: Camera capture: A structured light camera emits invisible infrared laser of a specific wavelength, and the distortion of the returned coded pattern is calculated using the Harris corner detection algorithm to obtain the object's position information, which is used to determine key feature points in the image. These corner points are stable under different viewpoints and can be used for object localization and pose estimation. A vision camera is used to simulate the human eye to acquire 3D data through two ordinary cameras at a certain distance. A time-of-flight camera is used to calculate the distance information of the object by measuring the time of flight of light. The known target template is compared with the image to find the most similar area, thereby determining the position of the target. This completes the identification of areas with drastic brightness changes in the image, determines the reference contour of the reference object, and realizes the operation of measuring the tool tip position. Step S3: Image reception and preprocessing: The signal receiving module is used to receive image data stream information from the structured light camera, vision camera and optical time-of-flight camera; the preprocessing module is used to preprocess the image by using mean filtering to smooth it, using median filtering to effectively remove salt-and-pepper noise, using Gaussian filtering to process Gaussian noise, and using histogram equalization to enhance contrast and highlight noise. Step S4: Visual Learning and Tool Compensation Adjustment: The visual algorithm module receives data from the preprocessing module and uses edge learning technology to accurately identify the shape, position, and movement trajectory of the tool during tool changing. During the tool compensation adjustment process, the acquisition marker remains unchanged, the relative position of the tool does not change, and only the camera posture is adjusted. When the robot moves to different positions, the tool and marker that do not change relative position are photographed. The camera obtains the corresponding data through the algorithm, and the precise position of the tool edge on the machine tool is obtained through the corresponding tool compensation calculation formula. Step S5: Tool Collision Avoidance Safety Handling: During the robot's movement, the collision avoidance unit, based on information such as the position and speed of other objects captured by visual imaging, uses a tool collision avoidance algorithm to interact with the robot and adjust the robot's own speed to avoid collisions. Assume there is an agent A and an obstacle B, both represented by circles: the center represents the position, and the radius represents the size; A and B each have a speed... The two circles run and, after a period of time, arrive at new positions A′ and B′ respectively; the two circles intersect here, indicating a collision; to solve this problem, it is necessary to predict the new positions a certain time later and adjust the current speed in response to possible collisions. Step S6: Tool Safety Warning: Utilizing the fusion safety module, receive visual anomaly information captured by the vision algorithm module and quickly respond. Visual anomalies include any of the following: tool cracks, tool wear exceeding a threshold, incorrect tool clamping position in the tool changer, excessively long tool change time, or robotic arm movement stagnation. The response includes: First, triggering the machine tool's emergency braking system to pause the tool change process, preventing the problem from worsening and causing more serious mechanical failures or safety accidents; Second, clearly displaying the anomaly details on the external human-machine interface of the controller, providing operators with intuitive troubleshooting information. Step S7: Product processing: After completing the automatic calculation of tool compensation parameters in steps S1-S6 above, the machine tool program is called to start the automated operation of processing the product.

Images