An industrial vision-based machining component positioning guidance system and method

Abstract

The present application relates to the field of industrial vision technology, and discloses a machining assembly positioning and guiding system and method based on industrial vision, and the machining assembly positioning and guiding system based on industrial vision comprises: obtaining a window micro-mark; collecting an original image; calculating absolute distortion displacement and time increment displacement; calculating absolute distortion displacement field and increment displacement field; calculating a pose-sensitive scalar; calculating a distortion-removed image; extracting a center point coordinate; and generating a guiding quantity. The present application expands discrete distortion displacement into a continuous displacement field through thin plate spline interpolation, constructs a solution weight in combination with the spatial distribution difference of distortion and the lever arm effect of camera imaging, simultaneously extracts a pose-sensitive scalar to generate a continuous scaling factor, makes the modulation of pose error fit the dynamic change state of distortion, makes the machining assembly pose detection result more fit the actual space state, and makes the positioning and guiding parameters match the actual distortion state, so as to fit the actual application scene of industrial machine tool machining.

Description

Technical Field

[0001] This invention relates to the field of industrial vision technology, and more specifically, to a system and method for positioning and guiding processing components based on industrial vision. Background Technology

[0002] Industrial vision technology has been widely applied to the positioning and guidance of CNC machine tool components. By capturing images of the machined components using industrial cameras, extracting the coordinates of feature points, and combining this with coordinate system transformations to solve for the pose, the machine tool is guided to complete the positioning process. This replaces traditional manual positioning methods and significantly improves positioning efficiency in the early stages of machining. Machine tool machining areas are typically protected by viewing windows. However, during machining, droplets, liquid films, oil stains, and other contaminants easily adhere to the surface of these windows. These contaminants cause light refraction and distortion, leading to shifts in the pixel coordinates of feature points in the camera-captured images, directly affecting the accuracy of pose detection.

[0003] The deposits on the viewport surface undergo slight flow and shape changes over time, making refractive distortion not a static feature but rather a short-term dynamic change. Existing technologies mostly employ fixed static distortion correction methods, which cannot capture the dynamic changes in distortion and are therefore difficult to achieve real-time and effective distortion correction. Furthermore, existing localization methods assign the same calculation weight to each feature point, failing to consider the different distributions of distortion in different image regions and ignoring the lever arm effect of camera imaging. Pixels farther from the camera's principal point experience a greater impact on pose due to distortion, and this omission further exacerbates pose detection errors.

[0004] In existing technologies, the pose error is not modulated in conjunction with the dynamic influence of distortion, resulting in a mismatch between the output positioning guidance amount and the actual distortion state. This ultimately leads to insufficient positioning guidance accuracy of the machine tool for the processed components, affecting the product quality of subsequent processing steps. Furthermore, existing technologies have not yet proposed an effective solution to simultaneously address the series of positioning problems caused by dynamic refraction distortion of the viewport, full-image correction, and adaptation to the distortion state. This has become a pressing technical challenge in the field of industrial vision-guided machine tool component positioning. Summary of the Invention

[0005] This invention provides a processing component positioning and guidance system and method based on industrial vision, which solves the technical problems in the background art mentioned above.

[0006] This invention provides a processing component positioning and guidance system based on industrial vision, comprising: The reference coordinate acquisition module establishes the transformation relationship between the camera coordinate system and the machine tool coordinate system, and obtains the reference pixel coordinates of the window micro-markers; The original image acquisition module, with the processing components clamped in and the machine tool stationary, continuously acquires the first and second original images at fixed time intervals; The distortion displacement calculation module detects the current pixel coordinates of the window micro-markers in the first and second original images, calculates the difference between the current pixel coordinates and the reference pixel coordinates to obtain the absolute distortion displacement, and calculates the time increment displacement generated within a fixed time interval. The displacement field extension module uses an interpolation algorithm to extend the absolute distortion displacement and the time increment displacement into an absolute distortion displacement field and an increment displacement field, respectively. The pose-sensitive scalar calculation module calculates the spatial gradient of the incremental displacement field to construct a strain rate tensor field, and calculates the strain rate moment by combining the camera principal point lever arm and extracts the maximum eigenvalue as the pose-sensitive scalar. The distortion-reduced image calculation module uses the absolute distortion displacement field to reverse resample the first original image to obtain the distortion-reduced image; The center point extraction module extracts the center point coordinates of the reference hole of the machining component from the distortion-free image; The guidance quantity generation module constructs perspective multi-point positioning solution weights based on the strain rate tensor field and the center point coordinates, solves the initial pose error by combining the transformation relationship, generates a continuous scaling factor using the pose-sensitive scalar, modulates the initial pose error using the continuous scaling factor to generate the guidance quantity, and outputs it to the CNC system.

[0007] This invention provides a method for positioning and guiding processing components based on industrial vision, comprising the following steps: Step S201: Establish the transformation relationship between the camera coordinate system and the machine tool coordinate system, and obtain the reference pixel coordinates of the window micro-marker; Step S202: Clamp the machining component and, while the machine tool is stationary, continuously acquire the first and second original images at fixed time intervals; Step S203: Detect the current pixel coordinates of the window micro-markers in the first original image and the second original image, calculate the difference between the current pixel coordinates and the reference pixel coordinates to obtain the absolute distortion displacement, and calculate the time increment displacement generated within a fixed time interval. Step S204: The absolute distortion displacement and time increment displacement are extended into an absolute distortion displacement field and an increment displacement field, respectively, using an interpolation algorithm. Step S205: Calculate the spatial gradient of the incremental displacement field to construct the strain rate tensor field, combine it with the camera principal point lever arm to calculate the strain rate moment and extract the maximum eigenvalue as the pose-sensitive scalar. Step S206: Reverse resampling of the first original image using the absolute distortion displacement field to obtain the distortion-free image; Step S207: Extract the center point coordinates of the reference hole of the machining component from the distorted image; Step S208: Construct perspective multi-point positioning solution weights based on strain rate tensor field and center point coordinates, solve initial pose error by combining transformation relationship, generate continuous scaling factor using pose sensitive scalar, apply continuous scaling factor to modulate initial pose error to generate guiding quantity and output to CNC system.

[0008] The beneficial effects of this invention are as follows: Addressing the positioning problem of machining components caused by dynamic refraction distortion in machine tool windows, this invention achieves precise quantification of window distortion and distortion correction across all pixels of the image. Through thin-plate spline interpolation, discrete distortion positions are expanded into a continuous displacement field, allowing distortion correction to cover any pixel position in the image. In the positioning calculation, the spatial distribution differences of distortion and the lever arm effect of camera imaging are combined to construct calculation weights, ensuring that the calculation weights for different feature points match the actual degree of distortion influence. Simultaneously, pose-sensitive scalars are extracted to generate continuous scaling factors, making the modulation of pose error conform to the dynamic changes in distortion. This achieves parameter interoperability between the camera coordinate system and the machine tool coordinate system, adapting the output guidance quantity to the control requirements of the machine tool CNC system. This makes the pose detection results of the machining components more closely resemble the actual spatial state, and the positioning guidance parameters match the actual distortion state, thus fitting the actual application scenarios of industrial machine tool processing and providing a reliable positioning basis for subsequent processing steps. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of a processing component positioning and guidance system based on industrial vision according to the present invention; Figure 2 This is a flowchart of a processing component positioning and guidance method based on industrial vision according to the present invention; Figure 3 This is a schematic diagram of the computational scenario of the present invention.

[0010] In the figure: Reference coordinate acquisition module 101, original image acquisition module 102, distortion displacement calculation module 103, displacement field expansion module 104, pose sensitive scalar calculation module 105, distortion-free image calculation module 106, center point extraction module 107, and guide quantity generation module 108. Detailed Implementation

[0011] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0012] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of the present invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in one or more embodiments of the present invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" indicate that the element or object preceding the term encompasses the elements or objects listed following the term and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0013] like Figures 1-3 As shown, a processing component positioning and guidance system based on industrial vision includes: The reference coordinate acquisition module 101 is used to establish the transformation relationship between the camera coordinate system and the machine tool coordinate system, and to acquire the reference pixel coordinates of the window micro-marker; The original image acquisition module 102 is used to clamp the processing component and continuously acquire the first original image and the second original image at fixed time intervals while the machine tool is stationary. The distortion displacement calculation module 103 is used to detect the current pixel coordinates of the window micro-markers in the first original image and the second original image, calculate the difference between the current pixel coordinates and the reference pixel coordinates to obtain the absolute distortion displacement, and calculate the time increment displacement generated within a fixed time interval. The displacement field extension module 104 is used to extend the absolute distortion displacement and the time increment displacement into an absolute distortion displacement field and an increment displacement field, respectively, using an interpolation algorithm. The pose-sensitive scalar calculation module 105 is used to calculate the spatial gradient of the incremental displacement field to construct the strain rate tensor field, and to calculate the strain rate moment by combining the camera principal point lever arm and extract the maximum eigenvalue as the pose-sensitive scalar. The distortion-reduced image calculation module 106 is used to reverse resample the first original image using the absolute distortion displacement field to obtain a distortion-reduced image. Center point extraction module 107 is used to extract the center point coordinates of the reference hole of the machining component in the distortion-free image; The guide quantity generation module 108 is used to construct perspective multi-point positioning solution weights based on the strain rate tensor field and the center point coordinates, solve the initial pose error by combining the transformation relationship, generate a continuous scaling factor using the pose sensitive scalar, apply the continuous scaling factor to modulate the initial pose error to generate guide quantities and output them to the CNC system.

[0014] In one embodiment of the present invention, the camera intrinsic parameter matrix is ​​determined by minimizing the reprojection error satisfying the following equation. With camera principal point : in To calibrate the pixel coordinates of pattern feature points, To calibrate the 3D coordinates of feature points in the camera coordinate system, This is a perspective projection operator used to map points in three-dimensional space to two-dimensional pixels. For the camera intrinsic parameter matrix, The pixel coordinate vector of the camera principal point; Find the homogeneous transformation matrix from the machine coordinate system to the camera coordinate system that minimizes the following expression. : The transformation relationship between the camera coordinate system and the machine tool coordinate system can be obtained from the following formula. : in To calibrate the homogeneous coordinate vectors of the tooling's three-dimensional feature points in the machine tool coordinate system, To obtain the first three components of the homogeneous transformation result to form a three-dimensional vector, To solve for the independent variables when minimizing the objective function, we need to find the homogeneous transformation matrix parameters that minimize the sum of squared reprojection errors. For L2 norm operations, This is the homogeneous transformation matrix from the camera coordinate system to the machine tool coordinate system; Acquire reference images The reference pixel coordinates of the window micro-marks are calculated according to the following formula. : in For the reference image at pixel coordinates The brightness value at that location, For the first A set of pixels in the reference image for each window micro-marker. This is the matrix transpose symbol.

[0015] It should be noted that the pixel coordinates of the feature points in the calibration pattern are the positional information of the feature points on the calibration pattern in the image pixel coordinate system. These coordinates can be obtained by extracting feature points from the acquired calibration pattern image using the Harris corner detection algorithm and the Hitomasi corner detection algorithm. The camera intrinsic parameter matrix is ​​a matrix describing the geometric characteristics of camera imaging, reflecting the combination relationship of inherent imaging parameters such as camera focal length, principal point, and pixel scaling. The camera principal point pixel coordinate vector is the coordinate information of the intersection point of the camera's ideal optical axis and the imaging plane in the pixel coordinate system, reflecting the position of the center pixel in the camera image. The 3D feature point coordinates of the calibration fixture are the 3D spatial position information of the feature points on the calibration fixture in the machine tool coordinate system. The preferred arrangement is a rectangular array of feature points, with adjacent points spaced 50 mm apart on the X and Y axes and a uniform 100 mm on the Z axis. This arrangement covers the field of view of the industrial camera and ensures the uniqueness and stability of the coordinate transformation matrix solution. The calibration fixture image is an image acquired by an industrial camera of a fixed calibration fixture in the machine tool coordinate system, reflecting the imaging characteristics of the calibration fixture within the camera's field of view. The homogeneous transformation matrix from the machine tool coordinate system to the camera coordinate system is a matrix characterizing the rigid body transformation relationship between the two coordinate systems. The reference image is an image acquired by the industrial camera when the viewport is in a reference state, reflecting the imaging background characteristics under distortion-free conditions. It can be obtained by taking a single shot with the industrial camera when the machine tool is stationary and the viewport is in the reference state. The pixel set of the viewport micro-marker is the set of all pixels covered by a single viewport micro-marker in the reference image. It can be obtained by segmenting and extracting the viewport micro-marker region in the reference image using an image segmentation algorithm. The pixel brightness value is the quantized brightness value of a single pixel in the image, obtained by analog-to-digital conversion of the light intensity signal of the imaging pixel using the industrial camera's image acquisition module. The reference pixel coordinates of the window micromarker are the baseline pixel position information of the window micromarker in the reference image, reflecting the pixel spatial position of the micromarker in the distortion-free state of the window.

[0016] It should be noted that the specific criteria for defining the reference state of the viewing window are as follows: the surface of the machine tool viewing window is free of droplets, liquid films, oil stains, dust, and other adhering substances, and the installation position of the viewing window is neither offset nor tilted. The cleanliness of the viewing window surface can be judged by visual inspection; the absence of visible adhering substances meets the cleanliness requirements. Simultaneously, the perpendicularity and parallelism between the viewing window's mounting surface and the machine tool's reference surface can be detected by laser ranging; deviations within 0.01 mm indicate no installation position deviation. Meeting both criteria constitutes the reference state of the viewing window. A checkerboard calibration board is used, with each checkerboard square measuring 20 mm. The calibration board has 9 rows and 12 columns, with alternating black and white squares. The substrate of the calibration board is a rigid metal plate with a matte finish to avoid glare affecting imaging. This specification of calibration pattern is compatible with the standard field of view of industrial cameras, and the corner features are clearly defined, facilitating extraction. The specific method for extracting the pixel coordinates of feature points in the calibration pattern is as follows: First, the acquired calibration pattern image is converted to grayscale and subjected to Gaussian filtering. Grayscale conversion converts the color image into a single-channel grayscale image, and Gaussian filtering uses a 5×5 filter kernel to reduce image noise. Then, the corner points of the checkerboard are detected using the Hitomasi corner detection algorithm, with a corner response value threshold of 200 set to remove false corner points with low response values. Finally, the detected corner points are refined at the sub-pixel level, and the accuracy of the corner coordinates is improved to 0.01 pixels using a quadratic interpolation method. The specific method for identifying the pixel set corresponding to each window micro-marker in the reference image is as follows: First, the reference image is converted to grayscale and binarized, with a binarization threshold of 128 set to divide the image into foreground and background regions. Then, the connected component analysis algorithm is used to identify connected components in the image. Based on the preset size and shape of the window micro-marker, connected components that meet the micro-marker features are selected. Finally, all pixels in each connected component that meets the features are grouped into a set, which is the pixel set corresponding to a single window micro-marker.

[0017] It should be noted that the homogeneous transformation matrix from the machine coordinate system to the camera coordinate system is solved by combining the perspective n-point algorithm with nonlinear optimization. First, the transformation matrix is ​​initially estimated using the EPnP algorithm. The initial value of the homogeneous transformation matrix is ​​obtained by using the coordinates of the three-dimensional feature points of the calibration fixture and the corresponding pixel coordinates. Then, with the goal of minimizing the reprojection error, the initial matrix is ​​iteratively refined using the Levenburg-Marquardt algorithm. The iteration terminates when the change in reprojection error is less than 0.001 pixel units. The matrix obtained at this point is the homogeneous transformation matrix from the machine coordinate system to the camera coordinate system. The calibration fixture is a rectangular metal base. The mounting surface of the base is in contact with the reference surface of the machine tool table. The upper surface of the base is covered with circular raised feature points. The feature points are 5 mm in diameter and 2 mm in height. The feature points are arranged in a rectangular array with 4 rows and 4 columns. The center-to-center distance between adjacent feature points is 50 mm in both the X and Y axes. The Z-axis coordinate of all feature points is uniformly set to 100 mm above the reference surface of the machine tool table. The surface of the feature points is treated with a highlighting process to facilitate camera recognition.

[0018] It should be noted that this invention obtains the intrinsic parameter matrix and principal point coordinates characterizing the camera's inherent imaging properties through camera calibration, laying the foundation for subsequent geometric analysis of the image; establishes a rigid body transformation relationship between the camera coordinate system and the machine tool coordinate system through hand-eye calibration, realizing the conversion of pixel information from visual detection to spatial coordinate information for machine tool motion control; and provides a fixed reference anchor point for the subsequent quantification and correction of window refraction distortion by acquiring the micro-marker pixel coordinates under the window reference state. These three elements form the basic geometric parameter system for visual positioning guidance. Specifically, solving for the camera's intrinsic parameters and principal point reduces the inherent system error of camera imaging and improves the mapping accuracy between pixel coordinates and spatial coordinates; the establishment of the coordinate system transformation relationship enables parameter exchange between the vision system and the machine tool CNC system, allowing the visual positioning results to directly serve the machine tool's motion control; and the acquisition of the window micro-marker reference coordinates provides a unified reference benchmark for subsequent dynamic distortion correction, giving distortion quantification a clear basis for comparison.

[0019] In one embodiment of the present invention, when the CNC machine tool is in a stationary state, the acquisition time of the first original image is set to... This triggers the industrial camera to capture the first raw image. ; Set the acquisition time of the second original image as The data collection time satisfies the following formula: At the time of collection Trigger the industrial camera to acquire a second raw image ;in This is the acquisition time of the first original image. This refers to the acquisition time of the second original image. The first original image, For the second original image, It is a fixed time interval.

[0020] It should be noted that the fixed time interval is preferably set between 100 and 500 milliseconds. This range can capture the short-term continuous dynamic changes in window refraction distortion. If the interval is too short, there will be no effective difference between two frames; if the interval is too long, the distortion changes will exceed the short-term linear characteristics, making it impossible to accurately quantify the rate of change. The specific triggering method for the industrial camera to acquire images is that the machine tool CNC system sends a TTL level trigger signal to the industrial camera. After the CNC system detects that the machine tool is in a stationary state, it outputs a high-level signal at the preset acquisition time. After receiving the high-level signal, the camera immediately starts the exposure process. The transmission delay of the trigger signal is controlled within 1 millisecond to avoid time reference deviation. The first original image acquisition time is set to the moment after the processing component is clamped and left to stand still for 3 to 5 seconds. During the clamping process, the clamping force of the fixture will cause the processing component to make a slight position adjustment, and the fixture itself will also produce a small elastic deformation. This process is a transient change. After standing still for 3 to 5 seconds, the clamping force and deformation tend to stabilize. The image acquired at this time can truly reflect the actual clamping position of the processing component. The specific parameter settings for the industrial camera to acquire the first and second raw images are as follows: exposure time is set to 10 to 50 microseconds, gain is set to 1 to 2 times, image resolution is 2048×2048 pixels, acquisition frame rate is 10 to 30 frames per second, white balance mode is set to automatic, and the acquired image format is 8-bit grayscale. This parameter combination can avoid overexposure due to reflections from the machine tool's lighting source, while ensuring the grayscale resolution of the image, which is convenient for subsequent detection of micro-markers and workpiece features. The specific criteria for determining the machine tool's stationary state are: the positional deviation of all moving axes of the machine tool is less than 0.001 mm, the axis feed rate is 0 mm / s, the clamping force detection value of the fixture remains stable with a deviation of less than 5%, and the vibration amplitude detected by the vibration sensor in the acquisition area of ​​the industrial camera is less than 0.0005 mm. The determination is made jointly by the axis status feedback of the machine tool CNC system and the detection data of the external sensor. When all conditions are met, the machine tool is considered to be in a stationary state.

[0021] It should be noted that images are acquired while the machine tool is stationary after the machining component is clamped. This eliminates interference from machine tool movement, vibration, and transient deformation of the component during clamping, ensuring that the imaging features accurately reflect the actual position of the machining component and the current state of the viewing window. By setting a custom acquisition time and a fixed time interval, two raw images are continuously acquired to capture the short-term dynamic changes in the window's refractive distortion, providing observational data with a unified time reference for subsequent quantification of the distortion rate. The requirement of acquiring images while the machine tool is stationary eliminates imaging errors such as motion blur and positional offset, ensuring the authenticity and effectiveness of the image features. The continuous acquisition at a fixed time interval allows the difference between the two images to be directly attributed to the short-term changes in window distortion, providing accurate incremental data for subsequent strain rate tensor calculations. The acquisition time setting after clamping and stationary avoids imaging deviations caused by transient deformation. The two raw images acquired provide fundamental and effective image data for subsequent distortion correction guided by positioning and dynamic risk quantification, perfectly meeting the requirements of the detection scenario of dynamic refractive distortion of the in-machine viewing window.

[0022] In one embodiment of the present invention, reference pixel coordinates are used. Determine the set of pixels centered on The current pixel coordinates of the window micro-marker in the first original image are calculated according to the following formula. : in The first original image at pixel coordinates The brightness value at that location, For the first The set of pixels corresponding to each window's micro-mark. The current pixel coordinates of the viewport micro-marker in the first original image, This is the matrix transpose symbol; The current pixel coordinates of the window micro-marker in the second original image are calculated according to the following formula. : in For the second original image at pixel coordinates The brightness value at that location, Set the current pixel coordinates of the viewport micro-marker in the second original image; Calculate the absolute distortion displacement using the following formula : Calculate the time increment displacement according to the following formula : in For the first The absolute distortion displacement of each window's micro-marker Provide reference pixel coordinates for the window micro-markers. This refers to the time increment displacement generated by the micromarker in the window within a fixed time interval.

[0023] It should be noted that the current pixel coordinates of the micro-marker in the first original image window are the pixel position information obtained by calculating the brightness-weighted centroid of the micro-marker in the first original image, reflecting the actual pixel position of the micro-marker under the window distortion state at the first acquisition time. The current pixel coordinates of the micro-marker in the second original image window are the pixel position information obtained by calculating the brightness-weighted centroid of the micro-marker in the second original image, reflecting the actual pixel position of the micro-marker under the window distortion state at the second acquisition time. The absolute distortion displacement is the difference between the current pixel coordinates of the micro-marker in the first original image and the reference pixel coordinates, reflecting the degree of pixel offset of the current window state relative to the undistorted reference state. The time increment displacement is the difference between the current pixel coordinates of the micro-marker in the second original image and the first original image, reflecting the amount of change in the micro-marker pixel position caused by window distortion within a fixed time interval.

[0024] It should be noted that the reference pixel coordinates of the window micro-marker are the baseline positions under distortion-free conditions. Defining the pixel set centered on these coordinates directly limits the detection range of the micro-marker in the image, avoiding the inclusion of invalid pixels from the background area in the calculation. Simultaneously, it ensures a high degree of match between the detection area and the actual pixel coverage of the micro-marker. For example, if the micro-marker covers 15×15 pixels in the reference image, a 15×15 pixel set centered on the reference coordinates can completely encompass all effective pixels of the micro-marker. The specific range for determining the pixel set centered on the reference pixel coordinates is preferably a 10×10 to 20×20 pixel square area centered on the reference pixel coordinates of the window micro-marker. This range can completely cover the pixel coverage of the window micro-marker in the image while avoiding the inclusion of too many background pixels. A 10×10 pixel set is used when the micro-marker pixel size is 10×10, and a 20×20 pixel set is used when the micro-marker pixel size is 15×15, which is compatible with pixel range variations caused by slight distortion of the micro-marker.

[0025] It should be noted that this invention defines a fixed set of pixels centered on the reference pixel coordinates of the window micro-marker. Using a unified brightness-weighted centroid calculation method, the current pixel coordinates of the micro-marker in two original frames of images are obtained. Coordinate difference calculations are then used to obtain the absolute distortion displacement reflecting static window distortion and the time increment displacement reflecting dynamic window distortion, transforming window refraction distortion into quantifiable pixel coordinate difference data. The fixed pixel set and unified calculation method ensure consistency in the detection of micro-marker pixel coordinates at different times, reducing detection and calculation deviations. The subtraction rule of coordinate components enables separate quantification of distortion in the horizontal and vertical directions. The acquisition of absolute distortion displacement and time increment displacement provides discrete reference data for the subsequent construction of the full-image distortion displacement field, allowing for accurate quantification of both static offset and short-term dynamic changes in window distortion, which will not be elaborated further here.

[0026] In one embodiment of the invention, the current pixel coordinates in the first original image are marked with a window micro-marker. As interpolation control points, with absolute distortion displacement As the target displacement at the interpolation control point, for any pixel coordinate Constructing an absolute distortion displacement field The components satisfy the following formula: The absolute distortion displacement field coefficients are determined by the following interpolation constraint: : in The current pixel coordinates of the viewport micro-marker in the first original image, For absolute distortion displacement, For absolute distortion displacement field, For the horizontal component of the absolute distortion displacement field. For the vertical component of the absolute distortion displacement field, Let be the kernel function, and its expression is: , It is a very small positive number. It is the natural logarithm. This is a Euclidean distance calculation used to calculate the spatial distance between any pixel coordinates and interpolation control points; The current pixel coordinates in the first original image are marked with a window micro-marker. As interpolation control points, displacement is based on time increments. As the target displacement at the interpolation control point, for any pixel coordinate Constructing an incremental displacement field The components satisfy the following formula: The incremental displacement field coefficients are determined by the following interpolation constraint. : in Displacement by time increment. For incremental displacement field, For the horizontal component of the incremental displacement field, This represents the vertical component of the incremental displacement field. The kernel function definition is consistent with the absolute distortion displacement field construction stage.

[0027] It should be noted that the absolute distortion displacement component calculation formula is an expression for calculating the horizontal and vertical components of absolute distortion displacement corresponding to any pixel coordinate, established through a thin-plate spline interpolation function. This reflects the correlation between distortion displacement and interpolation control points. The absolute distortion displacement field coefficients are undetermined coefficient values ​​in the absolute distortion displacement component calculation formula, reflecting the fitting characteristics of the interpolation function to discrete distortion displacement data. The absolute distortion displacement field is a continuous distortion displacement distribution system across the entire image, composed of the absolute distortion displacement component calculation formulas and their corresponding coefficients, reflecting the absolute distortion displacement characteristics corresponding to each pixel coordinate in the image. The incremental displacement component calculation formula is an expression for calculating the horizontal and vertical components of incremental displacement corresponding to any pixel coordinate, established through a thin-plate spline interpolation function. This reflects the correlation between incremental displacement and interpolation control points. The incremental displacement field coefficients are undetermined coefficient values ​​in the incremental displacement component calculation formula, reflecting the fitting characteristics of the interpolation function to discrete incremental displacement data. The incremental displacement field is a continuous incremental displacement distribution system across the entire image, composed of the incremental displacement component calculation formulas and their corresponding coefficients, reflecting the time incremental displacement characteristics corresponding to each pixel coordinate in the image.

[0028] It should be noted that thin-plate spline interpolation is based on the physical model of elastic thin-plate bending. It can fit a smooth and continuous surface using the numerical values ​​of discrete control points, adapting to the two-dimensional planar distribution characteristics of image pixels. The discrete micro-marker distortion displacement is data from a finite number of control points. This algorithm can generate continuous distortion displacement values ​​corresponding to all pixels in the image using this data. For example, 10 discrete micro-marker control points can fit a continuous displacement field for a 2048×2048 pixel image. The coefficients of the interpolation function are undetermined. An equation can be established for the pixel coordinates and corresponding target displacement of each interpolation control point. The number of control points must be greater than the number of coefficients to form an overdetermined system of equations, providing a sufficient data foundation for solving the coefficients. For example, if there are 12 undetermined coefficients, at least 13 control points are needed to establish the system of equations to ensure the uniqueness of the solution. The interpolation constraint is that the calculated displacement at the control point is equal to the actual target displacement. This condition gives the overdetermined system of equations a clear solution criterion. Unique coefficient values ​​can be obtained by solving the system using algorithms such as the least squares method, avoiding multiple solutions for the coefficient values ​​and ensuring the consistency of the displacement field calculation results. Discrete micro-marker distortion displacement can only reflect the distortion state at control points. Since most workpiece feature points in the image are non-control point pixels, a continuous displacement field can directly obtain the distortion displacement of any pixel through calculation, fully characterizing the distortion distribution features of the entire image and adapting to the distortion correction needs of any feature point of the workpiece. Furthermore, the interpolation control points are the current pixel coordinates of the window micro-markers, and their positions remain fixed. By simply changing the target displacement to absolute distortion displacement or time-increment displacement, two displacement fields can be constructed using the same set of control points without needing to redefine the control points, ensuring the spatial reference of the two displacement fields is consistent and facilitating subsequent joint calculations.

[0029] It should be noted that this invention uses the current pixel coordinates of the window micro-marker as a fixed interpolation control point, and takes the absolute distortion displacement and time increment displacement as the target displacements of the control point, respectively. A thin-plate spline interpolation algorithm is used to establish the displacement component calculation formula, and the field coefficients are solved by combining interpolation constraints. This expands the discrete micro-marker distortion displacement into a continuous absolute distortion displacement field and an increment displacement field covering the entire image, realizing the quantification of the continuous distribution of window distortion from discrete points to the entire image. The thin-plate spline interpolation ensures the smooth continuity of the displacement field, adapting to the natural distribution characteristics of window refraction distortion; the same control point adapts to different target displacements, ensuring the consistency of the spatial reference of the two displacement fields; independent component calculation formulas realize separate fitting of horizontal and vertical distortions; the continuous displacement field of the entire image can directly give the distortion displacement of any pixel, meeting the needs of distortion correction and dynamic change analysis of any feature point of the workpiece, and providing continuous data support for subsequent image distortion removal and strain rate tensor calculation.

[0030] In one embodiment of the present invention, in the workpiece pixel region Inner incremental displacement field Find the spatial gradient matrix : in It is a pixel coordinate vector. and These are the partial derivatives of the horizontal component of the incremental displacement field with respect to the horizontal and vertical pixel coordinates, respectively. and These are the partial derivatives of the vertical component of the incremental displacement field with respect to the horizontal and vertical pixel coordinates, respectively. Construct the strain rate tensor field based on the following formula : in For a fixed time interval, It is the transpose of the spatial gradient matrix. For matrix-valued functions, This is the matrix transpose. For the workpiece pixel area arbitrary pixel coordinates Calculate the length of the camera principal point lever arm and weight function The strain rate moment is calculated according to the following formula. : in Let the principal point pixel coordinate vector of the camera be denoted as . This is a measure of the area of ​​the pixel region of the workpiece. This is a double integral operator used to perform weighted accumulation operations over a continuous pixel domain. This is a 2-norm operation used to calculate the Euclidean distance between a pixel and the camera principal point; The pose-sensitive scalar PS-NSRM is extracted according to the following formula: in This is a maximum eigenvalue operation used to calculate the maximum eigenvalue corresponding to the strain rate moment. It is a pose-sensitive scalar.

[0031] It should be noted that the spatial gradient matrix is ​​a two-dimensional matrix obtained by taking the partial derivatives of each component of the incremental displacement field within the workpiece pixel region with respect to the pixel coordinates, reflecting the local variation trend of the incremental displacement in the pixel space. The local rate of change of the incremental displacement field components are the element values ​​in the spatial gradient matrix, reflecting the instantaneous rate of change of the horizontal and vertical components of the incremental displacement field in the horizontal and vertical directions of the pixel coordinates. The strain rate tensor field is a symmetric matrix field distribution constructed by combining the spatial gradient matrix with a fixed time interval, reflecting the short-term dynamic change characteristics of the window distortion of each pixel within the workpiece pixel region. The weight function value is the square of the distance from the pixel to the camera principal point within the workpiece pixel region, reflecting the difference in the pose influence weight of different pixel positions due to the lever arm of the camera principal point. The strain rate moment is the matrix obtained by weighted integration and normalization of the strain rate tensor field through the weight function, reflecting the overall dynamic change characteristics of the window distortion and the pose influence weight of the workpiece pixel region. The pose-sensitive scalar is the largest eigenvalue of the strain rate moment, reflecting the overall influence of the dynamic change of window distortion on the positioning and guiding pose of the machining component.

[0032] It should be noted that the specific method for defining the workpiece pixel region is as follows: the contour of the processing component in the first original image before distortion removal is identified by an image segmentation algorithm, and all pixels within the contour are extracted to form the workpiece pixel region. If the processing component is a regular shape, it can be defined by a preset pixel coordinate range. At the same time, the pixel dilation value of the region boundary is set to 5 pixels to avoid edge errors in contour recognition and ensure that the region completely covers the processing component. The incremental displacement field spatial gradient matrix is ​​solved by the central difference method in numerical differentiation. For the displacement component of any pixel, the displacement values ​​of its one adjacent pixel in the horizontal and vertical directions are taken and the difference is calculated with a step size of 1 pixel. The calculation formula is the difference between the displacements of adjacent pixels divided by the pixel step size. The four elements of the gradient matrix are solved by this method, and the calculation results are retained to 6 decimal places to avoid truncation errors in numerical differentiation. The weighted integration of the strain rate tensor field employs a discrete summation method to achieve continuous integration. For each pixel within the workpiece pixel region, its weight function value is multiplied by the corresponding strain rate tensor to obtain a weighted tensor. Then, the weighted tensors of all pixels are summed element-wise to obtain the total tensor, which is the weighted integration result. The maximum eigenvalue of the strain rate moment is determined using the Jacobi iterative algorithm. The convergence threshold is set to 10^-6, and the maximum number of iterations is 1000. If convergence is not achieved after reaching the maximum number of iterations, the strain rate moment is recalculated. After obtaining all eigenvalues, the maximum value is selected as the pose-sensitive scalar.

[0033] It should be noted that the distortion caused by window refraction is a continuous small deformation. A symmetric tensor field can eliminate the non-deformation components caused by rigid body rotation, retaining only the strain rate information of pure deformation, thus accurately reflecting the actual distortion characteristics of the window. An asymmetric tensor introduces irrelevant rotational components, leading to distorted distortion representation and failing to accurately reflect the actual distortion state of the window. Camera imaging is a central projection; the farther a pixel is from the principal point, the greater the spatial coordinate deviation caused by the same pixel offset, i.e., the lever arm effect. The weight of the squared distance can quantify this effect, allowing the distortion changes of pixels farther from the principal point to have a more reasonable weight in the calculation. For example, edge pixels have a higher weight than center pixels, reflecting the geometric characteristics of the image. This invention calculates the spatial gradient of the incremental displacement field within the workpiece pixel region to obtain the local rate of change of displacement. A symmetric strain rate tensor field is constructed using a fixed time interval, eliminating irrelevant rotational components and retaining only short-term dynamic changes in window distortion. The strain rate tensor field is weighted and integrated using the squared distance between the camera's principal point and the lever arm as weights, conforming to the imaging characteristics of the camera's central projection. Area normalization is then applied to obtain the strain rate moment characterizing the overall distortion of the workpiece region. The maximum eigenvalue of the strain rate moment is extracted, reducing the multidimensional distortion information to a single-valued pose-sensitive scalar, thus quantifying the impact of dynamic distortion changes on pose. The symmetric tensor field allows the distortion representation to more closely resemble realistic small deformation characteristics; the lever arm weights implement differentiated weighting at different pixel positions, conforming to imaging geometry; the weighted integration and normalization integrate the overall distortion information of the workpiece region, making the results comparable; and the extraction of the maximum eigenvalue reduces the dimensionality of multidimensional information, providing a concise and effective quantitative indicator for subsequent pose error modulation.

[0034] In one embodiment of the present invention, for any pixel coordinates within the first original image The distortion-free coordinates are calculated according to the following formula. : in To correct the distortion coordinates, It is a pixel coordinate vector. For absolute distortion displacement field, This is the matrix transpose. Pixel brightness value mapping is established based on the following formula to generate a distortion-free image. : in To correct the distortion in the image, The first original image, This represents the brightness value of the distorted image at the distorted coordinate position. This represents the brightness value of the first original image at its pixel coordinates.

[0035] It should be noted that the distortion-corrected coordinate mapping relationship is a one-to-one correspondence between the pixel coordinates of the first original image and the corresponding distortion-corrected coordinates, reflecting the spatial transformation law from the original distorted pixel position to the distortion-free pixel position. Distortion-corrected coordinates are the distortion-free pixel position information obtained by subtracting the corresponding absolute distortion displacement from the pixel coordinates of the first original image, reflecting the ideal coordinate position of the pixel in the distortion-free state of the viewport. The distortion-corrected image is the image obtained by performing inverse resampling on the first original image, eliminating the refraction distortion of the viewport, reflecting the true imaging characteristics of the processing component in the distortion-free state of the viewport. A bilinear interpolation algorithm is used to perform inverse resampling. The first step is to traverse all pixel coordinates of the distortion-corrected image; the second step is to find the corresponding position of that coordinate in the first original image based on the distortion-corrected coordinate mapping relationship; the third step is to directly extract the brightness value if the corresponding position is an integer pixel; the fourth step is to calculate the brightness value at that position using bilinear interpolation if the corresponding position is a non-integer pixel; the fifth step is to assign the calculated brightness value to the current pixel of the distortion-corrected image, completing the full image resampling. A boundary determination rule for the pixel range is set. If the x and y coordinates of the distortion-reduced image are less than 0 or greater than the maximum pixel coordinates of the original image, it is determined to be out of range. For pixels out of range, edge pixel brightness values ​​are used for filling. The edge pixel brightness values ​​of the corresponding directions in the original image are extracted and assigned to the distortion-reduced image coordinates to ensure the integrity of the distortion-reduced image and that no pixel information is missing. In addition, the pixel resolution of the distortion-reduced image is kept consistent with the first original image, and no scaling is performed to ensure that the spatial scale of the image remains unchanged.

[0036] It should be noted that this invention utilizes the continuous displacement features of the entire image from the absolute distortion displacement field to perform point-by-point subtraction on the coordinates of each pixel in the first original image, obtaining the corresponding distortion-free coordinates and establishing a spatial mapping relationship between the original pixels and the undistorted pixels. Through a reverse resampling algorithm, the pixel brightness values ​​of the original image are precisely allocated to the distortion-free coordinate positions according to this mapping relationship, compensating for brightness loss due to coordinate mismatch, and ultimately generating a distortion-free image that eliminates window refraction distortion, restoring the true imaging features of the processing component. Specifically, point-by-point calculation of the distortion-free coordinates achieves distortion correction for all pixels in the image, with no local correction blind spots; reverse resampling avoids pixel holes, ensuring image continuity; bilinear interpolation makes the brightness value mapping more accurate, preserving the imaging quality of the original image; edge filling processing ensures the integrity of the distortion-free image; and the resolution and format are consistent with the original image, requiring no additional conversion for direct use in subsequent feature extraction, providing a distortion-free image foundation for the accurate detection of the reference hole center coordinates, thus meeting the actual needs of industrial visual feature detection.

[0037] In one embodiment of the present invention, a distortion-corrected image is received. Identify the edge pixels of each reference hole to obtain the set of coordinates of the hole edge points. ,in The coordinates of the point on the edge of the hole; Based on the set of coordinates of the hole edge points, the following circle fitting equation is constructed: The circle fitting equation is then transformed into a system of linear equations as follows: Where the matrix Parameter vector to be determined and vectors Defined as: Solve for the least squares solution according to the following formula and calculate the coordinates of the center point of the reference hole of the machining component. : in To correct the distortion in the image, For the first Coordinates of the edge points of each hole The total number of points on the edge of the hole. These are the parameters for circle fitting. This is a coefficient matrix composed of the coordinate components of the hole edge points. Let be the vector of parameters to be determined, composed of the circle fitting parameters. Let the observation vector be the sum of the squares of the coordinates of the points at the edge of the borehole. The horizontal component of the reference hole center. The vertical component of the reference hole center. The coordinates of the center point of the reference hole for machining the component. For matrix transpose, The matrix inversion operation is used to calculate the inverse of a matrix in order to solve a system of linear equations. The vertical ellipsis symbol is used to indicate that matrix elements extend vertically according to a fixed pattern.

[0038] It should be noted that the set of hole edge point coordinates is the set of coordinates of all effective pixels at the edge of the reference hole in the distorted image, reflecting the pixel spatial distribution characteristics of the circular contour of the reference hole. The circle fitting equation is a mathematical equation representing the circular contour constructed based on the set of hole edge point coordinates, reflecting the correlation between the reference hole edge point coordinates and the circular geometric features. The linear equation system is a linear form of the circle fitting equation obtained through mathematical transformation, reflecting the linear correlation between the circle fitting parameters and the hole edge point coordinates. The circle fitting parameters are parameter values ​​representing the geometric features of the fitted circle obtained from the least squares solution of the linear equation system, reflecting the center and radius characteristics of the fitted circle. The center point coordinates of the reference hole of the machining component are the pixel coordinates of the reference hole center derived using the circle fitting parameters, reflecting the actual pixel position of the reference hole of the machining component in the image.

[0039] It should be noted that the specific method for determining the edge pixel position of the reference hole in the distorted image is as follows: First, the distorted image is converted to grayscale and Gaussian filtered with a 3×3 filter kernel to reduce image noise; then, the Cannibal edge detection algorithm is used, with a low threshold of 50 and a high threshold of 150, to detect all edge pixels in the image; finally, based on the preset pixel range of the reference hole, edge pixels within this range are selected and determined as the edge pixel positions of the reference hole. The rules for eliminating false edge points and selecting valid points in the hole edge point coordinate set are as follows: First, the distance from all edge points to the preset reference hole center is calculated, and a distance threshold of 0.3 to 1.2 times the preset hole diameter pixel value is set, eliminating false edge points that exceed this range; then, roundness detection is used to calculate the roundness of the local area where each edge point is located, eliminating points with a roundness less than 0.8; finally, the retained points constitute the valid hole edge point coordinate set, with no less than 30 valid points. When the pixel aperture of the reference aperture is 50 to 100 pixels, the number of edge points extracted is 30 to 50; when the pixel aperture is 100 to 200 pixels, the number of edge points extracted is 50 to 80; when the pixel aperture is greater than 200 pixels, the number of edge points extracted is 80 to 120. The basis for these values ​​is to match the number of edge points with the aperture, which ensures the fitting accuracy while avoiding the computational efficiency loss caused by too many points.

[0040] It should be noted that this invention extracts the edge pixel coordinates of the reference hole from the distortion-free image after distortion correction, constructs a set of effective edge point coordinates, and eliminates false edge points to ensure data validity. Based on circular geometric features, a circular fitting equation is constructed, which is then transformed into a system of linear equations to simplify the solution process. The optimal circular fitting parameters are obtained through the least squares solution, suppressing deviations caused by image noise. The center point coordinates of the reference hole are directly derived using the fitting parameters, achieving sub-pixel level precision control and obtaining the true pixel position of the reference hole in the image. Specifically, the Cannibal edge detection combined with screening rules ensures the validity and integrity of edge points; the circular fitting equation adapts to the circular contour of the reference hole, conforming to its physical characteristics; the linear transformation simplifies the calculation process and improves solution efficiency; and the least squares solution effectively suppresses the influence of image noise, improving fitting accuracy. Further details are omitted here.

[0041] In one embodiment of the invention, a strain rate tensor field is received. coordinates of the center point The strain rate intensity at the center point coordinates is calculated according to the following formula. : Receive camera master point The pose sensitivity risk is calculated according to the following formula. : Further construct the perspective multi-point positioning solution weights based on the following formula. : in is the Frobenius norm of the matrix, used to measure the overall energy intensity of the tensor matrix. The squared Euclidean distance is used to calculate the squared spatial distance from the center point to the camera's principal point. It is a very small positive number; Receive camera intrinsic parameter matrix Solve for the rotation matrix according to the following formula. With translation vector : By rotation matrix With translation vector Construct the pose of the machining component in the camera coordinate system And combine the transformation relationship between the camera coordinate system and the machine tool coordinate system The pose of the machining component in the machine tool coordinate system is obtained according to the following formula. : in The objective function is to minimize the variables used to find the attitude parameters that minimize the sum of squared weighted projection errors. For a three-dimensional rotational group space, For perspective projection operators, This is the three-dimensional coordinate vector of the center of the reference hole of the machining component in the coordinate system of the machining component; nominal workpiece pose in the receiver machine tool coordinate system The translation amount is obtained according to the following formula. With error rotation matrix : The rotation amount is obtained according to the following formula. : in For the error rotation matrix, the first... Line number The numerical components of the column, This is a matrix inverse operation used to convert the nominal pose matrix into an inverse transformation matrix; Receive the pose-sensitive scalar PS-NSRM and generate a continuous scaling factor according to the following formula. : The modulated guiding quantity is generated according to the following formula. and And output: in It is a dimensionless shrinkage coefficient used to dynamically adjust the magnitude of location compensation based on environmental risks.

[0042] It should be noted that the perspective multi-point positioning solution weights are coefficient values ​​calculated by combining the strain rate tensor field and the coordinates of the reference hole center points, reflecting the weight ratio characteristics of different reference hole center points in the positioning solution of the machining component. The rotation matrix is ​​a three-dimensional square matrix representing the rotational attitude of the machining component in the camera coordinate system, reflecting the angular deflection characteristics of the machining component relative to the camera coordinate system. The translation vector is a three-dimensional vector representing the positional offset of the machining component in the camera coordinate system, reflecting the spatial positional offset characteristics of the machining component relative to the camera coordinate system. The machining component pose in the camera coordinate system is an information body composed of the rotation matrix and translation vector, reflecting the actual spatial attitude and position state of the machining component in the camera coordinate system. The machining component pose in the machine tool coordinate system is the pose information after transformation by the camera and machine tool coordinate system transformation relationship, reflecting the actual spatial state of the machining component in the machine tool coordinate system. The nominal workpiece pose in the machine tool coordinate system is the preset standard spatial attitude and position information of the machining component in the machine tool coordinate system, reflecting the ideal positioning state of the machining component. The initial pose error transformation matrix is ​​a homogeneous transformation matrix constructed from the deviation between the actual and nominal poses of the machining component, reflecting the overall deviation characteristics of the actual pose relative to the nominal pose. The initial pose error is the translation and rotation deviation information extracted from the initial pose error transformation matrix, reflecting the degree of positioning deviation of the machining component in the machine coordinate system. The translation amount is a three-dimensional value representing the spatial position deviation in the initial pose error, reflecting the positional offset of the machining component along the X, Y, and Z axes of the machine coordinate system. The rotation amount is a three-dimensional value representing the angular attitude deviation in the initial pose error, reflecting the angular deflection of the machining component along the X, Y, and Z axes of the machine coordinate system. The continuous scaling factor is a linear modulation coefficient calculated using the pose-sensitive scalar, reflecting the proportional characteristics of the pose error modulation caused by dynamic changes in window distortion. The guidance amount is the pose correction value obtained after modulation by the continuous scaling factor, reflecting the specific correction amount by which the CNC system guides the positioning of the machining component.

[0043] It should be noted that the strain rate tensor field characteristics at different reference hole center points are different, corresponding to different degrees of influence of the positioning error caused by window distortion. Combining these two factors allows for the quantification of the positioning reliability of each center point. Points with high reliability are assigned high weights, and points with low reliability are assigned low weights, making the positioning solution more closely reflect the actual distortion effect. The pose-sensitive scalar is a single-valued quantification index of the influence of dynamic changes in window distortion on pose. Its value is positively correlated with the degree of distortion influence. Based on this, a continuous scaling factor is generated, which allows the modulation ratio of pose error to dynamically change with the degree of distortion influence, achieving linear correction to adapt to the distortion state. The specific method for obtaining the nominal pose of the workpiece in the machine tool coordinate system can be directly retrieved from the machining process file of the machine tool CNC system. The standard position and orientation data of the design reference of the machining component in the machine tool coordinate system are pre-entered in the process file, and this data is the nominal pose of the workpiece. In addition, the guidance quantity adopts a six-dimensional numerical format of three-dimensional translation and three-dimensional rotation in the machine tool coordinate system, with the numerical units being millimeters and degrees, respectively; the transmission method adopts the MODBUSTCP protocol of industrial Ethernet, with the vision system acting as a client to send guidance quantity data to the CNC system server, and the data frame format is a standard industrial control frame, with each frame containing a check bit to ensure the accuracy of transmission.

[0044] It should be noted that this invention combines the strain rate tensor field and the coordinates of the reference hole center point to construct a perspective multi-point positioning solution weight, allowing the positioning solution to adapt to the degree of distortion influence at each point. Then, the weighted solution is used to solve the pose of the machining component in the camera coordinate system, and the pose is transformed to the machine tool coordinate system through coordinate system transformation. The initial pose error is then calculated by comparing it with the nominal pose in the machine tool coordinate system. A continuous scaling factor is generated using the pose-sensitive scalar to linearly modulate the initial pose error, generating a guiding quantity adapted to the distortion state and transmitting it to the CNC system in a standard format, providing a quantitative basis for positioning guidance. The weighted solution reduces the interference of low-precision distortion points on positioning, improving the rationality of the pose solution; the coordinate system transformation enables parameter exchange between visual inspection and machine tool control, allowing pose data to directly serve machine tool motion; the continuous scaling factor allows pose error correction to dynamically adjust with distortion influence, adapting to the dynamic distortion characteristics of the viewport; the standard format and reliable transmission of the guiding quantity ensure that the CNC system can directly receive and execute it, making the positioning guidance of the machining component conform to the actual control requirements of industrial processing.

[0045] In one embodiment of the present invention, such as Figure 2 As shown, a method for positioning and guiding processing components based on industrial vision includes the following steps: Step S201: Establish the transformation relationship between the camera coordinate system and the machine tool coordinate system, and obtain the reference pixel coordinates of the window micro-marker; Step S202: Clamp the machining component and, while the machine tool is stationary, continuously acquire the first and second original images at fixed time intervals; Step S203: Detect the current pixel coordinates of the window micro-markers in the first original image and the second original image, calculate the difference between the current pixel coordinates and the reference pixel coordinates to obtain the absolute distortion displacement, and calculate the time increment displacement generated within a fixed time interval. Step S204: The absolute distortion displacement and time increment displacement are extended into an absolute distortion displacement field and an increment displacement field, respectively, using an interpolation algorithm. Step S205: Calculate the spatial gradient of the incremental displacement field to construct the strain rate tensor field, combine it with the camera principal point lever arm to calculate the strain rate moment and extract the maximum eigenvalue as the pose-sensitive scalar. Step S206: Reverse resampling of the first original image using the absolute distortion displacement field to obtain the distortion-free image; Step S207: Extract the center point coordinates of the reference hole of the machining component from the distorted image; Step S208: Construct perspective multi-point positioning solution weights based on strain rate tensor field and center point coordinates, solve initial pose error by combining transformation relationship, generate continuous scaling factor using pose sensitive scalar, apply continuous scaling factor to modulate initial pose error to generate guiding quantity and output to CNC system.

[0046] It should be noted that the hardware deployment of this invention involves fixing an industrial camera to an unobstructed position outside the machine tool's viewing window, with the camera's optical axis perpendicular to the machine tool's worktable. The camera connects to the machine tool's CNC system and industrial computer via an industrial Ethernet network. A calibration fixture is fixed to the machine tool's worktable reference position. The machine tool is equipped with vibration sensors and clamping force sensors, which communicate with the industrial computer. The industrial computer has the algorithm program of this invention built-in. Once the hardware network is complete, the invention can be used. In actual use, the initial calibration process is executed first. The industrial computer controls the camera to acquire multiple calibration pattern images, extracts feature points, and solves for the camera's intrinsic parameter matrix and principal point coordinates. Then, the calibration fixture is fixed to a known position in the machine tool's coordinate system, images of the calibration fixture are acquired, and the pixel coordinates of feature points are extracted. The homogeneous transformation matrix between the camera and machine tool coordinate systems is solved. Finally, in a reference state where the viewing window is free of attachments and the installation is without offset, a reference image is acquired, and micro-markers on the viewing window are identified. The reference pixel coordinates of the micro-markers are calculated, and the calibration parameters are stored in the industrial computer for direct retrieval in subsequent use. After calibration, the machining component positioning and guidance process begins. After the machining component is clamped, the sensor determines that the machine tool has reached a stationary state. The industrial control computer triggers the camera to continuously acquire two frames of original images at a set fixed time interval. The algorithm program automatically executes subsequent calculations: detects the current pixel coordinates of the micro-marker, calculates the absolute distortion displacement and time increment displacement, constructs the full-image displacement field through thin plate spline interpolation, solves the strain rate tensor field and pose-sensitive scalar, extracts the coordinates of the reference hole center point after distortion removal of the first original image, constructs perspective multi-point positioning solution weights and solves the pose of the machining component, obtains the initial pose error by comparing with the nominal pose, generates the guiding quantity after modulation by the pose-sensitive scalar, and finally outputs it to the machine tool CNC system by the industrial control computer.

[0047] It should be noted that the final output of this invention is a six-dimensional guiding quantity in the machine tool coordinate system, including three-dimensional translation and three-dimensional rotation, with numerical units of millimeters and degrees, respectively, transmitted in a standard industrial control frame format. For example, for a precision bearing machining component for automobiles, which has four reference holes, after the entire process calculation of this invention system, the output guiding quantities are: translation 0.018 mm on the X-axis, 0.022 mm on the Y-axis, and -0.009 mm on the Z-axis; and rotation 0.025 degrees on the X-axis, -0.018 degrees on the Y-axis, and 0.03 degrees on the Z-axis. After receiving these guiding quantities, the CNC system automatically drives the motion axes to complete the pose correction, and machining can begin after correction. The overall deployment of this invention does not require modification of the original machine tool structure. During use, except for the initial calibration and clamping of the machining component, all other steps are executed automatically, and the output results are directly adapted to the motion control requirements of the machine tool CNC system.

[0048] It should be noted that the interval and threshold sizes are set for ease of comparison. The size of the threshold depends on the amount of sample data and the base number set by those skilled in the art for each set of sample data, as long as it does not affect the proportional relationship between the parameter and the quantized value. Furthermore, the above formulas are all dimensionless calculations, and the formulas are derived from software simulations using a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0049] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A processing component positioning and guidance system based on industrial vision, characterized in that, include: The reference coordinate acquisition module establishes the transformation relationship between the camera coordinate system and the machine tool coordinate system, and obtains the reference pixel coordinates of the window micro-markers; The original image acquisition module, with the processing components clamped in and the machine tool stationary, continuously acquires the first and second original images at fixed time intervals; The distortion displacement calculation module detects the current pixel coordinates of the window micro-markers in the first and second original images, calculates the difference between the current pixel coordinates and the reference pixel coordinates to obtain the absolute distortion displacement, and calculates the time increment displacement generated within a fixed time interval. The displacement field extension module uses an interpolation algorithm to extend the absolute distortion displacement and the time increment displacement into an absolute distortion displacement field and an increment displacement field, respectively. The pose-sensitive scalar calculation module calculates the spatial gradient of the incremental displacement field to construct a strain rate tensor field, and calculates the strain rate moment by combining the camera principal point lever arm and extracts the maximum eigenvalue as the pose-sensitive scalar. The distortion-reduced image calculation module uses the absolute distortion displacement field to reverse resample the first original image to obtain the distortion-reduced image; The center point extraction module extracts the center point coordinates of the reference hole of the machining component from the distortion-free image; The guidance quantity generation module constructs perspective multi-point positioning solution weights based on the strain rate tensor field and the center point coordinates, solves the initial pose error by combining the transformation relationship, generates a continuous scaling factor using the pose-sensitive scalar, modulates the initial pose error using the continuous scaling factor to generate the guidance quantity, and outputs it to the CNC system.

2. The industrial vision-based processing component positioning and guidance system according to claim 1, characterized in that, Establish the transformation relationship between the camera coordinate system and the machine tool coordinate system, and obtain the reference pixel coordinates of the viewport micro-markers, including: Multiple images containing calibration patterns are acquired and the pixel coordinates of feature points of the calibration patterns are extracted. The camera intrinsic parameter matrix and the camera principal point pixel coordinate vector are calculated by minimizing the reprojection error. A calibration fixture with fixed three-dimensional feature point coordinates in the machine tool coordinate system is used to acquire an image of the calibration fixture and extract the pixel coordinates corresponding to the three-dimensional feature points of the calibration fixture. The homogeneous transformation matrix from the machine tool coordinate system to the camera coordinate system is solved, and the inverse of the homogeneous transformation matrix from the machine tool coordinate system to the camera coordinate system is obtained to generate the homogeneous transformation relationship from the camera coordinate system to the machine tool coordinate system. A reference image is acquired when the window is in reference state. The set of pixels corresponding to each window micro-marker in the reference image is identified. The reference pixel coordinates of the window micro-marker are determined by using the pixel brightness weighted centroid calculation method.

3. The industrial vision-based processing component positioning and guidance system according to claim 1, characterized in that, With the machining components clamped and the machine tool stationary, first and second raw images are continuously acquired at fixed time intervals, including: Clamp the machining components and keep the CNC machine tool stationary. Set the acquisition time of the first original image and trigger the industrial camera to acquire the first original image at the acquisition time of the first original image. Set the acquisition time of the second original image so that the acquisition time of the second original image is equal to the acquisition time of the first original image plus a fixed time interval, and trigger the industrial camera to acquire the second original image at the acquisition time of the second original image.

4. The industrial vision-based processing component positioning and guidance system according to claim 1, characterized in that, The current pixel coordinates of the viewport micro-markers in the first and second original images are detected. The difference between the current pixel coordinates and the reference pixel coordinates is calculated to obtain the absolute distortion displacement. The time increment displacement generated within a fixed time interval is also calculated, including: A pixel set is determined with the reference pixel coordinates of the window micro-marker as the center, and the current pixel coordinates of the window micro-marker in the first original image are obtained by calculating the weighted centroid of the pixel brightness in the pixel set within the first original image. In the second original image, perform pixel brightness weighted centroid calculation on the same set of pixels to obtain the current pixel coordinates of the window micro-marker in the second original image; The absolute distortion displacement is obtained by subtracting the reference pixel coordinates of the window micromark from the current pixel coordinates of the window micromark in the first original image, and the time increment displacement is obtained by subtracting the current pixel coordinates of the window micromark in the first original image from the current pixel coordinates of the window micromark in the second original image.

5. The industrial vision-based processing component positioning and guidance system according to claim 1, characterized in that, The absolute distortion displacement and time increment displacement are extended into an absolute distortion displacement field and an increment displacement field, respectively, using an interpolation algorithm, including: Using the current pixel coordinates of the window micromark in the first original image as the interpolation control point, and the absolute distortion displacement as the target displacement at the interpolation control point, the calculation formula for the absolute distortion displacement component corresponding to any pixel coordinate is established through the thin plate spline interpolation function, and the absolute distortion displacement field coefficient is determined by using the interpolation constraint conditions. Using the current pixel coordinates of the window micromark in the first original image as the interpolation control point and the time increment displacement as the target displacement at the interpolation control point, the incremental displacement component calculation formula corresponding to any pixel coordinate is established through the thin plate spline interpolation function, and the incremental displacement field coefficient is determined by using the interpolation constraint conditions.

6. The industrial vision-based processing component positioning and guidance system according to claim 1, characterized in that, The spatial gradient of the incremental displacement field is calculated to construct a strain rate tensor field. The strain rate moment is then calculated using the camera principal point lever arm, and the maximum eigenvalue is extracted as a pose-sensitive scalar, including: Within the pixel region of the workpiece, the spatial gradient matrix of the incremental displacement field is obtained, and the local rate of change of the incremental displacement field components with respect to the pixel coordinate components is calculated. A symmetric strain rate tensor field that varies with pixel coordinates is constructed using a fixed time interval and a spatial gradient matrix. The strain rate moment is obtained by combining the weight function value formed by the squared distance from the pixel coordinates to the camera principal point within the pixel region of the workpiece with the calculation of the pixel coordinates of the camera principal point, performing weighted integration on the strain rate tensor field and normalizing it with the area metric of the pixel region of the workpiece. Extract the maximum eigenvalue of the strain rate moment to determine the pose-sensitive scalar.

7. The industrial vision-based processing component positioning and guidance system according to claim 1, characterized in that, The distortion-free image is obtained by inverse resampling the first original image using the absolute distortion displacement field, including: Obtain the first original image and the absolute distortion displacement field. Calculate the dedistorted coordinates by subtracting the absolute distortion displacement value at the pixel coordinate position from the pixel coordinates. Based on the correspondence between distortion correction coordinates and pixel coordinates, the pixel brightness value at the pixel coordinate position of the first original image is assigned to the pixel at the distortion correction coordinate position to generate a distortion correction image.

8. The industrial vision-based processing component positioning and guidance system according to claim 1, characterized in that, Extract the center point coordinates of the reference hole of the machining component from the distorted image, including: In the distorted image, the edge pixel position is determined for each reference hole, and a set consisting of the coordinates of multiple hole edge points is obtained; A circular fitting equation is constructed using a set of hole edge point coordinates. The circular fitting equation is then transformed into a system of linear equations. The least squares solution of the system of linear equations is used to obtain the circular fitting parameters. The horizontal center component and the vertical center component are calculated using the circular fitting parameters, and then combined to obtain the center point coordinates of the reference hole of the machining component.

9. The industrial vision-based processing component positioning and guidance system according to claim 1, characterized in that, Based on the strain rate tensor field and the center point coordinates, a perspective multi-point positioning solution weight is constructed. The initial pose error is solved by combining the transformation relationship. A continuous scaling factor is generated using the pose-sensitive scalar. This continuous scaling factor is applied to modulate the initial pose error to generate a guiding quantity, which is then output to the CNC system. This includes: The strain rate tensor field and the coordinates of the center point are received. The strain rate intensity is calculated at the coordinates of the center point and mapped using the Frobenius norm. The pose-sensitive risk quantity is calculated by combining the camera principal point. The pose-sensitive risk quantity and the minimum positive number are used to construct the denominator to calculate the perspective multi-point positioning solution weight. By combining the camera intrinsic parameter matrix, using the center point coordinates as the observation input and the perspective multi-point positioning solution weight as the weighting coefficient, the rotation matrix and translation vector are solved to construct the pose of the machining component in the camera coordinate system. The pose of the machining component in the machine tool coordinate system is then calculated by combining the transformation relationship between the camera coordinate system and the machine tool coordinate system. The initial pose error transformation matrix is ​​calculated by combining the nominal pose of the workpiece in the machine tool coordinate system. The translation amount in the initial pose error is extracted, and the vector representation of the rotation amount is extracted by using the element difference of the error rotation matrix. Receive the pose-sensitive scalar to calculate the continuous scaling factor, use the continuous scaling factor to linearly modulate the translation and rotation in the initial pose error, generate the modulated guide quantity and output it to the CNC system.

10. A method for positioning and guiding processing components based on industrial vision, characterized in that, Implementing a processing component positioning and guidance system based on industrial vision as described in any one of claims 1 to 9 includes the following steps: Step S201: Establish the transformation relationship between the camera coordinate system and the machine tool coordinate system, and obtain the reference pixel coordinates of the window micro-marker; Step S202: Clamp the machining component and, while the machine tool is stationary, continuously acquire the first and second original images at fixed time intervals; Step S203: Detect the current pixel coordinates of the window micro-markers in the first original image and the second original image, calculate the difference between the current pixel coordinates and the reference pixel coordinates to obtain the absolute distortion displacement, and calculate the time increment displacement generated within a fixed time interval. Step S204: The absolute distortion displacement and time increment displacement are extended into an absolute distortion displacement field and an increment displacement field, respectively, using an interpolation algorithm. Step S205: Calculate the spatial gradient of the incremental displacement field to construct the strain rate tensor field, combine it with the camera principal point lever arm to calculate the strain rate moment and extract the maximum eigenvalue as the pose-sensitive scalar. Step S206: Reverse resampling of the first original image using the absolute distortion displacement field to obtain the distortion-free image; Step S207: Extract the center point coordinates of the reference hole of the machining component from the distorted image; Step S208: Construct perspective multi-point positioning solution weights based on strain rate tensor field and center point coordinates, solve initial pose error by combining transformation relationship, generate continuous scaling factor using pose sensitive scalar, apply continuous scaling factor to modulate initial pose error to generate guiding quantity and output to CNC system.

Images