Micro-cutting point-by-point visual closed-loop dynamic correction control method and device

Abstract

The application discloses a kind of micro-cutting point-by-point visual closed-loop dynamic deviation correction control method and device, comprising: calibration stage, mobile execution end is moved to multiple preset position points, pixel coordinates are collected and synchronous record mechanical pulse coordinates, obtain affine transformation matrix;Preparation stage, according to target cutting area to determine the pixel coordinates of each cutting point to be cut and the pixel coordinates of each cutting point to be cut, corresponding mechanical pulse coordinates are generated by affine transformation matrix mapping;Execution stage, execution end moves according to the mechanical pulse coordinates of the cutting point, and the positioning of the cutting point is completed by iterative compensation through visual feedback;Then according to the mechanical pulse coordinates sequence of each cutting point to be cut, point by point is pushed forward, each point executes coarse positioning, visual measurement, deviation compensation and precision check, until all cutting points to be cut are positioned.The application independently executes visual measurement and pulse compensation at each target point, cuts off the transmission path of error in time sequence, eliminates the influence of return gap, and ensures that each target point of the whole cutting trajectory has consistent positioning accuracy.

Description

Technical Field

[0001] This invention belongs to the field of mechanical motion and micro-cutting technology, specifically relating to a point-by-point visual closed-loop dynamic correction control method and device for micro-cutting. Background Technology

[0002] In precision micro-cutting operations, accurately mapping the target position in an image to the mechanical execution coordinate system is the core foundation for achieving high-precision control. However, in practical engineering applications, the calibration process is affected by the coupling of various complex factors, and the system generally suffers from the following problems: The backlash of the lead screw in the mechanical transmission system, the friction difference of the guide rail, and the slight deformation of the needle tip during the movement process cause positional deviations when the needle tip approaches the same target point from different directions. This manifests as inconsistent positioning accuracy in both forward and reverse directions, directly affecting the closure and contour accuracy of the cutting trajectory.

[0003] In existing technologies, such as the micro-cutting motion control method based on adaptive calibration of the execution end disclosed in CN122346001A, the pixel deviation vector between the execution end and the target point in the real-time image is converted into mechanical drive pulse increments through a mapping matrix and superimposed on the current mechanical absolute coordinates to generate absolute execution commands. However, the deviation compensation in the above scheme relies on the accumulation of previous execution results. When error sources such as mechanical backlash and control delay are propagated point by point in time, the positioning accuracy of subsequent target points will inevitably be affected by the previous errors and decrease. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a point-by-point visual closed-loop dynamic deviation correction control method and device for micro-cutting.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: In a first aspect, the present invention discloses a point-by-point visual closed-loop dynamic deviation correction control method for micro-cutting, comprising: a calibration stage, a preparation stage and an execution stage; The calibration phase includes: Step A1: Move the micro-cutting end of the microscope to multiple preset position points, acquire images at each position point and identify the pixel coordinates of the end of the microscope, and simultaneously record the corresponding mechanical pulse coordinates to construct calibration data pairs; Step A2: Based on the calibration data pairs of multiple preset position points, establish the affine transformation mapping relationship between the image pixel coordinate system and the mechanical pulse coordinate system to obtain the affine transformation matrix; The preparatory stage includes: Step B1: Determine the pixel coordinates of the micro-cutting entry point based on the target cutting area, and generate a sequence of pixel coordinates of each point to be cut on the contour of the target area according to the preset cutting parameters; Step B2: Based on the affine transformation matrix obtained in the calibration stage, map the pixel coordinates of the cutting point and the pixel coordinate sequence of each cutting point to the mechanical pulse coordinate system to generate the mechanical pulse coordinates of the cutting point and the mechanical pulse coordinate sequence of each cutting point. The execution phase includes: Step E1: The cutting task is started, and the execution end moves to the cutting point according to the mechanical pulse coordinates of the cutting point; After locking in place, identify the current pixel coordinates of the execution end and calculate the pixel deviation vector between the current pixel coordinates of the execution end and the pixel coordinates of the cut-in point; Determine if the magnitude of the pixel deviation vector is less than a preset allowable threshold: If so, the entry point location is complete, proceed to step E2; Otherwise, the linear part of the affine transformation matrix obtained in the calibration stage maps the pixel deviation vector to a pulse compensation amount. The execution end moves towards the entry point according to the pulse compensation amount. After reaching the entry point, the pulse compensation amount mapping, movement compensation, pixel coordinate recognition, and pixel deviation vector calculation are repeated until the magnitude of the pixel deviation vector is determined to be less than the preset allowable threshold. The entry point positioning is then completed, and the process proceeds to step E2. Step E2: Starting from the currently located point, move towards the current target point to be cut according to the mechanical pulse coordinate sequence of each point to be cut. Calculate the pixel distance vector between the current position of the execution end and the current target point to be cut in real time, and convert it into the pulse remaining feed amount through the linear part of the affine transformation matrix obtained in the calibration stage. When the magnitude of the remaining pulse feed is less than the preset reserved margin threshold, the movement stops and proceeds to step E3. Step E3: Identify the current pixel coordinates of the execution end, calculate the pixel deviation vector between the current position of the execution end and the current target to be cut point, and convert it into a pulse compensation amount through the linear part of the affine transformation matrix obtained in the calibration stage; The execution end moves towards the current target cutting point according to the pulse compensation amount; After the movement is completed, the current pixel coordinates of the execution end are identified again, the pixel deviation vector between the current position of the execution end and the current target cutting point is calculated, and it is determined whether the magnitude of the pixel deviation vector is less than a preset allowable threshold. If so, the current cutting point has been located. Otherwise, repeat the pulse compensation mapping, motion compensation, pixel coordinate recognition, and pixel deviation vector calculation until the magnitude of the pixel deviation vector is less than the preset allowable threshold, and complete the positioning of the current cutting point. Step E4: According to the mechanical pulse coordinate sequence of each point to be cut, select the next point to be cut as the new current target point to be cut, and repeat steps E2 to E3 until all points to be cut are located.

[0006] Based on the above technical solution, the following improvements can be made: As a preferred embodiment, the multiple preset location points in step A1 include the four corner points, the midpoints of the four sides, and the center point of the rectangular area distributed within the field of view, and the number of multiple preset location points is not less than nine.

[0007] As a preferred embodiment, step A2 includes: Step A2.1: Construct an absolute calibration dataset or a relative calibration dataset based on calibration data pairs from multiple preset location points; The absolute calibration dataset consists of the set of pixel coordinates at the end of each preset position and the corresponding set of mechanical pulse coordinates. The relative calibration dataset is: with the reference origin as a reference, the pixel coordinates and corresponding mechanical pulse coordinates of each preset position point at the end of the execution are converted into offsets relative to the reference origin, forming a set of relative pixel coordinate points and a set of relative pulse coordinate points. Step A2.2: Based on the absolute calibration dataset or the relative calibration dataset, establish the overdetermined system of equations, as follows: ; in: For the first The pixel coordinates of the end of the execution at each preset position point; For the first Mechanical pulse coordinates of a preset position point; These are the linear parameters of the affine transformation matrix; and These are the translation parameters of the affine transformation matrix; Solve the least squares solution of the overdetermined system of equations to obtain the parameters of the affine transformation matrix.

[0008] As a preferred option, if the execution end needs to be replaced, the calibration stage after step A2 also includes the following: Select any position point as the reference point, and obtain the pixel coordinates of the old execution end of the reference point and the corresponding mechanical pulse coordinates; After the execution end is replaced, the new execution end moves to the reference point according to the mechanical pulse coordinates of the reference point, and the pixel coordinates of the new execution end are identified; Calculate the pixel offset vector between the pixel coordinates of the new execution end and the pixel coordinates of the old execution end. The pixel deviation vector is transformed by the linear part of the affine transformation matrix. Mapped to pulse compensation quantity ; Subtract the pulse compensation amount from the translation vector formed by the translation parameters of the affine transformation matrix. This yields a new translation vector; Based on the new translation vector, update the translation parameters of the affine transformation matrix.

[0009] As a preferred embodiment, the calibration stage, after step A2, further includes performing end-shape error compensation, specifically including: The execution end is regarded as a geometric model, and several simulation parameters for the trajectory of the execution end are preset. The simulation parameters include one or more of the shape feature parameters and motion posture parameters used to characterize the geometric model. Adjust the simulation parameters and simulate the motion trajectory of the execution end under different simulation parameters in the visualization interface or simulation environment to make the simulation trajectory fit the contour of the target cutting area. When the deviation between the simulated trajectory and the target cutting contour is minimized, the current simulation parameters are the optimal compensation parameters for the end-shape error. Based on the optimal compensation parameters, determine the offset between the visual detection point at the execution end and the actual contact point in the image pixel coordinate system. The offset is obtained by using the linear part of the affine transformation matrix. Mapped to pulse compensation quantity ; Subtract the pulse compensation amount from the translation vector formed by the translation parameters of the affine transformation matrix. This yields a new translation vector; Based on the new translation vector, update the translation parameters of the affine transformation matrix.

[0010] As a preferred option, when shape error compensation has been performed during the calibration phase... The offset in the image pixel coordinate system determined in the shape error compensation step during the preparatory stage is based on the amount of the offset. The pixel coordinates of the entry point and each point to be cut are corrected for closed-loop target to generate closed-loop target pixel coordinates. During the execution phase, the target pixel coordinates of the closed loop are used as the target reference.

[0011] As a preferred approach, after the cutting is completed, one or more of the following error quantification indicators are used to evaluate the cutting results; Error quantification metrics include: cutting area, GT area, miscut area, IoU, Dice coefficient, centroid offset distance, centroid offset in the X direction, and centroid offset in the Y direction.

[0012] In a second aspect, the present invention discloses a micro-cutting point-by-point visual closed-loop dynamic deviation correction control device, comprising: a calibration module, a preparation module and an execution module; The calibration module includes: The data pair construction unit is used to move the micro-cutting execution end to multiple preset position points, acquire images at each position point and identify the pixel coordinates of the execution end, and synchronously record the corresponding mechanical pulse coordinates to construct calibration data pairs; The matrix acquisition unit is used to establish an affine transformation mapping relationship between the image pixel coordinate system and the mechanical pulse coordinate system based on calibration data pairs of multiple preset position points, and to obtain the affine transformation matrix. The preparatory modules include: The pixel sequence generation unit is used to determine the pixel coordinates of the micro-cutting entry point based on the target cutting area, and generate a pixel coordinate sequence of each point to be cut on the contour of the target area according to the preset cutting parameters. The mechanical sequence generation unit is used to map the pixel coordinates of the cutting point and the pixel coordinate sequence of each cutting point to the mechanical pulse coordinate system based on the affine transformation matrix obtained by the calibration module, and generate the mechanical pulse coordinates of the cutting point and the mechanical pulse coordinate sequence of each cutting point. The execution module includes: The entry point positioning unit is used to initiate the cutting task, and the execution end moves to the entry point according to the mechanical pulse coordinates of the entry point; After locking in place, identify the current pixel coordinates of the execution end and calculate the pixel deviation vector between the current pixel coordinates of the execution end and the pixel coordinates of the cut-in point; Determine if the magnitude of the pixel deviation vector is less than a preset allowable threshold: If so, the entry point positioning is completed, triggering the cutting point positioning unit; Otherwise, the linear part of the affine transformation matrix obtained by the calibration module maps the pixel deviation vector to a pulse compensation amount. The execution end moves towards the cutting point according to the pulse compensation amount. After reaching the position, the pulse compensation amount mapping, movement compensation, pixel coordinate recognition, and pixel deviation vector calculation are repeated until the magnitude of the pixel deviation vector is less than the preset allowable threshold. The cutting point positioning is then completed, and the cutting point positioning unit is triggered. The cutting point positioning unit is used to move from the currently positioned point to the current target cutting point according to the mechanical pulse coordinate sequence of each cutting point. It calculates the pixel distance vector between the current position of the execution end and the current target cutting point in real time, and converts it into the pulse remaining feed amount through the linear part of the affine transformation matrix obtained by the calibration module. When the magnitude of the remaining pulse feed is less than the preset reserved margin threshold, the movement stops, the current pixel coordinates of the execution end are identified, the pixel deviation vector between the current position of the execution end and the current target to be cut point is calculated, and it is converted into pulse compensation amount through the linear part of the affine transformation matrix obtained by the calibration module. The execution end moves towards the current target cutting point according to the pulse compensation amount; After the movement is completed, the current pixel coordinates of the execution end are identified again, the pixel deviation vector between the current position of the execution end and the current target cutting point is calculated, and it is determined whether the magnitude of the pixel deviation vector is less than a preset allowable threshold. If so, the current cutting point has been located. Otherwise, repeat the pulse compensation mapping, motion compensation, pixel coordinate recognition, and pixel deviation vector calculation until the magnitude of the pixel deviation vector is less than the preset allowable threshold, and complete the positioning of the current cutting point. The point-to-point loop unit is used to sequentially select the next point to be cut as the new current target point to be cut according to the mechanical pulse coordinate sequence of each point to be cut, and repeatedly trigger the cutting point positioning unit to execute the positioning process until all points to be cut are positioned.

[0013] Thirdly, the present invention discloses a computing device, comprising: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors, and the one or more programs include instructions for any of the above-described micro-cutting point-by-point visual closed-loop dynamic correction control methods.

[0014] Fourthly, the present invention discloses a storage medium storing one or more computer-readable programs, the one or more programs including instructions adapted to be loaded by a memory and executed by any of the above-described micro-cutting point-by-point visual closed-loop dynamic deviation correction control methods.

[0015] This invention discloses a point-by-point visual closed-loop dynamic deviation correction control method and device for microscopic cutting, which has the following beneficial effects: First, by employing point-by-point independent visual closed-loop dynamic correction, the propagation path of temporal errors is severed, solving the core problem of existing technologies where errors accumulate step by step and the positioning accuracy of subsequent points continuously deteriorates. This invention independently completes image acquisition, pixel deviation vector calculation, affine matrix mapping compensation, and iterative convergence positioning at the entry point and each point to be cut. The correction process at each point is independent of the execution results of previous points. This effectively avoids the defects of traditional compensation schemes where errors such as mechanical backlash, control delay, and dynamic drift accumulate point by point along the cutting trajectory, ensuring, in principle, the consistency of positioning accuracy for all points to be cut along the entire trajectory.

[0016] Secondly, by employing a segmented feeding strategy that involves reserving a margin for stopping and using minute visual compensation, the positioning deviation caused by mechanical transmission backlash and differences in motion friction is effectively eliminated, significantly improving the accuracy of micro-cutting point positioning and trajectory shaping. This invention first completes long-distance feeding based on a mechanical pulse coordinate sequence. When the magnitude of the remaining pulse feed is less than a preset reserved margin threshold, the main feed stops, and minute deviation compensation feeding is completed solely through visual closed-loop control. The entire process uses a unidirectional approach positioning method, avoiding repetitive positioning errors from mechanical forward and reverse movements, and significantly improving the closure and contour fit of the cutting trajectory.

[0017] Third, based on the construction of affine transformation mapping relationships using multiple calibration points, and combined with a static error compensation mechanism, high-precision matching between the pixel coordinate system and the mechanical pulse coordinate system is achieved. This invention uses at least nine grid calibration points to construct a calibration dataset and solve the affine transformation matrix. Simultaneously, matrix translation parameters are updated in real time through needle tip shape error compensation and needle replacement translation compensation, effectively correcting static deviations caused by needle tip shape differences and end-point replacement, ensuring the accuracy and stability of the coordinate mapping. When needle tip shape error compensation is effective, the compensation amount is simultaneously applied to the target reference for closed-loop deviation calculation, ensuring coordination between closed-loop control and the compensation mechanism, and preventing compensation from being canceled out by the closed loop.

[0018] Fourth, a multi-dimensional quantitative evaluation index system is configured to achieve accurate, comprehensive, and quantifiable evaluation of the cutting and positioning effect. Through multiple indicators such as cutting area, IoU, Dice coefficient, and centroid offset distance, the positioning and cutting effect can be quantified from multiple dimensions such as regional overlap, contour matching, and spatial offset, which facilitates accuracy calibration, parameter iterative optimization, and operation effect verification. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart of a control method provided in an embodiment of the present invention.

[0021] Figure 2 The upper left reference point provided in the embodiments of the present invention Visual recognition image.

[0022] Figure 3 The upper midpoint provided in the embodiments of the present invention Visual recognition image.

[0023] Figure 4 The upper right point provided in the embodiment of the present invention Visual recognition image.

[0024] Figure 5 The left midpoint provided in the embodiment of the present invention Visual recognition image.

[0025] Figure 6 Midpoint provided in the embodiments of the present invention Visual recognition image.

[0026] Figure 7 The right midpoint provided in the embodiment of the present invention Visual recognition image.

[0027] Figure 8 The lower left point provided in the embodiment of the present invention Visual recognition image.

[0028] Figure 9 The lower midpoint provided in the embodiments of the present invention Visual recognition image.

[0029] Figure 10 The lower right point provided in the embodiment of the present invention Visual recognition image.

[0030] Figure 11 The image provided in this embodiment of the invention is before compensation for the needle tip shape error.

[0031] Figure 12 The image provided in this embodiment of the invention is after compensation for needle tip shape error.

[0032] Figure 13 An experimental diagram showing the calibration deviation of the absolute affine matrix provided in an embodiment of the present invention.

[0033] Figure 14 An experimental diagram showing the relative affine matrix calibration deviation provided in an embodiment of the present invention.

[0034] Figure 15 The experimental diagram shows the manual calibration deviation provided in the embodiment of the present invention.

[0035] Figure 16 The experimental diagram shows the four-point calibration deviation provided in the embodiment of the present invention. Detailed Implementation

[0036] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] The expression “includes” is an “open-ended” expression, which means that there is a corresponding component or step, and should not be interpreted as excluding additional components or steps.

[0039] In some embodiments, the present invention discloses a point-by-point visual closed-loop dynamic deviation correction control method for microscopic cutting, such as... Figure 1 As shown, it includes: calibration stage, preparation stage and execution stage, with the micro-cutting execution end being a needle tip.

[0040] The calibration phase includes: Step A1: Move the micro-cutting needle tip to multiple preset position points, acquire images at each position point and identify the pixel coordinates of the needle tip, and simultaneously record the corresponding mechanical pulse coordinates to construct calibration data pairs; Step A2: Based on the calibration data pairs of multiple preset position points, establish the affine transformation mapping relationship between the image pixel coordinate system and the mechanical pulse coordinate system to obtain the affine transformation matrix.

[0041] The preparatory stage includes: Step B1: Determine the pixel coordinates of the micro-cutting entry point based on the target cutting area, and generate a sequence of pixel coordinates of each point to be cut on the contour of the target area according to the preset cutting parameters; Step B2: Based on the affine transformation matrix obtained in the calibration stage, map the pixel coordinates of the cutting point and the pixel coordinate sequence of each cutting point to the mechanical pulse coordinate system to generate the mechanical pulse coordinates of the cutting point and the mechanical pulse coordinate sequence of each cutting point.

[0042] The execution phase includes: Step E1: The cutting task is started, and the needle tip moves to the cutting point according to the mechanical pulse coordinates of the cutting point; After locking in place, identify the current pixel coordinates of the needle tip and calculate the pixel deviation vector between the current pixel coordinates of the needle tip and the pixel coordinates of the cutting point; Determine if the magnitude of the pixel deviation vector is less than a preset allowable threshold: If so, the entry point location is complete, proceed to step E2; Otherwise, the linear part of the affine transformation matrix obtained in the calibration stage maps the pixel deviation vector to a pulse compensation amount. The needle tip is controlled to move towards the cutting point according to the pulse compensation amount. After it is in place, the pulse compensation amount mapping, movement compensation, pixel coordinate recognition, and pixel deviation vector calculation are repeated until it is determined that the magnitude of the pixel deviation vector is less than the preset allowable threshold. The cutting point positioning is completed, and the process proceeds to step E2. Step E2: Starting from the currently located point, move towards the current target point according to the mechanical pulse coordinate sequence of each point to be cut. Calculate the pixel distance vector between the current position of the needle tip and the current target point to be cut in real time, and convert it into the pulse remaining feed amount through the linear part of the affine transformation matrix obtained in the calibration stage. When the magnitude of the remaining pulse feed is less than the preset reserved margin threshold, the movement stops and proceeds to step E3. Step E3: Identify the current pixel coordinates of the needle tip, calculate the pixel deviation vector between the current position of the needle tip and the current target cutting point, and convert it into a pulse compensation amount through the linear part of the affine transformation matrix obtained in the calibration stage. The needle tip is moved toward the current target cutting point according to the pulse compensation amount; After the needle tip is moved into position, its current pixel coordinates are identified again. The pixel deviation vector between the current position of the needle tip and the current target cutting point is calculated, and it is determined whether the magnitude of the pixel deviation vector is less than a preset allowable threshold. If so, the current cutting point has been located. Otherwise, repeat the pulse compensation mapping, motion compensation, pixel coordinate recognition, and pixel deviation vector calculation until the magnitude of the pixel deviation vector is less than the preset allowable threshold, and complete the positioning of the current cutting point. Step E4: According to the mechanical pulse coordinate sequence of each point to be cut, select the next point to be cut as the new current target point to be cut, and repeat steps E2 to E3 until all points to be cut are located.

[0043] The following section will elaborate on each step of each of the above stages.

[0044] For step A1 of the calibration phase, the needle tip is moved to the full field of view of the camera, and the pixel coordinates of the needle tip are accurately identified by a deep learning model based on a convolutional neural network. The pixel coordinates of the needle tip are then collected at nine preset locations. Corresponding mechanical drive pulse coordinates , .

[0045] like Figures 2 to 10 As shown, in this embodiment, an absolute coordinate modeling method is used, and the coordinates of the nine preset location points are as follows: Top left reference point The mechanical system is reset to the field of view reference origin, and the pulse coordinates of the mechanical origin are recorded. The camera captures the initial image and identifies the initial pixel coordinates of the needle tip. , which serve as the baseline sampling points for the dataset.

[0046] upper midpoint Send a fixed positive reference pulse to the X-axis motor. The pulse count (determined by the step angle, transmission ratio, and scale conversion factor of the mechanical system to ensure the needle tip remains within the camera's field of view after movement) is applied. After the mechanical system locks in place, the camera captures a second image to identify the needle tip pixel coordinates. Corresponding mechanical pulse coordinates .

[0047] Top right corner Maintaining the current X-axis position, send the same positive reference pulse to the X-axis motor again. After waiting for the position to be reached, a third image is captured to identify the pixel coordinates of the needle tip. Corresponding mechanical pulse coordinates .

[0048] left midpoint : Control the X-axis motor to retract to the reference position (i.e., pulse is ) Send a fixed positive reference pulse to the Y-axis motor. After the pulse is applied and the position is reached, a fourth image is captured to identify the pixel coordinates of the needle tip. Corresponding mechanical pulse coordinates .

[0049] midpoint Keep the Y-axis pulse as Keeping unchanged, send a positive reference pulse to the X-axis motor. After waiting for the needle tip to be positioned, a fifth image is captured to identify the pixel coordinates. Corresponding mechanical pulse coordinates .

[0050] right center point Continue sending positive reference pulses to the X-axis motor. After waiting for the needle tip to be positioned, a sixth image was captured to identify the new needle tip coordinates. Corresponding mechanical pulse coordinates .

[0051] Bottom left Keep the X-axis retracted to the reference position, and send another positive reference pulse to the Y-axis motor. After waiting for the needle tip to be positioned, a seventh image is captured to identify the pixel coordinates of the needle tip. Corresponding mechanical pulse coordinates .

[0052] lower midpoint Keep the Y-axis pulse as Send a positive reference pulse to the X-axis motor After waiting in position, the eighth image is captured to identify the pixel coordinates of the needle tip. Corresponding mechanical pulse coordinates .

[0053] Bottom right corner Continue sending positive reference pulses to the X-axis motor. After waiting for the needle tip to be positioned, a ninth image is captured to identify the pixel coordinates of the needle tip. Corresponding mechanical pulse coordinates .

[0054] The images captured after the above process and the results of needle tip recognition are as follows: Figures 2-10 As shown ( Figures 2-10 (The images and coordinate recognition results of the needle tip at nine calibration points are shown respectively).

[0055] By associating the pixel coordinates of the above nine locations with their corresponding mechanical pulse coordinates, an absolute calibration dataset for matrix solving is constructed, where the pixel coordinate set is defined as follows: The set of mechanical pulse coordinate points is defined as follows: .

[0056] In this embodiment, ; .

[0057] In another embodiment, a relative coordinate modeling strategy can also be used to construct a relative calibration dataset. Unlike absolute coordinate modeling, the relative coordinate method uses a fixed reference point within the calibration area (the upper left corner of the calibration grid in this embodiment). Using the reference point as the origin of the coordinate system, the coordinates of all calibration points are converted into offsets relative to this origin, thus establishing a mapping relationship of "pixel offset → pulse offset". Its core advantage is that the origin position can be stored and updated independently. When the needle tip is replaced or the mechanical reference is slightly offset, only the origin coordinates need to be updated without resolving the linear part of the affine transformation matrix, which significantly improves the maintainability of the system.

[0058] Let the coordinates of the selected reference point in the image pixel coordinate system be... The coordinates in the mechanical pulse coordinate system are The pixel coordinates of the nine calibration points collected above... With mechanical pulse coordinates Subtracting the origin coordinates from each, we obtain the relative coordinates: ; ; in: .

[0059] After the origin is zeroed out, the reference point The relative coordinates are always and In this embodiment, the reference origin is taken as the first calibration point. The coordinates, that is: ; ; Construct a relative calibration dataset, where the set of relative pixel coordinate points is defined as follows: The set of mechanical pulse coordinate points is defined as follows: .

[0060] In this embodiment, ; .

[0061] For step A2 in the calibration phase, an affine transformation model is used to describe the mapping relationship from the image pixel plane to the mechanical pulse plane. As a six-parameter linear transformation model, the affine transformation can simultaneously characterize the scaling, rotation, translation, and shearing transformations of image coordinates within a unified framework, making it particularly suitable for describing visual guidance and positioning scenarios with planar constraints.

[0062] The mapping equation is expressed as: ; Right now ; in, For the first The pixel coordinates of the end of the execution at each preset position point; These are the corresponding mechanical pulse coordinates.

[0063] The linear part of the affine transformation matrix consists of matrix elements. and The main diagonal item is responsible for independent scaling of each axis. and These are cross-coupling terms that jointly compensate for planar rotation and inter-axis non-orthogonality deviations. and These are the translation parameters of the affine transformation matrix.

[0064] An overdetermined linear system of equations is constructed based on an absolutely calibrated dataset. Singular value decomposition (SVD) is used to solve the least-squares solution of the overdetermined system of equations. The SVD method exhibits excellent numerical stability, ensuring robustness of the solution even when the nine location points show a certain degree of collinearity.

[0065] Finally, the six parameters obtained from the solution are reorganized into an affine transformation matrix of homogeneous form: .

[0066] This invention employs a nine-point calibration strategy, adding the midpoints of the four sides and the center point to a 2×2 rectangular grid, forming a 3×3 grid sampling layout covering the entire field of view. The nine location points simultaneously constitute an overdetermined system of 18 equations. After averaging the overdetermined system of equations, the contribution weight of the detection bias of a single location point to the final affine transformation matrix parameters is significantly reduced. The affine parameters are solved using least-squares fitting, and the parameter estimation results achieve statistical optimality across the entire field of view, thus ensuring stable mapping accuracy even when the needle tip moves to the center region of the field of view.

[0067] In some other embodiments, if the needle tip needs to be replaced, the calibration stage also includes a replacement compensation step after step A2.

[0068] In practical applications, needle tips are consumables and need to be replaced periodically. After replacement, the new needle tip's visual position in the image pixel coordinate system will shift due to installation position deviations or changes in its geometry, causing the translation parameters of the original absolute affine transformation matrix to become inapplicable. While re-executing the calibration process of steps A1-A2 can achieve accurate mapping, it is cumbersome, time-consuming, and impacts on-site operational efficiency.

[0069] Since tip replacement only changes the absolute position of the tool tip in image space, the scale, rotation, and inter-axis coupling relationships between image pixel coordinates and mechanical pulse coordinates (i.e., the linear part of the affine transformation matrix) remain unchanged. The matrix remains unchanged. Therefore, only the translation terms of the matrix need to be updated. This allows for the rapid correction of mapping relationships. The specific implementation steps are as follows: The first step is to select any point as a reference point and obtain the pixel coordinates of the needle tip at the reference point before replacement, as well as the corresponding mechanical pulse coordinates. After the initial calibration, record the original pixel coordinates of the needle tip in the image pixel coordinate system. This point is typically the reference point in the absolute calibration dataset. .

[0070] The second step involves replacing the needle tip, controlling the new tip to move to the reference point according to the mechanical pulse coordinates of the reference point, and identifying the pixel coordinates of the new tip at the reference point. .

[0071] The third step is to calculate the pixel deviation vector between the pixel coordinates of the new tip and the pixel coordinates of the tip before replacement: .

[0072] The fourth step is to transform the pixel deviation vector using the linear part of the affine transformation matrix. Mapping to pulse space yields the pulse compensation amount. .

[0073] ; in, It is a linear transformation submatrix of the affine transformation matrix.

[0074] This represents the equivalent offset of the new needle tip relative to the original needle tip in the mechanical pulse coordinate system.

[0075] The fifth step is to transform the translation vector in the original affine transformation matrix. Subtract the above pulse compensation amount This yields the translation vector of the new matrix: ; Right now: ; Finally, the updated affine transformation matrix is: .

[0076] The above compensation method works only if the pixel offset before and after tip replacement is small (usually on the order of tens of pixels), making the local linear approximation effective. If the tip deviation after replacement is too large (e.g., exceeding the range of the rectangular calibration coverage area), the scale factor of the linear part may have a non-negligible error due to the change in the field of view position. In this case, it is recommended to re-execute the complete calibration process.

[0077] Finally, the obtained affine transformation matrix The data is persistently stored in the system configuration module for use in the subsequent preparation and execution phases.

[0078] In some other embodiments, the calibration stage further includes a tip shape error compensation step after step A2.

[0079] In practical vision-guided positioning systems, the pixel coordinates of the needle tip output by the deep learning model typically correspond to a specific feature point (such as the tip tip itself) in the image, rather than the physical point of contact between the needle tip and the target plane. Due to geometric errors in the needle tip's manufacturing process (such as tip radius, axial skew, and asymmetrical cutting edges), and potential slight eccentricity or tilt during installation, a fixed systematic deviation occurs between the visual detection point and the actual contact point in the image pixel coordinate system. If this deviation is used directly for pulse coordinate mapping without compensation, it will lead to a systematic decrease in positioning accuracy.

[0080] This invention simulates trajectory offset under different needle tip shape characteristics by adjusting motion trajectory parameters, thereby observing the influence of needle tip shape error on the cutting path at a visual level and determining the optimal compensation parameters accordingly.

[0081] The specific method is as follows: The first step is to treat the needle tip as a geometric model and preset several simulation parameters for the needle tip trajectory. The simulation parameters include one or more of the shape feature parameters and motion posture parameters used to characterize the geometric model.

[0082] In a preferred embodiment, the geometric model is a rectangular surface model, and the simulation parameters include the rectangle width. Rectangle height Rotation angle Linear interpolation step size Among them, the width of the rectangle Rectangle height Simulate the equivalent dimensions of the needle tip in the direction of motion and its normal direction, and the rotation angle respectively. Characterizing the deflection attitude of the needle tip, linear interpolation step size Control the interpolation density of trajectory points.

[0083] By adjusting the combination of the above parameters, the trajectory offset characteristics under different needle tip geometries can be reproduced at the visualization level.

[0084] In other embodiments, the geometric model is a triangle or other polygonal model.

[0085] The second step involves the operator adjusting the simulation parameters ( , , , The simulation simulates the movement trajectory of the needle tip under different simulation parameters, so that the simulated trajectory fits the contour of the target cutting area.

[0086] When the deviation between the simulated trajectory and the contour of the target cutting area is minimized, the current simulation parameters are the optimal compensation parameters for the needle tip shape error.

[0087] This process is accomplished through visual verification or quantitative comparison of the deviation distance between the trajectory and the target cutting area contour.

[0088] The third step is to determine the offset between the visual detection point of the needle tip and the actual contact point in the image pixel coordinate system based on the optimal compensation parameters.

[0089] Treating the needle tip as a rectangular surface model, its geometric center is the visual detection point, and the length of its longer side is... (Equivalent dimensions in the direction of motion), the shorter side dimension is (Equivalent dimension in the corresponding normal direction), deflection angle is .

[0090] When the contact point is the center of the front edge of the rectangle, its position vector relative to the center of the rectangle is along the long side of the rectangle: ; in: It represents the offset in the image pixel coordinate system from the center of the rectangle (i.e., the visual detection point) to the actual contact point (i.e., the cutting action point), in pixels.

[0091] The above formula converts the trajectory simulation parameters into offsets in the image pixel coordinate system. It requires no additional fitting or empirical parameter tuning, and has clear geometric interpretability and engineering operability.

[0092] The fourth step is to use the linear part of the affine transformation matrix to shift the offset. Mapping to the mechanical pulse coordinate system yields the pulse compensation amount. : ; in, It is a linear transformation submatrix of the affine transformation matrix.

[0093] The translation vector in the current affine transformation matrix (Without changing the needle tip) Subtract the pulse compensation amount : ; ; Right now: ; Finally, the updated affine transformation matrix is: .

[0094] The above compensation method transforms the influence of needle tip shape error into an intuitive trajectory offset through trajectory visualization. Operators can quickly determine the optimal compensation value by adjusting the simulated trajectory parameters without the need for complex mathematical modeling or additional hardware support.

[0095] Finally, the obtained affine transformation matrix The data is persistently stored in the system configuration module for use in the subsequent preparation and execution phases.

[0096] The compensation results of the above process for static correction of the needle tip are as follows: Figures 11-12 As shown ( Figures 11-12 (Comparison of cutting results before and after compensation) Error results of different calibration methods are as follows Figures 13-16 As shown.

[0097] The following preparatory and execution phases will uniformly use the affine transformation matrix determined in the calibration phase. This is used when no compensation is triggered. Needle replacement but without shape error compensation. Used after shape error compensation .

[0098] When both compensation methods are applied, the needle should be changed first. Then, after shape error compensation, it is obtained Ultimately As the standard.

[0099] The following example uses the needles without replacement, but with shape error compensation. Let's take the affine transformation matrix, which is finally determined in the calibration phase, as an example to elaborate on this.

[0100] in, ; Affine transformation matrix The linear transformation submatrix.

[0101] For step B1 in the preparatory stage, before starting the cutting task, the initial position information of the needle tip is obtained, and a complete cutting path sequence is generated based on the target area. This initialization process provides reference data for subsequent entry point positioning and point-by-point correction, and is a prerequisite for ensuring accurate positioning throughout the entire cutting process.

[0102] Specifically, by selecting the target cutting region S through the interactive interface, the pixel coordinates of the cutting point of that region are automatically extracted. It generates a sequence of cutting points on the target contour based on preset cutting process parameters. ,in Let be the pixel coordinates of the k-th point to be cut in the image pixel coordinate system.

[0103] For step B2 in the preparatory stage, based on the affine transformation matrix... , the pixel coordinates of the cutting point and the pixel coordinate sequence of each cutting point Mapping to the mechanical pulse coordinate system generates the mechanical pulse coordinates of the entry point. and the mechanical pulse coordinate sequence of each cutting point The mechanical pulse coordinates corresponding to the k-th cutting point .

[0104] If tip shape error compensation has been performed during the calibration phase, then the offset in the image pixel coordinate system determined in the shape error compensation step is used. Closed-loop target correction is performed on the pixel coordinates of the cutting point and the pixel coordinates of each point to be cut, generating closed-loop target pixel coordinates: ; ; in, These are the pixel coordinates of the cut point after offset correction; These are the pixel coordinates of the k-th cut point after offset correction.

[0105] The minus sign in the above formula This means that the closed-loop target is offset and corrected along the direction from the visual detection point to the actual contact point, so that the closed-loop control can position the detection point to... When positioning, the actual contact point falls exactly on the original target. Place.

[0106] When tip shape error compensation is not performed during the calibration phase, the closed-loop target pixel coordinates are equal to the original target pixel coordinates, i.e. .

[0107] The closed-loop target pixel coordinates are persistently stored in the system configuration module and used for deviation calculation in execution stages E1 and E3.

[0108] For step E1 of the execution phase, the cutting task is initiated, and the needle tip follows the mechanical pulse coordinates of the entry point. Move to the entry point and perform the positioning operation.

[0109] After locking in place, identify the current pixel coordinates of the needle tip. And calculate the current pixel coordinates of the needle tip. Closed-loop target pixel coordinates of the entry point pixel deviation vector .

[0110] ; like If the magnitude is less than the preset allowable threshold (usually sub-pixel level, such as 0.5 pixels), then the positioning is determined to be complete, the needle tip has been precisely locked to the cutting point, and the process proceeds to step E2. Otherwise, the linear part of the affine transformation matrix obtained through the calibration phase pixel offset vector Mapped to pulse compensation quantity ; .

[0111] According to pulse compensation amount Control the needle tip to move towards the cutting point. Once in place, repeat the pulse compensation mapping, movement compensation, pixel coordinate recognition, and pixel deviation vector calculation until the magnitude of the pixel deviation vector is less than the preset allowable threshold. Then, complete the cutting point positioning and proceed to step E2.

[0112] For step E2 in the execution phase, after accurately locating the entry point, the current pixel coordinates of the needle tip are identified. And calculate the current pixel coordinates of the needle tip. The closed-loop target pixel coordinates of the first cutting point pixel distance vector between ; ; Through the linear part of the affine transformation matrix pixel distance vector Converted to pulse remaining feed This drives the needle tip to continue moving forward.

[0113] in, .

[0114] When the pulse remaining feed amount When the magnitude is less than the preset reserved margin threshold, stop moving and proceed to step E3.

[0115] The purpose of setting a reserved margin is twofold: firstly, to prevent the robotic arm from overshooting due to inertia or insufficient deceleration distance, causing the needle tip to exceed the target position; secondly, to reserve sufficient adjustment space for the subsequent visual fine-tuning stage, ensuring that the compensation motion during the fine-tuning stage is always in the positive feed direction, thereby eliminating the impact of return backlash on positioning accuracy.

[0116] Step E3 in the execution phase is the visual fine-tuning phase, which is the final correction step in the single-point positioning closed loop.

[0117] Once the needle tip stops, identify the current pixel coordinates of the needle tip. And calculate the current pixel coordinates of the needle tip. The closed-loop target pixel coordinates of the first cutting point pixel deviation vector ; ; Through the linear part of the affine transformation matrix pixel offset vector Converted to pulse compensation amount According to pulse compensation amount Control the needle tip to move towards the first point to be cut.

[0118] in, .

[0119] Unlike the entry point positioning stage, this pulse compensation amount Typically small and with limited mechanical movement, this avoids repetitive errors introduced by large-scale repositioning and effectively suppresses the impact of mechanical backlash on accuracy.

[0120] After the motion compensation is in place, the current pixel coordinates of the needle tip are identified again, and the current position of the needle tip and the closed-loop target pixel coordinates of the first cutting point are calculated. The pixel deviation vector between the two pixels is used to determine whether the magnitude of the pixel deviation vector is less than a preset allowable threshold. If so, the first cutting point has been located. Otherwise, repeat the above calculation and compensation process for iterative fine-tuning until the first cutting point is accurately positioned and the cutting is achieved.

[0121] After completing the precise positioning and cutting of the first cutting point, the mechanical pulse coordinate sequence of the cutting point is followed. It automatically advances to the next cutting point until all cutting points are positioned.

[0122] Specifically, for the k-th point to be cut... Perform the following sequence of operations: (1) After completing the precise positioning of the (k-1)th cutting point, calculate the current pixel coordinates of the needle tip and the closed-loop target pixel coordinates of the kth cutting point. The pixel distance vector between them is converted into the pulse residual feed amount through affine transformation, which drives the needle tip to move forward continuously.

[0123] (2) When the magnitude of the remaining pulse feed is less than the preset reserved margin threshold, the movement is stopped to avoid the influence of inertial overshoot and return gap.

[0124] (3) Enter the visual fine-tuning stage, identify the current pixel coordinates of the needle tip and the closed-loop target pixel coordinates of the k-th cutting point. The pixel deviation vector between the two is converted into a pulse compensation amount through affine transformation, and then a micro-feed is performed.

[0125] (4) Check whether the magnitude of the pixel deviation vector is less than the allowable threshold. If it is, the point positioning is completed; otherwise, continue iterative compensation until accurate positioning is achieved.

[0126] In other embodiments, after the cutting is completed, several error quantification indicators are used to evaluate the cutting results.

[0127] Error quantification metrics include: (1) Cutting area The area of ​​the closed region enclosed by the actual cutting trajectory of the pointer tip is the core geometric quantity for measuring the cutting execution result.

[0128] (2) GT area , refers to the area of ​​the target cutting region, which serves as a standard reference area, representing the ideal area to be cut, and is the benchmark for all error calculations.

[0129] (3) Area of ​​miscut ,Right now This indicator quantitatively describes the total deviation of the cutting result from the target cutting area. The larger the value, the worse the cutting accuracy. It includes the combined effects of missed cuts (undercutting) and overcutting (excessive cutting), and is an intuitive quantitative indicator for measuring the integrity of the cut.

[0130] (4) IoU is the ratio of the intersection to the union, i.e. The value range is [0,1]. IoU is the core evaluation index for measuring the degree of overlap between the cut region and the target cut region. The closer the value is to 1, the more accurate the cut is.

[0131] (5) Dice coefficient, The value range is [0,1]. The Dice coefficient is used to measure the degree of fit between the cutting area and the target cutting area. The closer the Dice is to 1, the higher the cutting quality.

[0132] (6) Centroid offset distance , refers to the Euclidean distance between the centroid of the cutting region and the centroid of the target cutting region. This indicator reflects the overall offset in spatial location, is unaffected by differences in area size, and is a key indicator for measuring the accuracy of needle tip positioning. It can provide quantitative information on positional deviation independently of area overlap.

[0133] (7) Centroid offset in the X direction offset of the centroid in the Y direction These are the projection differences between the centroid of the cut region and the centroid of the ground truth region in the X and Y directions of the image, respectively. Together, they describe the deviation components of the cut region relative to the target cut region in the horizontal and vertical directions.

[0134] This invention employs a deep learning model based on YOLOv8-pose to achieve needle tip target detection and key point localization. This model, built upon a convolutional neural network, can simultaneously output the detection bounding box of the needle tip and the coordinates of the tip's key points, providing accurate visual input for the calibration, preparation, and execution stages.

[0135] The training of a deep learning model specifically includes the following: First, a pinpoint detection dataset is established, and pinpoint detection boxes and keypoints in the images are manually labeled. Specifically, the dataset contains 504 images, divided into training, validation, and test sets in a 6:2:2 ratio. In some embodiments, the training set can be augmented to enhance the model's generalization ability and increase the number of samples.

[0136] Secondly, the model training is based on the YOLOv8-pose framework, using a keypoint detection task head for pinpoint localization. The loss function includes three components: bounding box regression loss, confidence loss, and keypoint regression loss, with the keypoint regression loss driving the model to learn the pinpoint position. During training, a validation set is used to monitor the model's convergence status, and the model weights with the lowest validation set loss are saved for subsequent testing.

[0137] Finally, the trained model is evaluated on the test set.

[0138] Furthermore, as new image data is continuously accumulated in actual cutting operations, newly added labeled samples can be continuously integrated into the training set. The model weights can be updated and iterated through retraining, so that the model can gradually adapt to the detection needs under different lighting conditions, different needle tip shapes, and different glass slide backgrounds.

[0139] To verify the effectiveness of the invention, experimental verification was conducted on a micro-dissection system.

[0140] Experiment 1: Comparison experiment before and after needle tip shape error compensation and correction.

[0141] like Figure 11 and Figure 12 The cut area is represented by a yellow outline, and its centroid is represented by a red pixel; the GT area is represented by a green outline, and its centroid is represented by a green pixel.

[0142] After compensation for the tip shape error, the centroid of the cutting area basically coincides with the centroid of the GT area, and the degree of fit between the cutting trajectory and the target contour is significantly improved.

[0143] Table 1 Comparison of Cutting Results

[0144] Table 1 further quantifies the comparison of cutting results before and after tip shape error compensation. The tip shape error compensation method proposed in this invention can effectively eliminate the fixed deviation between the visual detection point and the actual contact point, significantly improving cutting accuracy.

[0145] Experiment 2: Comparison Experiment of Different Calibration Methods To verify the superiority of the method of this invention, a comparative experiment was conducted between the absolute coordinate calibration method (9-point calibration + least squares solution of overdetermined equations), the relative coordinate calibration method (9-point calibration + least squares solution of overdetermined equations), the manual calibration method (operator manually selects 9 points for calibration), and the four-point calibration method (only selecting 4 points at the four corners of the field of view for calibration). All methods were performed on the same microdissection system under the same environmental conditions, with a target area of ​​10.02 mm². The experimental results are as follows: Figures 13 to 16 And as shown in Table 2.

[0146] Table 2 Comparison of different calibration methods

[0147] As shown in Table 2, the absolute coordinate calibration method proposed in this invention uses 9 calibration points to form an overdetermined system of 18 equations. After the detection deviation of a single calibration point is averaged by least squares fitting, the contribution weight of the deviation to the final matrix parameters is significantly reduced. The parameter estimation results achieve statistical optimality across the entire field of view. Therefore, its positioning accuracy is significantly better than the four-point calibration method using only 4 calibration points, the manual calibration method relying on operators to manually select points, and the relative coordinate calibration method using relative coordinate modeling.

[0148] In other embodiments, the present invention discloses a micro-cutting point-by-point visual closed-loop dynamic deviation correction control device, comprising: a calibration module, a preparation module, and an execution module; The calibration module includes: The data pair construction unit is used to move the micro-cutting execution end to multiple preset position points, acquire images at each position point and identify the pixel coordinates of the execution end, and synchronously record the corresponding mechanical pulse coordinates to construct calibration data pairs; The matrix acquisition unit is used to establish an affine transformation mapping relationship between the image pixel coordinate system and the mechanical pulse coordinate system based on calibration data pairs of multiple preset position points, and to obtain the affine transformation matrix. The preparatory modules include: The pixel sequence generation unit is used to determine the pixel coordinates of the micro-cutting entry point based on the target cutting area, and generate a pixel coordinate sequence of each point to be cut on the contour of the target area according to the preset cutting parameters. The mechanical sequence generation unit is used to map the pixel coordinates of the cutting point and the pixel coordinate sequence of each cutting point to the mechanical pulse coordinate system based on the affine transformation matrix obtained by the calibration module, and generate the mechanical pulse coordinates of the cutting point and the mechanical pulse coordinate sequence of each cutting point. The execution module includes: The entry point positioning unit is used to initiate the cutting task, and the execution end moves to the entry point according to the mechanical pulse coordinates of the entry point; After locking in place, identify the current pixel coordinates of the execution end and calculate the pixel deviation vector between the current pixel coordinates of the execution end and the pixel coordinates of the cut-in point; Determine if the magnitude of the pixel deviation vector is less than a preset allowable threshold: If so, the entry point positioning is completed, triggering the cutting point positioning unit; Otherwise, the linear part of the affine transformation matrix obtained by the calibration module maps the pixel deviation vector to a pulse compensation amount. The execution end moves towards the cutting point according to the pulse compensation amount. After reaching the cutting point, the pulse compensation amount mapping, movement compensation, pixel coordinate recognition, and pixel deviation vector calculation are repeated until it is determined that the pixel deviation vector is less than the preset allowable threshold. The cutting point positioning is then completed, and the cutting point positioning unit is triggered. The cutting point positioning unit is used to move from the currently positioned point to the current target cutting point according to the mechanical pulse coordinate sequence of each cutting point. It calculates the pixel distance vector between the current position of the execution end and the current target cutting point in real time, and converts it into the pulse remaining feed amount through the linear part of the affine transformation matrix obtained by the calibration module. When the magnitude of the remaining pulse feed is less than the preset reserved margin threshold, the movement stops, the current pixel coordinates of the execution end are identified, the pixel deviation vector between the current position of the execution end and the current target to be cut point is calculated, and it is converted into pulse compensation amount through the linear part of the affine transformation matrix obtained by the calibration module. The execution end moves towards the current target cutting point according to the pulse compensation amount; After the movement is completed, the current pixel coordinates of the execution end are identified again, the pixel deviation vector between the current position of the execution end and the current target cutting point is calculated, and it is determined whether the magnitude of the pixel deviation vector is less than a preset allowable threshold. If so, the current cutting point has been located. Otherwise, repeat the pulse compensation mapping, motion compensation, pixel coordinate recognition, and pixel deviation vector calculation until the pixel deviation vector is determined to be less than the preset allowable threshold, and complete the current cutting point positioning; The point-to-point loop unit is used to sequentially select the next point to be cut as the new current target point to be cut according to the mechanical pulse coordinate sequence of each point to be cut, and repeatedly trigger the cutting point positioning unit to execute the positioning process until all points to be cut are positioned.

[0149] Furthermore, it should be noted that the micro-cutting point-by-point visual closed-loop dynamic correction control device provided in the above embodiments is only illustrated by the division of the above functional modules when performing cutting control. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of a micro-cutting point-by-point visual closed-loop dynamic correction control device can be divided into different functional modules to complete all or part of the functions described above.

[0150] Furthermore, the embodiments of the micro-cutting point-by-point visual closed-loop dynamic correction control device and the micro-cutting point-by-point visual closed-loop dynamic correction control method provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0151] In other embodiments, the present invention discloses a computing device comprising: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors, and the one or more programs include instructions for any of the above-described micro-cutting point-by-point visual closed-loop dynamic correction control methods.

[0152] In other embodiments, the present invention discloses a storage medium storing one or more computer-readable programs, the programs including instructions adapted to be loaded by memory and executed by any of the above-described micro-cutting point-by-point visual closed-loop dynamic deviation correction control methods.

[0153] This invention discloses a point-by-point visual closed-loop dynamic deviation correction control method and device for microscopic cutting, which has the following beneficial effects: First, by employing point-by-point independent visual closed-loop dynamic correction, the propagation path of temporal errors is severed, solving the core problem of existing technologies where errors accumulate step by step and the positioning accuracy of subsequent points continuously deteriorates. This invention independently completes image acquisition, pixel deviation vector calculation, affine matrix mapping compensation, and iterative convergence positioning at the entry point and each point to be cut. The correction process at each point is independent of the execution results of previous points. This effectively avoids the defects of traditional compensation schemes where errors such as mechanical backlash, control delay, and dynamic drift accumulate point by point along the cutting trajectory, ensuring, in principle, the consistency of positioning accuracy for all points to be cut along the entire trajectory.

[0154] Secondly, by employing a segmented feeding strategy that involves reserving a margin for stopping and using minute visual compensation, the positioning deviation caused by mechanical transmission backlash and differences in motion friction is effectively eliminated, significantly improving the accuracy of micro-cutting point positioning and trajectory shaping. This invention first completes long-distance feeding based on a mechanical pulse coordinate sequence. When the magnitude of the remaining pulse feed is less than a preset reserved margin threshold, the main feed stops, and minute deviation compensation feeding is completed solely through visual closed-loop control. The entire process uses a unidirectional approach positioning method, avoiding repetitive positioning errors from mechanical forward and reverse movements, and significantly improving the closure and contour fit of the cutting trajectory.

[0155] Third, based on the construction of affine transformation mapping relationships using multiple calibration points, and combined with a static error compensation mechanism, high-precision matching between the pixel coordinate system and the mechanical pulse coordinate system is achieved. This invention uses at least nine grid calibration points to construct a calibration dataset and solve the affine transformation matrix. Simultaneously, matrix translation parameters are updated in real time through needle tip shape error compensation and needle replacement translation compensation, effectively correcting static deviations caused by needle tip shape differences and end-point replacement, ensuring the accuracy and stability of the coordinate mapping. When needle tip shape error compensation is effective, the compensation amount is simultaneously applied to the target reference for closed-loop deviation calculation, ensuring coordination between closed-loop control and the compensation mechanism, and preventing compensation from being canceled out by the closed loop.

[0156] Fourth, a multi-dimensional quantitative evaluation index system is configured to achieve accurate, comprehensive, and quantifiable evaluation of the cutting and positioning effect. Through multiple indicators such as cutting area, IoU, Dice coefficient, and centroid offset distance, the positioning and cutting effect can be quantified from multiple dimensions such as regional overlap, contour matching, and spatial offset, which facilitates accuracy calibration, parameter iterative optimization, and operation effect verification.

[0157] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope. All such changes and modifications fall within the scope of the present invention as claimed, which is defined by the appended claims and their equivalents.

Claims

1. A point-by-point visual closed-loop dynamic deviation correction control method for micro-cutting, characterized in that, include: The process can be divided into three phases: calibration, preparation, and execution. The calibration phase includes: Step A1: Move the micro-cutting end of the microscope to multiple preset position points, acquire images at each position point and identify the pixel coordinates of the end of the microscope, and simultaneously record the corresponding mechanical pulse coordinates to construct calibration data pairs; Step A2: Based on the calibration data pairs of multiple preset position points, establish the affine transformation mapping relationship between the image pixel coordinate system and the mechanical pulse coordinate system, and obtain the affine transformation matrix; The preparatory stage includes: Step B1: Determine the pixel coordinates of the micro-cutting entry point based on the target cutting area, and generate a sequence of pixel coordinates of each point to be cut on the contour of the target area according to the preset cutting parameters; Step B2: Based on the affine transformation matrix obtained in the calibration stage, map the pixel coordinates of the cutting point and the pixel coordinate sequence of each cutting point to the mechanical pulse coordinate system to generate the mechanical pulse coordinates of the cutting point and the mechanical pulse coordinate sequence of each cutting point. The execution phase includes: Step E1: The cutting task is started, and the execution end moves to the cutting point according to the mechanical pulse coordinates of the cutting point; After locking in place, identify the current pixel coordinates of the execution end and calculate the pixel deviation vector between the current pixel coordinates of the execution end and the pixel coordinates of the cut-in point; Determine whether the magnitude of the pixel deviation vector is less than a preset allowable threshold: If so, the entry point location is complete, proceed to step E2; Otherwise, the pixel deviation vector is mapped to a pulse compensation amount by the linear part of the affine transformation matrix obtained in the calibration stage. The execution end moves towards the entry point according to the pulse compensation amount. After reaching the entry point, the pulse compensation amount mapping, movement compensation, pixel coordinate recognition, and pixel deviation vector calculation are repeated until it is determined that the magnitude of the pixel deviation vector is less than the preset allowable threshold. The entry point positioning is completed, and the process proceeds to step E2. Step E2: Starting from the currently located point, move towards the current target point to be cut according to the mechanical pulse coordinate sequence of each point to be cut. Calculate the pixel distance vector between the current position of the execution end and the current target point to be cut in real time, and convert it into the pulse remaining feed amount through the linear part of the affine transformation matrix obtained in the calibration stage. When the magnitude of the remaining pulse feed is less than the preset reserved margin threshold, the movement stops and proceeds to step E3. Step E3: Identify the current pixel coordinates of the execution end, calculate the pixel deviation vector between the current position of the execution end and the current target to be cut point, and convert it into a pulse compensation amount through the linear part of the affine transformation matrix obtained in the calibration stage; The execution end is controlled to move towards the current target cutting point according to the pulse compensation amount; After the movement is completed, the current pixel coordinates of the execution end are identified again, the pixel deviation vector between the current position of the execution end and the current target cutting point is calculated, and it is determined whether the magnitude of the pixel deviation vector is less than a preset allowable threshold. If so, the current cutting point has been located. Otherwise, repeat the pulse compensation mapping, motion compensation, pixel coordinate recognition, and pixel deviation vector calculation until it is determined that the magnitude of the pixel deviation vector is less than a preset allowable threshold, and complete the current cutting point positioning. Step E4: According to the mechanical pulse coordinate sequence of each point to be cut, select the next point to be cut as the new current target point to be cut, and repeat steps E2 to E3 until all points to be cut are located.

2. The method according to claim 1, characterized in that, The plurality of preset location points in step A1 include the four corner points, the midpoints of the four sides, and the center point of the rectangular area distributed within the field of view, and the number of the plurality of preset location points is not less than nine.

3. The method according to claim 1, characterized in that, Step A2 includes: Step A2.1: Construct an absolute calibration dataset or a relative calibration dataset based on calibration data pairs from multiple preset location points; The absolute calibration dataset is: the set of pixel coordinates at the end of execution at each preset position point and the corresponding set of mechanical pulse coordinates; The relative calibration dataset is: with the reference origin as a reference, the pixel coordinates and corresponding mechanical pulse coordinates of the execution end of each preset position point are converted into offsets relative to the reference origin, forming a set of relative pixel coordinate points and a set of relative pulse coordinate points. Step A2.2: Based on the absolute calibration dataset or relative calibration dataset, establish an overdetermined system of equations, as follows: ； in: For the first The pixel coordinates of the end of the execution at each preset position point; For the first Mechanical pulse coordinates of a preset position point; These are the linear parameters of the affine transformation matrix; and These are the translation parameters of the affine transformation matrix; Solve the least squares solution of the overdetermined system of equations to obtain the parameters of the affine transformation matrix.

4. The method according to claim 1, characterized in that, If the execution terminal needs to be replaced, the calibration stage after step A2 also includes the following: Select any position point as a reference point, and obtain the pixel coordinates of the old execution end of the reference point and the corresponding mechanical pulse coordinates; After the execution end is replaced, the new execution end moves to the reference point according to the mechanical pulse coordinates of the reference point, and the pixel coordinates of the new execution end are identified; Calculate the pixel deviation vector between the pixel coordinates of the new execution end and the pixel coordinates of the old execution end. The pixel deviation vector is transformed by the linear part of the affine transformation matrix. Mapped to pulse compensation quantity ; Subtract the pulse compensation amount from the translation vector formed by the translation parameters of the affine transformation matrix. This yields a new translation vector; Based on the new translation vector, update the translation parameters of the affine transformation matrix.

5. The method according to claim 1, characterized in that, The calibration stage, following step A2, further includes performing end-shape error compensation, specifically including: The execution end is regarded as a geometric model, and several simulation parameters for the trajectory of the execution end are preset. The simulation parameters include one or more of the shape feature parameters and motion posture parameters used to characterize the geometric model. Adjust the simulation parameters to simulate the motion trajectory of the execution end under different simulation parameters in a visualization interface or simulation environment, so that the simulation trajectory fits the contour of the target cutting area. When the deviation between the simulated trajectory and the target cutting contour is minimized, the current simulation parameters are the optimal compensation parameters for the end-shape error. Based on the optimal compensation parameters, determine the offset between the visual detection point at the execution end and the actual contact point in the image pixel coordinate system. The offset is obtained through the linear part of the affine transformation matrix. Mapped to pulse compensation quantity ; Subtract the pulse compensation amount from the translation vector formed by the translation parameters of the affine transformation matrix. This yields a new translation vector; Based on the new translation vector, update the translation parameters of the affine transformation matrix.

6. The method according to claim 1, characterized in that, When the shape error compensation has been performed during the calibration phase The preparatory stage is based on the offset in the image pixel coordinate system determined in the shape error compensation step. The pixel coordinates of the cutting point and each cutting point are corrected for closed-loop target to generate closed-loop target pixel coordinates. The execution phase uses the closed-loop target pixel coordinates as the target reference.

7. The method according to claim 1, characterized in that, After the cutting is completed, one or more of the following error quantification indicators are used to evaluate the cutting results; The error quantification indicators include: cutting area, GT area, miscut area, IoU, Dice coefficient, centroid offset distance, centroid offset in the X direction, and centroid offset in the Y direction.

8. A micro-cutting point-by-point visual closed-loop dynamic deviation correction control device, characterized in that, include: The module consists of a calibration module, a preparation module, and an execution module. The calibration module includes: The data pair construction unit is used to move the micro-cutting execution end to multiple preset position points, acquire images at each position point and identify the pixel coordinates of the execution end, and synchronously record the corresponding mechanical pulse coordinates to construct calibration data pairs; The matrix acquisition unit is used to establish an affine transformation mapping relationship between the image pixel coordinate system and the mechanical pulse coordinate system based on calibration data pairs of multiple preset position points, and to obtain the affine transformation matrix. The preparatory module includes: The pixel sequence generation unit is used to determine the pixel coordinates of the micro-cutting entry point based on the target cutting area, and generate a pixel coordinate sequence of each point to be cut on the contour of the target area according to the preset cutting parameters. The mechanical sequence generation unit is used to map the pixel coordinates of the cutting point and the pixel coordinate sequence of each cutting point to the mechanical pulse coordinate system based on the affine transformation matrix obtained by the calibration module, and generate the mechanical pulse coordinates of the cutting point and the mechanical pulse coordinate sequence of each cutting point. The execution module includes: The entry point positioning unit is used to initiate the cutting task, and the execution end moves to the entry point according to the mechanical pulse coordinates of the entry point; After locking in place, identify the current pixel coordinates of the execution end and calculate the pixel deviation vector between the current pixel coordinates of the execution end and the pixel coordinates of the cut-in point; Determine whether the magnitude of the pixel deviation vector is less than a preset allowable threshold: If so, the entry point positioning is completed, triggering the cutting point positioning unit; Otherwise, the linear part of the affine transformation matrix obtained by the calibration module maps the pixel deviation vector to a pulse compensation amount. The execution end moves towards the cutting point according to the pulse compensation amount. After reaching the cutting point, the pulse compensation amount mapping, movement compensation, pixel coordinate recognition, and pixel deviation vector calculation are repeated until it is determined that the magnitude of the pixel deviation vector is less than a preset allowable threshold. The cutting point positioning is then completed, and the cutting point positioning unit is triggered. The cutting point positioning unit is used to move from the currently positioned point to the current target cutting point according to the mechanical pulse coordinate sequence of each cutting point, calculate the pixel distance vector between the current position of the execution end and the current target cutting point in real time, and convert it into the pulse remaining feed amount through the linear part of the affine transformation matrix obtained by the calibration module. When the magnitude of the remaining pulse feed is less than the preset reserved margin threshold, the movement stops, the current pixel coordinates of the execution end are identified, the pixel deviation vector between the current position of the execution end and the current target to be cut point is calculated, and it is converted into pulse compensation amount through the linear part of the affine transformation matrix obtained by the calibration module. The execution end is controlled to move towards the current target cutting point according to the pulse compensation amount; After the movement is completed, the current pixel coordinates of the execution end are identified again, the pixel deviation vector between the current position of the execution end and the current target cutting point is calculated, and it is determined whether the magnitude of the pixel deviation vector is less than a preset allowable threshold. If so, the current cutting point has been located. Otherwise, repeat the pulse compensation mapping, motion compensation, pixel coordinate recognition, and pixel deviation vector calculation until it is determined that the magnitude of the pixel deviation vector is less than a preset allowable threshold, and complete the current cutting point positioning. The point-to-point loop unit is used to sequentially select the next point to be cut as the new current target point to be cut according to the mechanical pulse coordinate sequence of each point to be cut, and repeatedly trigger the cutting point positioning unit to execute the positioning process until all points to be cut are positioned.

9. A computing device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include instructions for the micro-cutting point-by-point visual closed-loop dynamic correction control method according to any one of claims 1-7.

10. A storage medium, characterized in that, The storage medium stores one or more computer-readable programs, the programs including instructions adapted to be loaded by memory and executed as described in any of claims 1-7: the micro-cutting point-by-point visual closed-loop dynamic correction control method.

Images