Visual guidance track deviation correction system based on dynamic tracking

Abstract

The invention relates to the technical field of industrial automation and machine vision control, in particular to a visual guidance track deviation correction system based on dynamic tracking, which comprises a state cache module for acquiring a real-time encoder position, a real-time planning reference position and a system clock at a preset sampling rate and writing the positions into a state annular cache region; the visual resolving module is used for acquiring exposure instant image data and a latched global clock timestamp and resolving a target state quantity; the state backtracking module is used for extracting a historical encoder position and a historical planning reference position according to the timestamp; the error generation module verifies the target state quantity based on the historical planning reference position and generates an absolute visual error; and the feed-forward compensation module is used for generating an error compensation transition curve and superposing the error compensation transition curve to a bottom layer control instruction of the current control loop. According to the invention, a visual large time delay problem is converted into a historical state alignment problem, stable correction of real trajectory deviation is realized, and the influence of time-varying delay on control stability is reduced.

Description

Technical Field

[0001] This invention relates to the field of industrial automation and machine vision control technology, specifically to a vision-guided trajectory deviation correction system based on dynamic tracking. Background Technology

[0002] In existing vision-guided motion control systems, the control system typically includes a controlled actuator and a vision sensor. The controlled actuator moves continuously along a preset trajectory, and the vision sensor collects target markings, edge contours, or process features during the movement and uses the recognition results to correct trajectory deviations.

[0003] However, visual sensors typically consume time from exposure and image transmission to image processing, resulting in visual processing results that correspond to the spatial state at a certain point in the past. Existing solutions often directly compare or compensate for this visual result with the current position of the actuator, which can easily lead to misjudging the normal displacement of the actuator during the time delay as a trajectory deviation, causing problems such as time misalignment compensation, jitter correction, and decreased control stability.

[0004] In addition, existing solutions often lack consistency in backtracking of historical states and validity verification of visual results when using visual results for compensation. When there is reflection, occlusion, misidentification or unstable data updates in the image, it is easy to send the wrong target state quantity directly into the control loop, which leads to abrupt compensation changes and poor trajectory continuity, making it difficult to balance the correction accuracy and operational stability during dynamic tracking. Summary of the Invention

[0005] The purpose of this invention is to provide a vision-guided trajectory deviation correction system based on dynamic tracking, and to solve the following technical problems: It avoids trajectory misalignment and abnormal oscillation caused by directly using observation results with large visual time delay, and makes it easier to achieve spatiotemporal decoupling between the internal state of servo execution and external visual measurement, thereby completing the smoothing and stable correction of the deviation of the real trajectory.

[0006] The objective of this invention can be achieved through the following technical solutions: A vision-guided trajectory deviation correction system based on dynamic tracking is applied to control systems with communication connections between the controlled actuator and the vision sensor, including: The state buffer module is used to collect the real-time encoder position, real-time planning reference position and system clock of the controlled actuator at a preset sampling rate, and continuously push the real-time encoder position, the real-time planning reference position and the system clock into the state ring buffer. The visual processing module is used to acquire the image data generated by the visual sensor at the moment of exposure and the latched global clock timestamp, and to calculate the target state quantity of the controlled actuator with the global clock timestamp based on the image data. The state backtracking module is used to respond to the target state quantity output by the visual calculation module, and retrieve and extract the historical encoder position and historical planning reference position of the controlled actuator corresponding to the global clock timestamp in the state ring buffer according to the global clock timestamp. The error generation module is used to verify the validity of the target state quantity based on the historical planning reference position, and after the verification is passed, to subtract the target state quantity from the historical encoder position to generate an absolute visual error. The feedforward compensation module is used to generate an error compensation transition curve based on the absolute visual error, and to superimpose the error compensation transition curve with a preset outer loop visual compensation gain into the underlying control command of the current control loop of the controlled actuator.

[0007] Optionally, the state caching module is specifically used for: According to a preset clock cycle, the real-time encoder position, the real-time planning reference position, and the corresponding system clock are packaged into a status data frame; The state data frame is written into the state circular buffer allocated in the preset underlying random access memory according to the first-in-first-out principle.

[0008] Optionally, the visual processing module obtains the global clock timestamp in the following ways: The trigger clock of the vision sensor is synchronized with the system clock using a preset precise time protocol. At the physical instant when the visual sensor performs image exposure, the current system clock is latched by a preset hardware trigger as the global clock timestamp; The global clock timestamp is bound to the image data.

[0009] Optionally, the step of the state backtracking module extracting the historical encoder position includes: Determine whether the global clock timestamp precisely matches the system clock recorded in the state ring buffer; If the global clock timestamp precisely matches the system clock recorded in the state ring buffer, then the corresponding real-time encoder position is directly extracted as the historical encoder position. If the global clock timestamp does not precisely match the system clock recorded in the state ring buffer, then search in the state ring buffer for a first system clock and a second system clock that are earlier and later than the global clock timestamp, respectively. Extract the position of the first encoder corresponding to the first system clock and the position of the second encoder corresponding to the second system clock; The historical encoder position corresponding to the global clock timestamp is calculated by combining the first encoder position and the second encoder position using a linear interpolation algorithm.

[0010] Optionally, the process by which the error generation module generates the absolute visual error includes: Calculate the spatial positional deviation between the target state quantity and the historical planning reference position to obtain the state deviation degree; Determine whether the deviation of the state is greater than a preset error threshold; If the state deviation is less than or equal to the preset error threshold, the verification is deemed successful, and the target state quantity is subtracted from the historical encoder position to generate the absolute visual error. If the deviation of the state is greater than the preset error threshold, the verification is deemed to have failed, the target state quantity is discarded, and an out-of-bounds alarm signal is output.

[0011] Optionally, the feedforward compensation module includes: The curve generation unit is used to generate the error compensation transition curve based on a smooth curve through a preset kinematic forward model, using the absolute visual error as a state variable. The instruction overlay unit is used to extract the compensation increment corresponding to the error compensation transition curve within a preset continuous clock cycle. Combined with the outer ring visual compensation gain, the compensation increment is added to the underlying control command cycle by cycle.

[0012] Optionally, the error compensation transition curve is a smooth curve; The starting tangent direction of the smooth curve is consistent with the current motion direction of the controlled actuator; The endpoint of the smooth curve converges to a state where the absolute visual error is fully compensated.

[0013] Optional, also includes: A gain adjustment module is used to obtain the update frequency of the absolute visual error and determine whether the update frequency is higher than a preset frequency threshold. If the update frequency is higher than the preset frequency threshold, the outer ring visual compensation gain is set to a preset high gain value. If the update frequency is equal to the preset frequency threshold, then the current outer loop visual compensation gain is maintained. If the update frequency is lower than the preset frequency threshold, the outer ring visual compensation gain will be reduced to a preset low gain value.

[0014] Optionally, the controlled actuator is a general motion platform; The system's control architecture is divided into an instruction scheduling layer, a visual processing layer, and a server execution layer. The visual processing layer runs on a preset independent computing node, and the servo execution layer and the state ring buffer run on a preset underlying motion controller.

[0015] Optionally, after receiving the underlying control command superimposed with the error compensation transition curve, the controlled actuator performs a state correction action within the physical space. The vision sensor continuously captures the corrected physical space state and generates new target state quantities to form an asynchronous closed-loop control data stream with multiple sampling rates.

[0016] The beneficial effects of this invention are: 1. This invention uses state caching and historical backtracking to precisely align the target state quantity with the historical encoder position at the moment of exposure; this mechanism eliminates the time misalignment caused by visual processing delay, avoids misjudging the normal displacement of the actuator as trajectory deviation, and effectively improves the correction accuracy and control stability. 2. Before performing the difference processing, this invention uses the historical planning reference position of the same time section to verify the validity of the target state quantity; this mechanism can effectively isolate erroneous data such as reflection, occlusion or misidentification from entering the control loop, avoiding unreasonable reverse correction and trajectory deviation; 3. The feedforward compensation module of this invention transforms the absolute visual error into a smooth transition curve consistent with the current direction of motion, and superimposes it into the control command cycle by cycle; this avoids the instantaneous acceleration surge and mechanical jitter caused by the step injection of asynchronous visual error, and maintains the stability of operation. 4. This invention dynamically adjusts the outer loop visual compensation gain based on the update frequency of absolute visual error, using high gain when updates are frequent and reducing gain when updates are sparse; this effectively prevents instability caused by excessive correction when visual link fluctuations or frame drops occur, achieving dynamic balance. 5. When the timestamp does not precisely match the cache system clock, the system calculates the historical encoder position by searching for the clocks of two adjacent frames and using a linear interpolation algorithm. This establishes a fine-grained time bridge between high- and low-frequency modules, enabling high-precision reconstruction of the physical execution state. Attached Figure Description

[0017] The invention will now be further described with reference to the accompanying drawings; Figure 1 This is a schematic diagram of a visual guidance trajectory deviation correction system based on dynamic tracking provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1 A vision-guided trajectory deviation correction system based on dynamic tracking is applied to a control system with a controlled actuator and a vision sensor connected by communication. It includes a state buffer module, which is used to collect the real-time encoder position, real-time planning reference position and system clock of the controlled actuator at a preset sampling rate, and continuously push the real-time encoder position, the real-time planning reference position and the system clock into the state ring buffer. The visual processing module is used to acquire the image data generated by the visual sensor at the moment of exposure and the latched global clock timestamp, and to calculate the target state quantity of the controlled actuator with the global clock timestamp based on the image data. The state backtracking module is used to respond to the target state quantity output by the visual calculation module, and retrieve and extract the historical encoder position and historical planning reference position of the controlled actuator corresponding to the global clock timestamp in the state ring buffer according to the global clock timestamp. The error generation module is used to verify the validity of the target state quantity based on the historical planning reference position, and after the verification is passed, to subtract the target state quantity from the historical encoder position to generate an absolute visual error. The feedforward compensation module is used to generate an error compensation transition curve based on the absolute visual error, and to superimpose the error compensation transition curve with a preset outer loop visual compensation gain into the underlying control command of the current control loop of the controlled actuator.

[0020] This embodiment provides a vision-guided trajectory deviation correction mechanism based on dynamic tracking. Specifically, the system is deployed in a high-speed re-inspection station for semiconductor packaging substrates. The general-purpose multi-axis motion platform in the station moves continuously according to the pre-issued scanning trajectory, and the vision sensor above captures the alignment marks, edge contours, or process features on the substrate during the platform's movement. Due to the inherent time consumption of exposure, image transmission and image processing in the visual link, the target state quantity obtained by the visual processing layer at the current moment essentially reflects the spatial state of the controlled actuator at a certain exposure moment in the past, rather than the current state. To avoid directly using the location observation results corresponding to the exposure history as the basis for compensation at the current time, this embodiment completes spatiotemporal decoupling through a continuous pipeline of state caching, timestamp binding, historical backtracking, validity verification, and feedforward smooth compensation; The state buffer module continuously reads the real-time encoder position, real-time planning reference position and system clock from the underlying motion controller, and writes the three into the state ring buffer according to a unified sampling rhythm. Among them, the encoder position reflects the actual internal state of the platform in the high-frequency servo loop, the planned reference position reflects the target position expected by the control system at that moment, and the system clock serves as the common time reference for both states; the vision calculation module latches the global timestamp when the vision sensor is exposed, so that each frame of the image carries a clear generation time. The visual processing layer calculates the target state variables, such as the target edge center, marker coordinates, or target contour pose, based on the positional relationship of features in the image, and outputs them along with the exposure time. After obtaining the target state quantity, the state backtracking module does not directly compare it with the platform position at the current moment. Instead, it retrieves the historical encoder position and historical planning reference position at the corresponding moment in the state circular buffer based on the exposure moment. The technical effect of doing so is that the vision measures the platform state reflected by the external space at that historical moment, and the encoder backtracks to obtain the execution state recorded inside the platform at that historical moment. The two are on the same time section, which can eliminate the time misalignment caused by the continuous movement of the platform. The error generation module first verifies the validity of the target state quantity using the historical planning reference position. If the visual result deviates from the planned position at that time by more than the preset error threshold, it is more likely to be due to misidentification, occlusion, reflection, image contamination, or geometric consistency abnormalities, and should not be directly sent into the control loop. Only when the verification is passed will the target state quantity be subtracted from the historical encoder position to obtain the absolute visual error. This error represents the real spatial deviation relative to the actual execution state, rather than a pseudo deviation caused by visual delay.

[0021] Furthermore, to ensure that the historical planning reference position used for validity verification and the historical encoder position are at the same historical time segment, the state backtracking module preferably adopts a consistent time recovery strategy; specifically, it can be understood as: if the global clock timestamp corresponds precisely to the system clock of a certain state data frame, then the planning reference position in that state data frame is directly read as the historical planning reference position; If there is no precise correspondence, two adjacent complete state data frames around the timestamp are selected, and the historical planned reference position corresponding to the exposure time is restored according to the time ratio based on the planned reference position in the two frames. After this processing, the historical encoder position and the historical planned reference position both point to the same exposure time, and there will be no problem that the encoder position and the planned reference position correspond to different time slices. This allows the subsequent validity verification and error generation to be based on a unified historical time reference. The feedforward compensation module does not directly add the absolute visual error to the underlying control command in the form of abrupt changes. Instead, it generates an error compensation transition curve based on the absolute visual error and gradually adds it to the underlying control command of the current control loop in combination with the outer loop visual compensation gain. The physical reason for this is that the underlying servo loop usually has a high sampling rate and a narrow permissible disturbance window. If the low-frequency asynchronous visual result is injected in a step manner, it is easy to manifest as a sudden increase in acceleration, current fluctuation and structural jitter in the actuator. Smooth injection can make visual correction a kind of outer loop slow variable compensation that can be accepted by the servo system. In abnormal or boundary situations, if the vision calculation module fails to output the target state quantity with a valid timestamp, such as image frame loss, time synchronization failure, or feature not being successfully identified, the state backtracking module will not start historical retrieval, and the current control loop will continue to operate according to the original servo closed loop formed by the real-time planning reference position and encoder feedback. If there is no historical interval covering the timestamp in the state ring buffer, such as the buffer depth is insufficient to cover the vision calculation delay, the current vision result will be discarded, and a buffer overflow or delay over-limit prompt may be triggered. If the error generation module determines that the vision result is invalid, it will not output the absolute vision error, but only retain the underlying servo control, thereby avoiding error correction.

[0022] Furthermore, if the historical encoder position can be recovered but the historical planning reference position cannot be recovered due to missing items in the corresponding state frame, incomplete download of the trajectory segment, or the data in that interval being marked as invalid, the system will also terminate this visual correction to avoid directly using the target state quantity for compensation in the absence of a verification benchmark.

[0023] For example, in a high-speed re-inspection station for semiconductor packaging substrates, the platform continuously passes through multiple inspection areas along a serpentine scanning trajectory; the top-mounted camera captures the reference line at the edge of the substrate at a certain exposure moment, but the image processing requires a preset processing period (ms) to complete; if the system directly compares the image with the current platform position, it will misjudge the normal displacement of the platform during this period as a trajectory deviation, especially at high speeds, which will manifest as a misalignment exceeding the preset tolerance range; in this embodiment, the global clock is latched at the time of image exposure, and after the processing is completed, the system returns to the exposure moment to extract the encoder position and planned reference position of the platform at that time, and only performs error comparison at the same historical moment, and then smoothly superimposes the obtained true deviation into the current control command; It should be noted that, in order to avoid ambiguity caused by multiple names for the same technical object throughout the text, this embodiment further stipulates the following: the camera and top-mounted camera mentioned in this workstation scenario are specific implementation names of the vision sensor; the platform, motion platform, and general multi-axis motion platform mentioned are specific forms of the controlled actuator in this implementation scenario; the underlying motion controller mentioned is the underlying execution control node in the control system, and when it is not specifically distinguished from the controller, it does not change its technical meaning of undertaking servo execution and state cache writing; The planning reference position mentioned is an abbreviation for the real-time planning reference position or the historical planning reference position. If the statement involves both the current time and the exposure history time, it is distinguished by the prefix real-time or historical. If the statement only discusses a single time segment, the planning reference position refers to the reference position under that time segment by default. Through the above terminology conventions, the descriptions of vision sensors, controlled actuators, controllers and reference positions in the whole text can be understood as the same object, without changing the technical features defined by each embodiment.

[0024] The purpose of this step is to transform the problem of large visual time delay from a current state prediction problem into a historical state alignment problem, thereby achieving stable correction of the deviation of the true trajectory and reducing the damage of visual time-varying delay to phase margin and control stability.

[0025] In this embodiment of the invention, the state cache module is specifically used to: package the real-time encoder position, the real-time planning reference position and the corresponding system clock into a state data frame according to a preset clock cycle; and write the state data frame into the state ring cache area allocated in a preset underlying random access memory according to the first-in-first-out principle.

[0026] This embodiment provides a data organization mechanism for state caching. Specifically, in the aforementioned workstation, if the encoder position, planning reference position, and system clock are only stored separately without being solidified into a unified data frame, mismatches can easily occur during high-speed operation and concurrent multi-task reading, where the position comes from one control cycle and the clock comes from another control cycle. This mismatch may be below the system's allowable lower limit at low speeds, but it will be amplified to a time alignment error exceeding a preset threshold in high-speed trajectory segments. Therefore, this embodiment encapsulates the encoder position, planning reference position, and system clock collected in each sampling cycle into a complete state data frame and then writes it into the state circular buffer in the underlying random access memory.

[0027] The controller generates a frame of state data after each preset clock cycle. This can be understood as follows: at a certain sampling time T1, a data frame F1 is formed, which includes the encoder position E1, the planning reference position R1, and the system clock T1; at the next sampling time T2, a data frame F2 is formed, which includes the encoder position E2, the planning reference position R2, and the system clock T2. During subsequent historical retrieval, the system reads the complete frame F1 or F2, rather than splicing data from multiple independent arrays. This ensures that the three types of information in each frame naturally belong to the same physical time. The circular buffer is written in a first-in, first-out manner. When the buffer is full, the oldest data is overwritten by the newest data, thus continuously retaining the high-frequency historical state of the most recent period within the limited storage space.

[0028] The technical advantage of this organization method is that the visual latency is usually in the range of the first preset time (ms) to the second preset time (ms), while the servo state sampling is at a higher frequency; as long as the depth of the ring buffer covers the maximum processing latency that may occur in the visual link, the system can find the internal state of the corresponding historical moment after the visual result is returned, without having to perform complex cross-module reorganization for each retrieval.

[0029] As an extreme abnormal operating condition, if jitter occurs in the sampling period, the controller prioritizes marking the actual write time of each frame with the system clock, rather than assuming that they are absolutely equal in interval; if the underlying random access memory space is tight, the temporal continuity of the status frame can be prioritized, and real-time performance can be maintained by reducing the cache retention window or adjusting the write bit width; if a sudden load occurs and causes a complete status frame to not be formed in some periods, the frame is marked as invalid and skipped during backtracking to avoid using missing data for error reconstruction.

[0030] For example, in the high-speed re-inspection station of the semiconductor packaging substrate, when the platform enters a corner transition trajectory, the speed and direction change rapidly; the system forms state frames F101, F102, and F103 at multiple consecutive sampling moments and writes them into the circular buffer in sequence; when the vision processing layer returns a frame of image result with a timestamp later, the state backtracking module only needs to read the adjacent complete state frames around the timestamp to obtain the encoder position and planning reference position at the same moment, without the misalignment phenomenon that the timestamp belongs to F102 but the encoder position is mistakenly taken from F103.

[0031] The purpose of this step is to provide a unified, continuous, and traceable data carrier for historical state backtracking, thereby achieving a consistent binding between clock, reference, and execution state.

[0032] In this embodiment of the invention, the method by which the visual processing module obtains the global clock timestamp includes: synchronizing and aligning the trigger clock of the visual sensor with the system clock through a preset precise time protocol; at the physical instant when the visual sensor performs image exposure, latching the current system clock as the global clock timestamp by a preset hardware trigger; and binding the global clock timestamp with the image data.

[0033] This embodiment provides a hardware-level binding mechanism between images and time. Specifically, in the previous embodiment, even if a state circular buffer has been established, if the timestamp of the visual image is inaccurate, historical backtracking will still fall at the wrong time position. Common software timestamping methods usually occur when the image reaches the processing node or when the algorithm starts solving. These moments are already later than the actual exposure moment and are greatly affected by bus congestion, processor scheduling, and operating system latency. Therefore, this embodiment introduces a precise time protocol for clock synchronization and uses a hardware trigger to latch the system clock at the physical moment of image exposure to form a global clock timestamp. Specifically, the trigger clock of the vision sensor is first synchronized with the system master clock, so that the camera side and the motion control side operate on the same time reference. When the vision sensor begins exposure, the hardware trigger immediately latches the system clock value at this moment and uses this value as the global clock timestamp of the current frame image. This timestamp is transmitted to the vision processing layer along with the image data. In this way, any subsequent image processing time will only affect when the result is obtained, and will not change the physical fact that the image was generated. For the control system, this is equivalent to establishing a fixed time reference for each frame of visual observation. From the perspective of the system's working principle, the moment of exposure is the actual sampling moment when visual measurement is established; the controlled actuator may continue to move before and after exposure, while the target position in the image only corresponds to the scene state within the exposure window; therefore, only by binding the timestamp to the physical moment of exposure can subsequent backtracking be accurately aligned with the high-frequency state in the circular buffer; if the timestamp is set at the time when the image arrives in memory, when decoding is completed, or when recognition ends, the timestamp will be mixed with unpredictable processing delays, which will reintroduce time delay uncertainty; As an extreme abnormal operating condition, if the precise time protocol synchronization temporarily fails, the system can mark the visual link as untrusted, disable vision-based deviation correction, and only maintain the operation of the underlying servo. If the hardware trigger fails to return the latch value, the image data can be used for monitoring or offline analysis, but it will not be included in the closed-loop error calculation. If the binding relationship between image data and timestamp is damaged, such as frame number disorder or cache overwriting, the frame result will be directly discarded to avoid the erroneous time mapping from interfering with the control loop.

[0034] For example, in the high-speed re-inspection station for semiconductor packaging substrates, the camera completes the exposure when the substrate edge mark enters the center of the field of view; although the image will subsequently undergo transmission and feature extraction processes, the system has already latched the global clock at the moment of exposure; later, when the vision processing layer outputs the target edge center position, the controller backtracks the platform state based on the exposure time rather than the algorithm completion time, so even if the network load fluctuates during the shift and the image processing time fluctuates non-periodicly, the time reference for error calculation remains consistent.

[0035] The purpose of this mechanism is to fix the starting point of visual measurement at the moment of actual physical sampling, thereby achieving unified data alignment across clock domains and reducing the impact of visual computing link jitter on control accuracy.

[0036] In this embodiment of the invention, the step of the state backtracking module extracting the historical encoder position includes: determining whether the global clock timestamp precisely matches the system clock recorded in the state circular buffer; if the global clock timestamp precisely matches the system clock recorded in the state circular buffer, then the corresponding real-time encoder position is directly extracted as the historical encoder position; If the global clock timestamp does not precisely match the system clock recorded in the state ring buffer, then search in the state ring buffer for a first system clock and a second system clock that are earlier and later than the global clock timestamp, respectively; extract the first encoder position corresponding to the first system clock and the second encoder position corresponding to the second system clock; use a linear interpolation algorithm to combine the first encoder position and the second encoder position to calculate the historical encoder position corresponding to the global clock timestamp.

[0037] This embodiment provides a backtracking extraction step for the historical encoder position; specifically, in the aforementioned scheme, although the image timestamp has been precisely bound to the exposure moment, the visual exposure moment may not fall exactly on the sampling moment of the state buffer; If only the timestamp must be exactly equal to the cache clock to read the historical state, a large number of visual results will be unusable due to different sampling phases. Therefore, this embodiment directly takes the value when a precise match is possible, and uses linear interpolation between adjacent historical states to recover the historical encoder position corresponding to the exposure time when a precise match is not possible. Specifically, when the state backtracking module receives the global clock timestamp, it first searches the state circular buffer to see if the same system clock record exists. If it does, for example, if the image timestamp is exactly equal to the clock in state frame F200, the encoder position in that frame is directly extracted as the historical encoder position, which has the shortest processing path and is closest to the underlying original sampling result. If it does not exist, it searches the two state frames before and after the timestamp, such as the earlier state frame Fa and the later state frame Fb, and extracts the encoder positions Ea and Eb from them. Since the movement of the controlled actuator between adjacent high-frequency sampling cycles is usually continuous, especially when the servo loop is working normally, the position change within a short time window can be approximated as a smooth transition along a continuous trajectory. Therefore, linear interpolation can be used to recover the historical encoder position corresponding to the exposure time.

[0038] The linear interpolation here is not for long-distance prediction, but to reconstruct a nearest neighbor state between two adjacent high-frequency sampling points. It is suitable for situations with short time intervals and continuous mechanical motion. For example, if the timestamp is between Fa and Fb, the recovered historical encoder position is between the states corresponding to Ea and Eb, and is closer to the actual execution position at the moment of exposure. Compared with directly taking the nearest sampling point, this method can reduce the time quantization error caused by the sampling phase difference.

[0039] Furthermore, in order to establish subsequent error verification on a unified time base, the state backtracking module can perform synchronous recovery of the planned reference positions in the same pair of adjacent state data frames when it completes the extraction of historical encoder positions. When the timestamps are precisely matched, the historical planning reference position and the historical encoder position are directly taken from the same state data frame. When the timestamps are not precisely matched, the historical planning reference position is preferably based on the two planning reference positions corresponding to the same first system clock and second system clock for same-scale time recovery. In this way, the target state quantity, the historical encoder position, and the historical planning reference position are all constrained within the historical time section corresponding to the same exposure time, which can avoid the introduction of new pseudo-biases due to different approximation rules for different variables. This supplementary step is a refinement of the implementation of extracting the historical planning reference position, and its execution order can be completed in parallel or serially with the extraction of the historical encoder position. As an extreme abnormal condition, if the global clock timestamp is earlier than the earliest record in the buffer or later than the latest record, it indicates that the current buffer depth or processing timing is abnormal. The system will not perform interpolation and will directly discard the visual result. If the time interval between two adjacent frames is abnormally amplified, such as when sampling is interrupted, the historical data segment can be marked as unsuitable for interpolation, changed to an alarm, and exited from this correction. If a jump or fault flag is detected in the encoder reading itself, no historical recovery will be performed to avoid mistaking underlying abnormalities as the basis for visual correction.

[0040] Furthermore, if any adjacent state data frame used for synchronously restoring the historical planning reference position has missing fields, trajectory segment switching markers, or invalid reference position flags, the state backtracking module will preferentially abandon this backtracking and will not send the partially restored data to the error generation module.

[0041] For example, in the high-speed re-inspection station for semiconductor packaging substrates, the camera's exposure time is located between two servo sampling cycles, and the platform is continuously scanning along the edge of the substrate; the state backtracking module retrieves the states of two frames before and after the exposure time, where the platform is located to the left of the edge mark in the first frame and has moved further forward in the second frame. Since the platform movement is continuous between these two moments, the system can recover the internal position of the platform at the exposure moment and compare it with the target state quantity calculated in the image frame, thereby avoiding the amplification of deviation caused by inconsistent time granularity. Correspondingly, the system can also recover the planning reference position of the exposure moment based on the same pair of adjacent state frames, so that the rationality verification and execution error calculation are carried out around the same historical moment.

[0042] The purpose of this step is to establish a fine-grained temporal bridge between the high-frequency buffer and the low-frequency vision, thereby achieving an approximate reconstruction of the internal execution state at the exposure moment and improving the accuracy of historical alignment.

[0043] In this embodiment of the invention, the process of generating the absolute visual error by the error generation module includes: calculating the spatial positional deviation between the target state quantity and the historical planning reference position to obtain the state deviation degree; determining whether the state deviation degree is greater than a preset error threshold; wherein, the preset error threshold is obtained by tuning based on the maximum effective field of view of the visual sensor and the maximum allowable motion deviation of the controlled actuator within the maximum visual calculation delay. If the state deviation is less than or equal to the preset error threshold, the verification is deemed successful, and the target state quantity is subtracted from the historical encoder position to generate the absolute visual error; if the state deviation is greater than the preset error threshold, the verification is deemed unsuccessful, the target state quantity is discarded, and an out-of-bounds alarm signal is output.

[0044] This embodiment provides a mechanism for validating the validity of visual results based on historical planning reference positions. Specifically, in the aforementioned scheme, the system is already able to align the visual results with the historical execution states. However, if the visual image itself is contaminated, such as by substrate surface reflection, process residue occlusion, edge defocus, incorrect features being identified as targets, or reconstruction deviations, then even if the time alignment is correct, the obtained target state quantity may still be incorrect. If this incorrect result is directly sent into the control loop, it will cause the platform to correct in the wrong direction. Therefore, this embodiment first uses the historical planning reference positions to screen the reasonableness of the target state quantity before generating the absolute visual error. Specifically, the historical planning reference position represents the trajectory position that the system originally planned the platform to reach at the exposure time. Since the scanning trajectory in the aforementioned workstation is pre-planned and continuously executed, under normal circumstances, the target state quantity obtained by visual calculation should fall near the historical planning reference position, and there will be no step deviation beyond the preset effective field of view. The error generation module determines whether the visual result is consistent with the process scenario by examining the degree of deviation between the target state quantity and the historical planning reference position. If the deviation is less than or equal to the set threshold, it indicates that the visual result does indeed come from the currently tracked process object. At this time, the difference between the target state variable and the historical encoder position can be used to obtain the absolute visual error representing the deviation of the true trajectory. If the deviation is greater than the set threshold, it indicates that the visual result may correspond to erroneous features, abnormal reflections, image distortion, or geometric inconsistency. At this time, the system discards the result and outputs an out-of-bounds alarm signal.

[0045] As an example: assuming the historical planning reference position at the exposure time is near trajectory segment Rk, and the target state quantity obtained by visual calculation also falls near Rk, it indicates that the object seen by vision is consistent with the planned scanning object. Conversely, if the target state quantity falls into an area exceeding the preset distance threshold of Rk, it usually indicates that stray edges or non-target structures have been identified. In this case, although the absolute visual error can be calculated, it has no controllable significance. If the historical planning reference position is missing or the trajectory segment is in an uncertain adjustment phase, the validity verification can be paused, and the system only retains basic servo control. Visual correction can be reactivated after the planning information stabilizes. If multiple consecutive frames show out-of-bounds alarm signals, visual link degradation, workpiece repositioning, or process suspension can be further triggered. If the deviation is in the fuzzy range near the threshold, a conservative strategy can also be set, such as reducing the compensation weight this time to avoid excessive correction when the confidence level is insufficient.

[0046] For example, in the high-speed re-inspection station for semiconductor packaging substrates, the platform scans a substrate edge line according to a predetermined trajectory. Under normal circumstances, the edge center obtained by image processing should be located near the edge line. However, at a certain moment, due to local reflection on the substrate surface, the visual algorithm mistakenly takes a bright scratch as an edge feature, and the obtained target state quantity deviates significantly from the trajectory area that should have been traversed at that time. Based on this, the system determines that the frame is inconsistent with the historical planning reference position, directly discards the result and issues an out-of-bounds alarm, and does not use it to correct the platform position.

[0047] The purpose of this mechanism is to add a layer of process rationality verification before visual correction enters the control loop, thereby isolating erroneous visual results and avoiding misidentification that could lead to reverse correction or abnormal oscillations.

[0048] In this embodiment of the invention, the feedforward compensation module includes: a curve generation unit, used to generate the error compensation transition curve based on a smooth curve using the absolute visual error as a state variable and a preset kinematic forward model; and an instruction superposition unit, used to extract the compensation increment corresponding to the error compensation transition curve within a preset continuous clock cycle; and to superimpose the compensation increment into the underlying control instruction cycle by cycle, in conjunction with the outer loop visual compensation gain.

[0049] Specifically, the calculation method of the cycle-by-cycle superposition is as follows: the extracted compensation increment is multiplied by the outer loop visual compensation gain to obtain the effective position correction amount for a single cycle; the effective position correction amount is summed with the original position control command of the underlying servo loop as an additional value, and the corrected underlying control command is output; the error compensation transition curve is a smooth curve; the starting tangent direction of the smooth curve is consistent with the current motion direction of the controlled actuator; the endpoint of the smooth curve converges to the state of complete compensation of the absolute visual error.

[0050] This embodiment provides an error feedforward projection and smooth injection mechanism. Specifically, in the aforementioned scheme, the system has obtained the absolute visual error that has been time-aligned and validated. However, if this error is directly written into the underlying control command as a one-step step correction, it will form discontinuous step acceleration and deceleration commands on the high-speed precision platform. Such abrupt commands will disrupt the original trajectory continuity, causing the servo motor, current loop, and mechanical transmission structure to be subjected to instantaneous impacts, manifested as vibration, contour glitch, or control loop overshoot. Therefore, this embodiment generates a smooth error compensation transition curve through a kinematic forward model and gradually superimposes the compensation increment within the continuous control cycle.

[0051] Specifically, after receiving the absolute visual error, the curve generation unit does not simply output a fixed bias, but combines the current motion trend of the controlled actuator to form a compensation curve that smoothly transitions from the current trajectory state to the state of error compensation. As a specific implementation method, the preset kinematic forward model adopts a fifth-order polynomial interpolation algorithm or an S-shaped acceleration / deceleration control algorithm, uses the absolute visual error as the displacement boundary condition, and uses the velocity and acceleration corresponding to the current motion direction as the initial kinematic constraints to generate a continuous and smooth position compensation sequence as the error compensation transition curve. The curve can be understood as a transitional path that gradually approaches the target deviation correction amount. In the initial stage, the current motion inertia continues; in the middle stage, visual errors are gradually absorbed; and in the final stage, it converges to the state of complete compensation of absolute visual errors. In the subsequent several consecutive control cycles, the instruction superposition unit extracts the compensation increment corresponding to each moment of the curve and superimposes it into the underlying control instruction in combination with the outer loop visual compensation gain. Thus, the visual error is transformed into a continuous and executable compensation process, rather than a sudden event.

[0052] Furthermore, in order to avoid sudden changes in velocity direction at the trajectory splicing point, the error compensation transition curve is preferably a smooth curve, and its starting tangent direction is consistent with the current motion direction of the controlled actuator. The technical effect of doing so is that when the platform starts to receive visual compensation, it will not be suddenly required to turn sharply in a direction that conflicts with the original motion trend, but will naturally bring in the correction component in the original motion direction. The curve converges to the state of complete compensation for absolute visual error at its endpoint, indicating that after a finite control period, the outer loop compensation should have completed the absorption of the visual deviation, rather than leaving an unclosed bias for a long time.

[0053] To avoid the kinematic forward model being misunderstood as a functional description with an unclear processing procedure, this embodiment further limits its input, processing constraints, and output methods as follows: the curve generation unit shall receive at least one or more of the following: absolute visual error, current direction of motion, current velocity range, allowed number of compensation cycles, and upper limit of single-cycle compensation; First, determine how many consecutive control cycles the compensation should be completed within. Then, determine the upper limit of the compensation increment that can be released in each cycle. Under the conditions that the starting tangent is continuous, the compensation increment changes gradually, and the endpoint converges, an error compensation transition curve is formed. In other words, the role of this model is not to replan the entire process trajectory, but to generate an additional, small-amplitude, executable correction component trajectory outside the original planned trajectory.

[0054] Furthermore, the instruction overlay unit can extract the compensation increment in the following order: extract the initial compensation amount of the first preset amplitude in the first continuous control cycle to make the injection process start smoothly; extract the compensation amount that is gradually increased or released smoothly in the middle several cycles to improve the error absorption efficiency; gradually reduce the increment when the compensation is about to be completed until the remaining uncompensated amount converges to zero or falls into the preset residual tolerance range; after such settings, even if the absolute visual error corresponding to different batches of workpieces is different, the system can generate a compensation process with different amplitudes but consistent continuity along the same control logic.

[0055] As an example: Suppose that an absolute visual error manifests as a first rightward offset of the platform relative to the target edge. If the platform were to correct this offset all at once, it might immediately apply a correction amount to the left that exceeds the set smoothing threshold, resulting in an uneven edge segment scan. However, by using a smoothing curve, the system will gradually correct to the left over the next few control cycles, preserving the original forward continuity while gradually returning to the correct contour. For another example, when the current motion direction is mainly along the positive X-axis and the visual error is mainly reflected in the lateral offset of the Y-axis, the curve generation unit does not change the continuous forward trend of the main X-axis motion. Instead, it gradually adds the correction amount in the Y-axis direction according to a limited slope, thereby ensuring a continuous transition between the starting tangent direction and the current overall motion direction. To ensure that the relationship between the compensation increment, absolute visual error, and fully compensated state remains unique and clear throughout the text, this embodiment further explains as follows: Absolute visual error represents the total spatial deviation at a verified historical moment; the error compensation transition curve represents the release path of this total amount in multiple subsequent continuous control cycles; and the compensation increment represents the single-cycle increment value corresponding to the release path within a certain continuous control cycle. Therefore, the compensation increment is not a new definition of error, nor is it a second control objective independent of the absolute visual error. Rather, it is merely the periodic execution amount after the same absolute visual error is divided in time. The fully compensated state indicates that after the absolute visual error is continuously released, its cumulative injection amount reaches the total amount to be compensated or falls within the preset residual tolerance range. Based on this agreement, all references to absorbing visual errors, completing error compensation, and releasing compensation increments in this application refer to different processing stages of the same compensation closed loop, rather than different physical quantities.

[0056] Furthermore, to make the execution rules of the kinematic forward model more reproducible without introducing complex formulas, its processing logic can be understood as follows: first, read the remaining uncompensated amount before the current control cycle; then, based on the current speed range and the single-cycle compensation limit, determine the maximum compensation amount allowed to be injected in this cycle. The remaining uncompensated amount is compared with the allowed injection amount, and the smaller one is taken as the candidate release amount. Combining the continuity of the starting tangent and the smooth convergence constraint, the slope of the candidate release amount is limited and output as the compensation increment for this period. If there is still a remaining uncompensated amount after this period, the above process is repeated in the next period until the fully compensated state is reached or the residual tolerance range is entered. In this way, the data flow relationship between the curve generation unit and the instruction superposition unit remains fixed: the former is responsible for forming the continuous release rule, and the latter is responsible for executing the rule periodically. In abnormal or boundary situations, if the absolute visual error is greater than the preset maximum compensation threshold for a single time and exceeds the one-time absorption range allowed by the current process, the curve generation unit can extend the transition period or reduce the single-cycle compensation slope. If necessary, the large error can be split into multiple compensation segments. If the currently controlled actuator is close to the speed reversal point, mechanical limit position or other sensitive working conditions, the curve smoothness and command amplitude can be constrained first to avoid the compensation action from conflicting with the original kinematic constraints. If a new visual error is received before the compensation is completed, the subsequent transition curves can be replanned in chronological order, or incremental merging can be performed to avoid conflicting instructions between multiple compensation curves. Furthermore, when multiple visual errors arrive consecutively, it is preferable to use the remaining compensation amount that has not been released and the latest absolute visual error together to form the input for the next round of curve generation, so that the system maintains the same compensation coordinate system for old and new errors, rather than simply covering each other.

[0057] For example, in the high-speed re-inspection station for semiconductor packaging substrates, the platform continuously scans along the edge of the substrate contour. A visual result of a certain frame shows that the edge tracking value has a lateral offset within a preset tolerance. The system does not directly write the offset into the position command all at once, but generates a smooth transition curve according to the current forward direction of the platform, so that the compensation action unfolds naturally from the current scanning direction, and gradually corrects the contour to the correct position in several subsequent control cycles. If the platform is in a rounded contour transition area at this time, it is particularly important that the starting tangent direction is consistent with the current movement direction, because this can avoid the production of a broken line correction in the rounded corner segment. The purpose of this mechanism is to transform absolute visual errors into compensation commands that conform to the continuous motion characteristics of the actuator, thereby achieving flexible coupling between visual correction and high-frequency servo circuits and reducing jitter and shock caused by asynchronous data injection.

[0058] In this embodiment of the invention, it further includes: a gain adjustment module, used to obtain the update frequency of the absolute visual error and determine whether the update frequency is higher than a preset frequency threshold; The preset frequency threshold is a reference switching frequency set based on the empirical ratio of the control bandwidth of the underlying servo loop and the rated sampling frame rate of the vision sensor. If the update frequency is higher than the preset frequency threshold, the outer loop visual compensation gain is set to a preset high gain value; if the update frequency is equal to the preset frequency threshold, the current outer loop visual compensation gain is maintained. If the update frequency is lower than the preset frequency threshold, the outer ring visual compensation gain will be reduced to a preset low gain value.

[0059] This embodiment provides a gain adaptive adjustment mechanism based on error update frequency. Specifically, in the aforementioned compensation scheme, if the outer loop visual compensation gain remains fixed for a long period, the compensation strength may be at risk of lag in correcting the response when the visual results are updated frequently and stably. However, when the visual results are updated sparsely, the delay increases, or there are intermittent frame drops, the originally appropriate high gain may lead to overcompensation of the system and cause overshoot, which may easily amplify the control risks caused by outdated information. Therefore, this embodiment uses the update frequency of the absolute visual error as the basis for adjusting the outer loop compensation gain. Specifically, the error update frequency reflects the ability of the visual outer loop to provide the control system with the latest absolute visual error per unit time. When the update frequency is high, it indicates that the visual link is working stably and the effective results are continuously output. The control system can obtain new absolute deviation information more frequently. At this time, a preset high gain value can be used to make the platform more actively absorb visual errors and improve the low-frequency contour tracking accuracy. When the update frequency is lower than the preset threshold, it indicates that the visual results are relatively sparse and the absolute deviation information obtained by the outer loop is updated more slowly. If the gain is maintained at this time, the system may react too strongly to older errors. Therefore, it should be reduced to the preset low gain value to enhance robustness. If the update frequency is near the threshold and basically stable, the current gain should be maintained to avoid introducing additional disturbances due to frequent gain switching. If multiple frames of valid absolute visual errors are received consecutively within a certain time window, such as error sequences D1, D2, and D3 arriving stably, the outer loop has a high information refresh capability and can increase the correction intensity. If only a small number of discrete errors are received within the same time window, such as no new results for a long time after D1 arrives, the visual outer loop should be regarded as a low-frequency feedback source, and its compensation action should be weakened accordingly.

[0060] As an extreme abnormal operating condition, if a large number of invalid frames or out-of-bounds alarm frames are mixed in within the update frequency statistics window, the gain adjustment module can calculate the frequency based solely on the effective absolute visual error. If the visual link is temporarily interrupted, the outer loop visual compensation gain can be reduced to a low gain value, or the visual compensation can be frozen directly, retaining only the underlying servo closed loop. If the update frequency fluctuates frequently around the high and low thresholds, a hysteresis range or a smooth transition range can be set to avoid gain switching back and forth.

[0061] For example, in the high-speed re-inspection station for semiconductor packaging substrates, when the substrate surface texture is clear and the illumination is stable, the vision processing layer can continuously output effective target state quantities. The system counts that the absolute visual error update frequency remains at a high level, so the outer ring visual compensation gain is increased, enabling the platform to fit the real edge more quickly. After entering an area where local reflection exceeds the preset light intensity threshold, some images are discarded due to insufficient credibility, the effective error update frequency decreases, and the system automatically switches the gain to a lower level, so that the platform mainly relies on the original servo trajectory to run, and only performs fine-tuning corrections within a preset range.

[0062] The purpose of this mechanism is to dynamically adjust the outer loop compensation intensity based on the freshness of the visual information, thereby achieving a balance between accuracy and stability and avoiding control instability caused by excessive gain when the visual link fluctuates.

[0063] In this embodiment of the invention, the controlled actuator is a general motion platform; the control architecture of the system is divided into an instruction scheduling layer, a vision processing layer, and a servo execution layer; the vision processing layer runs on a preset independent computing node, and the servo execution layer and the state ring buffer run on a preset underlying motion controller.

[0064] This embodiment provides a hierarchical control architecture. Specifically, in the aforementioned scheme, if trajectory planning, vision processing, servo execution, and historical caching are centrally deployed on the same processing node, fluctuations in the vision calculation task will directly consume the underlying control resources, affecting the determinism of the high-frequency servo cycle. Especially in high-speed precision workstations, the underlying motion controller needs to stably maintain high-frequency position loops, velocity loops, and state cache writing, while the vision processing layer is affected by image complexity, lighting changes, and communication load. The two have different requirements for computing resources. Therefore, this embodiment divides the system into an instruction scheduling layer, a vision processing layer, and a servo execution layer, and deploys the vision processing layer on an independent computing node. The instruction scheduling layer is responsible for generating or issuing process scanning trajectories, operation phase switching, and work flow instructions, and can be located in the upper-level control unit; the vision processing layer runs on an independent computing node, such as an industrial computer or a dedicated vision controller, and is responsible for receiving image data with global timestamps and solving target state variables; the servo execution layer runs in the underlying motion controller and continuously performs high real-time tasks such as encoder sampling, planning reference execution, state buffer writing, history backtracking, and low-level control instruction output; the state ring buffer area is deployed together with the servo execution layer in the underlying motion controller, which can ensure that historical state acquisition and control execution are in the same deterministic timing environment.

[0065] The technical advantage of this architecture is that even if there are instantaneous load fluctuations in visual computing, the periodic operation of the servo execution layer will not be interrupted; the underlying motion controller can stably save high-frequency historical states and quickly complete state backtracking and compensation injection after receiving the target state quantity returned by the visual processing layer; at the same time, the instruction scheduling layer maintains overall management of the entire process trajectory and cycle time, which makes it easy to reuse the same set of control logic in different production batches or process modes.

[0066] It should be noted that, in order to maintain the uniqueness of the technical object when referring to the control system controller, motion controller, independent computing node, etc. throughout this application, this embodiment further explains that: the control system is an overall system concept that includes an instruction scheduling layer, a vision processing layer, and a servo execution layer; The underlying motion controller is the specific hardware node in the control system that carries the servo execution layer and the state ring buffer; the independent computing node is the specific hardware node in the control system that carries the vision processing layer; the upper-level control unit is the upper-level node in the control system that carries the instruction scheduling layer; in statements that do not involve the distinction of hardware deployment location, the controller mentioned in the text can be used as an abbreviation for the underlying motion controller. When emphasizing layered collaboration, the upper-level control unit independently computes the underlying motion controller corresponding to the instruction scheduling layer, vision processing layer, and servo execution layer. Through this correspondence, the descriptions of task allocation, data flow, and resource isolation throughout the text are all based on the same layered architecture, and there is no situation where a module repeatedly undertakes different responsibilities across layers.

[0067] Furthermore, to avoid the data path and control path being misunderstood as being interchangeable, this embodiment provides the following supplementary explanation of the inter-layer interaction relationship: the instruction scheduling layer sends the planning trajectory and operation stage information to the servo execution layer, the vision processing layer sends the target state quantity with a global clock timestamp back to the servo execution layer, and the servo execution layer outputs the underlying control instructions to the controlled actuator and writes them locally into the state ring buffer. The state ring buffer is continuously written to without relying on the existence of the vision processing layer. The vision processing layer does not directly rewrite the underlying servo cycle, but participates in the control through the path of target state quantity, historical backtracking, error generation and feedforward compensation. With such limitations, the conclusion that the load fluctuation of the vision processing layer does not affect the determinism of high-frequency servo is consistent with the hardware connection relationship and division of responsibilities in this embodiment. In abnormal or boundary situations, if an independent computing node restarts, experiences abnormal latency, or experiences image processing congestion, the servo execution layer can still continue to run according to the predetermined reference trajectory; if the upper-level instruction scheduling layer temporarily interrupts the issuance of new tasks, the lower-level motion controller can safely complete or stop the currently downloaded trajectory segment. If communication between the vision processing layer and the servo execution layer is interrupted, the vision correction link will be suspended, but the state cache writing will not be affected, thus providing a historical data basis for the closed-loop correction to continue after the system recovers.

[0068] For example, in the high-speed re-inspection station for semiconductor packaging substrates, the host industrial control computer is responsible for arranging substrate entry, scanning trajectory switching, and process segment scheduling; the independent vision computing node is responsible for processing camera images and outputting the target edge state with timestamps. The underlying motion controller continuously drives the general motion platform to run along the scanning trajectory and writes the encoder position, reference position and system clock of each control cycle into the state ring buffer. When the visual processing node experiences a short-term increase in load due to the complexity of a batch of substrate images, the underlying motion controller still maintains a stable servo cycle and will not lose historical states due to visual task jitter. The purpose of this mechanism is to ensure high-frequency servo determinism through functional layering and resource isolation, thereby achieving decoupling between visual high-latency processing and underlying real-time control.

[0069] In this embodiment of the invention, after receiving the underlying control command superimposed with the error compensation transition curve, the controlled actuator performs a state correction action in the physical space; the vision sensor continuously captures the corrected physical space state and generates a new target state quantity to form an asynchronous closed-loop control data stream with multiple sampling rates.

[0070] This embodiment provides a asynchronous, multi-sampling-rate closed-loop control data loop mechanism. Specifically, in the aforementioned scheme, the error compensation transition curve has been superimposed on the underlying control command. However, the control closed loop of this system is not a traditional single-sampling-rate, strictly synchronous closed loop, but is composed of a high-frequency servo inner loop and a low-frequency vision outer loop. If it is still understood as a synchronous closed loop, it will be mistakenly believed that every platform action must wait for a new vision confirmation, which will significantly reduce dynamic performance. Therefore, this embodiment explicitly adopts a asynchronous and multi-sampling-rate data flow mechanism: the controlled actuator continuously executes state correction actions in the physical space according to high-frequency low-level control commands, while the vision sensor continuously observes the corrected spatial state according to its own sampling rhythm, and generates new target state quantities in subsequent moments to drive the next round of historical backtracking and error update; The servo execution layer typically operates at a higher frequency, and its core responsibility is to ensure that the platform continuously follows the current control commands. The vision sensor, on the other hand, acquires images at a lower frequency, and each frame of the image is processed and delayed to form a new target state quantity. The two are not aligned in time, but through global timestamps and historical backtracking pipelines, the low-frequency vision outer loop can still form an effective closure with the high-frequency servo inner loop. After the platform performs a correction action, the contour position, edge alignment state, or target pose in physical space changes. The vision sensor will observe these changes again in subsequent exposures and feed them back to the system as new target state quantities. This cycle forms a continuously rolling closed-loop data stream. From an engineering perspective, asynchronous and multi-sampling rate are inherent characteristics of this type of vision-guided control system. The high-frequency servo loop is responsible for suppressing rapid dynamic errors and maintaining basic stability, while the low-frequency vision outer loop is responsible for providing absolute spatial correction without cumulative drift. The two work together through a time alignment mechanism to maintain execution continuity while continuously absorbing new absolute position information.

[0071] As an extreme abnormal operating condition, if the vision sensor fails to continuously output new target state quantities at a certain stage, the platform can still maintain trajectory operation by relying on the underlying servo control, only the absolute correction capability of the outer loop will be temporarily weakened; if the vision update recovers, the system will continue to send new target state quantities into the backtracking, verification and compensation process; if no effective results are obtained within several consecutive vision cycles, the vision outer loop degradation, alarm or process switching can be triggered; if the actuator approaches the process boundary during the correction process, the controller can limit the subsequent compensation range to prioritize operational safety and trajectory continuity.

[0072] In the high-speed re-inspection station for the semiconductor packaging substrate, the platform continuously moves along the edge of the substrate and performs minute lateral corrections under the drive of the underlying servo commands. The upper camera does not provide new measurement results in every servo cycle, but outputs a new frame of the target edge state only after several servo cycles. The system backtracks the platform's historical state based on the exposure time of this frame, generates the next round of absolute visual error, and forms a new compensation transition curve. The entire process is characterized by continuous operation of the inner servo loop and asynchronous refresh of the outer visual loop, which together form a stable closed-loop correction link. The inherently low frequency, time delay, and asynchronous characteristics of visual feedback enable continuous coordination between high-frequency execution and low-frequency absolute correction, ensuring that the platform can continuously correct trajectory deviations during dynamic tracking.

[0073] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A vision-guided trajectory deviation correction system based on dynamic tracking, applied to a control system with a controlled actuator and a vision sensor connected via communication, characterized in that, include: The state buffer module is used to collect the real-time encoder position, real-time planning reference position and system clock of the controlled actuator at a preset sampling rate, and continuously push the real-time encoder position, the real-time planning reference position and the system clock into the state ring buffer. The visual processing module is used to acquire the image data generated by the visual sensor at the moment of exposure and the latched global clock timestamp, and to calculate the target state quantity of the controlled actuator with the global clock timestamp based on the image data. The state backtracking module is used to respond to the target state quantity output by the visual calculation module, and retrieve and extract the historical encoder position and historical planning reference position of the controlled actuator corresponding to the global clock timestamp in the state ring buffer according to the global clock timestamp. The error generation module is used to verify the validity of the target state quantity based on the historical planning reference position, and after the verification is passed, to subtract the target state quantity from the historical encoder position to generate an absolute visual error. The feedforward compensation module is used to generate an error compensation transition curve based on the absolute visual error, and to superimpose the error compensation transition curve with a preset outer loop visual compensation gain into the underlying control command of the current control loop of the controlled actuator.

2. The visual guidance trajectory deviation correction system based on dynamic tracking according to claim 1, characterized in that, The state caching module is specifically used for: According to a preset clock cycle, the real-time encoder position, the real-time planning reference position, and the corresponding system clock are packaged into a status data frame; The state data frame is written into the state circular buffer allocated in the preset underlying random access memory according to the first-in-first-out principle.

3. The visual guidance trajectory deviation correction system based on dynamic tracking according to claim 1, characterized in that, The visual processing module obtains the global clock timestamp in the following ways: The trigger clock of the vision sensor is synchronized with the system clock using a preset precise time protocol. At the physical instant when the visual sensor performs image exposure, the current system clock is latched by a preset hardware trigger as the global clock timestamp; The global clock timestamp is bound to the image data.

4. The visual guidance trajectory deviation correction system based on dynamic tracking according to claim 1, characterized in that, The step of the state backtracking module to extract the historical encoder position includes: Determine whether the global clock timestamp precisely matches the system clock recorded in the state ring buffer; If the global clock timestamp precisely matches the system clock recorded in the state ring buffer, then the corresponding real-time encoder position is directly extracted as the historical encoder position. If the global clock timestamp does not precisely match the system clock recorded in the state ring buffer, then search in the state ring buffer for a first system clock and a second system clock that are earlier and later than the global clock timestamp, respectively. Extract the position of the first encoder corresponding to the first system clock and the position of the second encoder corresponding to the second system clock; The historical encoder position corresponding to the global clock timestamp is calculated by combining the first encoder position and the second encoder position using a linear interpolation algorithm.

5. The visual guidance trajectory deviation correction system based on dynamic tracking according to claim 1, characterized in that, The process by which the error generation module generates the absolute visual error includes: Calculate the spatial positional deviation between the target state quantity and the historical planning reference position to obtain the state deviation degree; Determine whether the deviation of the state is greater than a preset error threshold; If the state deviation is less than or equal to the preset error threshold, the verification is deemed successful, and the target state quantity is subtracted from the historical encoder position to generate the absolute visual error. If the deviation of the state is greater than the preset error threshold, the verification is deemed to have failed, the target state quantity is discarded, and an out-of-bounds alarm signal is output.

6. The visual guidance trajectory deviation correction system based on dynamic tracking according to claim 1, characterized in that, The feedforward compensation module includes: The curve generation unit is used to generate the error compensation transition curve based on a smooth curve through a preset kinematic forward model, using the absolute visual error as a state variable. The instruction overlay unit is used to extract the compensation increment corresponding to the error compensation transition curve within a preset continuous clock cycle. Combined with the outer ring visual compensation gain, the compensation increment is added to the underlying control command cycle by cycle.

7. The visual guidance trajectory deviation correction system based on dynamic tracking according to claim 6, characterized in that, The error compensation transition curve is a smooth curve; The starting tangent direction of the smooth curve is consistent with the current motion direction of the controlled actuator; The endpoint of the smooth curve converges to a state where the absolute visual error is fully compensated.

8. The visual guidance trajectory deviation correction system based on dynamic tracking according to claim 1, characterized in that, Also includes: A gain adjustment module is used to obtain the update frequency of the absolute visual error and determine whether the update frequency is higher than a preset frequency threshold. If the update frequency is higher than the preset frequency threshold, the outer ring visual compensation gain is set to a preset high gain value. If the update frequency is equal to the preset frequency threshold, then the current outer loop visual compensation gain is maintained. If the update frequency is lower than the preset frequency threshold, the outer ring visual compensation gain will be reduced to a preset low gain value.

9. The visual guidance trajectory deviation correction system based on dynamic tracking according to claim 1, characterized in that, The controlled actuator is a general motion platform; The system's control architecture is divided into an instruction scheduling layer, a visual processing layer, and a server execution layer. The visual processing layer runs on a preset independent computing node, and the servo execution layer and the state ring buffer run on a preset underlying motion controller.

10. The visual guidance trajectory deviation correction system based on dynamic tracking according to claim 1, characterized in that, After receiving the underlying control command superimposed with the error compensation transition curve, the controlled actuator performs a state correction action in the physical space. The vision sensor continuously captures the corrected physical space state and generates new target state quantities to form an asynchronous closed-loop control data stream with multiple sampling rates.

Images