Automatic focusing system based on FPGA and ARM heterogeneous architecture

Abstract

The application relates to automatic optical detection, in particular to an automatic focusing system based on an FPGA and ARM heterogeneous architecture; the system adopts a layered heterogeneous architecture, high real-time image preprocessing, feature operation and micro-drive tasks are sunk to FPGA hardware implementation, meanwhile, motion control tasks are split to the hard real-time and soft real-time units of ARM according to real-time levels, so that the decoupling of hard and soft real-time tasks and the reasonable allocation of computing power resources are realized; under the heterogeneous architecture, high-speed image data and sensing data can be directly accessed to the FPGA for preprocessing without being transferred through the ARM, and cooperating with the micro-drive output channel directly connected with the FPGA, the single-frame image processing and closed-loop response time delay can be compressed to the microsecond level; meanwhile, the physical isolation design of the macro-drive and micro-drive links avoids the resource occupation of different priority control signals, and in combination with the layered task scheduling mechanism, the stability and anti-interference ability of long-term operation of the system are improved.

Description

Technical Field

[0001] This invention relates to automated optical inspection, and in particular to an autofocus system based on a heterogeneous architecture of FPGA and ARM. Background Technology

[0002] In applications such as industrial panel inspection and automated optical inspection of precision workpieces, the autofocus system is a core component to ensure imaging quality and inspection accuracy. It acquires spot images through image acquisition equipment, calculates the defocus amount by combining sensor data, and then drives the motion actuator to complete closed-loop focus adjustment.

[0003] Current mainstream autofocus control systems mainly adopt a single ARM processor architecture or an independent FPGA architecture, which have obvious limitations in hardware architecture design and task computing power allocation. Among them, the single ARM master control scheme entrusts all tasks such as image acquisition, feature calculation, motion control, and host computer interaction to the ARM processor, and relies on the operating system for software scheduling to process image data. In the independent FPGA scheme without ARM collaboration, the FPGA only undertakes basic image acquisition functions and lacks an independent master control unit to coordinate the overall operation process of the equipment, parameter calibration, and fault diagnosis.

[0004] The aforementioned existing technical solutions have many shortcomings in practical industrial applications.

[0005] First, the system lacks real-time performance, and the focusing latency is difficult to meet the needs of high-speed production lines. In the single ARM solution, image processing, matrix filtering and other operations rely on software execution. The operating system scheduling will introduce millisecond-level jitter, which cannot match the high-frequency image sampling rhythm. The single-frame image processing latency is high, which can easily lead to focusing lag and image defocusing. On the other hand, the pure FPGA solution lacks a main control scheduling unit and cannot carry out background management tasks such as temperature compensation, zero-point calibration and log storage.

[0006] Secondly, the allocation of computing power is unreasonable. High-real-time tasks such as high-speed image acquisition, spot feature extraction, and closed-loop control share the bus and computing resources with low-speed tasks such as parameter configuration and human-computer interaction. Data flow and control flow compete for hardware resources, which can easily lead to problems such as image frame loss and motion control command delay.

[0007] Furthermore, the system has weak scalability and maintainability. The hardware functions are not decoupled in layers, and the sensing, control, driving, and interaction modules are highly coupled. Replacing cameras, lasers, and motion modules requires large-scale modifications to the underlying logic, resulting in high debugging and upgrade costs. In addition, there is a lack of standardized high-speed bus interfaces with peripherals, limiting hardware compatibility. Summary of the Invention

[0008] In view of the shortcomings of the prior art, the purpose of this invention is to provide an autofocus system based on a heterogeneous architecture of FPGA and ARM, which solves the technical problems of operating system scheduling jitter and excessive focusing latency in the traditional single ARM architecture, as well as the lack of master control scheduling and inability to carry out background management tasks in the pure FPGA architecture.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: An autofocus system based on a heterogeneous architecture of FPGA and ARM includes a sensing and detection module, an FPGA hardware acceleration module, an ARM main control scheduling module, and a drive execution module. The sensing and detection module acquires optical image data and position sensing data required for the focusing process, and its output is connected to the acquisition input of the FPGA hardware acceleration module. The FPGA hardware acceleration module preprocesses the acquired data and communicates with the ARM main control scheduling module via a high-speed bus. Its drive output is connected to the micro-motion control of the drive execution module. The ARM main control scheduling module includes a hard real-time task unit and a soft real-time task unit. The hard real-time task unit performs motion path planning and vibration suppression calculations, and its output is connected to the macro-motion control of the drive execution module. The soft real-time task unit performs parameter drift compensation and equipment fault diagnosis.

[0010] The system also includes a host interaction module, and the ARM main control scheduling module further includes an interaction communication management unit; the interaction communication management unit is interconnected with the hard real-time task unit and the soft real-time task unit respectively; the interaction communication management unit and the host interaction module establish a connection through a communication link, which is used to coordinate the data transmission, protocol parsing and instruction distribution between the ARM main control scheduling module and the host interaction module.

[0011] In the system described above, the sensing and detection module includes an image acquisition unit and a sensor data acquisition unit; the image acquisition unit is used to acquire optical image data required for the focusing process, and its output terminal is connected to the acquisition input terminal of the FPGA hardware acceleration module; the sensor data acquisition unit is used to acquire position sensing data required for the focusing process, and its output terminal is connected to the acquisition input terminal of the FPGA hardware acceleration module.

[0012] In the system described above, the image acquisition unit includes a first imaging optical path, a second imaging optical path, and an optical path switching unit. The first imaging optical path is used to detect low-reflectivity workpieces, and the second imaging optical path is used to detect high-reflectivity, large-angle workpieces. The acquisition output terminals of the first imaging optical path and the second imaging optical path are both connected to the input terminal of the optical path switching unit, and the output terminal of the optical path switching unit is correspondingly connected to the acquisition input terminal of the FPGA hardware acceleration module.

[0013] In the system described, the FPGA hardware acceleration module includes an image access unit and a data access unit. The image access unit's acquisition input terminal is connected to the corresponding output terminal of the image acquisition unit. The image access unit is used to receive optical image data and complete interface protocol parsing and data format conversion. The data access unit's acquisition input terminal is connected to the corresponding output terminal of the sensor data acquisition unit. The data access unit is used to receive position sensor data and complete timing synchronization and data verification.

[0014] In the system described, the FPGA hardware acceleration module further includes a ping-pong buffer unit, an image processing core unit, a computation acceleration core unit, a control computation unit, a drive signal output unit, and a DDR memory unit. The output terminals of the image access unit and the data access unit are both connected to the input terminal of the ping-pong buffer unit, which is used to buffer acquired data through a dual-buffer read-write parallel mechanism. The first output terminal of the ping-pong buffer unit is connected to the input terminal of the image processing core unit, which is used to extract feature data from the spot image. The second output terminal of the ping-pong buffer unit is connected to the input terminal of the computation acceleration core unit, which is used to... The system independently performs filtering operations and defocus mapping matrix calculations. The outputs of both the image processing core unit and the computation acceleration core unit are connected to the input of the control computation unit. The control computation unit combines feature data and sensor data to calculate the defocus amount and generate corresponding drive control quantities. The output of the control computation unit is connected to the input of the drive signal output unit, and the output of the drive signal output unit is connected to the micro-motion control terminal of the drive execution module to output micro-motion control pulse signals. The DDR storage unit is connected to both the image processing core unit and the computation acceleration core unit to store processed image copy data and intermediate data from the computation process.

[0015] In the system described above, the image processing core unit is used to complete image processing based on a multi-level pipelined parallel mechanism. The image processing flow is divided into multiple serial processing steps, including an image receiving step, a noise reduction preprocessing step, a ROI spot cropping step, and a grayscale centroid feature extraction step. Each step is implemented based on an independent hardware circuit, and each level of hardware circuit processes image data of different frames in parallel.

[0016] In the system described, the hard real-time task unit includes a path planning unit, a vibration suppression unit, and a real-time control unit. The path planning unit generates macro-motion trajectory planning parameters, and the vibration suppression unit generates vibration suppression compensation parameters. The outputs of both the path planning unit and the vibration suppression unit are connected to the input of the real-time control unit. The output of the real-time control unit is connected to the macro-motion control terminal of the drive execution module to output macro-motion control commands. The soft real-time task unit includes a fault diagnosis unit and a low-speed calculation unit. The fault diagnosis unit executes equipment operating status detection and fault diagnosis logic, and the low-speed calculation unit executes parameter drift compensation calculations. The inputs of both the fault diagnosis unit and the low-speed calculation unit interact with the FPGA hardware acceleration module via a high-speed bus to obtain equipment operating status data. The outputs of both the fault diagnosis unit and the low-speed calculation unit are connected to the correction input of the hard real-time task unit to correct motion control parameters.

[0017] In the system described, the drive execution module includes a stepper motor drive unit, a piezoelectric ceramic drive unit, and a light source drive unit. The stepper motor drive unit is used to implement coarse focus adjustment, and its control terminal is connected to the output terminal of the hard real-time task unit. The piezoelectric ceramic drive unit is used to implement nanometer-level fine focus adjustment, and its control terminal is connected to the drive output terminal of the FPGA hardware acceleration module. The light source drive unit is used to regulate the output power of the focusing laser light source, and its control terminal is connected to the output terminal of the ARM main control scheduling module.

[0018] In the system described, three levels of isolated clock domains are defined based on the real-time performance level, response cycle requirements, and interrupt priority of the tasks: a high-speed clock domain, a medium-speed clock domain, and a low-speed clock domain. The high-speed clock domain is deployed in the FPGA hardware acceleration module and is used to carry the highest-priority hard real-time tasks, including image acquisition, feature extraction, matrix operations, and drive output. The medium-speed clock domain is deployed in the hard real-time task unit of the ARM main control scheduling module and is used to carry the second-priority hard real-time tasks, including motion path planning, vibration suppression, and real-time control. The low-speed clock domain is deployed in the soft real-time task unit of the ARM main control scheduling module and is used to carry the low-priority soft real-time tasks, including parameter drift compensation and fault diagnosis.

[0019] Beneficial effects: This invention provides an autofocus system based on a heterogeneous architecture of FPGA and ARM. Employing a hierarchical heterogeneous structure, it offloads high-real-time image preprocessing, feature computation, and micro-motion drive tasks to the FPGA hardware. Simultaneously, motion control tasks are split according to real-time requirements, distributed across the ARM's hard real-time and soft real-time units, achieving decoupling of hardware and software real-time tasks and rational allocation of computing resources. Under this architecture, high-speed image data and sensor data can be directly accessed for preprocessing on the FPGA without passing through the ARM intermediary. Combined with the FPGA's direct-connected micro-motion drive output channel, single-frame image processing and closed-loop response latency can be compressed to the microsecond level, matching a 6kHz high-frequency sampling rate. This effectively solves the problems of traditional single-ARM architectures, such as operating system scheduling jitter and excessive focusing latency, as well as the shortcomings of pure FPGA architectures, such as the lack of master control scheduling and inability to handle background management tasks. Furthermore, the physical isolation design of the macro-motion and micro-motion drive links avoids resource contention between control signals of different priorities. Combined with the hierarchical task scheduling mechanism, this improves the system's long-term stability and anti-interference capabilities, adapting to the continuous operation requirements of industrial scenarios. Attached Figure Description

[0020] Figure 1 The structural block diagram of the autofocus system based on FPGA and ARM heterogeneous architecture provided by the present invention; Figure 2 This is a block diagram of the internal structure of the FPGA hardware acceleration module provided by the present invention.

[0021] Key component symbols: 1-Sensing and detection module, 2-FPGA hardware acceleration module, 21-Image access unit, 22-Data access unit, 23-Ping-pong buffer unit, 24-Image processing core unit, 25-Arithmetic acceleration core unit, 26-Control and arithmetic unit, 27-Drive signal output unit, 28-DDR memory unit, 3-ARM main control scheduling module, 31-Hard real-time task unit, 32-Soft real-time task unit, 33-Interactive communication management unit, 4-Drive execution module, 5-Upper-level interactive module. Detailed Implementation

[0022] This invention provides an autofocus system based on a heterogeneous architecture of FPGA and ARM. To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention.

[0023] In the description of this invention, it should be understood that the terms "middle," "inner side," "outer side," etc., indicate the orientation or positional relationship of this invention based on the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0024] like Figure 1 and Figure 2 As shown, this invention provides an autofocus system based on a heterogeneous architecture of FPGA and ARM, including a sensing and detection module 1, an FPGA hardware acceleration module 2, an ARM main control scheduling module 3, and a drive execution module 4. The sensing and detection module 1 is used to collect optical image data and position sensing data required for the focusing process, and the output of the sensing and detection module 1 is connected to the acquisition input of the FPGA hardware acceleration module 2. The FPGA hardware acceleration module 2 is used to preprocess the input acquired data, and the FPGA hardware acceleration module 2 is interconnected with the ARM main control scheduling module 3 through a high-speed bus. The drive output of the FPGA hardware acceleration module 2 is connected to the micro-motion control of the drive execution module 4. The ARM main control scheduling module 3 includes a hard real-time task unit 31 and a soft real-time task unit 32. The hard real-time task unit 31 is used to perform motion path planning and vibration suppression calculations, and the output of the hard real-time task unit 31 is connected to the macro-motion control of the drive execution module 4. The soft real-time task unit 32 is used to perform parameter drift compensation and equipment fault diagnosis.

[0025] In this embodiment, the system can be logically divided into a three-layer structure: a peripheral layer, an FPGA hardware acceleration layer, and an ARM main control layer. This structure is designed to decouple tasks based on their real-time performance level and functional attributes, enabling hardware-based high-real-time tasks and software-based medium- and low-real-time tasks. Each layer interacts via a dedicated bus, reducing functional coupling and improving hardware scalability. Furthermore, the ARM main control scheduling module 3 adopts a SoC (System-on-a-Chip) architecture. This means the ARM main control scheduling module 3 uses an ARM processor core as its core and integrates peripheral interface resources such as AXI / PCIe high-speed on-chip bus interfaces, DDR memory controllers, Ethernet interfaces, and SPI. It can also be configured with processor cores of different running priorities to respectively handle hard real-time control tasks and soft real-time management tasks. The ARM SoC architecture improves system integration, reduces the number of peripheral discrete components, and lowers hardware complexity. Simultaneously, low-latency data interaction with the FPGA hardware acceleration module 2 via the on-chip high-speed bus ensures orderly scheduling and efficient interaction of tasks with different priorities, adapting to the hierarchical control requirements of the autofocus system.

[0026] In actual operation, the optical image data and position sensing data collected by the sensing and detection module 1 are directly transmitted to the FPGA hardware acceleration module 2. After data preprocessing and feature calculation, the FPGA hardware acceleration module 2 generates micro-motion control quantities, which are directly output to the micro-motion control terminal of the drive execution module 4. At the same time, the FPGA hardware acceleration module 2 interacts with the ARM main control scheduling module 3 through the high-speed on-chip bus. The hard real-time task unit 31 on the ARM side generates macro-motion control quantities and outputs them to the macro-motion control terminal of the drive execution module 4. The soft real-time task unit 32 completes background tasks such as parameter compensation and fault diagnosis.

[0027] The autofocus system based on a heterogeneous architecture of FPGA and ARM provided by this invention adopts a hierarchical heterogeneous structure. By offloading high-real-time image preprocessing, feature calculation, and micro-motion driving tasks to the FPGA hardware, and splitting motion control tasks according to real-time level to the hard real-time and soft real-time units of the ARM, the decoupling of hard and soft real-time tasks and the rational allocation of computing resources are achieved. Under this architecture, high-speed image data and sensor data can be directly connected to the FPGA for preprocessing without going through the ARM. With the micro-motion drive output channel directly connected to the FPGA, the single-frame image processing and closed-loop response latency can be compressed to the microsecond level, matching a 6kHz high-frequency sampling rate. This effectively solves the defects of traditional single-ARM architecture, such as operating system scheduling jitter and excessive focusing latency, as well as the lack of master control scheduling and inability to handle background management tasks in pure FPGA architecture. At the same time, the physical isolation design of the macro-motion and micro-motion drive links avoids resource contention between control signals of different priorities. Combined with the hierarchical task scheduling mechanism, it improves the long-term stability and anti-interference capability of the system, and adapts to the continuous operation requirements in industrial scenarios.

[0028] In some embodiments, the system further includes a host interaction module 5, and the ARM main control scheduling module 3 further includes an interactive communication management unit 33; the interactive communication management unit 33 is interconnected with the hard real-time task unit 31 and the soft real-time task unit 32 respectively; the interactive communication management unit 33 establishes a connection with the host interaction module 5 through a communication link, and is used to coordinate the data transmission, protocol parsing and instruction distribution between the ARM main control scheduling module 3 and the host interaction module 5.

[0029] In this embodiment, the host interaction module 5 includes a human-machine interface unit, a data storage unit, and a remote command issuing unit. The data storage unit can use industrial-grade solid-state storage media, SD card storage modules, etc., to store operation logs, detection data, and configuration parameters. The remote command issuing unit can use host computer configuration software, industrial touch screen control interface, etc., to issue focusing parameters, control commands, and calibration commands. The interactive communication management unit 33 can use an Ethernet MAC controller, on-chip bus arbitrator, etc., to coordinate data transmission, protocol parsing, and command distribution between the ARM main control scheduling module 3 and the host interaction module 5. In actual operation, all host computer interactive data is uniformly scheduled and distributed by the interactive communication management unit 33. Non-real-time services do not directly intervene in the hard real-time control loop, which can reduce the interference of non-real-time data transmission on the hard real-time control timing. At the same time, it realizes system parameter configuration, status monitoring, and data backtracking functions, improving the maintainability and ease of use of the system.

[0030] In some embodiments, the sensing and detection module 1 includes an image acquisition unit and a sensor data acquisition unit; the image acquisition unit is used to acquire optical image data required for the focusing process, and its output terminal is connected to the acquisition input terminal of the FPGA hardware acceleration module 2; the sensor data acquisition unit is used to acquire position sensing data required for the focusing process, and its output terminal is connected to the acquisition input terminal of the FPGA hardware acceleration module 2.

[0031] In this embodiment, the image acquisition unit includes an imaging optical component and an image sensor, used to acquire optical image data required for the focusing process; the sensing data acquisition unit includes a laser sensor and a grating ruler position encoder. The laser sensor is used to acquire position offset data of the laser beam, and the grating ruler position encoder is used to acquire real-time position data of the motion mechanism. Together, they provide multi-dimensional feedback for closed-loop focusing; the image data and sensing data are acquired in parallel through independent links and synchronously input into the corresponding access unit of the FPGA hardware acceleration module 2, providing multi-source data support for defocus calculation. That is, the multi-source data required for defocus calculation are acquired synchronously and transmitted independently, which can reduce mutual interference of the acquisition links and improve the accuracy of defocus calculation and data acquisition efficiency.

[0032] In some embodiments, the image acquisition unit includes a first imaging optical path, a second imaging optical path, and an optical path switching unit. The first imaging optical path is used to detect low-reflectivity workpieces, and the second imaging optical path is used to detect high-reflectivity, large-angle workpieces. The acquisition output terminals of the first imaging optical path and the second imaging optical path are both connected to the input terminal of the optical path switching unit, and the output terminal of the optical path switching unit is correspondingly connected to the acquisition input terminal of the FPGA hardware acceleration module 2.

[0033] In this embodiment, the first imaging optical path includes a laser, a rectangular mask, an apochromatic microscope objective, a tube mirror, and a CCD image sensor. The rectangular mask encodes the laser beam, acquiring spot centroid shift data, adapting to low-reflectivity workpiece inspection scenarios. The second imaging optical path includes a laser, two sets of orthogonal cylindrical mirrors, and a linear CCD image sensor. This imaging optical path modulates the circular laser beam into a linear spot, calculating the defocus amount through peak intensity shift, adapting to high-reflectivity, large-angle workpiece inspection scenarios. The optical path switching unit supports both manual configuration and automatic switching modes. The preset threshold for the automatic switching mode includes... The system features a light spot contrast threshold range of 30:1 to 50:1, an edge distortion rate threshold range of 5% to 10%, and a light intensity uniformity threshold range of 80% to 85%. When the real-time detected light spot characteristic parameters exceed the corresponding threshold range, the system determines that the imaging quality of the current optical path is insufficient and automatically switches to another optical path. Both optical paths share the back-end FPGA image processing logic. By setting two imaging optical paths, the system's adaptability to different working conditions can be broadened. This ensures high-precision focusing on conventional low-reflectivity workpieces while also being compatible with the detection requirements of high-reflectivity and large-angle workpieces, reducing imaging distortion and calculation errors of a single optical path under special working conditions.

[0034] In some implementations, such as Figure 2 As shown, the FPGA hardware acceleration module 2 includes an image access unit 21 and a data access unit 22. The acquisition input terminal of the image access unit 21 is connected to the output terminal of the image acquisition unit, and the image access unit 21 is used to receive optical image data and complete interface protocol parsing and data format conversion. The acquisition input terminal of the data access unit 22 is connected to the output terminal of the sensor data acquisition unit, and the data access unit 22 is used to receive position sensor data and complete timing synchronization and data verification.

[0035] In this embodiment, the image access unit 21 includes a CoaXPress / CameraLink interface circuit and a data bit width conversion logic circuit, used to receive optical image data and complete interface protocol parsing and data format conversion; the data access unit 22 includes a multi-channel synchronous acquisition circuit and a data verification logic circuit, used to receive position sensing data and complete timing synchronization and data verification; when the image access unit 21 receives optical image data, it writes it to the subsequent buffer unit after interface parsing and format conversion; when the data access unit 22 receives sensing data, it writes it to the subsequent buffer unit after multi-channel synchronous alignment and data verification. The two types of data are preprocessed through independent channels to reduce mutual interference between different data links and ensure the data quality and timing consistency of subsequent operations.

[0036] In some implementations, such as Figure 2 As shown, the FPGA hardware acceleration module 2 further includes a ping-pong buffer unit 23, an image processing core unit 24, a computation acceleration core unit 25, a control computation unit 26, a drive signal output unit 27, and a DDR storage unit 28. The output terminals of the image access unit 21 and the data access unit 22 are both connected to the input terminal of the ping-pong buffer unit 23. The ping-pong buffer unit 23 is used to buffer acquired data through a dual-buffer read-write parallel mechanism. The first output terminal of the ping-pong buffer unit 23 is connected to the input terminal of the image processing core unit 24. The image processing core unit 24 is used to extract feature data from the spot image. The second output terminal of the ping-pong buffer unit 23 is connected to the input terminal of the computation acceleration core unit 25. Unit 25 is used to independently complete filtering operations and defocus mapping matrix calculations; the output terminals of the image processing core unit 24 and the computation acceleration core unit 25 are both connected to the input terminal of the control computation unit 26. The control computation unit 26 is used to combine feature data and sensor data to calculate the defocus amount and generate corresponding drive control quantities; the output terminal of the control computation unit 26 is connected to the input terminal of the drive signal output unit 27, and the output terminal of the drive signal output unit 27 is connected to the micro-motion control terminal of the drive execution module 4 to output micro-motion control pulse signals; the DDR storage unit 28 is connected to the image processing core unit 24 and the computation acceleration core unit 25 respectively, and is used to store processed image copy data and intermediate data during the computation process.

[0037] In this embodiment, the ping-pong cache unit 23 includes two independent FPGA on-chip BlockRAM storage areas, which cache the acquired data through a parallel mechanism of alternating read and write operations; the image processing core unit 24, the computation acceleration core unit 25, and the control computation unit 26 are all hardware acceleration logic IP cores implemented inside the FPGA using a hardware description language, respectively responsible for spot feature extraction, matrix operations, and defocus calculation; the DDR storage unit 28 includes external DDR3 / DDR4 SDRAM chips, used to store processed image copy data and intermediate data during the computation process; in actual operation, the received image data and sensor data are written into one of the storage areas of the ping-pong cache unit 23. In the storage area, another set of storage areas simultaneously outputs data to the processing unit, enabling parallel execution of read and write operations; the image processing core unit 24 extracts the spot feature parameters, and the computation acceleration core unit 25 performs filtering operations and defocus mapping calculations in parallel, realizing the parallel execution of image operations and matrix operations, and controlling the total processing delay of a single frame image to within 18μs; the two operation results are fed into the control operation unit 26 to generate drive control quantities, which are converted into PWM pulse signals by the drive signal output unit 27 and then output to the micro-motion drive end; the processed image copy and intermediate operation data are written to the DDR storage unit 28 for the ARM main control scheduling module 3 to read as needed, ensuring the continuity of data transmission and the real-time performance of drive output.

[0038] In some implementations, the image processing core unit 24 is used to complete image processing based on a multi-level pipelined parallel mechanism. The image processing flow is divided into multiple serial processing steps, including an image receiving step, a noise reduction preprocessing step, a ROI spot cropping step, and a grayscale centroid feature extraction step. Each step is implemented based on an independent hardware circuit, and each level of hardware circuit processes image data of different frames in parallel.

[0039] In this embodiment, the image receiving step is used to complete the bit width alignment and frame synchronization detection of image data to ensure the timing consistency of subsequent processing; the noise reduction preprocessing step is used to remove image noise through Gaussian filtering or median filtering logic to improve the signal-to-noise ratio of the spot features; the ROI spot cropping step is used to extract the effective area of ​​the spot in the image, remove invalid background areas, and reduce the amount of data for subsequent calculations; the gray-level centroid feature extraction step is used to calculate the gray-level centroid coordinates of the spot as the core feature basis for defocus calculation; the independent hardware circuit refers to a dedicated computing circuit with independent logic resources and wiring channels configured separately for each step inside the FPGA. Each level of the circuit is driven by a synchronous clock, and the data is passed step by step in pipeline order. Each level can process the corresponding step data of different frames simultaneously; by splitting the complete image processing flow into multiple independent serial subtasks, each subtask is executed by a dedicated hardware circuit, forming a pipelined parallel processing mode, which can improve the throughput efficiency of image processing in continuous sampling scenarios, reduce the effective processing latency of single frame data, match the 6kHz high-frequency image sampling beat, and alleviate the latency accumulation problem caused by serial processing.

[0040] In some embodiments, the hard real-time task unit 31 includes a path planning unit, a vibration suppression unit, and a real-time control unit; the path planning unit is used to generate macro-motion trajectory planning parameters, the vibration suppression unit is used to generate vibration suppression compensation parameters, and the output terminals of the path planning unit and the vibration suppression unit are both connected to the input terminal of the real-time control unit; the output terminal of the real-time control unit is connected to the macro-motion control terminal of the drive execution module 4 for outputting macro-motion control commands; the soft real-time task unit 32 includes a fault diagnosis unit and a low-speed calculation unit; the fault diagnosis unit is used to execute equipment operating status detection and fault diagnosis logic, and the low-speed calculation unit is used to execute parameter drift compensation calculations; the input terminals of the fault diagnosis unit and the low-speed calculation unit interact with the FPGA hardware acceleration module 2 through a high-speed bus to obtain equipment operating status data; the output terminals of the fault diagnosis unit and the low-speed calculation unit are both connected to the correction input terminal of the hard real-time task unit 31 for correcting motion control parameters.

[0041] In this embodiment, a hard real-time task refers to a task with a response period requirement of ≤1ms, which must ensure timing accuracy and cannot be interrupted by low-priority tasks; a soft real-time task refers to a task with a response period requirement of ≤5ms, which allows for limited timing jitter and can be preempted by a hard real-time task. The path planning unit is used to implement a trajectory planning algorithm for S-curve or trapezoidal velocity curves to generate macro motion trajectory planning parameters. The vibration suppression unit is used to implement a feedforward vibration suppression algorithm to generate vibration suppression compensation parameters. The real-time control unit is used to implement PID closed-loop control logic to output macro motion control commands. The fault diagnosis unit is used to execute equipment operating status detection and fault diagnosis logic. The low-speed calculation unit is used to perform parameter drift compensation calculations such as temperature drift and zero-point drift. The aforementioned trajectory planning algorithm, vibration suppression algorithm, PID control logic, and parameter drift compensation algorithm are all well-known conventional algorithms in the field of industrial motion control. In actual operation, the device status data uploaded by the FPGA hardware acceleration module 2 is input to the fault diagnosis unit and the low-speed calculation unit respectively. The calculated correction parameters are output to the hard real-time task unit 31 to correct the path planning and control parameters. The real-time control unit outputs the final control command to the macro-motion drive unit, which can realize parameter compensation and device status monitoring while ensuring the real-time performance of macro-motion control, taking into account both control accuracy and system reliability.

[0042] In some embodiments, the drive execution module 4 includes a stepper motor drive unit, a piezoelectric ceramic drive unit, and a light source drive unit; the stepper motor drive unit is used to implement coarse focus adjustment, and its control terminal is connected to the output terminal of the hard real-time task unit 31; the piezoelectric ceramic drive unit is used to implement nanometer-level fine focus adjustment, and its control terminal is connected to the drive output terminal of the FPGA hardware acceleration module 2; the light source drive unit is used to regulate the output power of the focusing laser light source, and its control terminal is connected to the output terminal of the ARM main control scheduling module 3.

[0043] In this embodiment, the stepper motor drive unit includes a stepper motor driver and a stepper motor body, with a cross roller guide mechanism, a stroke of 30mm, and a positioning accuracy of ±1μm, used to achieve a wide range of coarse focus adjustment; the piezoelectric ceramic drive unit includes a piezoelectric ceramic drive power supply and a piezoelectric ceramic micro-motion platform, a stroke of 65μm, a displacement resolution of 2.5nm, and an idle resonant frequency of 5000Hz, used to achieve nanometer-level high-precision fine focus adjustment; the light source drive unit includes a laser constant current drive circuit, used to regulate the output power of the focusing laser light source; in actual operation, the micro-motion PWM control signal output by the FPGA hardware acceleration module 2 is directly input to the piezoelectric ceramic drive unit, the macro-motion control signal output by the ARM main control scheduling module 3 is input to the stepper motor drive unit, and the power control signal output by the ARM main control scheduling module 3 is input to the light source drive unit. The three control links are independent of each other and complete the corresponding drive control respectively, which can take into account the coordinated control of large stroke coarse focus and nanometer-level fine focus, reduce signal interference between different drive links, and achieve an overall focusing accuracy of ±1.2μm, while ensuring the stable and controllable output power of the laser light source.

[0044] In some implementations, the system is divided into three isolated clock domains based on the real-time level, response cycle requirements, and interrupt priority of the tasks: a high-speed clock domain, a medium-speed clock domain, and a low-speed clock domain. The high-speed clock domain is deployed in the FPGA hardware acceleration module 2 and is used to carry the highest priority hard real-time tasks, including image acquisition, feature extraction, matrix operations, and drive output. The medium-speed clock domain is deployed in the hard real-time task unit 31 of the ARM main control scheduling module 3 and is used to carry the second-priority hard real-time tasks, including motion path planning, vibration suppression, and real-time control. The low-speed clock domain is deployed in the soft real-time task unit 32 of the ARM main control scheduling module 3 and is used to carry the low-priority soft real-time tasks, including parameter drift compensation and fault diagnosis.

[0045] In this embodiment, the high-speed clock domain has a main frequency of 100MHz and a clock period of 10ns, corresponding to a task execution period of ≤18μs. It carries the highest priority hard real-time tasks such as image acquisition, feature extraction, matrix operations, and drive output, triggered by the hardware frame synchronization signal, and cannot be preempted by lower priority tasks. The medium-speed clock domain corresponds to a task execution period of 1ms, carrying secondary priority hard real-time tasks such as motion path planning, vibration suppression, and real-time control, and is scheduled for execution only between image frames. The low-speed clock domain corresponds to a task execution period of 5ms, carrying low priority soft real-time tasks such as parameter drift compensation and fault diagnosis, and can be preempted by higher priority tasks. The three clock domains operate independently of each other. The system uses three layers of means—physical isolation between the clock source and clock tree, logical resource routing partition constraints, and interface boundary limitations—to achieve real-time operation from the physical layer to the logical layer. The system implements timing and resource isolation across three clock domains: high-speed, medium-speed, and low-speed. For hard real-time cross-domain interaction between the high-speed and medium-speed clock domains, a two-level register synchronization structure is used to suppress metastability in single-bit control signals. Multi-bit batch data is implemented using an asynchronous FIFO based on Gray code pointers. A handshake protocol is added to real-time control data to ensure reliability. Configuration parameter data uses a register group plus pulse synchronizer scheme. For cross-domain interaction between the medium-speed and low-speed clock domains, shared memory plus mailbox interrupt mechanism is used for transmitting status and log data from medium speed to low speed. Compensation and correction parameters are transmitted from low speed to medium speed using register mapping plus a handshake protocol, executed only between image frames. Simultaneously, mechanisms such as metastability suppression, CRC data verification, and FIFO level / flow control are implemented to ensure the security and accuracy of cross-domain data interaction.

[0046] It is understood that those skilled in the art can make equivalent substitutions or modifications to the technical solution and inventive concept of the present invention, and all such substitutions or modifications should fall within the protection scope of the appended claims.

Claims

1. An autofocus system based on a heterogeneous architecture of FPGA and ARM, characterized in that, The system includes a sensing and detection module, an FPGA hardware acceleration module, an ARM main control scheduling module, and a drive execution module. The sensing and detection module acquires optical image data and position sensing data required for the focusing process, and its output is connected to the acquisition input of the FPGA hardware acceleration module. The FPGA hardware acceleration module preprocesses the acquired data and communicates with the ARM main control scheduling module via a high-speed bus. Its drive output is connected to the micro-motion control of the drive execution module. The ARM main control scheduling module includes a hard real-time task unit and a soft real-time task unit. The hard real-time task unit performs motion path planning and vibration suppression calculations, and its output is connected to the macro-motion control of the drive execution module. The soft real-time task unit performs parameter drift compensation and equipment fault diagnosis.

2. The system according to claim 1, characterized in that, It also includes a host interaction module, and the ARM main control scheduling module further includes an interaction communication management unit; the interaction communication management unit is interconnected with the hard real-time task unit and the soft real-time task unit respectively; the interaction communication management unit and the host interaction module are connected through a communication link, and are used to coordinate the data transmission, protocol parsing and instruction distribution between the ARM main control scheduling module and the host interaction module.

3. The system according to claim 1, characterized in that, The sensing and detection module includes an image acquisition unit and a sensor data acquisition unit; the image acquisition unit is used to acquire optical image data required for the focusing process, and its output terminal is connected to the acquisition input terminal of the FPGA hardware acceleration module; the sensor data acquisition unit is used to acquire position sensing data required for the focusing process, and its output terminal is connected to the acquisition input terminal of the FPGA hardware acceleration module.

4. The system according to claim 3, characterized in that, The image acquisition unit includes a first imaging optical path, a second imaging optical path, and an optical path switching unit. The first imaging optical path is used to detect low-reflectivity workpieces, and the second imaging optical path is used to detect high-reflectivity, large-angle workpieces. The acquisition output terminals of the first imaging optical path and the second imaging optical path are both connected to the input terminal of the optical path switching unit, and the output terminal of the optical path switching unit is correspondingly connected to the acquisition input terminal of the FPGA hardware acceleration module.

5. The system according to claim 3, characterized in that, The FPGA hardware acceleration module includes an image access unit and a data access unit. The acquisition input terminal of the image access unit is connected to the output terminal of the image acquisition unit. The image access unit is used to receive optical image data and complete interface protocol parsing and data format conversion. The acquisition input terminal of the data access unit is connected to the output terminal of the sensor data acquisition unit. The data access unit is used to receive position sensor data and complete timing synchronization and data verification.

6. The system according to claim 5, characterized in that, The FPGA hardware acceleration module further includes a ping-pong buffer unit, an image processing core unit, a computation acceleration core unit, a control computation unit, a drive signal output unit, and a DDR memory unit. The output terminals of the image access unit and the data access unit are both connected to the input terminal of the ping-pong buffer unit, which is used to buffer acquired data through a dual-buffer read-write parallel mechanism. The first output terminal of the ping-pong buffer unit is connected to the input terminal of the image processing core unit, which is used to extract feature data from the spot image. The second output terminal of the ping-pong buffer unit is connected to the input terminal of the computation acceleration core unit, which is used to independently complete... The system performs filtering operations and defocus mapping matrix calculations. The outputs of both the image processing core unit and the computation acceleration core unit are connected to the input of the control operation unit. The control operation unit calculates the defocus amount by combining feature data and sensor data, and generates corresponding drive control quantities. The output of the control operation unit is connected to the input of the drive signal output unit, and the output of the drive signal output unit is connected to the micro-motion control terminal of the drive execution module to output micro-motion control pulse signals. The DDR storage unit is connected to both the image processing core unit and the computation acceleration core unit, and stores processed image copy data and intermediate data from the computation process.

7. The system according to claim 6, characterized in that, The image processing core unit is used to complete image processing based on a multi-level pipeline parallel mechanism. The image processing flow is divided into multiple serial processing steps, including image receiving step, noise reduction preprocessing step, ROI spot cropping step, and grayscale centroid feature extraction step. Each step is implemented based on an independent hardware circuit, and the hardware circuits at each level process image data of different frames in parallel.

8. The system according to claim 1, characterized in that, The hard real-time task unit includes a path planning unit, a vibration suppression unit, and a real-time control unit. The path planning unit is used to generate macro motion trajectory planning parameters, and the vibration suppression unit is used to generate vibration suppression compensation parameters. The output terminals of the path planning unit and the vibration suppression unit are both connected to the input terminal of the real-time control unit. The output terminal of the real-time control unit is connected to the macro motion control terminal of the drive execution module and is used to output macro motion control commands. The soft real-time task unit includes a fault diagnosis unit and a low-speed computing unit; The fault diagnosis unit is used to perform equipment operation status detection and fault diagnosis logic, and the low-speed calculation unit is used to perform parameter drift compensation calculation. The input terminals of the fault diagnosis unit and the low-speed calculation unit interact with the FPGA hardware acceleration module through a high-speed bus to obtain equipment operation status data. The output terminals of the fault diagnosis unit and the low-speed calculation unit are connected to the correction input terminal of the hard real-time task unit to correct motion control parameters.

9. The system according to claim 1, characterized in that, The drive execution module includes a stepper motor drive unit, a piezoelectric ceramic drive unit, and a light source drive unit. The stepper motor drive unit is used to implement coarse focus adjustment, and its control terminal is connected to the output terminal of the hard real-time task unit. The piezoelectric ceramic drive unit is used to implement nanometer-level fine focus adjustment, and its control terminal is connected to the drive output terminal of the FPGA hardware acceleration module. The light source drive unit is used to regulate the output power of the focusing laser light source, and its control terminal is connected to the output terminal of the ARM main control scheduling module.

10. The system according to claim 1, characterized in that, The system is divided into three levels of isolated clock domains based on the real-time level, response cycle requirements, and interrupt priority of the tasks: a high-speed clock domain, a medium-speed clock domain, and a low-speed clock domain. The high-speed clock domain is deployed in the FPGA hardware acceleration module and is used to carry the highest priority hard real-time tasks, including image acquisition, feature extraction, matrix operations, and drive output. The medium-speed clock domain is deployed in the hard real-time task unit of the ARM main control scheduling module and is used to carry the second-priority hard real-time tasks, including motion path planning, vibration suppression, and real-time control. The low-speed clock domain is deployed in the soft real-time task unit of the ARM main control scheduling module and is used to carry the low-priority soft real-time tasks, including parameter drift compensation and fault diagnosis.

Images