FPGA scan image frame average processing method and system based on recursive average and ping-pong prefetching

Abstract

The application discloses an FPGA scanning image frame average processing method and system based on recursive average and ping-pong prefetching, and belongs to the field of electron microscope scanning imaging and FPGA signal processing. Scanning waveform generation, multi-channel signal acquisition and three-stage cascade average processing are integrated in FPGA programmable logic. In view of the defect that N frame data needs to be stored in the existing frame average, recursive average is used to update the frame data in situ, only one frame average value is stored in the external memory, the maximum image size is not limited by the average number, working cache and standby cache are set and used alternately, the current frame is written into the external memory and is simultaneously pushed to the standby cache for the next frame to be directly read, so that the inter-frame delay is eliminated, point-level average uses a register, line-level average uses on-chip memory, and frame-level average uses external memory, storage consumption does not increase with the average number, and real-time frame-by-frame preview is supported.

Description

Technical Field

[0001] This invention relates to the fields of electron microscope scanning imaging and FPGA signal processing technology, and in particular to a method and system for averaging FPGA scan image frames based on recursive averaging and ping-pong prefetching. Background Technology

[0002] Scanning electron microscopy (SEM) is an important high-resolution imaging tool in materials science, semiconductor manufacturing, biomedicine, and other fields. It uses a scanning generator to produce a deflection signal that drives an electron beam to scan the sample surface point-by-point along a pre-set grating trajectory. Simultaneously, it acquires multi-channel signals, including secondary electrons and backscattered electrons generated by the electron beam excitation, and reconstructs the scanned image after digital processing. The accuracy and stability of the scanning waveform directly determine the spatial resolution and geometric fidelity of the image, while the throughput and signal-to-noise ratio of the signal processing link determine the final image quality.

[0003] In scanning electron microscopy (SEM) imaging, the signal-to-noise ratio (SNR) of the raw image obtained from a single scan is often insufficient to meet the requirements of high-precision analysis due to the influence of multiple sources of noise, such as electron beam noise, detector noise, and amplification link noise. Therefore, in engineering practice, it is common to perform multiple repeated scans of the same field of view and average the acquired results to suppress random noise and improve the image SNR. Averaging typically involves two levels: row averaging, which averages the data from multiple repeated scans of the same scan line to suppress intra-row random noise; and frame averaging, which averages the data from multiple repeated scans of the same complete frame to further improve the overall image SNR. Especially under imaging conditions of low beam current, low accelerating voltage, or electron beam-sensitive samples, the SNR of a single frame image is extremely low, often requiring averaging of tens or even hundreds of frames to obtain image quality that meets analytical requirements.

[0004] In existing scanning electron microscope imaging systems, the mainstream approach to frame averaging is as follows: First, the full-frame image data obtained from each complete frame scan is stored frame by frame in an external DDR memory. After all N frames of data have been acquired, all N frames are read from the DDR, summed pixel by pixel, and then divided by the frame averaging frequency N to obtain the final average frame image. A significant drawback of this approach is that the DDR needs to store all N complete frame image data simultaneously. Let the data size of a single frame be S, then the DDR storage capacity required for N-frame averaging is N×S. Given a fixed total DDR capacity, the maximum supported image size is the total DDR capacity / N. This means that the frame averaging frequency is strictly inversely proportional to the maximum image size; that is, the higher the averaging frequency, the smaller the supported scan image size. For example, with a total DDR capacity of 1GB, when the frame averaging frequency is 64, the maximum data size per frame is only about 16MB, severely limiting the application requirements for high-resolution, large-field-of-view scanning imaging. Some systems mitigate this problem by using a block-based processing strategy, which involves dividing a large image into multiple sub-blocks and averaging them separately. However, this method prevents image acquisition and averaging from proceeding continuously, requiring frequent interruptions and reconfiguration of scanning parameters. This significantly increases the system's control complexity and overall imaging time, and is still fundamentally constrained by the total DDR capacity.

[0005] On the other hand, existing solutions typically employ a similar full-storage averaging method, which involves storing all row data generated by repeated row scans first, and then calculating the average value after the row repetitions are completed. This also faces the problem of on-chip storage resources being consumed linearly with the number of row repetitions, limiting the hardware feasibility under high row repetition counts.

[0006] Furthermore, existing scanning electron microscopes typically distribute scanning waveform generation, signal acquisition, and image processing across different hardware modules or processors, with each stage relying on software scheduling for coordination and synchronization. This loosely coupled architecture has the following inherent drawbacks: the timing accuracy of waveform generation is limited by software response latency and processor scheduling jitter, making it difficult to achieve nanosecond-level precision in scanning timing control; the synchronization alignment between deflection signal output and echo signal acquisition relies on software calibration, which cannot dynamically compensate for propagation delay fluctuations on the physical link; and the parallel acquisition and processing of multi-channel signals is limited by the communication bandwidth of the host computer and the software processing capabilities, making it difficult to meet the requirements of real-time high-throughput imaging. These issues collectively constrain the overall performance ceiling of the scanning imaging system.

[0007] Besides the software scheduling architecture mentioned above, there are also hardware-level attempts to use a dual-buffer structure in FPGAs for image frame processing. For example, Chinese patent application CN202110778224.5 discloses an SCMOS imaging circuit and method with in-system real-time image processing capabilities. It uses a dual DDR buffer in an FPGA to implement ping-pong storage of frame images, and achieves image difference processing by subtracting background frames from data frames line by line. In this scheme, the dual DDR buffer is used to alternately store complete data frames and background frames. Its core function is to provide the data source for inter-frame difference operations, but it does not involve frame averaging. The purpose and technical means of its ping-pong storage are different from the frame averaging processing of this invention. As another example, Chinese patent application CN201010251661.3 discloses an FPGA-based SDRAM dual-port image data read / write control system and method. Its dual-buffer structure is also only used for alternating buffering of image data, without involving recursive averaging processing. Furthermore, each frame of data overwrites the previous frame after being written, and it does not have the function of multi-frame cumulative averaging. None of the aforementioned patent documents involve a technical solution that combines a recursive averaging formula with a ping-pong prefetching mechanism for averaging scanned image frames.

[0008] Therefore, there is an urgent need for a complete solution that integrates digital scanning generation, multi-channel synchronous acquisition, and efficient image averaging processing into the same FPGA chip to overcome the aforementioned DDR memory bottleneck and on-chip resource consumption issues. At the same time, it can get rid of the timing accuracy limitations of software scheduling architecture, so that the maximum size of the scanned image is not constrained by the number of averaging operations, and provide nanosecond-level scanning timing accuracy and real-time multi-channel signal processing capabilities. Summary of the Invention

[0009] To achieve the above-mentioned objectives and other advantages of the present invention, a first objective of the present invention is to provide an FPGA scan image frame averaging processing method based on recursive averaging and ping-pong prefetching, comprising the following steps: The scanning control step involves generating a scanning waveform and a scanning position tag synchronized with the scanning waveform in the FPGA programmable logic. The scanning waveform is used to drive the electron beam to perform two-dimensional grating scanning. The acquisition and alignment step involves acquiring sampled data from multi-channel echo signals and aligning the sampled data with the scan position labels in time to obtain sampled data with position labels. The point averaging step involves accumulating the sampled data corresponding to the same scan position and dividing by the sample count to obtain the point average value for that scan position. The row averaging step, on a per-scan-line basis, uses on-chip memory to successively accumulate the average value of points corresponding to multiple repeated scans of each scan position within the same scan line. After the last repeated scan of the scan line is completed, the accumulated result is divided by the number of row repetitions to obtain the row average value. At any given time, the on-chip memory stores only the accumulated intermediate value of one scan line. The frame averaging step uses external memory to recursively update the frame average value. The frame average value of the (N+1)th frame is calculated based on the frame average value of the Nth frame and the row average value of the current frame according to the recursive averaging formula. At any given time, only one frame average value is stored in the external memory, and the frame average value of each frame is written to the same storage area in the external memory to achieve in-situ coverage. In the prefetching step, a working buffer and a backup buffer are set up. During the processing of the current frame, the average frame value of the previous frame is read from the working buffer for use in the frame averaging step. At the same time, while the average frame value of the current frame is being written to the external memory, the average frame value is synchronously written to the backup buffer. After a frame is processed, the working buffer and the backup buffer exchange roles, so that the frame averaging step of the next frame can directly read the average frame value of the previous frame from the pre-filled buffer.

[0010] Furthermore, when the current frame is the first frame, the frame averaging step directly stores the line average of the current frame as the frame average in the external memory.

[0011] Furthermore, in the row averaging step, the write enable of the on-chip memory is controlled as follows: When a row is not being scanned repeatedly, write enable is active, and the sum of the current point average value and the historical accumulated value read from the on-chip memory is written back to the same address. During the last row repetition scan, the write enable is disabled, and the accumulated result in the on-chip memory is output to the division unit for row averaging.

[0012] Furthermore, in the scanning control step, waveform data of multiple scanning sub-points and corresponding scanning position labels are generated in parallel within each clock cycle.

[0013] A second objective of this invention is to provide an FPGA scan image frame averaging processing system based on recursive averaging and ping-pong prefetching, comprising: The scanning waveform generation module, located in the FPGA programmable logic, is used to generate the scanning waveform and the scanning position label synchronized with the scanning waveform. The acquisition interface module is used to receive the sampling data acquired by the multi-channel analog-to-digital converter and align the sampling data with the scanning position label; The point averaging module is used to accumulate the sampled data corresponding to the same scan position and divide it by the sample count to output the point average value; The row averaging module includes an on-chip memory and a first division operation unit. The row averaging module accumulates the average value of points corresponding to multiple repeated scans of each scan position within the same scan line, and temporarily stores it in the on-chip memory. After the last repeated scan of the row is completed, the first division operation unit divides the accumulated result by the number of row repetitions and outputs the row average value. The frame averaging module includes a second division operation unit. The frame averaging module is used to update the frame average value according to the recursive averaging formula. The frame average value of the (N+1)th frame is calculated based on the frame average value of the Nth frame and the row average value of the current frame. The updated frame average value is written to a fixed storage area of ​​the external memory. The write address of each frame is the same to achieve in-situ overwriting. Only one frame average value is stored in the external memory at any given time. The ping-pong buffer module includes a working buffer and a backup buffer. The working buffer is used to provide the frame average value of the previous frame to the frame averaging module. The backup buffer is used to receive the synchronous writing of the frame average value while the frame average value of the current frame is written to the external memory. After a frame is processed, the working buffer and the backup buffer exchange roles. The bus interface module is used to write the frame average value output by the frame averaging module into the external memory, and to read the frame average value from the external memory into the ping-pong buffer module.

[0014] Furthermore, the system is implemented based on an FPGA SoC platform. The FPGA programmable logic contains multiple clock domains that are asynchronous with each other, including a core processing clock domain for scan control and averaging calculation, an acquisition clock domain for receiving sampled data, a waveform clock domain for waveform output, and a bus clock domain for accessing external memory. Data is exchanged between the clock domains through an asynchronous FIFO.

[0015] Furthermore, the data bit width of the on-chip memory in the row averaging module is automatically adapted at compile time according to the number of channels enabled by the system, and the storage depth of the on-chip memory is fixed, so its storage resource consumption does not change with the increase of the number of row repetitions.

[0016] Furthermore, the scanning waveform generation module includes a line scanning state machine and a frame scanning state machine. The line scanning state machine includes at least an idle state, a scanning state, and a retrace state. The frame scanning state machine includes at least an idle state and a scanning state. The line scanning state machine and the frame scanning state machine work together in a nested manner. The line scanning state machine is executed cyclically when the frame scanning state machine is in the scanning state, so as to realize the point-by-point and line-by-line output of the two-dimensional raster scanning waveform.

[0017] Furthermore, the point averaging module includes a sampling counter, which is used to count the number of valid samples corresponding to the same scanning position. When the scanning position is switched, the point averaging module sends the accumulated result and the sampling count value together into the division operation unit.

[0018] Furthermore, it also includes a gain bias correction module, which configures an independent pipeline processing unit for each acquisition channel to sequentially perform gain multiplication, bias addition and saturation clamping operations.

[0019] Furthermore, it also includes a fault detection module for monitoring FIFO overflow, FIFO empty read, AXI bus error and divider overflow events, and latching fault information and triggering pipeline drain when an anomaly is detected.

[0020] Furthermore, the scanning waveform generation module also includes an affine transformation unit, which performs affine matrix transformation on the scanning coordinates to achieve rotation and correction of the scanning pattern.

[0021] Compared with the prior art, the beneficial effects of the present invention are: This invention employs a recursive averaging method for in-situ updates of frame data. Only one frame's cumulative average data is always stored in the external memory, and each frame's write address starts from the same base address, achieving in-situ overwriting. Therefore, regardless of the frame averaging count N, the maximum supported single-frame image size remains constant at the available capacity of the external memory, rather than the available capacity / N of traditional solutions. Taking 1GB of external memory as an example, when the frame averaging count is 64, the maximum single-frame image size in the traditional solution is only about 16MB, while this invention can still support about 944MB; when the averaging count is 512, the supported capacity of this invention is 512 times that of the traditional solution. This advantage increases linearly with the averaging count, completely eliminating the constraint of the averaging count on image size. Similarly, in the row averaging step, the on-chip memory always stores only the accumulated intermediate value of one scan row, and storage resource consumption does not increase with the row repetition count.

[0022] This invention employs a ping-pong switching and replay push mechanism between the working buffer and the standby buffer. While the average value of the current frame is being written to external memory, it is simultaneously pushed to the standby buffer. At the start of the next frame, the standby buffer switches to the working buffer, and the frame averaging module can directly read the average data from the previous frame without needing to reread it from external memory via the bus. This mechanism eliminates the latency of waiting for reads to complete between frames, while significantly reducing the bandwidth usage of external memory reads, reserving all bandwidth resources for the write operation of the frame averaging result, thus improving the overall data throughput efficiency of the system.

[0023] This invention employs recursive averaging, ensuring that after each frame scan is completed, the external memory contains the complete average result up to the current frame. The processing system can read intermediate results at any point during the scanning process for real-time preview. Operators can observe the gradual improvement of the signal-to-noise ratio as the number of frames increases, promptly assess image quality, and decide whether to terminate the scan early or adjust parameters. Traditional full-capacity storage schemes require waiting for all N frames to be acquired and the summation and averaging to be completed before obtaining the final result, during which time no preview of any frame average image can be provided.

[0024] Meanwhile, all calculations for line averaging and frame averaging are completed in real time in the FPGA programmable logic in a hardware pipeline manner. The processing system only needs to read the final result image and does not need to perform a large number of pixel-level calculations and data transfers, which significantly reduces the computing pressure and software complexity of the host computer.

[0025] This invention implements scanning waveform generation, multi-channel signal acquisition, cross-clock domain tag alignment, and three-level cascaded averaging processing all within the programmable logic of a single FPGA chip. Each stage is directly coupled through hardware logic, without relying on software scheduling or coordination. Scanning waveform generation employs a parallel state machine architecture, where the waveform output timing accuracy depends solely on the FPGA core clock frequency, enabling nanosecond-level precision scanning timing control. Scan position tags and sampled data are aligned sample-by-sample in the hardware pipeline. A configurable delay compensation pipeline dynamically compensates for physical propagation delays, avoiding the limitations of traditional software calibration methods that cannot dynamically compensate for delay fluctuations. Multi-channel signal acquisition and processing are performed in parallel within the hardware pipeline, unrestricted by host computer communication bandwidth, enabling real-time high-throughput imaging.

[0026] This invention utilizes FPGA storage resources in a tiered manner based on different levels of averaging: point-level averaging uses registers, offering fast response speeds suitable for real-time accumulation of high-frequency sampling; row-level averaging uses on-chip memory, leveraging its random access capabilities and high bandwidth to support point-to-point read / write; and frame-level averaging uses external memory, utilizing its large capacity to meet the storage requirements of large-size images. These three storage levels each perform their specific functions, achieving rational allocation and efficient utilization of storage resources and avoiding the problems of over-occupancy or insufficient capacity of a single storage resource.

[0027] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0028] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 The flowchart shows the FPGA scan image frame averaging processing method based on recursive averaging and ping-pong prefetching. Figure 2 This is a block diagram of the overall architecture of a scanning electron microscope image processing system; Figure 3 This provides a complete data path from parameter configuration to image output. Figure 4 Here is a flowchart of the recursive frame averaging processing method; Figure 5 A schematic diagram of the timing sequence for prefetching and playback push in a ping-pong FIFO format; Figure 6 This is a diagram illustrating the cumulative row average BRAM. Figure 7 A comparison chart of DDR usage with existing technologies; Figure 8 This is a state transition diagram for a row scan state machine; Figure 9 This is the state transition diagram for the frame scan state machine. Detailed Implementation

[0029] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0030] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0031] The drawing numbers in this application are only used to distinguish the steps in the scheme and are not used to limit the execution order of the steps. The specific execution order is as described in the specification.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0033] To address the technical problems in existing scanning electron microscope (SEM) image frame averaging processing, such as the need to store all N frames of data, leading to a decrease in maximum image size with increasing averaging counts, and the separation of scanning, signal acquisition, and image processing stages, making tight coupling and real-time processing difficult, this invention provides an FPGA-based scanning image frame averaging processing method and system based on recursive averaging and ping-pong prefetching. This method is applicable to scanning imaging systems such as scanning electron microscopes (SEM) and focused ion beam (FIB). The specific solution is as follows: Example 1 A method for averaging FPGA scanned image frames based on recursive averaging and ping-pong prefetching, such as... Figure 1 , Figure 3 As shown, it includes the following steps: S1. Scanning control step: Generate a scanning waveform and a scanning position label synchronized with the scanning waveform in the FPGA programmable logic. The scanning waveform is used to drive the electron beam to perform two-dimensional grating scanning. In some embodiments, the scanning waveform generation module employs a nested two-layer state machine to generate a two-dimensional raster scanning waveform. For example... Figure 8 As shown, the line scan state machine contains five states: LINE_IDLE (idle, waiting for scan to start), LINE_START_DELAY (start delay, waiting for a preset setup time to allow the deflector to stabilize), LINE_SCAN (scan, outputting the X-direction deflection value point by point), LINE_BEAMRT (line retrace, shifting the X-deflection signal back to the beginning of the next line), and LINE_FRAMERT (frame retrace, shifting the X-deflection signal back to the beginning of the frame). Figure 9 As shown, the frame scan state machine contains three states: FRAME_IDLE (idle), FRAME_SCAN (scan), and FRAME_FRAMERT (frame retrace). The two state machines work together in a nested manner, orthogonally combined to control the entire two-dimensional scanning process.

[0034] After the frame scan state machine enters the scan state, the line scan state machine sequentially goes through the start delay state and the scan state. In the scan state, during forward scanning, the line direction DAC code value is generated point-by-point according to X_raw = line_offset + line_step × cnt_line_point; during reverse scanning, it is generated according to X_raw = line_offset + line_step × (last_point - cnt_line_point), where line_offset is the line direction start offset, line_step is the line direction step, cnt_line_point is the current point count, and last_point is the maximum number of points in the line direction. The frame direction DAC code value is generated line-by-line according to Y_raw = frame_offset + frame_step × cnt_frame_point, where frame_offset is the frame direction start offset, frame_step is the frame direction step, and cnt_frame_point is the current line count. The frame direction DAC code value remains unchanged within the current line. After each line scan is completed, the line scan state machine enters the line retrace state, shifting the line direction deflection signal back to the starting position of the next line. After all rows are scanned and repeated, the frame scan state machine enters the frame retrace state, shifting the row direction and frame direction deflection signals back to the frame start position, respectively.

[0035] The dwell time of each scan point is determined by both the setup time and the hold time. In scan mode, each scan point first undergoes a setup time phase, during which the deflector is stabilizing and the sampling enable signal is invalid. After the setup time is complete, the hold time phase begins, during which the deflector has stabilized at the target position, the sampling enable signal is valid, and the analog-to-digital converter (ADC) is allowed to acquire the signal. The sampling enable signal must also satisfy the condition that the current point index exceeds the number of pre-scan points. Pre-scan points are used to provide additional deflector stabilization time before entering the effective imaging area, improving the imaging quality of image edge regions.

[0036] In some embodiments, the scanning waveform generation module generates waveform data of N scanning sub-points and corresponding scanning position labels in parallel within each core clock cycle. N is a configurable parallelism parameter; in this embodiment, N is 4, but it can also be configured to be 8 or 16 depending on the ratio of the FPGA core clock frequency to the digital-to-analog converter clock frequency. The parallel-generated multi-sub-point data is output to the digital-to-analog converter after parallel-to-serial conversion to match the operating clock frequency of the digital-to-analog converter, achieving high-speed serial output. The serialized 16-bit dual-channel DAC data is latched in the DCO transmit clock domain through the IOB register in the digital-to-analog converter interface module to ensure that the setup and hold time requirements of the DAC chip are met. The ODDR primitive generates a differential clock output signal with the DAC reference clock, which, together with the data signal, drives an external dual-channel 16-bit DAC chip such as AD9747 to generate analog deflection voltages in the X and Y directions. This deflection voltage drives the scanning deflection coil or electrostatic deflection plate of the electron microscope through a high-voltage amplifier, controlling the scanning position of the electron beam on the sample surface.

[0037] In some other embodiments, the parallel-to-serial conversion module employs a double-buffered prefetch pipeline: while the current data packet is being output point by point, the next data packet has been prefetched from the asynchronous FIFO into the buffer register and is waiting to be switched, ensuring continuous and uninterrupted output.

[0038] The scan location label contains structured information corresponding to each scan sub-point, including at least the scan X coordinate, scan Y coordinate, row repetition count, sub-point index, point synchronization signal, row synchronization signal, and frame synchronization signal. The point synchronization signal is valid at the beginning of each scan point, the row synchronization signal is valid at the beginning of each row scan, and the frame synchronization signal is valid at the beginning of each frame scan. The scan coordinate information consists of logical pixel coordinates after subtracting the pre-scan point count, ensuring that image coordinates start from zero, facilitating address mapping in subsequent averaging processing.

[0039] In some embodiments, step S1 further includes an affine transformation step: performing a 2×2 affine matrix transformation on the generated original scan coordinates to achieve rotation, tilt, and skew correction of the scanned graphic. The original coordinates are first centered by subtracting the intermediate code (2^15), and then the rotated coordinates are calculated through four fixed-point multiplication and addition operations. The transformation formula is as follows: x_rotated=(coe1×x_center−coe2×y_center)>>>Q y_rotated=(coe3×x_center+coe4×y_center)>>>Q Where x_center and y_center are the centered coordinates after subtracting the intermediate code (2^15) from the original coordinates, coe1~coe4 are composed of sine, cosine and correction coefficients k1~k4, b1~b4, Q is the number of fixed-point decimal places, and >>>Q means arithmetic right shift by Q bits for decimal alignment. After the rotated coordinates are added back to the intermediate code and saturated and clamped to the effective range of the 16-bit DAC, they are output as the final DAC code value to the digital-to-analog converter.

[0040] In some embodiments, before the scan waveform is generated, a parameter configuration and task initiation step is also included: the processing system (ARM Cortex-A9) writes scan parameters to the programmable logic side through a register file mapped to block memory, with the register file address mapped to the base address 0x40000000. Key parameters include: line offset (lineOffsetVal), step size (lineStepVal), and number of points (linePointCnt); frame offset (frameOffsetVal), step size (frameStepVal), and number of points (framePointCnt); pre-scan points (preScanStepCountX) and preScanStepCountY); line retrace points (lineBeamRtPointCnt) and retrace step size (lineBeamRtStepVal); frame retrace parameters; line repetition count (lineAveragingCount); frame repetition count (frameAveragingCount); average enable (averageEn); gain and offset coefficients for each channel; rotation enable (RotationEn); and the sine, cosine, and correction coefficients k1~k4 and b1~b4 of the 2×2 affine matrix; and channel enable mask (ChannelEn).

[0041] After the parameters are written, the processing system sets bit 0 of the scanStartSingleFlag register to 1. Upon detecting the write pulse on this bit, the programmable logic side first performs a start-up condition gating check: the system is in a ready state, ADC initialization is complete, the previous task has ended, there is no fault latching, and there is no residual data clearing operation. Once all conditions are met, this bit is automatically cleared, triggering a self-reset, and simultaneously, all parameters from the register file are latched into the internal working register. After a 10-clock-cycle start delay (used for DDR address calculation and internal state initialization), a rising edge of the task_start signal is generated, officially starting the scan process. The task_start signal is simultaneously synchronized to the 100MHz AXI clock domain through two flip-flops to reset the DDR write address counter.

[0042] S2. Acquisition Alignment Step: Acquire sampling data of multi-channel echo signals, and align the sampling data with the scanning position label in time to obtain sampling data with position label; This embodiment employs two LTC2282 dual-channel 12-bit ADCs operating at a 100MHz sampling frequency, providing a total of four ADC channels. The multi-channel analog-to-digital converter synchronously acquires echo signals from various detectors (such as secondary electron detectors and backscattered electron detectors) during the scanning process. Each ADC interface module drives the ADC's sampling clock input with a 100MHz clock frequency and phase offset via ODDR primitives. The data port uses the IOB priority register (IOB=TRUE attribute) for first-stage sampling latching at 100MHz, minimizing input path delay differences. The ADC can be configured for two's complement or unsigned output mode; in two's complement mode, negative values ​​are clamped to zero. After a 10-cycle warm-up period, the data_valid signal enables valid data output.

[0043] In some embodiments, the scan location tag is first transmitted from the core processing clock domain to the analog-to-digital converter clock domain via an asynchronous FIFO, and then passes through a configurable-stage sampling delay compensation pipeline. Each stage of the delay compensation pipeline is a delay unit of one clock cycle, and the total number of stages is configured based on the measured physical propagation delay of the system. The configuration value is determined during the product factory calibration phase or written by the user during system initialization. Through this delay compensation pipeline, the location tag is delayed by a preset number of clock cycles to match the propagation delay between waveform output and echo signal acquisition.

[0044] The receiving gearbox module operates in the analog-to-digital converter (ADC) clock domain, pairing each ADC sampling point with its corresponding delayed-compensated position tag. Since the ADC sampling clock frequency is typically higher than the core processing clock frequency, after accumulating two samples, the receiving gearbox module packages multiple sampling points and their corresponding position tags into a parallel data packet, which is then transmitted to the core processing clock domain via an asynchronous FIFO, forming a multi-channel sampling data stream with precise scan position markings.

[0045] In some embodiments, after the acquisition alignment step, a gain and bias correction step is further included: the sampled data of each acquisition channel undergoes gain and bias correction through an independent image processing module. This module has a three-stage pipeline structure: the first stage multiplies the sampled value by the channel gain coefficient (fixed-point Q10 format) to achieve linear scaling of the signal amplitude; the second stage adds the multiplication result to the channel bias value to achieve DC offset compensation; the third stage saturates and clamps the result to constrain it within a preset effective data range. The gain coefficient and bias value of each channel are independently configured by the processing system through a register file.

[0046] S3, the point averaging step, involves accumulating the sampled data corresponding to the same scanning position and dividing by the sampling count to obtain the point average value of that scanning position; Since the sampling rate of an analog-to-digital converter (ADC) is typically higher than the scan point rate, the same scan coordinate position may correspond to multiple ADC sampling points within the dwell time. The point-level averaging step is used to combine these multiple sampling points into a single point-level average.

[0047] The point-level averaging module independently sets up a register-level accumulator and a sampling counter for each enabled acquisition channel. Whenever a sampled data is received and the sampling enable signal for that sample is valid, the accumulator increments the sampled value, and the sampling counter increments by one. When a change in scan coordinate is detected (i.e., the point synchronization signal is valid), a flush event is triggered. The accumulated sum in the accumulator and the count value of the sampling counter are sent to the division unit to calculate the accumulated sum divided by the sampling count, obtaining the point-level average value for that scan coordinate position. After the flush is complete, the accumulator and sampling counter are reset, and the accumulation of samples for the next scan coordinate position begins.

[0048] Since zero, one, or more valid samples may be received within each core clock cycle, depending on the ratio of the analog-to-digital converter's sampling rate to the core clock frequency, the sampling counter dynamically records the actual number of valid samples received, ensuring the accuracy of the point-level average calculation. The division unit is implemented using a hardware divider, which can be implemented in the FPGA via a dedicated DSP slice or lookup table, and has a fixed computation delay. In this embodiment, the point-level hardware divider is configured to divide a 34-bit dividend by a 26-bit divisor.

[0049] S4. The row averaging step involves accumulating the average value of points corresponding to multiple repeated scans of each scan position within the same scan line using on-chip memory, with each scan line as a unit. After the last repeated scan of the scan line is completed, the accumulated result is divided by the number of row repetitions to obtain the row average value. At any given time, the on-chip memory stores only the accumulated intermediate value of one scan line. In some embodiments, such as Figure 6 As shown, the on-chip memory uses the FPGA's internal block RAM (BRAM) configured in a dual-port mode. One port is used to write the accumulated result, and the other port is used to read the historical accumulated value. In this embodiment, the BRAM depth is 16K, which can store at least all the pixels of one scan line. The data bit width of the block RAM is automatically adapted at compile time according to the number of channels enabled by the system. For example, the bit width is 64 bits in a 4-channel configuration, 128 bits in an 8-channel configuration, 192 bits in a 12-channel configuration, and 256 bits in a 16-channel configuration.

[0050] The specific control logic for row-level averaging is as follows: The row repetition counter tracks the current row repetition scan (counting from 0 to the row repetition count minus 1); when the row repetition counter is 0 (first row repetition), the point-level average is directly written to the corresponding address in the block memory, and the written value is the point-level average itself; when the row repetition counter is 1 to the row repetition count minus 2 (intermediate row repetition), the historical accumulated value is first read from the same address in the block memory, the current point-level average is summed with the historical accumulated value, and then written back to the same address: line_buffer_data_i = point_quotient + line_buffer_data_o; when the row repetition counter is the row repetition count minus 1 (last row repetition), the write enable is turned off, no data is written to the block memory, and the accumulated result output from the block memory is directly sent to the row-level average hardware divider (16-bit dividend ÷ 9-bit divisor), divided by the row repetition count, to obtain the row-level average of the scanned row; when the row ends, the write pointer of the block memory is reset to the beginning address of the row, preparing for the accumulation of the next row.

[0051] The row average divider receives the accumulated result and the number of row repetitions from the block memory during the last row repetition, calculates the quotient of the accumulated result divided by the number of row repetitions, and outputs the row-level average. The row average divider is a hardware divider with a fixed pipeline delay, and the division operation can be pipelined in parallel with the point averaging processing of the next row. In this embodiment, the row average hardware divider is configured as a 16-bit dividend divided by a 9-bit divisor. Regardless of the value of the number of row repetitions M (from 1 to 512), the BRAM usage is always one row × channel width and does not increase with increasing M.

[0052] In the control logic described above, the write enable of the block memory is controlled by the output of the row repetition counter, and is only effective when the row is not repeated for the last time. After each row scan is completed, the write pointer of the block memory is reset to the beginning address of the row, preparing for the accumulation of the next row.

[0053] The row-level average is output to the frame-level averaging step after each row scan is completed. Since the cumulative result of the row average is only finalized after the row scan is finished, the row-level average is output row by row in a row-synchronous manner.

[0054] S5, Frame averaging step: The frame average value is recursively updated using external memory. The frame average value of the (N+1)th frame is calculated based on the frame average value of the Nth frame and the row average value of the current frame according to the recursive averaging formula. At any given time, only one frame average value is stored in the external memory, and the frame average value of each frame is written to the same storage area in the external memory to achieve in-situ coverage. like Figure 4As shown, the frame-level averaging step updates the frame average value using a recursive averaging method. The recursive averaging formula is: Avg(N+1)=(Avg(N)×N+X) / (N+1), where Avg(N) is the frame average value of the Nth frame, and X is the row-level average value of the current frame. For each pixel position in each channel, the frame averaging module performs the following recursive operation: For the (N+1)th frame currently being processed (N starts counting from 0), the frame averaging module reads the frame average of the previous frame (the Nth frame) from the working buffer, multiplies this value by N, adds the current frame's row-level average, and then divides the sum by N+1 to obtain the updated frame average of the (N+1)th frame. The mathematical equivalence of this recursive formula is proven as follows: Let the average of the previous N frames be Avg(N) = (X1 + X2 + ... + X...). n ) / N, then Avg(N)×N+X_{N+1}=X1+X2+…+X n +X_{N+1}, then divide by N+1 to get (X1+X2+…+X_{N+1}) / (N+1)=Avg(N+1). Mathematically, this recursive formula is strictly equivalent to taking the arithmetic mean of all N+1 frames of data.

[0055] When N=0, i.e., processing the first frame, the data read enable of the ping-pong prefetch buffer module is gated off, the data_unsqueeze_dout output is zero or invalid, and the recursive formula degenerates to: frame_dividend[i]=0×0+line_quotient[i]=line_quotient[i], frame_quotient[i]=line_quotient[i] / 1=line_quotient[i], that is, the line-level average of the first frame is directly used as the frame average output, which is equivalent to the recursive formula degenerating into pass-through mode.

[0056] The output of the frame average divider is processed by a data width compression module, which packages the frame average values ​​of each enabled channel into a 128-bit AXI data width and sends it to the write FIFO (ad2axi FIFO). This data is then written to external memory via the bus interface in burst write mode. The write address to external memory is reset to the same base address at the beginning of each frame, ensuring that the frame average values ​​of each frame completely overlap in physical address, with the write result of the next frame directly overwriting the result of the previous frame. The AXI write-back module maintains a write address counter, cnt_axi_waddr, initially set to the DDR base address DDR_BASE_ADDR (0x0200_0000). After each AXI burst write operation, the write address is incremented by the burst length multiplied by the number of bytes per clock cycle. At the beginning of each frame, the write address counter is reset to DDR_BASE_ADDR: cnt_axi_waddr <= DDR_BASE_ADDR. The available address range for DDR is from DDR_BASE_ADDR to DDR_MAX_ADDR (0x3D00_0000), totaling approximately 944MB, all of which is used to store single-frame image data. Only one frame of cumulative average data is stored in the external memory at any given time.

[0057] The division operation in the frame averaging module is implemented using a hardware divider. To balance resource consumption and computational accuracy, the divider can use fixed-point division, truncating or rounding the output to the effective data bit width. Specifically, for each pixel position of each channel, the intermediate variable frame_dividend[i] = data_unsqueeze_dout[i] × frame_frame_cnt + line_quotient[i], frame_quotient[i] = frame_dividend[i] / (frame_frame_cnt + 1), where data_unsqueeze_dout[i] is the average value of the i-th channel of the previous frame obtained from the ping-pong prefetch FIFO, frame_frame_cnt is the current frame counter, line_quotient[i] is the row-level average value of the current frame, and frame_quotient[i] is the updated frame-level average value. In a specific embodiment of the present invention, the divider uses a 16-bit dividend divided by a 9-bit divisor, and the quotient output is a 16-bit integer. The dividend has a sufficient bit width to accommodate the sum of the product term Avg(N)×N and the current frame line mean, ensuring no overflow during the operation. Each recursive division produces a truncation error of no more than one least significant bit (LSB), which is proportionally diluted in subsequent frame recursions and has a negligible impact on the final image quality. This recursive method trades truncation error for a reduction in DDR usage by a factor of N and the ability for real-time frame-by-frame preview.

[0058] S6. Prefetching step: Set up working buffer and standby buffer. During the processing of the current frame, read the average frame value of the previous frame from the working buffer for use in the frame averaging step. At the same time, while the average frame value of the current frame is being written to the external memory, the average frame value is synchronously written to the standby buffer. After a frame is processed, the working buffer and the standby buffer exchange roles, so that the frame averaging step of the next frame can directly read the average frame value of the previous frame from the pre-filled buffer.

[0059] The recursive averaging calculation in step S5 requires obtaining the average value of the previous frame for each frame. To avoid the latency and bandwidth consumption caused by rereading the data of the previous frame from external memory for each frame, this invention uses a ping-pong prefetching and replay push mechanism in step S6 to ensure that the data of the previous frame is already in the cache when the current frame is processed.

[0060] like Figure 5 As shown, the specific implementation of the ping-pong prefetch and playback push mechanism is as follows: Two buffer units are set up: a working buffer and a standby buffer. Each buffer unit is a FIFO buffer with a first-in, first-out characteristic. In this embodiment, the two FIFOs are 128-bit wide asynchronous FIFO buffers with configurable depth (e.g., 2048), operating in the 100MHz AXI clock domain. The roles of the two buffer units are controlled by working selection signals: role allocation is controlled by two one-bit wide selection signals. `fifo_work_sel` points to the "working FIFO" currently consumed by the 50MHz processing side, and `fifo_fetch_sel` points to the "prefetch FIFO" currently filled by the 100MHz AXI side. Initially, `fifo_work_sel` = 1, `fifo_fetch_sel` = 0, and both FIFOs are in a reset state. When a task starts or a new frame begins, both selection signals are inverted simultaneously, the FIFO is reset and released, and the roles are swapped: fifo_fetch_sel<=~fifo_fetch_sel; fifo_work_sel<=~fifo_work_sel.

[0061] During the processing of the current frame, the frame averaging module sequentially reads the frame average of the previous frame from the current working buffer. Simultaneously, while the frame average of the current frame is written to external memory via the bus interface, each time a data frame is successfully written to external memory, that data frame is simultaneously pushed to the standby buffer. The push is triggered by three conditions: the frame repetition count is greater than 1 (frame_repeat_cnt ≠ 1), meaning there are subsequent frames that need to be prefetched; the read enable of the write FIFO is valid (ad2axi_rd_en is high), meaning data is indeed being written; and the target non-working FIFO is not full (axi2ad_full[~fifo_work_sel] is low), meaning there is space to receive data. When all three conditions are met, the playback push register is set: axi2ad_replay_push <= 1; axi2ad_replay_sel <= ~fifo_work_sel; axi2ad_replay_data <= ad2axi_dout; the current frame data is copied and written to the standby buffer.

[0062] After all data for the current frame has been written to external memory, the empty working buffer is reset and cleared. At the start of the next frame, the working selection signal is inverted, and the two buffer units switch roles. The original spare buffer becomes the new working buffer, which has the complete average value of the previous frame pre-stored. The frame averaging module can directly read data from it without initiating an external memory read operation. The original working buffer becomes the new spare buffer, used to receive playback pushes of the current frame data.

[0063] Through the aforementioned ping-pong switching and replay push mechanism, starting from the second frame, the frame averaging step of each frame does not need to wait for external memory to be read, and the inter-frame processing latency is minimized.

[0064] During the scanning process, since the external memory stores the complete average result up to the current frame after each scan frame is completed, the processing system can read the current frame's average value at any time via the bus for real-time preview display. Operators can use the preview image to assess the progress of signal-to-noise ratio improvement and decide whether to continue increasing the number of frames or adjusting the scanning parameters.

[0065] The FPGA scanning image frame averaging processing method based on recursive averaging and ping-pong prefetching provided by this invention executes six processing stages in sequence in the FPGA programmable logic in a hardware pipeline manner: scanning waveform generation, multi-channel signal acquisition, point-level averaging, row-level averaging, frame-level averaging and ping-pong prefetching. Each stage is tightly coupled through data flow to form a complete processing link from scanning drive to averaged image output.

[0066] Example 2 Based on the same concept, this embodiment also provides an FPGA scan image frame averaging processing system based on recursive averaging and ping-pong prefetching. This system is used to execute the method provided in Embodiment 1. For a detailed description of the method provided in Embodiment 1, please refer to the corresponding description in the above method embodiments, which will not be repeated here.

[0067] It is understood that the FPGA scan image frame averaging processing system based on recursive averaging and ping-pong prefetching provided in this embodiment includes corresponding hardware structures and / or software modules to perform each function in order to achieve the above-mentioned functions. Combining the units and algorithm steps of the various examples disclosed in this embodiment, this embodiment can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solution of this embodiment.

[0068] An FPGA-based scanning image frame averaging processing system based on recursive averaging and ping-pong prefetching, such as Figure 2 As shown, it includes: The scanning waveform generation module, located in the FPGA programmable logic, is used to generate the scanning waveform and the scanning position label synchronized with the scanning waveform. Specifically, the scanning waveform generation module includes a line scanning state machine, a frame scanning state machine, a coordinate calculation unit, and a label generation unit. The line scanning state machine and the frame scanning state machine work together in a nested manner. The line scanning state machine executes cyclically when the frame scanning state machine is in the scanning state, realizing point-by-point and line-by-line two-dimensional raster scanning control.

[0069] Under the control of the state machine, the coordinate calculation unit calculates the DAC code value for each scan point in the row and frame directions based on preset scan offset, step size, and number of scan points. The coordinate calculation supports both forward and reverse scan modes, which can be switched at runtime via configuration registers. The DAC code value for each scan point is output to the parallel-to-serial conversion module and simultaneously to the tag generation unit.

[0070] The tag generation unit generates a corresponding scan position tag for each scan sub-point, including scan X coordinate, scan Y coordinate, line repeat count, sub-point index, and point synchronization, line synchronization, and frame synchronization signals. The tag information and DAC code value are output synchronously to ensure the time consistency between waveform data and position information.

[0071] In some embodiments, the scan waveform generation module generates waveform data and tag information for four scan sub-points in parallel within each core clock cycle. The parallel output multi-sub-point data is expanded into a point-by-point serial stream by the parallel-to-serial conversion module and output to the digital-to-analog converter interface. The parallelism parameter can be configured at compile time to adapt to different combinations of core clock frequency and digital-to-analog converter clock frequency.

[0072] In some other embodiments, the scanning waveform generation module further includes an affine transformation unit. The original scanning coordinates output by the coordinate calculation unit are first centered by subtracting the intermediate code (2^15), and then the rotated coordinates are calculated through four fixed-point multiplication and addition operations. The transformation formula is as follows: x_rotated=(coe1×x_center−coe2×y_center)>>>Q y_rotated=(coe3×x_center+coe4×y_center)>>>Q Where x_center and y_center are the centered coordinates (original coordinates minus the intermediate code), coe1~coe4 are composed of sine, cosine, and correction coefficients k1~k4, b1~b4, and Q is the fixed-point decimal place; >>>Q indicates an arithmetic right shift of Q bits for decimal place alignment. After the rotated coordinates are added back to the intermediate code and saturated and clamped to the effective range of the 16-bit DAC, they are output as the final DAC code value to the digital-to-analog converter. This affine transformation can correct the geometric distortion of the scanning system without changing the scanning parameters.

[0073] The acquisition interface module is used to receive the sampling data acquired by the multi-channel analog-to-digital converter and align the sampling data with the scanning position label; Specifically, the acquisition interface module includes an analog-to-digital converter interface unit, a delay compensation pipeline, and a receiving gearbox unit.

[0074] The analog-to-digital converter (ADC) interface unit performs first-stage sampling and latching of the ADC's data bus through the IOB input register, ensuring that the setup and hold time requirements of the ADC's data output are met. The interface unit also includes a start delay counter, which waits for a preset number of clock cycles after the ADC is powered on or exits reset to allow the ADC's internal circuitry to stabilize before enabling valid data output.

[0075] The delay compensation pipeline resides in the analog-to-digital converter (ADC) clock domain and consists of multiple cascaded delay registers, with the total number of stages controlled by the delay compensation configuration register. The scan position tags output by the scan waveform generation module are transmitted to the ADC clock domain via an asynchronous FIFO and then enter the delay compensation pipeline. The number of delay compensation stages is configured based on the measured physical propagation delay of the system, ensuring that the position tags are time-aligned with the corresponding echo sampling data.

[0076] The receiving gearbox unit operates in the analog-to-digital converter (ADC) clock domain, pairing each ADC sampling point with a delay-compensated position tag. Since the ADC clock frequency is typically higher than the core processing clock frequency, after accumulating a preset number of samples, the receiving gearbox packages multiple sampling points and their corresponding tags into a parallel data packet, which is then transmitted to the core processing clock domain via an asynchronous FIFO. Clock domain isolation on both sides of the asynchronous FIFO ensures reliable cross-domain data transmission.

[0077] In some embodiments, the system further includes a gain bias correction module. This module configures an independent three-stage pipelined processing unit for each acquisition channel: the first stage is a fixed-point multiplication unit, used to multiply the sampled value by the channel gain coefficient to achieve linear scaling of the signal amplitude; the second stage is an addition unit, used to add the multiplication result to the channel bias value to achieve DC offset compensation; and the third stage is a saturation clamping unit, used to constrain the addition result within a preset effective data range. The gain coefficient and bias value of each channel are independently configured by the processing system through a register file.

[0078] The point averaging module is used to accumulate the sampled data corresponding to the same scan position and divide it by the sample count to output the point average value; The point averaging module is configured independently for each enabled acquisition channel. Each channel's point averaging module includes an accumulator, a sampling counter, and control logic.

[0079] The accumulator is a register-level adder. Within each core clock cycle, if there is a valid sample in the current frame and the sampling enable signal is valid, the accumulator adds the sampled value. The sampling counter synchronously adds the number of valid samples. When the point synchronization signal is valid (indicating an upcoming switch in the scan coordinates), the control logic triggers a flush operation: the output values ​​of the accumulator and the sampling counter are fed into the division unit. After the division operation is completed, the point-level average value is output, and the accumulator and sampling counter are reset.

[0080] Since the ratio of the analog-to-digital converter's sampling rate to the core clock frequency may not be an integer, the number of valid samples received in each core clock cycle may be 0, 1, or more. The point averaging module dynamically records the actual number of samples through a sampling counter, ensuring that the divisor in the division operation accurately reflects the actual number of samples and guaranteeing the accuracy of the averaging result.

[0081] In some other embodiments, the point averaging module also includes overflow detection logic. When the accumulator is about to overflow, the detection logic outputs a warning signal and can be configured to either handle saturation or trigger a fail-safe stop.

[0082] The row averaging module includes an on-chip memory and a first division operation unit. The row averaging module accumulates the average value of points corresponding to multiple repeated scans of each scan position within the same scan line, and temporarily stores it in the on-chip memory. After the last repeated scan of the row is completed, the first division operation unit divides the accumulated result by the number of row repetitions and outputs the row average value. The row averaging module includes on-chip memory, a write address generator, a read address generator, accumulation logic, and a row averaging divider.

[0083] The on-chip memory is preferably the block RAM (BRAM) inside the FPGA, configured in a dual-port mode. Port A is used to write the accumulated result, and port B is used to read the historical accumulated value. The depth of the memory is at least equal to the maximum number of pixels in a single scan line, and the data bit width is adaptively configured at compile time according to the number of channels enabled by the system. Since the on-chip memory stores only the accumulated intermediate value of one scan line at any given time, the depth of the memory does not change with the increase of the number of line repetitions.

[0084] The write address generator and read address generator increment synchronously under the control of the row synchronization signal, progressing point by point from the beginning address to the end address of the row. At each address, the read operation occurs one clock cycle in advance, reading the historical accumulated value; the write operation is executed in the current clock cycle, summing the current point-level average value with the historical accumulated value and writing it to the same address.

[0085] The accumulation logic controls the source of written data and the write enable switch based on the value of the row repetition counter. The value of the row repetition counter is updated at the beginning of each row scan. During non-last row repetitions, the write enable is active, and the written data is the sum of the current point-level average and the historical accumulated value read; during the last row repetition, the write enable is inactive, and the block memory retains the output accumulation result.

[0086] The row average divider receives the accumulated result and the number of row repetitions from the block memory during the last row repetition, calculates the quotient of the accumulated result divided by the number of row repetitions, and outputs the row-level average. The row average divider is a hardware divider with a fixed pipeline delay, and the division operation and the point averaging processing of the next row can be pipelined in parallel.

[0087] The frame averaging module includes a second division operation unit. The frame averaging module is used to update the frame average value according to the recursive averaging formula. The frame average value of the (N+1)th frame is calculated based on the frame average value of the Nth frame and the row average value of the current frame. The updated frame average value is written to a fixed storage area of ​​the external memory. The write address is the same for each frame to achieve in-situ overwriting. Only one frame average value is stored in the external memory at any given time. The frame-level recursive averaging module uses the recursive averaging formula Avg(N+1)=(Avg(N)×N+X) / (N+1) to perform in-situ recursive updates on each frame of data, where Avg(N) is the frame average value of the Nth frame and X is the row average value of the current frame.

[0088] Specifically, the frame averaging module includes a multiplier, an adder, a frame averaging divider, and control logic.

[0089] The frame averaging module processes one pixel position within each pixel clock cycle. The multiplier multiplies the average value of the previous frame output from the working buffer with the current frame counter N; the adder adds the product to the row-level average value of the current frame; and the frame averaging divider divides the sum by N+1 to output the updated frame average value.

[0090] When the frame counter N=0 (i.e. the first frame), the average value of the previous frame output by the working buffer is invalid. The control logic of the frame averaging module bypasses the multiplier and divider, and directly outputs the row-level average value as the frame average value.

[0091] The frame averaging divider and the line averaging divider can use the same hardware divider core and be shared through time-division multiplexing to save FPGA resources; alternatively, a separate divider can be configured for frame averaging to achieve full pipelined parallelism. In this embodiment, the system uses the high 8 bits of the ADC as the effective data bit width (AD_USE_DATA_WIDTH=8), and the frame averaging quotient is written to external memory after the low 8 bits are extracted by the data compression module. In some embodiments, the frame averaging module is configured with a separate divider, independent of the line averaging divider, allowing the processing of line averaging and frame averaging to be executed in pipelined parallelism.

[0092] The frame average value output by the frame averaging module is converted by data bit width and channel packing, and then written into the transmit FIFO of the bus interface module, waiting for bus burst write.

[0093] The ping-pong buffer module includes a working buffer and a backup buffer. The working buffer is used to provide the frame average value of the previous frame to the frame averaging module. The backup buffer is used to receive the synchronous writing of the frame average value while the frame average value of the current frame is written to the external memory. After a frame is processed, the working buffer and the backup buffer exchange roles. Specifically, the ping-pong caching module includes two caching units (working cache and standby cache), role selection logic, and replay push control logic.

[0094] Each buffer unit is preferably an asynchronous FIFO, operating in the bus clock domain, used to buffer the average value data of one frame. The depth of the FIFO is at least equal to the total number of pixels in one frame of the image to ensure that the average value of one frame can be stored completely.

[0095] The role selection logic maintains a working selection register, the value of which determines which buffer unit is currently the working buffer and which is the standby buffer. The working selection register is inverted at the beginning of each frame to achieve the role swapping of the two buffer units.

[0096] The playback push control logic monitors the write operations of the bus interface module. During the process of writing the average frame value to external memory, whenever the bus interface module successfully completes the writing of a data frame (write acknowledgment is valid), and the frame repetition count is greater than 1, and the spare buffer is not full, the playback push control logic simultaneously writes the data frame to the spare buffer.

[0097] After the current frame is completely written, the empty working buffer is reset and cleared. At the start of the next frame, the working selection register is inverted, and the original spare buffer is switched to the working buffer, which has the complete frame average value of the previous frame pre-stored. The frame averaging module can read it directly without initiating an external memory read operation.

[0098] The bus interface module is used to write the frame average value output by the frame averaging module into the external memory, and to read the frame average value from the external memory into the ping-pong buffer module.

[0099] Specifically, the bus interface module includes a write channel and a read channel. The write channel contains a transmit FIFO, a write address counter, and an AXI write state machine. The transmit FIFO buffers the frame average data output by the frame averaging module. The write address counter is reset to the base address of the image storage area in external memory at the beginning of each frame and increments by the burst length after each burst write operation. The AXI write state machine reads data from the transmit FIFO and writes the data to external memory using the AXI burst write protocol.

[0100] The read channel includes a receive FIFO, a read address counter, and an AXI read state machine. The read channel is only used during the first frame or when the buffer expires, to read the average value of the previous frame from external memory into the ping-pong buffer module. During normal operation, the read channel is disabled and does not consume bus bandwidth because the replay push mechanism ensures that the previous frame's data is pre-stored in the backup buffer.

[0101] The write address of the bus interface module is reset to the same base address at the beginning of each frame, ensuring that the frame average values ​​of each frame have completely overlapping physical addresses in the external memory, and the write result of the next frame directly overwrites the result of the previous frame. Only one frame of cumulative average data is stored in the external memory at any given time.

[0102] In some other embodiments, frame repetition can only be initiated after the processing system writes an authorization signal to the register, ensuring the system's control over the scanning process.

[0103] After a frame scan is completed, the system checks the following completion conditions: frame scan data acquisition is finished (frame_sample_end_flag), all AXI burst write operations are completed (frame_trans_end_flag), all data FIFOs are emptied (frame_fifo_end_flag), and the ping-pong FIFO is ready for reset (axi2ad_rst_ready). Once all conditions are met, the system checks if the frame repetition counter is less than the frame repetition count minus 1. If so, provided the processing system is authorized via the ScanAveFrameTrig register, a frame repetition start signal is generated, the frame counter increments, the ping-pong FIFOs switch roles, and the scan restarts. When all frame repetitions are complete, the frame_task_end flag is set, and the processing system reads the final average result image from the DDR via the bus.

[0104] In some embodiments, the system further includes a fault detection module for monitoring abnormal events in the data path. The monitoring objects of the fault detection module include at least: FIFO overflow, FIFO empty read, AXI bus error response, and divider overflow. In this embodiment, the central controller has a built-in priority-coded fault detector that monitors 13 types of fault sources, including: setup and hold time constraint violation, point-level sampler counter overflow or multiple submissions, inter-channel synchronization mismatch, write FIFO overflow, AXI burst length FIFO overflow, synchronization FIFO overflow, AXI write / read error response, DAC parallel-to-serial converter overflow or undercurrent, ADC tag bridge overflow, and ADC gearbox overflow. When the first abnormal event occurs, the fault detection module latches the fault code and diagnostic context information and outputs a safe stop request signal to the central controller. In response to the safe stop request signal, the central controller suppresses the consumption enable of subsequent data paths, drains residual data in the pipeline, and prevents new tasks from starting until the processing system confirms recovery by writing a fault clearing command to the register file. The processing system obtains fault details by reading the ErrCodeExt and ErrContext registers for fault location and system debugging.

[0105] In some embodiments, the system described above is implemented based on the Xilinx Zynq-7000 SoC platform (model xc7z020clg484-1), including a processing system (PS) side and a programmable logic (PL) side. The PS side configures scan parameters to the PL side via a register file mapped from block memory (BRAM) and reads processing results from DDR via the AXI4 bus. The PL side comprises multiple asynchronous clock domains working collaboratively, specifically divided as follows: Core processing clock domain (50MHz): Scan waveform generation, point-level averaging, row-level averaging, frame-level recursive averaging, and control register file all operate in this clock domain. ADC acquisition clock domain (100MHz): The analog-to-digital converter interface module and the receiving gearbox module operate in this clock domain; DAC output clock domain (200MHz): The serial output interface of the digital-to-analog converter and the parallel-to-serial conversion module operate in this clock domain; AXI bus clock domain (100MHz): The AXI4 bus write-back module and prefetch module operate in this clock domain.

[0106] Data exchange between clock domains is achieved through asynchronous FIFOs, enabling clock isolation and cross-domain data transmission. The system's register file contains 67 PS-accessible 32-bit registers, mapped to base address 0x40000000, used to configure scan offset, step size, number of points per line, number of lines, line repetition count, frame repetition count, gain / offset coefficients for each channel, rotation matrix parameters, channel enable masks, etc.

[0107] The processing system configures the scan and averaging parameters to the programmable logic side via a register file mapped from block memory. After the processing system writes the start flag, the programmable logic side latches all parameters and starts the scan process. After one frame of scan and averaging is completed, the programmable logic side sets a status flag. The processing system can detect the completion of processing through polling or interrupts and read the final averaged result image from external memory via the bus.

[0108] For multi-frame averaging, the above process is repeated frame by frame. The frame average value for each frame is continuously updated through recursive averaging, and the cumulative average value in external memory is optimized frame by frame until the set number of frame repetitions is reached.

[0109] During the scan, the processing system can read the current frame average from external memory at any time for real-time preview display. Operators can use the preview image to assess the progress of signal-to-noise ratio improvement and decide whether to continue increasing the number of frames or adjusting scan parameters.

[0110] The FPGA scanning image frame averaging processing system based on recursive averaging and ping-pong prefetching provided by this invention integrates scanning waveform generation, multi-channel signal acquisition, three-level cascaded averaging processing and ping-pong prefetching buffer into the programmable logic of the same FPGA chip. The modules are tightly coupled through data flow and control signals to form a complete scanning imaging processing pipeline.

[0111] like Figure 7 As shown, the DDR storage usage and maximum image size of this invention and the traditional full-capacity storage scheme are compared under different average frame counts. Taking a DDR usable capacity of 944MB as an example: It can be seen that the maximum image size of the traditional scheme decreases inversely with the frame average number N, and the maximum image size is only 1.84MB when N=512. However, because the DDR always stores only the cumulative average of one frame, the maximum image size of this invention remains unchanged at 944MB under all average numbers, which is 512 times that of the traditional scheme when N=512.

[0112] For row-average data, the BRAM consumption of this invention remains constant: taking a 4-channel configuration as an example, the storage width of one row of scan data in the BRAM is 64 bits, the depth is 16K, and it occupies approximately 128KB of BRAM resources, accounting for about 20% of the total BRAM of the xc7z020 chip (630KB). The BRAM consumption remains unchanged regardless of whether the row repetition count is 1 or 512 times. In a traditional solution, if all M row repetitions of data need to be stored in the BRAM, the BRAM usage would be 128KB × M, which would exceed the total on-chip BRAM capacity of the FPGA when the row repetition count is large.

[0113] Comparison of calculation errors between recursive averaging and one-time averaging: The frame-by-frame recursive division method used in this invention is mathematically equivalent to the traditional one-time summation division method, but there is a difference in truncation error in fixed-point hardware implementation. The one-time averaging scheme accumulates N frames of data pixel by pixel and then performs a single division, generating only a truncation error of 1 LSB, but it requires storing all N frames of data and cannot provide intermediate results. The recursive method of this invention generates a truncation error δ_k (0≤δ_k<1 LSB) each time it is recursively calculated in each frame. The cumulative error after N frames of recursion is ε_N=Σ_{k=2}^{N}(k / N)·δ_k, with the worst-case upper bound being approximately (N+1) / 2 LSBs. Under the 8-bit effective data precision of this embodiment (AD_USE_DATA_WIDTH=8, dynamic range 256 gray levels), the worst-case truncation error comparison of the two schemes under different frame averaging times is as follows: It should be noted that the worst-case scenario in the table above assumes that the recursive division in each frame precisely truncates the theoretical upper bound of the maximum error (δ_k approaches 1 LSB). In actual image data, the truncation error of each frame is approximately uniformly distributed within the [0,1) LSB interval and in the same direction (downward truncation). The actual cumulative error is significantly smaller than this upper bound. Furthermore, the average number of frames is typically 8 to 64 in typical scanning electron microscopy applications, corresponding to a worst-case error of 4.5 to 32.5 LSBs (1.76% to 12.70%), which is within an acceptable range.

[0114] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0115] For example, this embodiment is based on the Zynq-7000 platform, but the recursive averaging and ping-pong prefetching methods of this invention are also applicable to other FPGA platforms, such as Xilinx UltraScale, Intel / Altera, or ASIC implementations. The specific models, number of channels, sampling rates, and other parameters of the ADC and DAC can be adjusted according to actual application requirements without affecting the applicability of the core method of this invention. The recursive averaging formula can also be extended to variations such as weighted averaging.

[0116] The scanning imaging system described in this invention is not limited to scanning electron microscopes, but is also applicable to all devices that acquire images using grating scanning methods, such as focused ion beam (FIB) systems, scanning transmission electron microscopes (STEM), confocal laser scanning microscopes, ultrasonic scanning imaging, and radio frequency thermal scanning systems.

[0117] The number of devices and processing scale described herein are for the purpose of simplifying the description of the invention. Applications, modifications, and variations of the invention will be readily apparent to those skilled in the art.

[0118] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

[0119] The systems and methods provided in the embodiments of this specification are corresponding. Therefore, the systems also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding devices, computer equipment, and non-volatile computer storage media will not be repeated here.

[0120] Those skilled in the art will also know that, besides implementing the controller in the form of purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller take the form of logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices included within it for implementing various functions can also be considered structures within that hardware component. Alternatively, the devices for implementing various functions can be considered as both software units implementing the method and structures within a hardware component.

[0121] The systems, apparatuses, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above apparatuses are described separately as various units based on their functions. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0122] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0126] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0127] This specification may be described in the general context of computer-executable instructions, such as program units, that are executed by a computer. Generally, program units include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification may also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program units may reside in local and remote computer storage media, including storage devices.

[0128] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0129] The above description is merely an embodiment of this specification and is not intended to limit the scope of one or more embodiments of this specification. Various modifications and variations can be made to one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of one or more embodiments of this specification.

Claims

1. A method for averaging FPGA scanned image frames based on recursive averaging and ping-pong prefetching, characterized in that, Includes the following steps: The scanning control step involves generating a scanning waveform and a scanning position tag synchronized with the scanning waveform in the FPGA programmable logic. The scanning waveform is used to drive the electron beam to perform two-dimensional grating scanning. The acquisition and alignment step involves acquiring sampled data from multi-channel echo signals and aligning the sampled data with the scan position labels in time to obtain sampled data with position labels. The point averaging step involves accumulating the sampled data corresponding to the same scan position and dividing by the sample count to obtain the point average value for that scan position. The row averaging step, on a per-scan-line basis, uses on-chip memory to successively accumulate the average value of points corresponding to multiple repeated scans of each scan position within the same scan line. After the last repeated scan of the scan line is completed, the accumulated result is divided by the number of row repetitions to obtain the row average value. At any given time, the on-chip memory stores only the accumulated intermediate value of one scan line. The frame averaging step uses external memory to recursively update the frame average value. The frame average value of the (N+1)th frame is calculated based on the frame average value of the Nth frame and the row average value of the current frame according to the recursive averaging formula. At any given time, only one frame average value is stored in the external memory, and the frame average value of each frame is written to the same storage area in the external memory to achieve in-situ coverage. In the prefetching step, a working buffer and a backup buffer are set up. During the processing of the current frame, the average frame value of the previous frame is read from the working buffer for use in the frame averaging step. At the same time, while the average frame value of the current frame is being written to the external memory, the average frame value is synchronously written to the backup buffer. After a frame is processed, the working buffer and the backup buffer exchange roles, so that the frame averaging step of the next frame can directly read the average frame value of the previous frame from the pre-filled buffer.

2. The FPGA scan image frame averaging method based on recursive averaging and ping-pong prefetching as described in claim 1, characterized in that, When the current frame is the first frame, the frame averaging step directly stores the line average of the current frame as the frame average into the external memory.

3. The FPGA scan image frame averaging method based on recursive averaging and ping-pong prefetching as described in claim 1, characterized in that, In the row averaging step, the write enable of the on-chip memory is controlled as follows: When a row is not being scanned repeatedly, write enable is active, and the sum of the current point average value and the historical accumulated value read from the on-chip memory is written back to the same address. During the last row repetition scan, the write enable is disabled, and the accumulated result in the on-chip memory is output to the division unit for row averaging.

4. The FPGA scan image frame averaging method based on recursive averaging and ping-pong prefetching as described in claim 1, characterized in that, In the scanning control step, waveform data of multiple scanning sub-points and corresponding scanning position labels are generated in parallel within each clock cycle.

5. An FPGA scanning image frame averaging processing system based on recursive averaging and ping-pong prefetching, characterized in that, include: The scanning waveform generation module, located in the FPGA programmable logic, is used to generate the scanning waveform and the scanning position label synchronized with the scanning waveform. The acquisition interface module is used to receive the sampling data acquired by the multi-channel analog-to-digital converter and align the sampling data with the scanning position label; The point averaging module is used to accumulate the sampled data corresponding to the same scan position and divide it by the sample count to output the point average value; The row averaging module includes an on-chip memory and a first division operation unit. The row averaging module accumulates the average value of points corresponding to multiple repeated scans of each scan position within the same scan line, and temporarily stores it in the on-chip memory. After the last repeated scan of the row is completed, the first division operation unit divides the accumulated result by the number of row repetitions and outputs the row average value. The frame averaging module includes a second division operation unit. The frame averaging module is used to update the frame average value according to the recursive averaging formula. The frame average value of the (N+1)th frame is calculated based on the frame average value of the Nth frame and the row average value of the current frame. The updated frame average value is written to a fixed storage area of ​​the external memory. The write address of each frame is the same to achieve in-situ overwriting. Only one frame average value is stored in the external memory at any given time. The ping-pong buffer module includes a working buffer and a backup buffer. The working buffer is used to provide the frame average value of the previous frame to the frame averaging module. The backup buffer is used to receive the synchronous writing of the frame average value of the current frame while the frame average value of the current frame is written to the external memory. After a frame is processed, the working buffer and the backup buffer exchange roles. The bus interface module is used to write the frame average value output by the frame averaging module into the external memory, and to read the frame average value from the external memory into the ping-pong buffer module.

6. The FPGA scan image frame averaging processing system based on recursive averaging and ping-pong prefetching as described in claim 5, characterized in that, The system is implemented based on an FPGA SoC platform. The FPGA programmable logic contains multiple clock domains that are asynchronous with each other, including a core processing clock domain for scan control and averaging calculation, an acquisition clock domain for receiving sampled data, a waveform clock domain for waveform output, and a bus clock domain for accessing external memory. Data is exchanged between the clock domains through an asynchronous FIFO.

7. The FPGA scan image frame averaging processing system based on recursive averaging and ping-pong prefetching as described in claim 5, characterized in that, The data bit width of the on-chip memory in the row averaging module is automatically adapted at compile time according to the number of channels enabled by the system. The storage depth of the on-chip memory is fixed, and its storage resource consumption does not change with the increase of the number of row repetitions.

8. The FPGA scan image frame averaging processing system based on recursive averaging and ping-pong prefetching as described in claim 5, characterized in that, The scanning waveform generation module includes a line scanning state machine and a frame scanning state machine. The line scanning state machine includes at least an idle state, a scanning state, and a retrace state. The frame scanning state machine includes at least an idle state and a scanning state. The line scanning state machine and the frame scanning state machine work together in a nested manner. The line scanning state machine is executed cyclically when the frame scanning state machine is in the scanning state, so as to realize the point-by-point and line-by-line output of the two-dimensional raster scanning waveform.

9. The FPGA scan image frame averaging processing system based on recursive averaging and ping-pong prefetching as described in claim 5, characterized in that, The point averaging module includes a sampling counter, which is used to count the number of valid samples corresponding to the same scanning position. When the scanning position is switched, the point averaging module sends the accumulated result and the sampling count value together into the division operation unit.

10. The FPGA scan image frame averaging processing system based on recursive averaging and ping-pong prefetching as described in claim 5, characterized in that, It also includes a gain bias correction module, which configures an independent pipeline processing unit for each acquisition channel to sequentially perform gain multiplication, bias addition and saturation clamping operations.

11. The FPGA scan image frame averaging processing system based on recursive averaging and ping-pong prefetching as described in claim 5, characterized in that, It also includes a fault detection module, which monitors FIFO overflow, FIFO empty read, AXI bus error and divider overflow events, and latches fault information and triggers pipeline drain when an anomaly is detected.

12. The FPGA scan image frame averaging processing system based on recursive averaging and ping-pong prefetching as described in claim 5, characterized in that, The scanning waveform generation module also includes an affine transformation unit, which performs affine matrix transformations on the scanning coordinates to achieve rotation and correction of the scanning pattern.

Images