A high-speed image acquisition and processing method, system, terminal and storage medium

Abstract

The application relates to the technical field of image processing, and discloses a high-speed image acquisition and processing method, a high-speed image acquisition and processing system, a terminal and a storage medium.The method comprises the following steps: scanning a sample, collecting physical signals generated in the scanning, and converting the physical signals into digital image data; grouping the digital image data, adding encapsulation information, and obtaining a plurality of data packets; receiving the data packets, determining the boundary of an image frame according to a frame start flag, and sorting and recombining the data packets belonging to the same image frame according to a packet serial number to obtain image frame data corresponding to a preset scanning mode; performing image enhancement processing on the image frame data to obtain enhanced image data; and displaying the enhanced image data in a display channel independent of the image enhancement processing process according to a preset display refreshing strategy.The application realizes comprehensive improvement of a high-speed image acquisition and processing system in terms of transmission reliability, reconstruction accuracy, image quality and display fluency.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a high-speed image acquisition and processing method, system, terminal, and computer-readable storage medium. Background Technology

[0002] In precision equipment used in the semiconductor industry, such as electron beam lithography and inspection, high-resolution images of the sample surface need to be constructed in real time by scanning the electron beam and acquiring secondary electron or backscattered electron signals. This process places extremely high demands on the image data acquisition speed, transmission reliability, reconstruction accuracy, and real-time display.

[0003] Current technical solutions typically employ Field-Programmable Gate Arrays (FPGAs) for high-speed data acquisition and transmit the data to a host computer via a User Datagram Protocol (UDP) network for image reconstruction and display. However, this approach suffers from the following significant drawbacks: 1. Image reconstruction errors: The UDP protocol does not guarantee the order and reliability of data packets. In high-bandwidth, high-speed transmission scenarios, data packets are prone to arriving out of order or across frames, causing the host computer to be unable to correctly reconstruct the image.

[0004] 2. Poor adaptability to scanning modes: To improve efficiency, scanning often employs a serpentine (back-and-forth) path. Existing systems lack a data packaging and reconstruction mechanism that is deeply integrated with the scanning mode, resulting in misaligned reconstructed images that require additional post-processing correction.

[0005] 3. System resource contention and lag: In pursuit of real-time display, the graphical user interface (UI) needs to refresh the complete high-resolution image at high frequency, which seriously consumes CPU or GPU resources, causing interface lag and affecting the operator's real-time observation.

[0006] 4. Uncontrollable image quality: Raw electronic images have limited dynamic range, low contrast, and are difficult to identify key nanostructures. Existing systems lack real-time image enhancement capabilities embedded in high-speed pipelines.

[0007] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0008] The main objective of this invention is to provide a high-speed image acquisition and processing method, system, terminal, and computer-readable storage medium, aiming to solve problems such as image corruption, disconnect between scanning mode and reconstruction logic, display lag, and poor image quality caused by UDP transmission.

[0009] To achieve the above objectives, the present invention provides a method for acquiring and processing high-speed images, the method comprising the following steps: The electron beam is controlled to scan the sample according to a preset scanning mode, the physical signals generated by the scanning are collected, and the physical signals are converted into digital image data; The digital image data is grouped and encapsulation information is added to obtain multiple data packets. The encapsulation information includes: a frame start flag to identify the start of an image frame, a packet start flag to identify the start of a data packet, a frame sequence number, a packet sequence number, and a scan mode identifier to indicate the current image frame scanning mode. The data packet is received, the boundary of the image frame is determined according to the frame start flag, and the data packets belonging to the same image frame are sorted and reassembled according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode. The image frame data is subjected to image enhancement processing to obtain enhanced image data; In a display channel independent of the image enhancement processing flow, the enhanced image data is visualized according to a preset display refresh strategy.

[0010] Furthermore, the controlled electron beam scans the sample according to a preset scanning mode, acquires the physical signals generated by the scanning, and converts the physical signals into digital image data, specifically including: The scanning control signal generated by the digital-to-analog converter drives the electron beam to perform grating scanning on the sample according to the path corresponding to the preset scanning mode. The electron detector generates an analog voltage signal based on the grating scanning. The analog voltage signal is acquired and converted into digital image data under the synchronization of a high-speed clock signal.

[0011] Furthermore, the process of grouping the digital image data and adding encapsulation information to obtain multiple data packets specifically includes: The digital image data is temporarily stored in a buffer. When the amount of data in the buffer reaches a preset sampling point threshold, the encapsulation operation is triggered. During the triggering encapsulation operation, the data that reaches the threshold is grouped as a data group and encapsulation information is added to form a data packet.

[0012] Further, receiving the data packet, determining the boundary of the image frame based on the frame start flag, and sorting and reassembling data packets belonging to the same image frame according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode, specifically includes: Set a data buffer of appropriate size based on the width and height parameters of the image to be acquired; The data packet is received, its data length is checked, and its encapsulation information is parsed. The start of a new image frame is identified based on the frame start flag, and the data packets within the same image frame are sorted according to the packet sequence number to obtain the target data packet. Based on the scanning mode identifier and preset image parameters, the image coordinates corresponding to the payload data of the target data packet are calculated, and the payload data is written to the corresponding position in the data buffer to obtain the target image frame data.

[0013] Furthermore, the step of receiving the data packet, determining the boundary of the image frame according to the frame start flag, and sorting and reassembling data packets belonging to the same image frame according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode further includes: When the scanning mode indicator indicates a snake scan mode, the forward data segment and the reverse data segment are identified. The forward data segment is written to the corresponding position in the data buffer, and the reverse data segment is reversed and then written to the corresponding position in the data buffer.

[0014] Furthermore, the image enhancement processing of the image frame data to obtain enhanced image data specifically includes: Perform bilateral filtering on the image frame data to obtain the first target image frame data; Adaptive histogram equalization is performed on the first target image frame data to obtain the second target image frame data; Gamma correction or piecewise linear contrast stretching is performed on the second target image frame data to obtain enhanced image data.

[0015] Furthermore, the step of visually displaying the enhanced image data according to a preset display refresh strategy in a display channel independent of the image enhancement processing flow specifically includes: Set up a display channel that is independent of the data reception, image reconstruction, and image enhancement processing flow; In the display channel, the target image is obtained from the output buffer of the enhanced image data according to a preset display refresh strategy; The target image is rendered and displayed on the user interface.

[0016] Furthermore, to achieve the above objectives, the present invention also provides a high-speed image acquisition and processing system, which is used to implement the high-speed image acquisition and processing method described above, wherein the high-speed image acquisition and processing system includes: The data acquisition module is used to control the electron beam to scan the sample according to a preset scanning mode, acquire the physical signals generated by the scanning, and convert the physical signals into digital image data; The data grouping module is used to group the digital image data and add encapsulation information to obtain multiple data packets. The encapsulation information includes: a frame start flag to identify the start of an image frame, a packet start flag to identify the start of a data packet, a frame sequence number, a packet sequence number, and a scan mode identifier to indicate the current image frame scanning mode. The data reassembly module is used to receive the data packet, determine the boundary of the image frame according to the frame start flag, and sort and reassemble the data packets belonging to the same image frame according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode. The image enhancement module is used to perform image enhancement processing on the image frame data to obtain enhanced image data; The display module is used to visualize the enhanced image data in a display channel independent of the image enhancement processing flow, according to a preset display refresh strategy.

[0017] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a high-speed image acquisition and processing program stored in the memory and executable on the processor, wherein when the high-speed image acquisition and processing program is executed by the processor, it implements the steps of the high-speed image acquisition and processing method described above.

[0018] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a high-speed image acquisition and processing program, which, when executed by a processor, implements the steps of the high-speed image acquisition and processing method described above.

[0019] The beneficial effects of this invention are as follows: By controlling the electron beam and converting physical signals, this invention provides a high-quality, controllable raw data source for the entire system. Then, by adding encapsulation containing information such as frame start markers, intra-frame packet sequence numbers, and scan mode identifiers during data grouping, it lays a structural foundation for reliable and orderly data reconstruction in unreliable transmission channels. Furthermore, the receiving end determines frame boundaries based on the encapsulation information, sorts data packets, and reassembles the image according to the scan mode identifier, avoiding image corruption and achieving adaptive and accurate reconstruction for multiple scan modes. Next, the reconstructed image frames are enhanced, effectively improving the visual quality and detail clarity of the original electronic image. Finally, by employing a configurable refresh strategy, computationally intensive image processing is decoupled from user interface rendering, ensuring the integrity of image processing while avoiding interface stuttering caused by high-frequency refresh of high-resolution images, thus achieving smooth real-time visualization. In summary, this invention achieves a comprehensive improvement in transmission reliability, reconstruction accuracy, image quality, and display smoothness for high-speed image acquisition and processing systems. Attached Figure Description

[0020] Figure 1 This is a flowchart of a preferred embodiment of the high-speed image acquisition and processing method of the present invention; Figure 2 This is an image processing system architecture diagram based on FPGA and host computer collaboration, which is an embodiment of the high-speed image acquisition and processing method of the present invention. Figure 3 This is a pipeline architecture diagram of the entire high-speed image acquisition and processing system, which is an embodiment of the high-speed image acquisition and processing method of the present invention. Figure 4 This is a structural diagram of a preferred embodiment of the high-speed image acquisition and processing system of the present invention; Figure 5 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0021] This application provides a method, system, terminal, and storage medium for high-speed image acquisition and processing. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0022] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0023] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0024] The high-speed image acquisition and processing method described in the preferred embodiment of the present invention, such as... Figure 1 As shown, the high-speed image acquisition and processing method includes the following steps: S10. Control the electron beam to scan the sample according to the preset scanning mode, collect the physical signals generated by the scanning, and convert the physical signals into digital image data.

[0025] The purpose of this step is to achieve high-speed, precise acquisition and digitization of physical signals, providing the original data source for subsequent data transmission, reconstruction, and processing. This step utilizes precise electron beam control and high-speed analog-to-digital conversion to convert the physical properties of the sample surface (such as secondary electrons and backscattered electron signals) into an image data stream that can be processed by a digital system.

[0026] It should be noted that this step is typically implemented in hardware logic devices such as Field-Programmable Gate Arrays (FPGAs) to ensure real-time, deterministic control of the high-speed scanning process. The scanning beam is usually an electron beam, used in electron beam exposure and inspection equipment in the semiconductor industry.

[0027] Furthermore, the controlled electron beam scans the sample according to a preset scanning mode, acquires the physical signals generated by the scanning, and converts the physical signals into digital image data, specifically including: Step S11: The scanning control signal generated by the digital-to-analog converter drives the electron beam to perform grating scanning on the sample according to the path corresponding to the preset scanning mode. The electron detector generates an analog voltage signal according to the grating scanning.

[0028] Step S12: Acquire the analog voltage signal and convert it into digital image data under the synchronization of a high-speed clock signal.

[0029] In this embodiment, step S10 is executed by the FPGA as the core controller. The FPGA's internal logic precisely generates deflection control voltage signals for the X and Y axes through a 16-bit digital-to-analog converter (DAC). These voltage signals are applied to the scanning coils, thereby driving the electron beam to perform a raster scan on the sample surface along a preset, precise path. The preset scanning mode includes at least a progressive scan mode and a serpentine scan mode. In the progressive scan mode, the electron beam scans each row sequentially from left to right and from top to bottom; in the serpentine scan mode, the electron beam scans from left to right starting from the first row, then from right to left in the next row, and so on, alternating between forward and reverse directions in subsequent rows, thereby eliminating the electron beam's back-scan time, significantly improving scanning efficiency and helping to suppress the sample charging effect.

[0030] Meanwhile, when the electron beam bombards the sample surface, it excites physical signals such as secondary electrons or backscattered electrons. These signals are received by an electron detector (such as a scintillator-photomultiplier tube combination or a semiconductor detector) and converted into an analog voltage signal proportional to the signal intensity.

[0031] In step S12, the FPGA acquires the analog voltage signal in real time using a 12-bit analog-to-digital converter (ADC). The entire acquisition and conversion process is precisely synchronized by a highly stable 100MHz high-speed clock. This clock signal not only determines the ADC's sampling rate, but more importantly, it defines the residence time of the electron beam at each pixel. By programming and adjusting the clock division or counting, the scanning speed can be flexibly controlled, thereby achieving dynamic adjustment of the image acquisition frame rate, such as achieving a high frame rate higher than 50 fps to meet real-time observation requirements. The ADC converts the analog voltage value corresponding to each pixel into a 12-bit digital value, which represents the signal strength (i.e., grayscale value) of that pixel, thus forming a continuous digital image data stream. This high-speed, synchronous digitization process ensures the accuracy of the original image data in the time domain and amplitude, laying a high-quality data foundation for subsequent processing.

[0032] S20. The digital image data is grouped and encapsulation information is added to obtain multiple data packets. The encapsulation information includes: a frame start flag for identifying the start of an image frame, a packet start flag for identifying the start of a data packet, a frame sequence number, a packet sequence number, and a scan mode identifier for indicating the current image frame scanning mode.

[0033] The purpose of this step is to structure and package high-speed, continuous digital image data streams, embedding key metadata (encapsulation information) to address issues such as out-of-order packets and cross-frame arrivals encountered in unreliable network (such as UDP) transmission, and to provide necessary guidance for correct reconstruction at the receiving end.

[0034] It should be noted that this step is also implemented on the FPGA side. The packaging operation is the last processing step before data leaves the acquisition hardware and enters the transmission channel, and its robustness directly determines the reliability of the entire transmission chain.

[0035] Furthermore, the process of grouping the digital image data and adding encapsulation information to obtain multiple data packets specifically includes: Step S21: The digital image data is temporarily stored in a buffer. When the amount of data in the buffer reaches a preset sampling point threshold, the encapsulation operation is triggered.

[0036] Step S22: During the triggering encapsulation operation, the data that has reached the threshold is grouped into a data group and encapsulation information is added to form a data packet.

[0037] As can be seen from steps S21 to S22, inside the FPGA, the digital image data output in step S12 is first written into a high-speed first-in-first-out buffer. The purpose of setting up this buffer is to balance the continuous data flow generated by the ADC with the burst transmission behavior that may occur at the network interface. In this embodiment, the preset sampling point threshold is 200 pixels. When the amount of data accumulated in the buffer reaches 200 pixels (each pixel is 12 bits, which is about 300 bytes of raw data), the FPGA's packaging logic will be automatically triggered to group the data of these 200 pixels into a single data group.

[0038] In step S22, the FPGA adds a custom packet header to the data packet, thereby forming a complete UDP (User Datagram Protocol) packet that can be sent over Ethernet. This customized packet header is the encapsulation information, and its specific structure is as follows: Start of Frame Flag: 1 bit. This flag is set to logic '1' when this data packet is the first packet of a new image frame; otherwise, it is '0'. This is a crucial signal for the receiver to identify the start of an image frame.

[0039] Frame sequence number: e.g., 16 bits. This sequence number increments from 1 at the start of each new image frame. All data packets within the same frame share the same frame sequence number. This helps the receiver distinguish data from different frames, even in the event of severe cross-frame reordering.

[0040] Packet start flag: 1 bit. Set to '1' when this packet is the first packet in the current transmission session, or when it serves as a logical start identifier, but the more core mechanism involves coordination with the packet sequence number.

[0041] Packet sequence number: For example, 16 bits. This sequence number is counted independently within each frame and increments from 1. That is, the first data packet in each frame has a sequence number of 1, the second has 2, and so on. When the start-of-frame flag is detected, the packet sequence number counter is reset to 1. This mechanism is one of the key innovations of this invention, enabling the receiving end to accurately identify the order and continuity of data packets within a frame, fundamentally solving the problem of ineffective handling of cross-frame out-of-order packets in traditional schemes due to the use of continuously incrementing sequence numbers throughout the session.

[0042] Scan mode identifier: 1 bit. Used to indicate the scan mode used to generate this frame image. For example, '0' represents progressive scan mode, and '1' represents serpentine scan mode. This information is necessary for the receiving end to perform correct image reconstruction (especially the data inversion of serpentine scan lines).

[0043] Payload: The raw data of 200 pixels packed in step S21.

[0044] After packaging, the FPGA sends data packets to the host computer via a Gigabit Ethernet interface using the User Datagram Protocol (UDP). This dual sequence identification mechanism based on frames and packets provides a solid foundation for reliable and ordered image data reconstruction over unreliable UDP transmission.

[0045] S30. Receive the data packet, determine the boundary of the image frame according to the frame start flag, and sort and reassemble the data packets belonging to the same image frame according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode.

[0046] The purpose of this step is to reliably receive data packets transmitted over the network on the host computer, overcome the challenges of out-of-order delivery and packet loss brought about by the UDP protocol, and accurately reconstruct the data stream into two-dimensional image frames that are consistent with the original scan space based on the scanning mode knowledge embedded in the encapsulation information.

[0047] It should be noted that this step is executed in the data reception and reconstruction thread of the host computer software. Its core is a state-aware data packet parsing and image assembly engine.

[0048] Further, the process of receiving the data packet, determining the boundary of the image frame based on the frame start flag, and sorting and reassembling data packets belonging to the same image frame according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode specifically includes: Step S31: Set a data buffer of appropriate size according to the width and height parameters of the image to be acquired.

[0049] Step S32: Receive the data packet, check the data length of the data packet, parse the encapsulation information of the data packet, identify the start of a new image frame according to the frame start flag, and sort the data packets in the same image frame according to the packet sequence number to obtain the target data packet.

[0050] Step S33: Calculate the image coordinates corresponding to the payload data of the target data packet according to the scanning mode identifier and preset image parameters, and write the payload data into the corresponding position of the data buffer to obtain the target image frame data.

[0051] As can be seen from steps S31 to S33, before starting the acquisition, the host computer obtains the width and height of the image to be acquired based on user settings or system presets. In step S31, the host computer allocates an image data buffer area with a size of Width * Height in memory to temporarily store all pixel data of the current frame being reconstructed.

[0052] In step S32, the host computer's network receiving thread continuously listens to the UDP port. Whenever a data packet is received, it first performs a length check to exclude corrupted packets, and then parses the packet header. This parsing process is crucial for intelligent reconstruction. Frame boundary identification: Check the frame start flag. If it is '1', it indicates the start of a new image frame. At this time, the system will complete the final processing of the previous frame (if it has been fully received) and reset the frame reception state, including clearing the buffer used to temporarily store the current frame data packets, in preparation for receiving the data packets of the new frame. The frame sequence number is read and recorded for multi-frame tracking.

[0053] Intra-frame sorting and integrity processing: Reading packet sequence numbers. Since the FPGA restarts counting from 1 for each frame, the host computer can easily sort the received data packets belonging to the same frame (same frame sequence number) according to their packet sequence numbers, obtaining an ordered sequence of target data packets. This mechanism can effectively correct intra-frame out-of-order issues that may occur in UDP transmission.

[0054] Packet loss handling mechanism: During the sequential insertion of data packets, if a packet sequence number is found to be discontinuous (for example, packet 1 and packet 2 have been received, and the next packet to be received is packet 4), then packet loss is determined to have occurred. This invention designs a strategy to address this: If the number of consecutively lost data packets exceeds a preset threshold (e.g., 5%) of the total number of packets in the frame, the frame is considered to be severely corrupted, the entire frame is discarded, and the system waits for the next frame to begin in order to avoid displaying an erroneous image.

[0055] If the number of packet losses is within a threshold, a data imputation mechanism is activated. The system uses the pixel values ​​at the corresponding positions in the previous complete image frame to fill in the missing data in the current frame. This imputation method based on temporal redundancy can maximize the visual integrity and continuity of the reconstructed image even with minor packet loss, ultimately keeping the image reconstruction error rate within a preset range.

[0056] Step S33 is the core of image space reconstruction. For each payload (i.e., 200 consecutive pixel data points) in the sorted target data packet, the system needs to calculate their coordinates (Row, Col) in the final two-dimensional image. The parameters required for calculation include: the total image width (Width), the total height (Height), and the scan mode identifier parsed from the packet header.

[0057] The system calculates the starting position Index of this data in the original one-dimensional data stream based on the global sequence number of the data packet within the frame (derived from the packet sequence number) and the number of pixels contained in each packet (200).

[0058] Then, based on the Index, Width, and scan mode, the row coordinates (Row) and column coordinates (Col) corresponding to each pixel are calculated. The specific formulas are: Row = Index / Width, Col = Index % Width (for progressive scan). The calculated coordinates (Row, Col) are the target addresses of the pixel data in the image data buffer allocated in step S31.

[0059] The system writes the pixel data to the corresponding location in the buffer. Once all data packets for a frame have been received and processed, the buffer stores a complete target image frame data arranged spatially.

[0060] Further, the step of receiving the data packet, determining the boundary of the image frame according to the frame start flag, and sorting and reassembling data packets belonging to the same image frame according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode, also includes: When the scanning mode indicator indicates a snake scan mode, the forward data segment and the reverse data segment are identified. The forward data segment is written to the corresponding position in the data buffer, and the reverse data segment is reversed and then written to the corresponding position in the data buffer.

[0061] This step is another key innovation of the present invention, specifically addressing the characteristics of electron beam scanning. When the scanning mode identifier parsed in step S32 is '1' (serpentine scanning), the reconstruction algorithm requires special processing.

[0062] In serpentine scanning, the electron beam scans from left to right starting with the first row, then from right to left in the next row, and so on, alternating between forward and reverse directions in subsequent rows. If data packets arriving in chronological order are directly mapped to image rows, the images in the reverse scan rows will be mirrored horizontally, making it impossible to stitch the entire image correctly.

[0063] To address this issue, a horizontal order reversal operation was added between the coordinate calculation and data writing in step S33, as detailed below: Identifying Reverse Rows: After calculating the row number based on the global index, the scanning direction is determined based on the row number index starting from 0. Since the first row (Row=0) is scanned in the forward direction and the second row (Row=1) is scanned in the reverse direction, when Row % 2 == 1, the row is determined to be a reverse scan row.

[0064] Perform data reversal: For the identified reverse scan row, before calculating its column coordinate Col and preparing to write the data to the buffer location (Row, Col), first identify the pixel sub-segments spanning each row of the data packet, and then perform horizontal order reversal on the sub-segments corresponding to the reverse scan row. That is, reverse the order of pixels within the data block from [Pn, Pn+1, ..., Pm, Pm+1] to [Pm+1, Pm, ..., Pn+1, Pn].

[0065] Coordinate remapping and writing: After the inversion operation, the pixel data Pm+1, which originally corresponded to the rightmost position of the scan line, is now mapped to the leftmost column (Col=0) of that image row, and so on. The system then writes the data to the buffer according to the inverted pixel order and the correct column coordinate relationship.

[0066] This mechanism achieves deep collaboration between the scanning mode and the reconstruction logic. The FPGA only needs to package and send data in the physical scan time sequence, while the host computer reconstruction algorithm can automatically and intelligently complete the spatial rearrangement and orientation correction of the data based on the "scanning mode identifier," ultimately outputting an image with the correct orientation. This process is fully automated, requiring no manual intervention, effectively solving the image misalignment problem in traditional serpentine scanning, and ensuring the spatial consistency of the reconstructed image while improving scanning efficiency.

[0067] S40. Perform image enhancement processing on the image frame data to obtain enhanced image data.

[0068] The purpose of this step is to address the issues of limited dynamic range, low contrast, and significant noise in the original electronic images by embedding real-time image enhancement algorithms in a high-speed pipeline to improve image quality and highlight the nanoscale structural features of the sample (such as alignment marks and minute defects), thereby significantly improving the operator's visual observation experience and the input quality of subsequent automated algorithms.

[0069] It should be noted that this step is performed in the image processing channel of the host computer software. This channel runs in parallel with the data reception and image reconstruction channels, and is specifically responsible for image post-processing and optimization.

[0070] Furthermore, the step of performing image enhancement processing on the image frame data to obtain enhanced image data specifically includes: Step S41: Perform bilateral filtering on the image frame data to obtain the first target image frame data.

[0071] Step S42: Perform adaptive histogram equalization processing on the first target image frame data to obtain the second target image frame data.

[0072] Step S43: Perform gamma correction or piecewise linear contrast stretching on the second target image frame data to obtain enhanced image data.

[0073] As can be seen from steps S41 to S43, step S40 constructs a configurable multi-stage image enhancement pipeline. Users can choose to enable or adjust the parameters of each processor stage according to sample characteristics and observation requirements. Specific details are as follows: Step S41: Bilateral Filtering. Raw electronic images often contain Poisson noise. A bilateral filter is a nonlinear filter that smooths noise while effectively preserving edge details. It considers both the spatial distance between pixels and gray-level similarity, smoothing in flat areas and reducing smoothing at edges to maintain edge sharpness. This step effectively suppresses noise, preparing a cleaner input for subsequent contrast enhancement and avoiding noise amplification.

[0074] Step S42: Adaptive Histogram Equalization. Traditional histogram equalization may over-amplify noise in uniform regions. Adaptive histogram equalization divides the image into multiple small regions, performs histogram equalization independently within each region, and then eliminates boundary effects between regions through interpolation. This method significantly improves the local contrast of the image, making details (such as the edges and textures of nanoscale structures) in darker or brighter areas of the overall image clearly visible, and is particularly suitable for situations where signal intensity distribution is uneven in electronic images.

[0075] Step S43: Gamma correction or contrast stretching. This is the final user-adjustable visual optimization step.

[0076] Gamma correction: Adjusts the grayscale response curve of an image through a power-law transformation. For example, it enhances detail in dark areas or bright areas. Users can adjust the rendering of the region of interest in real time using a slider.

[0077] Piecewise linear contrast stretching: Users can define a segment [a, b] of the input grayscale range and linearly stretch it to the entire output grayscale range [0, 255], while truncating grayscale levels outside the range. This can strongly enhance the contrast of specific grayscale intervals (such as the signal strength corresponding to a certain material or structure), and in extreme cases, it can be used for binarization preview.

[0078] After processing steps S41 to S43, the raw, low-contrast "target image frame data" is transformed into high-quality "enhanced image data." This data is sent to the display channel for real-time observation by the operator, and can also be simultaneously stored on the hard drive for subsequent offline analysis, measurement, or archiving. This enhancement pipeline is seamlessly embedded in the high-speed processing flow, significantly improving image interpretability without increasing the inherent latency of image acquisition.

[0079] S50. In a display channel independent of the image enhancement processing flow, the enhanced image data is visualized according to a preset display refresh strategy.

[0080] The purpose of this step is to decouple computationally intensive image processing from user interface rendering. Through flexible refresh strategies, while ensuring the integrity and high quality of image processing, a smooth and lag-free user interface refresh is achieved, thereby improving the operator's real-time interactive experience.

[0081] It should be noted that this step is performed in a separate display thread (or display channel) of the host computer software. This thread is separate from background processing threads such as data reception, image reconstruction, and image enhancement, and is dedicated to display rendering. Its refresh rate is not absolutely limited by the background processing speed.

[0082] Furthermore, the step of visually displaying the enhanced image data according to a preset display refresh strategy in a display channel independent of the image enhancement processing flow specifically includes: Step S51: Set up a display channel that is independent of the data reception, image reconstruction, and image enhancement processing flow.

[0083] Step S52: In the display channel, the target image is obtained from the output buffer of the enhanced image data according to the preset display refresh strategy.

[0084] Step S53: Render the target image onto the user interface for display.

[0085] The display refresh strategies include: line-by-line refresh, interlaced refresh, frame-by-frame refresh, or frame rate reduction refresh.

[0086] As can be seen from steps S51 to S53, this invention innovatively constructs a dual-channel architecture of "full-frame fidelity processing + on-demand dynamic display," which can solve the UI (User Interface) lag problem. Specific details are as follows: Step S51: Establish an independent display channel. In terms of software architecture, create a dedicated display thread or display channel. This channel communicates with the background processing thread responsible for steps S20-S40 via a thread-safe buffer (such as a double-buffered or triple-buffered queue). The "enhanced image data" after image enhancement processing is completed is fed into the input of this buffer, while the display channel retrieves the latest available image from the output of the buffer. This producer-consumer model decouples processing and display resources. Regardless of whether the background processing task is busy or idle, the display channel operates at a fixed, high-priority rhythm, ensuring UI responsiveness.

[0087] Step S52: Acquire image according to strategy. The display channel does not blindly fetch images from the buffer at maximum speed. Instead, it follows a user-configurable display refresh strategy: Line-by-line refresh: Upon receiving a new line of data, image reconstruction and enhancement are performed immediately before the data is sent to the display channel to update and display the latest data line. This provides the fastest visual feedback, but the frequent refreshes consume the most system resources.

[0088] Alternating refresh: For example, refresh the display after receiving and processing every 2 or 4 rows of data. This is a balance between real-time performance and resource consumption.

[0089] Frame-by-frame refresh: After a complete frame of image has been received, reconstructed, and enhanced, the entire display screen is updated all at once. This is the most resource-efficient method, providing a smooth visual experience.

[0090] Reduced frame rate refresh: For example, even though the capture frame rate is 60fps, the display channel only fetches and displays the latest frame of image from the buffer every 2 or 5 frames. This significantly reduces the rendering load on the GPU and is an effective way to ensure UI smoothness when using low-end hardware or processing ultra-high-definition (such as 4K) images.

[0091] Step S53: Rendering and Display. The display channel periodically checks the buffer according to the selected strategy. When the refresh condition is met, it copies the latest "target image" from the buffer and then calls the graphics API to render it to the image display controls in the user interface. This copying and rendering process is very fast and independent of heavy background image processing tasks.

[0092] As can be seen from the above steps, the background processing thread can focus on ensuring the complete reception, correct reconstruction, and high-quality enhancement of each frame of image without being interrupted by high-frequency display refreshes. Simultaneously, the user interface remains consistently responsive, allowing operators to perform panning, zooming, parameter adjustments, and other interactive operations without lag, significantly improving system availability and user experience in high-bandwidth, high-resolution scenarios.

[0093] In one embodiment, such as Figure 2 As shown, the system receives data packets from the network, parses the frame start flag and packet sequence number in the encapsulation information, sorts data packets belonging to the same image frame to correct out-of-order errors, and performs horizontal reversal on the reverse scan line data of the serpentine scan based on the scan mode identifier, thereby reliably reconstructing the correct image frame data; subsequently, as... Figure 3 The system processing pipeline shown receives this reconstructed data and sequentially performs image enhancement processing such as bilateral filtering and adaptive histogram equalization. Finally, the enhanced image data is stored in a buffer, while an independent display channel retrieves and renders the image from the buffer according to a preset strategy (such as frame-by-frame refresh), thus achieving decoupling and coordination between high-quality processing and smooth display.

[0094] Furthermore, such as Figure 4 As shown, based on the above-described high-speed image acquisition and processing method, the present invention also provides a high-speed image acquisition and processing system, the high-speed image acquisition and processing system comprising: Data acquisition module 51 is used to control the electron beam to scan the sample according to a preset scanning mode, acquire the physical signals generated by the scanning, and convert the physical signals into digital image data; The data grouping module 52 is used to group the digital image data and add encapsulation information to obtain multiple data packets. The encapsulation information includes: a frame start flag for identifying the start of an image frame, a packet start flag for identifying the start of a data packet, a frame sequence number, a packet sequence number, and a scan mode identifier for indicating the current image frame scanning mode. The data reassembly module 53 is used to receive the data packet, determine the boundary of the image frame according to the frame start flag, and sort and reassemble the data packets belonging to the same image frame according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode. Image enhancement module 54 is used to perform image enhancement processing on the image frame data to obtain enhanced image data; Display module 55 is used to visualize the enhanced image data in a display channel independent of the image enhancement processing flow, according to a preset display refresh strategy.

[0095] Furthermore, such as Figure 5As shown, based on the above-mentioned high-speed image acquisition and processing method and system, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 5 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0096] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a high-speed image acquisition and processing program 40, which can be executed by the processor 10 to implement the high-speed image acquisition and processing method of this application.

[0097] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the high-speed image acquisition and processing method.

[0098] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0099] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a high-speed image acquisition and processing program, which, when executed by a processor, implements the steps of the high-speed image acquisition and processing method described above.

[0100] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0101] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0102] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for high-speed image acquisition and processing, characterized in that, The method for acquiring and processing high-speed images includes the following steps: The electron beam is controlled to scan the sample according to a preset scanning mode, the physical signals generated by the scanning are collected, and the physical signals are converted into digital image data; The digital image data is grouped and encapsulation information is added to obtain multiple data packets. The encapsulation information includes: a frame start flag to identify the start of an image frame, a packet start flag to identify the start of a data packet, a frame sequence number, a packet sequence number, and a scan mode identifier to indicate the current image frame scanning mode. The data packet is received, the boundary of the image frame is determined according to the frame start flag, and the data packets belonging to the same image frame are sorted and reassembled according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode. The image frame data is subjected to image enhancement processing to obtain enhanced image data; In a display channel independent of the image enhancement processing flow, the enhanced image data is visualized according to a preset display refresh strategy.

2. The high-speed image acquisition and processing method according to claim 1, characterized in that, The controlled electron beam scans the sample according to a preset scanning mode, collects the physical signals generated by the scan, and converts the physical signals into digital image data, specifically including: The scanning control signal generated by the digital-to-analog converter drives the electron beam to perform grating scanning on the sample according to the path corresponding to the preset scanning mode. The electron detector generates an analog voltage signal based on the grating scanning. The analog voltage signal is acquired and converted into digital image data under the synchronization of a high-speed clock signal.

3. The high-speed image acquisition and processing method according to claim 1, characterized in that, The process of grouping the digital image data and adding encapsulation information to obtain multiple data packets specifically includes: The digital image data is temporarily stored in a buffer. When the amount of data in the buffer reaches a preset sampling point threshold, the encapsulation operation is triggered. During the triggering encapsulation operation, the data that reaches the threshold is grouped as a data group and encapsulation information is added to form a data packet.

4. The high-speed image acquisition and processing method according to claim 1, characterized in that, The process of receiving the data packet, determining the boundary of the image frame based on the frame start flag, and sorting and reassembling data packets belonging to the same image frame according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode specifically includes: Set a data buffer of appropriate size based on the width and height parameters of the image to be acquired; The data packet is received, its data length is checked, and its encapsulation information is parsed. The start of a new image frame is identified based on the frame start flag, and the data packets within the same image frame are sorted according to the packet sequence number to obtain the target data packet. Based on the scanning mode identifier and preset image parameters, the image coordinates corresponding to the payload data of the target data packet are calculated, and the payload data is written to the corresponding position in the data buffer to obtain the target image frame data.

5. The high-speed image acquisition and processing method according to claim 4, characterized in that, The process of receiving the data packet, determining the boundary of the image frame based on the frame start flag, and sorting and reassembling data packets belonging to the same image frame according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode further includes: When the scanning mode indicator indicates a snake scan mode, the forward data segment and the reverse data segment are identified. The forward data segment is written to the corresponding position in the data buffer, and the reverse data segment is reversed and then written to the corresponding position in the data buffer.

6. The high-speed image acquisition and processing method according to claim 1, characterized in that, The image enhancement processing of the image frame data to obtain enhanced image data specifically includes: Perform bilateral filtering on the image frame data to obtain the first target image frame data; Adaptive histogram equalization is performed on the first target image frame data to obtain the second target image frame data; Gamma correction or piecewise linear contrast stretching is performed on the second target image frame data to obtain enhanced image data.

7. The high-speed image acquisition and processing method according to claim 1, characterized in that, The step of visually displaying the enhanced image data in a display channel independent of the image enhancement processing flow, according to a preset display refresh strategy, specifically includes: Set up a display channel that is independent of the data reception, image reconstruction, and image enhancement processing flow; In the display channel, the target image is obtained from the output buffer of the enhanced image data according to a preset display refresh strategy; The target image is rendered and displayed on the user interface.

8. A high-speed image acquisition and processing system, characterized in that, The high-speed image acquisition and processing system is used to implement the high-speed image acquisition and processing method as described in any one of claims 1-7, wherein the high-speed image acquisition and processing system comprises: The data acquisition module is used to control the electron beam to scan the sample according to a preset scanning mode, acquire the physical signals generated by the scanning, and convert the physical signals into digital image data; The data grouping module is used to group the digital image data and add encapsulation information to obtain multiple data packets. The encapsulation information includes: a frame start flag to identify the start of an image frame, a packet start flag to identify the start of a data packet, a frame sequence number, a packet sequence number, and a scan mode identifier to indicate the current image frame scanning mode. The data reassembly module is used to receive the data packet, determine the boundary of the image frame according to the frame start flag, and sort and reassemble the data packets belonging to the same image frame according to the packet sequence number to obtain image frame data corresponding to the preset scanning mode. The image enhancement module is used to perform image enhancement processing on the image frame data to obtain enhanced image data; The display module is used to visualize the enhanced image data in a display channel independent of the image enhancement processing flow, according to a preset display refresh strategy.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a high-speed image acquisition and processing program stored in the memory and executable on the processor. When the high-speed image acquisition and processing program is executed by the processor, it implements the steps of the high-speed image acquisition and processing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a high-speed image acquisition and processing program, which, when executed by a processor, implements the steps of the high-speed image acquisition and processing method as described in any one of claims 1-7.

Images