Large field of view image real-time acquisition enhancement and multi-stage registration splicing system and method thereof

Abstract

The application provides a large field of view image real-time acquisition enhancement and multi-stage registration splicing system and method, which comprises a laser, a resonance / galvanometer mirror, a three-axis motor, a high-speed acquisition card, a computer host, a hardware control and scanning module, a resonance scanning acquisition and sine resampling module, a multi-thread real-time image enhancement module, a flat field correction module, an adaptive contrast adjustment module, an adaptive sharpening module, a large field of view snake-shaped scanning and tile acquisition module, a multi-stage registration splicing module, a depth direction multi-layer and layer map scanning module and a case and report management module. The application comprises the following steps: real-time acquisition and enhancement, large field of view snake-shaped scanning and tile acquisition, multi-stage registration splicing and depth direction multi-layer and layer map scanning. Thus, the problems of small single frame field of view, resonance scanning geometric distortion, uneven illumination, obvious splicing mismatch and joint artifact, and difficult self-adaptation of real-time contrast and sharpening can be solved, and the image geometry is consistent, the brightness is uniform, and the joint is natural.

Description

Technical Field

[0001] This invention relates to the field of biomedical imaging and image processing technology, and in particular to a system and method for real-time acquisition, enhancement, and multi-stage registration and stitching of large-field-of-view images. Background Technology

[0002] Reflection confocal microscopy uses near-infrared laser light to illuminate skin tissue and receives reflected light from a specific focal plane through a confocal pinhole. This allows for the acquisition of near-histological resolution microscopic images of the skin in vivo, and it has been widely applied in clinical settings such as assessment of pigmented skin lesions, screening for skin tumors, and follow-up of inflammatory skin diseases. Compared to traditional biopsies, this method offers advantages such as being non-invasive, repeatable, and allowing for real-time observation.

[0003] However, existing reflective confocal microscopy and its accompanying software still have the following prominent problems in practical clinical applications: (1) The single-frame field of view is too small to cover the whole picture of the lesion: The single-frame field of view of a reflection confocal microscope is usually only about a few hundred micrometers (e.g., about 500 micrometers × 500 micrometers). Clinical lesions are often several millimeters to more than ten millimeters in size. Doctors need to manually move the probe repeatedly to "piece together" the whole picture of the lesion. The operation is cumbersome, the positioning is difficult, and it is impossible to form a large panoramic image under a unified coordinate system, which affects the integrity of diagnosis and the repeatability of follow-up comparison.

[0004] (2) Non-uniform sampling of resonant scanning leads to geometric distortion: In order to improve the imaging frame rate, resonant galvanometers often perform rapid scanning with an approximate sinusoidal law. The original sampling points are not evenly distributed in space. If the images are rearranged directly at equal intervals, geometric distortions such as lateral stretching or compression will occur, resulting in inaccurate subsequent measurement and calibration, and significantly reducing the registration success rate when stitching multi-tiles.

[0005] (3) Uneven optical illumination and detector dark current causing brightness artifacts: Factors such as Gaussian distribution of laser spot, vignetting of optical system, and detector dark current can cause uneven brightness phenomena such as bright center, dark edge, or local hot spots in the same focal plane image. When used directly for splicing, brightness jumps and visible seams are likely to occur at the tile joints; when used for diagnostic observation, it will weaken structural contrast and interfere with the doctor's interpretation.

[0006] (4) Traditional stitching algorithms are prone to mismatching under skin microscopic texture: Skin confocal images have strong texture repetition (such as cell structure, reticular dermal texture), and there may be mechanical positioning errors, slight deformation and noise in the overlapping area of ​​adjacent tiles. When relying solely on frequency domain phase correlation or simple template matching, the correlation peaks often compete with each other, leading to incorrect displacement estimation; after the incorrect local displacement is accumulated by simple grid, it will form systematic misalignment and "tear" artifacts in the large mosaic.

[0007] (5) Insufficient real-time display and clinical usability: Skin reflection intensity varies significantly with individual, location, and depth. Fixed contrast window width is prone to overexposure or underexposure; fixed sharpening parameters amplify noise on low signal-to-noise ratio frames. Most existing equipment software adopts a "first acquisition, then offline processing" workflow, lacking a multi-threaded real-time enhancement pipeline synchronized with hardware scanning, and also lacking an integrated solution that connects single-frame acquisition, large field-of-view mosaicking, multi-slice scanning in the depth direction, case management, and report output.

[0008] (6) The separation between depth direction information and two-dimensional large field of view: Some existing solutions only support manual focusing or simple depth direction step-by-step image acquisition, which cannot automatically complete the horizontal large field of view mosaic at each depth layer, making it difficult to form inspection data that combines "planar overall view + depth layer". Summary of the Invention

[0009] The present invention aims to at least partially solve one of the technical problems in the related art.

[0010] Therefore, the purpose of this invention is to propose a large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching system and method, which can solve the problems of small single-frame field of view, resonant scanning geometric distortion, uneven illumination, stitching mismatch and obvious seam artifacts, and difficulty in adaptive real-time contrast and sharpening, and ensure image geometric consistency, uniform brightness and natural seams.

[0011] To achieve the above objectives, this invention proposes a large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching system and method, including a laser, a resonant / galvanometer galvanometer, a triaxial motor, a high-speed acquisition card, a computer host, a hardware control and scanning module, a resonant scanning acquisition and sinusoidal resampling module, a multi-threaded real-time image enhancement module, a flat field correction module, an adaptive contrast adjustment module, an adaptive sharpening module, a large-field-of-view serpentine scanning and tile acquisition module, a multi-stage registration and stitching module, a depth-direction multi-layer and layer-by-layer scanning module, and a case and report management module; the hardware control and scanning module is connected to the laser, the resonant / galvanometer galvanometer, the triaxial motor, and the large-field-of-view serpentine scanning and tile acquisition module, and is used to transmit images to the large-field-of-view serpentine scanning and tile acquisition module. The serpentine scanning and tile acquisition module provides an on-state condition; the resonant scanning acquisition and sinusoidal resampling module is connected to the high-speed acquisition card and the multi-threaded real-time image enhancement module respectively, and is used to receive the raw data from the high-speed acquisition card and input it into the multi-threaded real-time image enhancement module; the large-field-of-view serpentine scanning and tile acquisition module is connected to the multi-stage registration and stitching module, and is used to acquire images, write them to storage, and trigger the multi-stage registration and stitching module; the depth-direction multi-layer and layer-map scanning module is connected to the hardware control and scanning module, the large-field-of-view serpentine scanning and tile acquisition module, and the multi-stage registration and stitching module respectively; the computer host is equipped with an enhancement pipeline, and the flat field correction module, the adaptive contrast adjustment module, and the adaptive sharpening module are sequentially embedded in the enhancement pipeline.

[0012] A method for real-time acquisition, enhancement, and multi-stage registration and stitching of large-field-of-view images includes the following steps: S1. Real-time acquisition and enhancement: Raw scan data is acquired through a high-speed acquisition card, and enhanced frames are then processed by sinusoidal resampling, dynamic range mapping, multi-frame averaging, and adaptive contrast update before being written to storage. S2, wide-field serpentine scanning and tile acquisition, setting the mosaic area and grid, generating a serpentine path, moving point by point, stabilizing, acquiring images, and triggering stitching; S3. Multi-stage registration and stitching: After layout and edge set construction, edge estimation of relative displacement, median step size constraint, global position solution, and seam fusion, a large-view stitched image is output and archived. S4. Multi-layer and tomographic scanning in depth direction: Complete the acquisition of all depth layers through multi-layer mode and tomographic mode to display the lesion range and infiltration depth.

[0013] In addition, the large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching system and method proposed in the application may also have the following additional technical features: Specifically, the multi-threaded real-time image enhancement module is used for multi-level pipeline parallel processing. Each level is decoupled through queues to ensure that the real-time preview image frame rate and subsequent image acquisition for mosaic do not block each other.

[0014] Specifically, the flat field correction module is used to display pipeline cascades in real time and suppress the effects of uneven optical illumination and dark current. The adaptive contrast adjustment module is used to collect grayscale mappings from the front end of the link to achieve dynamic window width adaptation to skin reflection intensity and reduce overexposure and underexposure. The adaptive sharpening module is used to improve structural clarity while suppressing noise amplification.

[0015] Specifically, the depth-direction multi-layer and layer-map scanning module includes multi-layer scanning and layer-map scanning. The multi-layer scanning is used to acquire a single frame image at each depth, and the layer-map scanning is used to obtain three-dimensional inspection data.

[0016] Specifically, the case and report management module is used to manage patients and examination records, supports length and area measurement annotations on images based on field of view calibration, and generates and exports examination reports based on report templates.

[0017] Specifically, step S1 includes the following steps: S11. Raw data acquisition: The high-speed acquisition card acquires one frame of raw scan data according to the set number of trigger points. S12. Sinusoidal resampling: Based on the resonant scanning sinusoidal model, the non-uniform sampling columns are rearranged into uniform target columns using inverse cosine mapping to obtain the geometrically corrected intermediate data. S13, Dynamic Range Mapping and Multi-Frame Averaging: Maps intermediate data to the display grayscale range and performs sliding average according to the set number of frames to reduce instantaneous noise; S14. Adaptive contrast update: Calculate the average gray level of the entire image. If the average gray level is higher than the upper limit threshold, increase the upper limit of the mapping according to the deviation level. If the average gray level is lower than the lower limit threshold, decrease the upper limit of the mapping according to the deviation level, and ensure that it is not lower than the safe lower limit. S15, Optional flat field correction, enable flat field correction and dark field and flat field reference maps are available, correct the current frame, and cache templates by resolution to reduce redundant calculations; S16, Optional contrast-limited adaptive histogram equalization and adaptive sharpening, performs contrast-limited adaptive histogram equalization on the corrected image, and then the adaptive sharpening module adjusts the sharpening intensity based on the Laplacian variance and noise estimation and outputs the display frame. S17, Display and Branch Output: The enhanced frame is sent to the display. If it is in the state of image capture, mosaic, or recording, it is synchronously written to the corresponding storage sequence.

[0018] Specifically, step S2 includes the following steps: S21. Set the puzzle area and grid, select the diagonal point of the area of ​​interest, and determine the number of puzzle rows and columns, the amount of overlap, and the equivalent motor pulse for each step of displacement; S22. Generate a serpentine path and generate the tile access order according to the serpentine rules. Even-numbered rows are from left to right, and odd-numbered rows are from right to left, ensuring consistency with the subsequent layout construction rules. S23, point-by-point movement, stabilization, and image acquisition: For each grid point in the path, control the X-axis and Y-axis motors to move to the target pulse position, wait for a configurable stabilization delay to eliminate residual mechanical vibration, trigger the acquisition of one frame, and save it as a tile file after processing by the enhancement pipeline. S24. Trigger stitching: After all tiles have been acquired, inject the input directory, number of rows and columns, overlap parameters, registration and fusion parameters into the multi-stage registration and stitching module to start stitching.

[0019] Specifically, step S3 includes the following steps: S31. Layout and edge set construction: Read all tiles, establish tile adjacency relationships according to the serpentine layout, and generate all horizontal and vertical edges. S32. Estimate the relative displacement for each edge, extract overlapping regions of interest by direction, remove edge whitespace, preprocess by high-pass filtering and normalization, obtain sub-pixel displacement, primary-secondary peak ratio and candidate peak list by discrete Fourier transform phase correlation, if the peak value is lower than the ambiguity threshold, perform spatial disambiguation on the first few candidate peak values, perform pattern search for the objective function, obtain the candidate with the highest score and score margin that meets the threshold as the final displacement, and record the displacement, error metric and weight correlation index of the edge. S33. Median step size constraint: Calculate the median step size by row or column for the coarse registration edge set, construct a step size model, and then perform fine registration to constrain the search center to the vicinity of the predicted displacement. Perform median step size replacement or weight reduction on the out-of-range edges. S34. Global position solution: With the starting tile as the origin of the coordinate system, establish a set of relative displacement constraint equations, assign weights according to the edge confidence, and use weighted least squares to solve the global coordinates of each tile. S35, seam blending: For overlapping seam areas, perform a linear transition of transparency based on the blending width; for non-overlapping areas, assign center priority based on the distance from the tile center; accumulate pixels according to weight and normalize; crop according to the strict inscribed effective area or bounding box; and output the maximum field of view stitched image. S36. Results archiving: Write the mosaic image into the inspection catalog for display, measurement, and reporting reference.

[0020] Specifically, step S4 includes the following steps: S41, Multi-layer mode: Set the starting depth, step size, and number of steps. After each step of moving the Z-axis, a single frame is captured and saved to form a depth sequence. S42, Layer Map Mode: For each depth layer, complete horizontal serpentine tile acquisition and stitching are performed. After completion, the Z-axis is moved to the next layer by step, until all depth layers are completed.

[0021] Compared with the prior art, the present invention has the following advantages: 1. It realizes the integration of resonant scanning geometric correction and real-time enhancement. By using inverse cosine resampling based on a sine scanning model, it eliminates lateral distortion caused by non-uniform sampling. Then, through a multi-threaded pipeline, it completes flat field correction, adaptive contrast and adaptive sharpening, so that the real-time preview and archived images have geometric consistency and readability at the same time, solving the problem of "processing after sampling and inconsistency between preview and archive".

[0022] 2. Achieve automatic coverage of a large field of view at the millimeter level, breaking through the limitation of a small field of view in a single frame. Automatically acquire multi-tile images by means of hardware serpentine grid scanning and strict alignment of software layout. It can cover an examination area of ​​about a few millimeters to nearly a centimeter, significantly improving the efficiency and repeatability of observing the whole picture of lesions.

[0023] 3. Through multi-stage registration and stitching, based on discrete Fourier transform sub-pixel registration, and considering the low peak-to-peak ratio and multi-peak competition in skin microscopic images, a spatial candidate disambiguation method based on gray-level zero-mean normalized cross-correlation and gradient zero-mean normalized cross-correlation is introduced to effectively reduce the mismatch rate and improve the structural continuity of large mosaics.

[0024] 4. Achieve global optimization and robust seam fusion to reduce misalignment and seam artifacts. Through confidence-weighted global position solving and optional robust iteration and median step-size bad edge backoff, suppress the propagation of local error displacement to the whole image. Furthermore, feathering fusion and center priority strategies are used to reduce seam brightness jumps and bilateral ghosting, thereby improving the visual consistency and diagnostic usability of large field-of-view images.

[0025] 5. Enables joint acquisition of two-dimensional large field of view and depth information, supports multi-layer single-frame sequences and layer map mode of "first stitching and then stepping" for each layer, so that the examination data has both planar coverage and depth information, which facilitates the comprehensive assessment of lesion range and infiltration depth.

[0026] 6. A clinical closed loop of data acquisition, enhancement, stitching, measurement, and reporting is formed. The case and report management module connects hardware control, image processing, case archiving, and report output, reducing the need for switching between multiple software programs and manual secondary processing, thereby improving the efficiency of clinical workflow and the standardization of data.

[0027] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0028] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a general structural block diagram of the present invention; Figure 2 This is a schematic diagram of the multi-threaded real-time acquisition enhancement pipeline of the present invention; Figure 3 This is a schematic diagram of the wide-field serpentine scanning path and tile numbering of the present invention; Figure 4 This is a flowchart of the multi-stage registration and stitching method of the present invention; Figure 5 This is a schematic diagram of frequency domain registration and spatial candidate disambiguation according to the present invention; Figure 6 This is a schematic diagram of global position solving and seam fusion in this invention; Figure 7 This is a schematic diagram of the multi-layer scanning and layer map scanning process of the present invention. Detailed Implementation

[0029] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein reference numerals such as AND or similar denote elements or elements having similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the invention, and should not be construed as limiting the invention. Rather, embodiments of the invention include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0030] The following description, in conjunction with the accompanying drawings, describes a large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching system and method according to an embodiment of the present invention.

[0031] Example 1

[0032] like Figure 1 As shown, the large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching system of this invention includes a laser, a resonant / galvanometer galvanometer, a triaxial motor, a high-speed acquisition card, a computer host, a hardware control and scanning module, a resonant scanning acquisition and sinusoidal resampling module, a multi-threaded real-time image enhancement module, a flat field correction module, an adaptive contrast adjustment module, an adaptive sharpening module, a large-field-of-view serpentine scanning and tile acquisition module, a multi-stage registration and stitching module, a depth-direction multi-layer and layer-by-layer scanning module, and a case and report management module.

[0033] It should be noted that the case and report management module manages patient and examination records, and archives examinations as single-frame images, mosaics, multi-layer images, layer images, and videos; it supports length and area measurement annotations on images based on field of view calibration; and it generates and exports examination reports based on report templates.

[0034] The hardware control and scanning module is connected to the laser, the resonant / galvanometer galvanometer mirror, the triaxial motor, and the large-field-of-view serpentine scanning and tile acquisition module, respectively, and is used to provide the positioning status to the large-field-of-view serpentine scanning and tile acquisition module.

[0035] It should be noted that the hardware control and scanning module controls the laser, resonant mirror, galvanometer mirror, and X-axis, Y-axis, and Z-axis motors respectively through serial communication protocol. In the jigsaw puzzle mode, the sample is scanned in a serpentine path according to the preset grid, so that the scanning direction of odd rows is opposite to that of even rows, in order to reduce idle travel and improve scanning efficiency. After the motor is in place, the image acquisition is triggered after a configurable stable delay to ensure that the tile exposure and mechanical positioning are synchronized.

[0036] The resonant scanning acquisition and sinusoidal resampling modules are connected to the high-speed acquisition card and the multi-threaded real-time image enhancement module, respectively, to receive the raw data from the high-speed acquisition card and input it into the multi-threaded real-time image enhancement module.

[0037] It should be noted that in the resonant scanning acquisition and sinusoidal resampling modules, the high-speed acquisition card acquires the original scan line data upon triggering; the sinusoidal resampling module, based on the sinusoidal motion model of the resonant galvanometer scan, uses inverse cosine mapping to rearrange the non-uniform sampling points into horizontally uniform target column pixels (e.g., 1024 columns), and maintains energy conservation during interpolation or accumulation, thereby eliminating the geometric distortion of the resonant scan and providing a geometrically consistent input image for subsequent measurement and stitching.

[0038] The multi-threaded real-time image enhancement module employs multi-stage pipelined parallel processing, typically including: a raw data acquisition thread, a sinusoidal resampling and cumulative averaging thread, a dynamic range mapping and multi-frame moving average thread, an image matrix writing and average grayscale statistics thread, and a flat field correction / contrast-limited adaptive histogram equalization / adaptive sharpening and display thread. Each stage is decoupled through queues to ensure that the real-time preview frame rate and subsequent image acquisition for stitching do not block each other.

[0039] The large-field serpentine scanning and tile acquisition module is connected to the multi-stage registration and stitching module, which is used to acquire images, write them to storage, and trigger the multi-stage registration and stitching module.

[0040] It should be noted that the large-field serpentine scanning and tile acquisition module automatically generates a scanning grid within the user-selected region of interest, according to a configurable number of rows and columns (e.g., 17 rows × 17 columns). Each grid point corresponds to a tile image, which is numbered and saved in serpentine order. The tile file names correspond one-to-one with the subsequent stitching layout, ensuring that the hardware scanning path is consistent with the software layout model.

[0041] Among them, the depth direction multi-layer and layer map scanning modules are connected to the hardware control and scanning module, the large field-of-view snake scanning and tile acquisition module and the multi-stage registration and stitching module, respectively. The computer host is equipped with an enhancement pipeline, and the flat field correction module, the adaptive contrast adjustment module and the adaptive sharpening module are embedded in the enhancement pipeline in sequence.

[0042] It should be noted that the flat-field correction module pre-calibrates and stores the dark-field reference image and the flat-field reference image; during runtime, it caches floating-point templates according to the current image resolution and corrects the input image using the following formula: Corrected image = (Source image - Dark-field image) / (Flat-field image - Dark-field image + Small constant) × (Flat-field image - Dark-field image) mean, where the small constant is used to prevent division by zero; the correction result is cropped to the effective grayscale range. This module can be cascaded with the real-time display pipeline to suppress optical illumination unevenness and dark current effects.

[0043] The adaptive contrast adjustment module calculates the average grayscale of the entire image in real time. When the average grayscale is higher than the upper threshold, the dynamic range mapping upper limit is increased according to the deviation; when the average grayscale is lower than the lower threshold, the mapping upper limit is decreased symmetrically while ensuring a safe lower limit boundary. This adjustment applies to the grayscale mapping at the front end of the acquisition link, achieving dynamic window width adaptation for different skin reflectance intensities and reducing overexposure and underexposure.

[0044] The adaptive sharpening module uses Laplacian variance to characterize sharpness and sets a target sharpness. It calculates the ideal sharpening intensity based on the ratio of the current sharpness to the target sharpness and applies a penalty to the sharpening intensity by combining flat area noise estimation. After exponential smoothing across frames, it uses anti-masking sharpening output to improve structural sharpness while suppressing noise amplification.

[0045] The depth-oriented multi-layer and layer-by-layer scanning modules include: Multi-layer scanning: Automatically controls Z-axis stepping according to the set starting depth and step size, and acquires a single frame image at each depth; Layer-by-layer scanning: Performs a complete horizontal large-field-of-view serpentine scan and stitching at each depth layer, and then steps to the next depth to obtain three-dimensional inspection data of "large field of view per layer".

[0046] Example 2

[0047] like Figure 2-7 As shown, the large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching method of this invention includes the following steps: Step 1, Real-time Acquisition and Enhancement, includes the following steps: Step 11: Raw data acquisition. The high-speed acquisition card acquires one frame of raw scan data according to the set number of trigger points.

[0048] Step 12: Sine resampling. Based on the resonant scanning sine model, the non-uniform sampling columns are rearranged into uniform target columns using inverse cosine mapping to obtain the geometrically corrected intermediate data.

[0049] Step 13: Dynamic range mapping and multi-frame averaging. The intermediate data is mapped to the display grayscale range and a sliding average is performed according to the set number of frames to reduce instantaneous noise.

[0050] Step 14: Adaptive contrast update. Calculate the average gray level of the entire image. If the average gray level is higher than the upper limit threshold, increase the upper limit of the mapping according to the deviation level. If the average gray level is lower than the lower limit threshold, decrease the upper limit of the mapping according to the deviation level, and ensure that it is not lower than the safe lower limit.

[0051] Step 15: Optional flat field correction. Enable flat field correction and make dark field and flat field reference maps available. Correct the current frame and cache templates by resolution to reduce redundant calculations.

[0052] Step 16: Optional contrast-limited adaptive histogram equalization and adaptive sharpening. Perform contrast-limited adaptive histogram equalization on the corrected image, and then the adaptive sharpening module adjusts the sharpening intensity based on the Laplacian variance and noise estimation and outputs the display frame.

[0053] Step 17: Display and Branch Output. Send the enhanced frame to the display. If it is in the state of image acquisition, mosaicking, or recording, write it to the corresponding storage sequence synchronously.

[0054] Step 2, wide-field serpentine scanning and tile acquisition, includes the following steps: Step 21: Set the puzzle area and grid, select the diagonal point of the area of ​​interest, and determine the number of puzzle rows and columns, the amount of overlap, and the equivalent motor pulse for each step of displacement.

[0055] Step 22: Generate a serpentine path. Generate the tile access order according to the serpentine rules, with even-numbered rows from left to right and odd-numbered rows from right to left, to ensure consistency with the subsequent layout construction rules.

[0056] Step 23: Move point by point, stabilize, and acquire data. For each grid point in the path, perform the following steps: (1) Control the X-axis and Y-axis motors to move to the target pulse position, (2) wait for the configurable stable delay to eliminate mechanical residual vibration, (3) trigger the acquisition of a frame and save it as a tile file after processing by the enhancement pipeline.

[0057] Step 24: Trigger stitching. After all tiles have been acquired, inject the input directory, number of rows and columns, overlap parameters, registration and fusion parameters into the multi-stage registration and stitching module to start stitching.

[0058] Step 3: Multi-stage registration and stitching, including the following steps: Step 31: Layout and edge set construction. Read all tiles, establish tile adjacency relationships according to the serpentine layout, and generate all horizontal and vertical edges. Step 32: Estimate the relative displacement for each edge, including the following steps: (1) Extract overlapping regions of interest by direction and remove edge white space. (2) Preprocessing: optional high-pass filtering, normalization, optional Hanning window. (3) Use discrete Fourier transform phase correlation to obtain sub-pixel displacement, primary-secondary peak ratio and candidate peak list. (4) If the peak value is lower than the ambiguity threshold, perform spatial disambiguation on the first few candidate peak values, perform pattern search for the objective function, and obtain the candidate with the highest score and score margin that meets the threshold as the final displacement. (5) Record the displacement, error metric and weight correlation index of the edge.

[0059] Step 33: Median step size constraint. Calculate the median step size by row or column for the coarse registration edge set, construct the step size model, and then perform fine registration. Constrain the search center to be near the predicted displacement, and perform median step size replacement or weight reduction for edges with out-of-range errors.

[0060] Step 34: Global position solution. With the starting tile as the origin of the coordinate system, establish a set of relative displacement constraint equations, assign weights according to the edge confidence, and use weighted least squares to solve the global coordinates of each tile.

[0061] Step 35: Seam blending. For overlapping seam areas, apply a linear transition of transparency based on the blending width. For non-overlapping areas, assign center priority based on the distance from the tile center. Accumulate pixels according to weight and normalize. Crop according to the strictly inscribed effective area or the bounding box. Output the stitched image with the maximum field of view.

[0062] Step 36: Archive the results and write the mosaic image into the inspection catalog for display, measurement, and reporting reference.

[0063] Step 4, multi-layer and layer map scanning in the depth direction, includes the following steps: Step 41: Multi-layer mode. Set the starting depth, step size, and number of steps. After each step, move along the Z-axis and capture a single frame and save it to form a depth sequence.

[0064] Step 42, Layer Map Mode: Perform complete horizontal serpentine tile acquisition and stitching for each depth layer. After completion, move the Z-axis to the next layer by step distance until all depth layers are completed.

[0065] Example 3

[0066] like Figure 1 As shown, the operation of the large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching system according to an embodiment of the present invention includes the following steps: Step 1: Initialize the check, create or select patient examination records, initialize serial communication and data acquisition card, enable laser and galvanometer, and complete motor return to zero or current position confirmation.

[0067] Step 2: Real-time preview and parameter confirmation. Start the multi-threaded acquisition enhancement pipeline. The doctor confirms the focal plane, laser power, galvanometer amplitude, contrast and sharpening effect in the preview window; enable flat field correction if necessary.

[0068] Step 3: Select the imaging mode. Options include: single frame capture, large field of view mosaic, multi-layer scanning, layer scan, and video recording.

[0069] Step 4: Perform the corresponding data collection and processing, following the steps for each mode below, and archive the results to the current inspection directory.

[0070] Step 5: Measurement, Diagnosis and Reporting. Mark measurements on the archived images, fill in diagnostic comments, and generate a report according to the template.

[0071] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the corresponding embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the features described in this specification that are not related to embodiments or examples.

[0072] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A system for real-time acquisition, enhancement, and multi-stage registration and stitching of large-field-of-view images, characterized in that, It includes a laser, a resonant / galvanometer galvanometer, a triaxial motor, a high-speed acquisition card, a computer host, a hardware control and scanning module, a resonant scanning acquisition and sinusoidal resampling module, a multi-threaded real-time image enhancement module, a flat field correction module, an adaptive contrast adjustment module, an adaptive sharpening module, a large field-of-view serpentine scanning and tile acquisition module, a multi-stage registration and stitching module, a depth-direction multi-layer and tomographic scanning module, and a case and report management module; The hardware control and scanning module is connected to the laser, the resonant / galvanometer galvanometer mirror, the triaxial motor, and the large field-of-view serpentine scanning and tile acquisition module, respectively, and is used to provide the positioning status to the large field-of-view serpentine scanning and tile acquisition module; The resonant scanning acquisition and sinusoidal resampling module is connected to the high-speed acquisition card and the multi-threaded real-time image enhancement module, respectively, and is used to receive the raw data from the high-speed acquisition card and input it into the multi-threaded real-time image enhancement module. The wide-field snake scanning and tile acquisition module is connected to the multi-stage registration and stitching module, which is used to acquire images, write them to storage, and trigger the multi-stage registration and stitching module. The depth-direction multi-layer and layer map scanning modules are respectively connected to the hardware control and scanning module, the large field-of-view snake scanning and tile acquisition module, and the multi-stage registration and stitching module. The computer host is equipped with an enhancement pipeline, and the flat field correction module, the adaptive contrast adjustment module, and the adaptive sharpening module are sequentially embedded in the enhancement pipeline.

2. The large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching system according to claim 1, characterized in that, The multi-threaded real-time image enhancement module is used for multi-level pipeline parallel processing. Each level is decoupled through queues to ensure that the real-time preview image frame rate and subsequent image acquisition for mosaic do not block each other.

3. The large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching system according to claim 1, characterized in that, The flat field correction module is used to display pipeline cascades in real time and suppress the effects of uneven optical illumination and dark current. The adaptive contrast adjustment module is used to collect grayscale mapping of the link front end to achieve dynamic window width adaptation to skin reflection intensity and reduce overexposure and underexposure. The adaptive sharpening module is used to improve structural clarity while suppressing noise amplification.

4. The large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching system according to claim 1, characterized in that, The depth-direction multi-layer and layer-map scanning module includes multi-layer scanning and layer-map scanning. The multi-layer scanning is used to acquire a single frame image at each depth, and the layer-map scanning is used to obtain three-dimensional inspection data.

5. The large-field-of-view image real-time acquisition enhancement and multi-stage registration and stitching system according to claim 1, characterized in that, The case and report management module is used to manage patients and examination records, supports length and area measurement annotation based on field of view calibration on images, and generates and exports examination reports based on report templates.

6. A method for real-time acquisition, enhancement, and multi-stage registration and stitching of large-field-of-view images, used in the real-time acquisition, enhancement, and multi-stage registration and stitching system for large-field-of-view images as described in any one of claims 1-5, characterized in that, Includes the following steps: S1. Real-time acquisition and enhancement: Raw scan data is acquired through a high-speed acquisition card, and enhanced frames are then processed by sinusoidal resampling, dynamic range mapping, multi-frame averaging, and adaptive contrast update before being written to storage. S2, wide-field serpentine scanning and tile acquisition, setting the mosaic area and grid, generating a serpentine path, moving point by point, stabilizing, acquiring images, and triggering stitching; S3. Multi-stage registration and stitching: After layout and edge set construction, edge estimation of relative displacement, median step size constraint, global position solution, and seam fusion, a large-view stitched image is output and archived. S4. Multi-layer and tomographic scanning in depth direction: Complete the acquisition of all depth layers through multi-layer mode and tomographic mode to display the lesion range and infiltration depth.

7. The method for real-time acquisition, enhancement, and multi-stage registration and stitching of large-field-of-view images according to claim 6, characterized in that, Step S1 includes the following steps: S11. Raw data acquisition: The high-speed acquisition card acquires one frame of raw scan data according to the set number of trigger points. S12. Sinusoidal resampling: Based on the resonant scanning sinusoidal model, the non-uniform sampling columns are rearranged into uniform target columns using inverse cosine mapping to obtain the geometrically corrected intermediate data. S13, Dynamic Range Mapping and Multi-Frame Averaging: Maps intermediate data to the display grayscale range and performs sliding average according to the set number of frames to reduce instantaneous noise; S14. Adaptive contrast update: Calculate the average gray level of the entire image. If the average gray level is higher than the upper limit threshold, increase the upper limit of the mapping according to the deviation level. If the average gray level is lower than the lower limit threshold, decrease the upper limit of the mapping according to the deviation level, and ensure that it is not lower than the safe lower limit. S15, Optional flat field correction, enable flat field correction and dark field and flat field reference maps are available, correct the current frame, and cache templates by resolution to reduce redundant calculations; S16, Optional contrast-limited adaptive histogram equalization and adaptive sharpening, performs contrast-limited adaptive histogram equalization on the corrected image, and then the adaptive sharpening module adjusts the sharpening intensity based on the Laplacian variance and noise estimation and outputs the display frame. S17, Display and Branch Output: The enhanced frame is sent to the display. If it is in the state of image capture, mosaic, or recording, it is synchronously written to the corresponding storage sequence.

8. The method for real-time acquisition, enhancement, and multi-stage registration and stitching of large-field-of-view images according to claim 6, characterized in that, Step S2 includes the following steps: S21. Set the puzzle area and grid, select the diagonal point of the area of ​​interest, and determine the number of puzzle rows and columns, the amount of overlap, and the equivalent motor pulse for each step of displacement; S22. Generate a serpentine path and generate the tile access order according to the serpentine rules. Even-numbered rows are from left to right, and odd-numbered rows are from right to left, ensuring consistency with the subsequent layout construction rules. S23, point-by-point movement, stabilization, and image acquisition: For each grid point in the path, control the X-axis and Y-axis motors to move to the target pulse position, wait for a configurable stabilization delay to eliminate residual mechanical vibration, trigger the acquisition of one frame, and save it as a tile file after processing by the enhancement pipeline. S24. Trigger stitching: After all tiles have been acquired, inject the input directory, number of rows and columns, overlap parameters, registration and fusion parameters into the multi-stage registration and stitching module to start stitching.

9. The method for real-time acquisition, enhancement, and multi-stage registration and stitching of large-field-of-view images according to claim 6, characterized in that, Step S3 includes the following steps: S31. Layout and edge set construction: Read all tiles, establish tile adjacency relationships according to the serpentine layout, and generate all horizontal and vertical edges. S32. Estimate the relative displacement for each edge, extract overlapping regions of interest by direction, remove edge whitespace, preprocess by high-pass filtering and normalization, obtain sub-pixel displacement, primary-secondary peak ratio and candidate peak list by discrete Fourier transform phase correlation, if the peak value is lower than the ambiguity threshold, perform spatial disambiguation on the first few candidate peak values, perform pattern search for the objective function, obtain the candidate with the highest score and score margin that meets the threshold as the final displacement, and record the displacement, error metric and weight correlation index of the edge. S33. Median step size constraint: Calculate the median step size by row or column for the coarse registration edge set, construct a step size model, and then perform fine registration to constrain the search center to the vicinity of the predicted displacement. Perform median step size replacement or weight reduction on the out-of-range edges. S34. Global position solution: With the starting tile as the origin of the coordinate system, establish a set of relative displacement constraint equations, assign weights according to the edge confidence, and use weighted least squares to solve the global coordinates of each tile. S35, seam blending: For overlapping seam areas, perform a linear transition of transparency based on the blending width; for non-overlapping areas, assign center priority based on the distance from the tile center; accumulate pixels according to weight and normalize; crop according to the strict inscribed effective area or bounding box; and output the maximum field of view stitched image. S36. Results archiving: Write the mosaic image into the inspection catalog for display, measurement, and reporting reference.

10. The method for real-time acquisition, enhancement, and multi-stage registration and stitching of large-field-of-view images according to claim 6, characterized in that, Step S4 includes the following steps: S41, Multi-layer mode: Set the starting depth, step size, and number of steps. After each step of moving the Z-axis, a single frame is captured and saved to form a depth sequence. S42, Layer Map Mode: For each depth layer, complete horizontal serpentine tile acquisition and stitching are performed. After completion, the Z-axis is moved to the next layer by step, until all depth layers are completed.

Images