Method for improving the quality of scanned images based on multi-frame registration and averaging algorithm

Abstract

The application discloses a method for improving the quality of scanning images based on multi-frame registration and average algorithm, comprising the following steps: inputting N B-scans; selecting a reference B-scan; preprocessing the B-scans; calculating the axial offset of each A-scan in the moving B-scan relative to the corresponding A-scan in the reference B-scan; determining the axial offset by calculating the normalized cross-correlation function between the A-scans; performing linear regression, obtaining the best curve through all points of the maximum cross-correlation, eliminating abnormal points in the distribution, and repeating the linear regression; calculating the conversion between the moving B-scan and the reference B-scan; selecting the best moving B-scan and sorting; creating a composite image with the first several moving B-scans and the reference B-scan; sorting the first n moving B-scans and the selected reference B-scan in the list for creating a composite image; and post-processing the composite image. The application effectively improves the quality of B-scan images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology. Background Technology

[0002] Combining multiple B-scan images obtained by repeatedly scanning the same tomographic location of the eye presents a problem: eye movement occurs both within and between consecutive scans, even during the short time required to collect the scans. This can lead to temporally separated scans being spatially desynchronized. Ideally, without eye movement during scanning, each frame, as a cross-section of the same location, would display the same tomographic structure. However, due to the spatial misalignment caused by eye movement, consecutive frames will display spatially shifted tomographic features. If this effect is not considered, the averaging process will average the different tomographic features from the scans, resulting in a degraded visual effect, such as blurring, false edges, and even damage to the information content. Figures 2-4 This displays the misalignment of three B-scans for a single frame of a 4x5 raster.

[0003] Error registration between three scans of a 1-frame B-scan using a 4x5 raster is displayed in a checkerboard format. Figure 2 , Figure 3 and Figure 4 The images show spatial misalignments between B-scan images 1 and 2, 1 and 3, and 1 and 4. Effective methods are needed to improve the quality of B-scan images. Summary of the Invention

[0004] The purpose of this invention is to provide a method for improving the quality of scanned images based on multi-frame registration and averaging algorithms, thereby effectively improving the quality of B-scan images.

[0005] The technical solution to achieve the above objectives is:

[0006] Methods for improving the quality of scanned images based on multi-frame registration and averaging algorithms include:

[0007] Input N B-scans (computed tomography scans);

[0008] Select a reference B-scan;

[0009] Preprocessing B-scan;

[0010] Calculate the axial offset of each A-scan (axial scan) in the moving B-scan relative to the corresponding A-scan in the reference B-scan; determine the axial offset by calculating the normalized cross-correlation function between these A-scans;

[0011] Perform linear regression to obtain the optimal curve through all points with the maximum cross-correlation, remove outliers in the distribution, and repeat the linear regression.

[0012] Calculate the conversion between the moving B-scan and the reference B-scan;

[0013] Select the best B-scan and sort it;

[0014] Create a composite image using the preceding moving B-scans and the reference B-scan;

[0015] The first n moving B-scans in the sorted list and the selected reference B-scan are used to create the composite image;

[0016] Post-processing of the synthesized image.

[0017] Preferably, the preprocessing of B-scan includes:

[0018] A set of uniformly distributed A-scans is selected from the reference B-scans to calculate the offset. Based on the range of the selected A-scans, a set of 3 B-scans are preprocessed: speckle is treated as a high-frequency component, and a speckle content image is generated by high-pass filtering of the frame. Then, the speckle content image is subtracted from the original image to obtain an image with higher visual quality. This process is repeated 3 times, and then a median image is constructed from this set.

[0019] Preferably, linear regression is performed to obtain the optimal curve through all points with the maximum cross-correlation, outliers in the distribution are removed, and linear regression is repeated, including:

[0020] A set of A-scans is used to find the translation, which indicates the maximum cross-correlation between the reference B-scan and the moving B-scan at each position; this is a set of P points spanning the length of the B-scan, indicating the vertical translation at that position; then linear regression is used to obtain the equation of the best-fit line through these points, after obtaining the best curve, the deviation of each point from this best curve is calculated, the distribution of these deviations is obtained, points that are further than a threshold are discarded, and a new regression line is derived from the remaining points.

[0021] Preferably, calculating the conversion between the moving B-scan and the reference B-scan includes:

[0022] The translation correction for the transformation is evaluated using the difference between the maximum normalized cross-correlation values ​​of the two A-scans; a pair of lines are sampled from the corresponding positions of each B-scan along the x and z directions; the intersection of these lines defines a rectangle ABCD, which corresponds to the reference B-scan coordinate system to be transformed; an axial translation correction is added to this rectangle to obtain a parallelogram A'B'C'D'.

[0023] Preferably, the best moving B-scan is selected, including:

[0024] The function formula for creating a composite image is expressed as follows:

[0025]

[0026] in and Let be the NCC between the A-scan of the moving B-scan and the A-scan of the reference B-scan; T is the set of moving B-scans, s is a subset of the B-scans of T, and r is the reference B-scan; if the sum of the absolute differences of the NCCs of the moving B-scans and the reference B-scans is minimized, and the sum of the NCCs is maximized, then it is considered the best subset for creating the synthetic image; calculate the NCC of each moving B-scan. And sort them in ascending order.

[0027] Preferably, it also includes: evaluating the sub-synthesized image before creating the synthesized image.

[0028] Preferably, post-processing of the synthesized image includes:

[0029] Adaptive histogram equalization is performed on the synthesized image to obtain a secondary synthesized image;

[0030] Create two weight functions, representing two opposite axial ramp functions;

[0031] The composite image is combined with the first weighting function to obtain the first result;

[0032] The image is synthesized twice using a multiple of the original image and a second weighting function to obtain the second result.

[0033] The weighted average of the first and second results. The weight of this algorithm is 0.5.

[0034] Adjusting the contrast of the weighted average image by stretching the image histogram;

[0035] Gamma correction is applied based on the gamma index specified by the user.

[0036] Preferably, N is a natural number and greater than 50.

[0037] The beneficial effects of this invention are: Based on acquiring multiple B-scan images at the same location, and utilizing frame averaging and averaging algorithms, this invention presents the user with a higher resolution B-scan image than other scanning modes. It is particularly suitable for combining a large number of B-scan images. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method for improving the quality of scanned images based on multi-frame registration and averaging algorithms according to the present invention;

[0039] Figure 2 This is a schematic diagram of the spatial misalignment between the first and second scans of a 4x5 grating frame B-scan.

[0040] Figure 3 This is a schematic diagram of the spatial misalignment between the first and third scans of a single frame of B-scan using a 4x5 grating.

[0041] Figure 4 This is a schematic diagram of the spatial misalignment between the first and fourth scans of a single frame B-scan using a 4x5 grating.

[0042] Figure 5 This is a schematic diagram of a set of uniformly distributed A-scans in this invention;

[0043] Figure 6a This is a schematic diagram showing the offset of the NCC at the A-scan position in this invention;

[0044] Figure 6b This is a schematic diagram showing the offset of the two B-scans at the A-scan position in this invention;

[0045] Figure 7a This is a schematic diagram of the reference frame in this invention;

[0046] Figure 7b This is a schematic diagram of the dynamic frame in this invention;

[0047] Figure 8a This is a schematic diagram of the NCC before reordering in this invention;

[0048] Figure 8b This is a schematic diagram of the reordered moving B-scan in this invention;

[0049] Figure 8c This is a schematic diagram of the NCC after reordering in this invention;

[0050] Figure 9a This is an image before compositing according to an embodiment of the present invention;

[0051] Figure 9b This is a synthesized image result according to an embodiment of the present invention. Detailed Implementation

[0052] The invention will now be further described with reference to the accompanying drawings.

[0053] Please see Figures 1 to 9bA method for improving the quality of scanned images based on multi-frame registration and averaging algorithms includes the following steps:

[0054] Step S1: Input N B-scans, where each B-scan may be 512×1024 pixels or 1024×1024 pixels. The acquisition time should be long enough for eye movements to cause spatial misalignment of consecutive B-scans, thus affecting image formation. N is a natural number, ideally greater than 50.

[0055] Step S2: Select a reference B-scan. Specifically, a reference B-scan is selected from a subset of M B-scans out of N B-scans to maximize the contrast of the sub-synthesized image. For example, every k-th B-scan is selected as a candidate reference B-scan; for instance, 100 B-scans are divided into 10 groups of 10, and the best reference image is selected from each group. K and M are both natural numbers.

[0056] Step S3, preprocess the B-scan. Select a set of uniformly distributed A-scans from the reference B-scan to calculate the offset, such as... Figure 5 As shown. The offset is determined by finding the maximum normalized cross-correlation between the dynamic image coordinate system and the reference image coordinate system at each A-scan position, and then performing linear regression to obtain the best-fit line. The offset and tilt are calculated, and the dynamic image is transformed to match the reference image.

[0057] Based on the selected A-scan range, a set of three B-scans are preprocessed, primarily to reduce the speckle content in the image. To achieve a fast and efficient speckle reduction method that preserves salient features, speckle is treated as a high-frequency component, and a speckle content image is generated through frame high-pass filtering (3x3 kernel matrix). This speckle content image is then subtracted from the original image to obtain an image with higher visual quality. It has been observed that repeating this process three times, and then constructing a median image from this set, achieves a good balance between speckle reduction and edge smoothing.

[0058] Extract all A-scans with reduced speckle from the B-scans and convolve them using a Gaussian smoothing function with a standard deviation of 3.

[0059] In the context of ophthalmic imaging, scans are typically performed at adjacent tomographic locations. This helps eliminate speckle noise between frames because it acquires more diverse speckles compared to wobbly beam scanning and assumes that the primary cause of image misregistration is eye movement. Once successfully registered, each pair of images is mutually registered and can be combined using an arithmetic mean or median, prioritizing enhancement of system intensity variations and eliminating random noise. Frame averaging is a method to improve B-scan image quality by combining multiple mutually registered images.

[0060] Step S4, Normalized Cross-Correlation: Calculate the axial offset of each A-scan in the moving B-scan relative to the corresponding A-scan in the reference B-scan. The axial offset is determined by calculating the normalized cross-correlation function between these A-scans. The axial offset relative to the center of the A-scan is determined when this function reaches its maximum value. The value of the normalized cross-correlation varies between 0 and 1. Normalized cross-correlation is a simple and effective similarity measure that is unaffected by linear changes in brightness and contrast. Below... Figure 6a and Figure 6b This shows the NCC (maximum correlation value in the cross-correlation) and the offsets of the two B-scans at the A-scan positions. Note that the difference in the offset of each A-scan position in the B-scan indicates that axial misalignment occurs in the B-scan.

[0061] Step S5: Perform linear regression to obtain the best curve through all points with the maximum cross-correlation, remove outliers from the distribution, and repeat the linear regression. A set of A-scans is used to find the translation, which indicates the maximum cross-correlation between the reference B-scan and the moving B-scan at each location. This is a set of P points spanning the length of the B-scan, indicating the vertical translation at that location. Linear regression is then used to obtain the equation of the best-fit line through these points. After obtaining the best curve, the deviation of each point from this best curve is calculated. The distribution of these deviations is obtained, and points further than a threshold (defined as a factor of the standard deviation of these values) are discarded. A new regression line is derived from the remaining points. This regression line is then used to calculate the vertical translation and tilt used in the restricted affine transformation between B-scans. The advantage of doing this is that rapidly changing features in the B-scan (such as small drusen) will not have a significant impact on the final transformation. These features may also change significantly with a small number of axial translations between consecutive B-scans.

[0062] Step S6: Calculate the transformation between the moving B-scan and the reference B-scan. The translation correction for the transformation is evaluated using the difference between the maximum normalized cross-correlation values ​​of the two A-scans. To apply the displacement to the entire B-scan, a pair of lines is sampled along the x and z directions from the corresponding position on each B-scan. The intersection of these lines defines a rectangle ABCD, corresponding to the reference B-scan coordinate system to which the transformation is to be performed. Adding an axial translation correction to this rectangle results in a parallelogram A'B'C'D', as follows: Figure 7a , 7b As shown. Thus, the registration problem is defined as transforming parallelogram A1B1C1D1 (dynamic B-scan) into rectangle ABCD (reference B-scan). The lateral position of the rectangle is the position of the selected A-scan in the reference B-scan, and the axial position of the rectangle has a fixed offset from the top and bottom of the B-scan image. Here, the dynamic frame is the moving frame.

[0063] Due to eye movements of the scanned object, B-scans acquired at the same location at very close time points may exhibit distortion. This problem can be simplified using affine transformations, which typically describe rotations, scaling, shearing, and translations that may exist between a pair of images. The process of geometrically aligning two images is called registration. Its goal is to achieve a consistent spatial correspondence between each image acquisition and a reference image by calculating and evaluating transformation parameters, and then using this registration method to combine the OCT images.

[0064] Step S7: Select the optimal moving B-scan. The composite image is created based on M images from all B-scans. Using all B-scans to composite images often fails to produce the optimal composite image. Therefore, a subset of B-scans that best register with the reference image is selected to create the best composite image. The selection of this subset is essentially an optimization problem. Intuitively, we aim to minimize the distance between two normalized cross-correlation coefficients (NCCs) while simultaneously maximizing the NCC. This yields the optimal registration parameters.

[0065] For some B-scans, there are large differences in NCC between B-scans. This may indicate that the registration between these B-scans and the reference B-scan may not be accurate enough to facilitate the creation of a synthetic image. In addition, low NCC values ​​indicate poor assumed registration parameters.

[0066] The function formula for finding a subset of B-scan data to create a composite image is expressed as follows:

[0067]

[0068] in and Let be the NCC between the A-scan of the moving B-scan and the A-scan of the reference B-scan. Let T be the set of moving B-scans, s be a subset of the B-scans of T, and r be the reference B-scan. The optimal subset for creating the composite image is considered to be the one whose sum of absolute differences between the moving B-scans and the reference B-scans is minimized, and whose sum of NCCs is maximized.

[0069] Calculate the B-scan for each movement They are then sorted in ascending order. The first few B-scans are then used to create a composite image, excluding those B-scans with an NCC less than a set threshold. Figure 8a , 8b 8c displays the NCC and reordered B-scan.

[0070] Step S8, Sub-synthesized Image Creation. A synthesized image is created using the preceding moving B-scans and a reference B-scan. The sub-synthesized image is created by obtaining the percentile (e.g., median) of the intensity of a subset of the B-scan at each position on the middle portion of the A-scan. Creating the sub-synthesized image based on the middle portion of the A-scan makes the algorithm faster, but may mask image defects, especially at the left or right edges. This may lead to the selection of a problematic B-scan as the reference B-scan; therefore, if the algorithm is found to produce poor-quality output, it should be re-run.

[0071] Step S9, evaluation of the sub-synthesized image. The sum of the absolute gradients of the sub-synthesized images represents the image sharpness, and the B-scan that maximizes the sharpness of the synthesized image is selected as the reference.

[0072] Step S10, Composite Image Creation. The first n moving B-scans in the sorted list and the selected reference B-scan are used to create the composite image. The choice of n depends on the scanning mode, field of view, sharpness, or blurriness of the composite image we want to create. The composite image is created by obtaining the percentile (e.g., median) of the intensity of a subset of B-scans at each location. (Next) Figure 9a , 9b An example result is shown.

[0073] Step S11, Post-processing of the synthesized image. Post-processing can further enhance the synthesized image. The goal of post-processing is to enhance important tomographic regions in the synthesized image. For example, the vitreous humor typically has a lower intensity level compared to other tomographic regions in the synthesized image. Image fusion-based algorithms can enhance the vitreous humor and choroidal regions. The following algorithm describes the post-processing:

[0074] 1) Perform adaptive histogram equalization on the synthesized image to obtain a secondary synthesized image.

[0075] 2) Create two weight functions, representing two opposite axial ramp functions.

[0076] 3) The composite image is multiplied and combined with the first weighting function (the top is dark) to obtain the first result.

[0077] 4) The image is synthesized twice and then combined with the second weighting function (the top is bright) to obtain the second result.

[0078] 5) Weighted average of the first and second results. The weight of this algorithm is 0.5.

[0079] 6) Adjust the contrast of the weighted average image by stretching the image histogram.

[0080] 7) Gamma correction is applied based on the user-specified gamma index. This ultimately improves the quality of the scanned image.

[0081] This approach is particularly suitable for B-scan images with a large number of combinations. This is because the transform group we are searching for is a subgroup of a general linear affine transformation. A small number of B-scans cannot guarantee that the selected reference B-scan can be found within the general linear affine transformation subgroup.

[0082] The above embodiments are for illustrative purposes only and are not intended to limit the invention. Those skilled in the art can make various changes or modifications without departing from the spirit and scope of the invention. Therefore, all equivalent technical solutions should also fall within the scope of the invention and should be defined by the claims.

Claims

1. A method for improving the quality of scanned images based on multi-frame registration and averaging algorithms, characterized in that, include: Input N B-scans; Select a reference B-scan; Preprocessing B-scan; The axial offset is determined by calculating the normalized cross-correlation function of each A-scan in the moving B-scan relative to the corresponding A-scan in the reference B-scan; Perform linear regression to obtain the optimal curve through all points with the maximum cross-correlation, remove outliers in the distribution, and repeat the linear regression. Calculate the conversion between the moving B-scan and the reference B-scan; Select the best B-scan and sort it; A sub-synthesized image is created using a series of moving B-scans arranged in front and a reference B-scan; The sub-synthesized image is evaluated, and the sum of the absolute gradients of the sub-synthesized images represents the image sharpness. The B-scan that maximizes the sharpness of the synthesized image is selected as the reference B-scan. The first n moving B-scans in the sorted list and the selected reference B-scan are used to create the composite image; Post-processing of the synthesized image.

2. The method for improving scanned image quality based on multi-frame registration and averaging algorithm according to claim 1, characterized in that, Preprocessing for B-scan includes: A set of uniformly distributed A-scans is selected from the reference B-scans to calculate the offset. Based on the range of the selected A-scans, a set of 3 B-scans is preprocessed: speckle is treated as a high-frequency component, and a speckle content image is generated by high-pass filtering of the frame. Then, the speckle content image is subtracted from the original image to obtain an image with higher visual quality. This set is formed, and this process is repeated 3 times. Then, a median image is constructed from this set.

3. The method for improving scanned image quality based on multi-frame registration and averaging algorithm according to claim 1, characterized in that, Perform linear regression to obtain the optimal curve through all points with the maximum cross-correlation, remove outliers from the distribution, and repeat the linear regression, including: A set of A-scans is used to find the translation, which indicates the maximum cross-correlation between the reference B-scan and the moving B-scan at each location; this is a set of P points spanning the length of the B-scan, indicating the vertical translation at that location; then linear regression is used to obtain the equation of the best-fit line through these points, after obtaining the best curve, the deviation of each point from this best curve is calculated, the distribution of these deviations is obtained, points that are further than a threshold are discarded, and a new regression line is derived from the remaining points.

4. The method for improving scanned image quality based on multi-frame registration and averaging algorithm according to claim 1, characterized in that, Calculate the conversion between the moving B-scan and the reference B-scan, including: The translation correction for the transformation is evaluated using the difference between the maximum normalized cross-correlation values ​​of the two A-scans; a pair of lines are sampled from the corresponding positions of each B-scan along the x and z directions; the intersection of these lines defines a rectangle ABCD, which corresponds to the reference B-scan coordinate system to be transformed; an axial translation correction is added to this rectangle to obtain a parallelogram A'B'C'D'.

5. The method for improving scanned image quality based on multi-frame registration and averaging algorithm according to claim 1, characterized in that, Select the best B-scan and sort it, including: The function formula for creating a composite image is expressed as follows: in, It is the NCC between the first target A-scan of the moving B-scan i and the corresponding A-scan of the reference B-scan r; Let be the normalized cross-correlation coefficient (NCC) between the second target A-scan of moving B-scan i and the corresponding A-scan of reference B-scan r; T is the set of moving B-scans, s is a subset of B-scans in T, and r is the reference B-scan; if the sum of the absolute differences of the NCCs of the moving B-scans and the reference B-scans is minimized, and the sum of the NCCs is maximized, then it is considered the optimal subset for creating the synthetic image; calculate the NCC of each moving B-scan. And sort them in ascending order.

6. The method for improving scanned image quality based on multi-frame registration and averaging algorithm according to claim 1, characterized in that, Post-processing of the synthesized image includes: Adaptive histogram equalization is performed on the synthesized image to obtain a secondary synthesized image; Create two weight functions, representing two opposite axial ramp functions; The composite image is combined with the first weighting function to obtain the first result; The image is synthesized twice using a multiple of the original image and a second weighting function to obtain the second result. The weighted average of the first and second results, with a weight of 0.5 in the weighted average algorithm; Adjusting the contrast of the weighted average image by stretching the image histogram; Gamma correction is applied based on the gamma index specified by the user.

7. The method for improving scanned image quality based on multi-frame registration and averaging algorithm according to claim 1, characterized in that, N is a natural number and greater than 50.

Images