Pathological image splicing method and device based on pathological section association

Abstract

The present application relates to pathological section image splicing technical field, and disclose pathological image splicing method and device based on pathological section association.The present application includes: obtaining and preprocessing serialization pathological sub-section image;Calculate the texture flow field representing the arrangement of microstructure of tissue, and construct multi-scale feature descriptor;Based on multi-scale feature, through the multi-scale correlation function of fusion gradient difference, texture distribution KL divergence and frequency domain mutual information, the adjacent relationship between sub-sections is quantitatively evaluated;With the correlation degree as the edge weight, the global splicing graph model is constructed, the energy function containing data fidelity and texture smoothing constraint is established, the optimal splicing parameter is solved by iterative optimization, and the iterative process is controlled according to the convergence criterion of comprehensive splicing stability, residual matching potential and progress;Finally, seamless panoramic pathological image is generated.The present application uses biological structure characteristics, improves the robustness and matching accuracy under complex conditions, and ensures the continuity and consistency of splicing results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pathological section image splicing, in particular to a pathological image splicing method and device based on pathological section association. BACKGROUND

[0002] In pathological diagnosis and research, in order to obtain a large range of tissue morphology panoramic view, it is often necessary to splice the continuous pathological sub-section digital images with high precision. In the prior art, the image splicing method is mostly based on feature point matching or specific region pixel correlation calculation. For example, the Chinese patent with the patent number CN115619634B and the name of "pathological image splicing method and device based on pathological section association" discloses a method of screening and iteratively splicing adjacent pathological sections by calculating the "cut registration degree". The method takes the cut position of the image as the reference, defines a registration degree calculation formula with the cut pixel matrix correlation coefficient as the core, finds the adjacent image most matched with the current reference image cut in the image set to be registered, and then gradually completes the splicing. To some extent, this method converts the splicing problem into a local pixel matching problem of the cut region.

[0003] However, the above-mentioned prior art solution has obvious limitations. First of all, its core assumption is that the cut edges of adjacent sections can maintain complete and regular correspondence on the image, but in the actual pathological section process, the tissue is prone to deformation, wrinkling or tearing, resulting in irregular physical cuts and incomplete or blurred cut edge information in the image, which makes the registration method relying on fixed cut region have poor stability and low fault tolerance. Secondly, this method only uses the local pixel value statistics correlation of the cut, and fails to use the more biologically meaningful tissue microstructure information contained in the pathological image, such as cell arrangement direction and fiber texture direction, etc. Therefore, in the area where the tissue texture is complex and the cut features are not obvious, it is easy to cause mismatching or matching problems. Finally, its iterative splicing strategy is essentially a local greedy selection, which lacks global optimization of the overall consistency and smoothness of the splicing result, and is prone to cumulative errors, making it difficult to ensure the continuity and authenticity of the final panoramic image in histological structure. Therefore, there is an urgent need for a consistent splicing method that does not rely on fragile cut features, can utilize the intrinsic structure information of the tissue, and can perform global optimization. SUMMARY

[0004] The technical problem to be solved by the present application is that the existing image splicing method is based on feature point matching or specific region pixel correlation calculation, which has the disadvantage of lack of practicality. Therefore, we propose a pathological image splicing method based on pathological section association.

[0005] In order to achieve the above-mentioned purpose, the following technical solution is adopted in the present application: a pathological image splicing method based on pathological section association, comprising the following steps:

[0006] Step one: pathological sub-section image acquisition and preprocessing: acquire the serialized pathological sub-section digital image, perform standardization preprocessing on the image, filter out interference information and extract the effective tissue area;

[0007] Step two: texture flow field calculation and multi-scale analysis: calculate the texture flow field in the effective tissue area, which represents the arrangement characteristics of the tissue microstructure, construct a multi-scale image pyramid, analyze the texture flow field characteristics in different scale spaces, and generate corresponding multi-scale texture feature descriptors;

[0008] Step three: adjacent relationship evaluation based on multi-scale texture correlation degree: based on the multi-scale texture feature descriptors, quantitatively calculate the texture distribution difference, structural consistency and frequency domain correlation between sub-sections, and obtain the adjacent relationship correlation degree score through the multi-scale structural similarity correlation degree function;

[0009] Step four: global stitching model construction and iterative optimization: construct a global stitching relationship graph model with sub-sections as nodes and correlation degree scores as edge weights, establish a global energy function containing data fidelity terms and prior smoothing terms, and solve the optimal stitching transformation parameters using an iterative optimization algorithm. In the iterative process, the global iterative convergence criterion is calculated based on the current stitching state, the remaining sub-section matching potential and the overall stitching progress;

[0010] Step five: panoramic image generation and output: when the iteration meets the convergence condition corresponding to the convergence criterion, project all sub-section images to a unified coordinate system, fuse the overlapping areas to generate a seamless panoramic pathology image, and output and store the final image and related stitching metadata.

[0011] Preferably, in step two, the texture flow field calculation and multi-scale analysis specifically includes:

[0012] Using the gradient structure tensor-based method, the local principal direction of each pixel point in the sub-section image is calculated to form an initial flow field;

[0013] The initial flow field is regularized and filtered to smooth noise and enhance the consistency of the flow direction, obtaining the final texture flow field;

[0014] The sub-section image and the corresponding texture flow field are down-sampled under the Gaussian pyramid to generate multiple image sequences and flow field sequences of different scales; at each scale, the following operations are performed:

[0015] From the image at the current scale, the local binary pattern histogram of its adjacent boundary region is extracted to describe the texture pattern distribution;

[0016] From the flow field at the current scale, the average gradient amplitude is calculated;

[0017] The Fourier transform is performed on the adjacent boundary region of the image at the current scale, and a Fourier amplitude spectrum is extracted for frequency domain structure analysis.

[0018] Preferably, the adjacent relationship evaluation based on the multi-scale texture correlation degree in step three specifically includes:

[0019] For the two sub-slice images to be evaluated, the texture feature descriptors generated in step two at each scale are obtained, including the local binary pattern histogram, the average gradient amplitude, and the Fourier amplitude spectrum.

[0020] The absolute difference of the average gradient amplitudes of the two images at each scale is calculated, and the KL divergence between the local binary pattern histograms of the two images at each scale is calculated to measure the texture distribution difference.

[0021] The mutual information between the Fourier amplitude spectra of the two images at each scale is calculated to measure the frequency domain structure similarity.

[0022] The calculation indicators of all scales are integrated, and the multi-scale structure similarity correlation degree function is substituted to solve the final correlation degree score between the two images.

[0023] Preferably, the global stitching model construction and iterative optimization in step four specifically includes:

[0024] The edge weights of a fully connected graph model are initialized based on the correlation degree scores calculated in step three, with all sub-slices as nodes.

[0025] A global energy function is defined, which includes a data fidelity term and a priori smoothing term. The data fidelity term is obtained by weighted summation of the reprojection errors between all pairs of images with determined adjacent relationships, and the weight is the corresponding correlation degree. The priori smoothing term is used to punish the discontinuous jump of the texture flow field in the adjacent regions of the stitched panoramic image.

[0026] The iteration state is initialized, and the set of successfully stitched local regions and the set of remaining un-stitched sub-slices are set. The iteration loop is entered, and the following operations are performed in each iteration:

[0027] In the remaining un-stitched set, the optimal sub-slice or matching pair is selected for stitching alignment according to the correlation degree, and the local transformation parameters are solved.

[0028] The set of stitched regions and the set of remaining un-stitched sub-slices are updated.

[0029] Based on all the solved transformation parameters, the value of the global energy function is calculated, and the optimization method is used to fine-tune the parameters to minimize the function.

[0030] The global iterative stitching convergence criterion under the current iteration is calculated.

[0031] Preferably, in the iterative loop of step four, the data required for computing the global iterative stitching convergence criterion includes:

[0032] The number of local region blocks that have been stitched so far, and the image states of each region block after the current iteration and the last iteration, for evaluating the stitching stability;

[0033] The correlation scores between each pair of sub-slices in the set of remaining un-stitched sub-slices, for evaluating the remaining matching potential;

[0034] The total pixel area of all sub-slices in the set of remaining un-stitched sub-slices, and the total pixel area of all sub-slices, for evaluating the overall stitching progress.

[0035] Preferably, step four further includes an iteration termination judging logic:

[0036] A pre-set convergence threshold and a variation threshold;

[0037] After computing the convergence criterion in each iteration, it is judged whether the pre-set convergence condition is met;

[0038] If the convergence condition is met, it is determined that the global stitching optimization process has converged, and the iteration is terminated and step five is entered; otherwise, the next round of iteration optimization is continued.

[0039] Preferably, the panoramic image generation and output in step five further includes a quality verification step:

[0040] After generating the preliminary fused panoramic image, the smoothness index of the overall texture flow field thereof is computed;

[0041] Detect and identify the regions where stitching misalignment or texture discontinuity may exist;

[0042] Archive the quality verification results together with the stitching metadata, and if serious quality problems are found, trigger an alarm and backtrack to the relevant steps for parameter adjustment and re-computation.

[0043] Preferably, the multi-scale structural similarity correlation function used in step three for computing the correlation score between two sub-slice images is:

[0044]

[0045] Wherein: N denotes the total number of scales; n denotes the scale index; denotes the scale The related adjustable parameters; denotes the absolute difference of the average gradient amplitude of the two images at scale n; denotes a very small positive number used to prevent the denominator from being zero; KL divergence between local binary pattern histograms of two images in scale KL divergence between local binary pattern histograms of two images in scale nonlinear adjustment exponent KL divergence between local binary pattern histograms of two images in scale mutual information between Fourier amplitude spectra of two images in scale correlation score between sub-slice image and .

[0046] In step four, the calculation formula of the global iteration stitching convergence criterion for evaluating the convergence state in the second iteration is preferably

[0047]

[0048] wherein: is the current iteration number; is the number of local region blocks that have been stitched; and are the images after the first iteration and the second iteration of the i-th region block, respectively; is the structural similarity index calculation function; is the mean square error calculation function; and are small positive numbers used to prevent the denominator from being zero; is the set of sub-slice indexes that remain un-stitched after the second iteration; is the correlation score between sub-slice and ; is the total pixel area of all sub-slices in the set ; is the total pixel area of all sub-slices to be stitched; is the decay coefficient; is the global convergence criterion index at the second iteration. Another technical solution is proposed in the present application: a pathological image stitching device based on pathological slice correlation, which comprises: An image acquisition and preprocessing module: used to control a digital scanning device to acquire pathological sub-slice sequence images, and to complete color correction, illumination equalization, and noise filtering standardization preprocessing;

[0049] Another technical solution is proposed in the present application: a pathological image stitching device based on pathological slice correlation, which comprises:

[0050] An image acquisition and preprocessing module: used to control a digital scanning device to acquire pathological sub-slice sequence images, and to complete color correction, illumination equalization, and noise filtering standardization preprocessing;

[0051] ​​​​​Texture flow field analysis module: used for calculating the texture flow field of each sub-slice image, constructing a multi-scale image pyramid, and extracting local binary pattern histogram, average gradient amplitude and Fourier amplitude spectrum texture feature descriptors at each scale;

[0052] Multi-scale correlation degree calculation module: used for receiving feature descriptors from the texture flow field analysis module, calculating the correlation degree score between any two sub-slice images through a multi-scale structural similarity correlation degree function, and forming a global correlation degree matrix;

[0053] Global stitching optimization module: used for taking the correlation degree matrix as input, constructing a global stitching relationship graph and an energy function, and executing an iterative optimization algorithm; in each iteration, a convergence index is calculated through a global iterative stitching convergence criterion formula, and it is determined whether to terminate the iteration according to a preset logic, and an optimal stitching transformation parameter sequence is output;

[0054] Panoramic image synthesis and output module: used for projecting, aligning and multi-band fusing all sub-slice images according to the final transformation parameters output by the global stitching optimization module, generating a final seamless panoramic pathology image, and packing the image and related metadata and storing them to a specified location.

[0055] Technical effects and advantages of the present application:

[0056] In the present application, the texture flow field is used as the core organizational correlation feature, and combined with multi-scale analysis and global optimization model, compared with the existing technology which excessively relies on fragile and easily deformed physical incision edge information, the present application uses the texture flow field reflecting the inherent biological structure of the tissue for calculation, the feature is more stable and has more biological significance, and the method robustness and application range are significantly improved under complex conditions such as tissue deformation, wrinkles or incision defects;

[0057] Secondly, by constructing a multi-scale pyramid and calculating the correlation degree by integrating gradient, texture distribution and frequency domain multi-dimensional features, the single problem of relying only on local pixel statistical correlation in the prior art is overcome, the adjacent relationship evaluation is more comprehensive and accurate, and the false matching is effectively reduced; in addition, by constructing a global graph model weighted by the correlation degree and establishing an energy function containing data fidelity and texture smoothing constraints for iterative optimization, global consistent solution of the stitching transformation parameters is realized, the cumulative error can be effectively smoothed, and the continuity of the tissue structure and the visual natural seamless of the panoramic pathology image generated finally is ensured. BRIEF DESCRIPTION OF DRAWINGS

[0058] The disclosed content of the present application is explained with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes, and are not intended to limit the scope of protection of the present application. In the drawings, the same reference signs are used to refer to the same parts:

[0059] Figure 1 The flowchart of the present application. DETAILED DESCRIPTION

[0060] It is easy to understand that, according to the technical solutions of the present application, those skilled in the art can propose a plurality of structure modes and implementation modes that can be replaced with each other without changing the essential spirit of the present application. Therefore, the following detailed description and the accompanying drawings are only exemplary descriptions of the technical solutions of the present application, and should not be regarded as the whole or as a limitation or restriction on the technical solutions of the present application.

[0061] Embodiment one: reference Figure 1 The pathological image splicing method based on pathological section association provided by the embodiment of the present application comprises the following steps:

[0062] Step one: pathological sub-section image acquisition and preprocessing:

[0063] Firstly, a series of pathological sub-section digital images that overlap with each other are obtained through a high-resolution digital scanning device; these images are usually stored in a serialized manner; after being obtained, standardization preprocessing is performed on these sub-section images to eliminate various interferences and noises introduced in the scanning process.

[0064] The preprocessing process can include but is not limited to:

[0065] Color correction: adjusting the color balance of the image, eliminating the color difference between different sub-sections, and ensuring color consistency.

[0066] Illumination homogenization: compensating for the uneven illumination that may occur in the scanning process, making the image brightness distribution more uniform.

[0067] Noise filtering: using Gaussian filtering, median filtering and other methods to remove random noise in the image.

[0068] Effective tissue area extraction: through image segmentation techniques such as threshold segmentation, region growing or deep learning segmentation models, the background area in the image is separated from the effective area containing pathological tissues, and only the effective tissue area is retained for subsequent processing to reduce the amount of calculation and avoid the interference of background noise.

[0069] Step two: texture flow field calculation and multi-scale analysis:

[0070] In the extracted effective tissue area, the texture flow field that characterizes the arrangement features of the tissue microstructure is calculated; the texture flow field can reflect important pathological information such as the arrangement direction of tissue cells and the fiber direction; specifically, this step includes:

[0071] Computing initial flow field: adopt the method based on gradient structure tensor to calculate the local principal direction of each pixel point in the sub-slice image, forming the initial flow field; the gradient structure tensor can capture the gradient information of the local region of the image, and its eigenvector can indicate the main direction of the texture.

[0072] Regularization filtering: the initial flow field is subjected to regularization filtering to smooth the noise and enhance the consistency of the flow direction, obtaining the final texture flow field; the regularization filtering can adopt Gaussian filtering or directional smoothing filter to eliminate the local irregular flow direction, making the overall flow direction more coherent.

[0073] Constructing multi-scale image pyramid: the sub-slice image and the corresponding texture flow field are subjected to Gaussian pyramid downsampling to generate a plurality of image sequences and flow field sequences of different scales; the Gaussian pyramid generates images of different resolutions through layer-by-layer smoothing and downsampling, so as to analyze the image features at different scales.

[0074] Extracting multi-scale texture feature descriptor: at each scale, the following operations are performed to extract the multi-scale texture feature descriptor:

[0075] From the image at the current scale, the local binary pattern histogram of its adjacent boundary region is extracted to describe the texture pattern distribution; LBP is an effective texture descriptor that can capture the microstructure of local texture;

[0076] From the flow field at the current scale, the average gradient amplitude is calculated. The average gradient amplitude reflects the intensity and contrast information of the texture;

[0077] Performing Fourier transform on the adjacent boundary region of the image at the current scale to extract its Fourier amplitude spectrum for frequency domain structure analysis; the Fourier amplitude spectrum can reflect the energy distribution of the image in the frequency domain, thereby characterizing the periodicity and directionality of the texture.

[0078] Step three: adjacent relationship evaluation based on multi-scale texture correlation degree:

[0079] Based on the multi-scale texture feature descriptor, the texture distribution difference, structural consistency and frequency domain correlation between the sub-slices are quantitatively calculated, and the adjacent relationship correlation degree score is obtained through the multi-scale structural similarity correlation degree function; specifically, this step includes:

[0080] Obtaining texture feature descriptor: for the two sub-slice images to be evaluated, for example, for image and image , obtain the texture feature descriptors generated at each scale in step two, including: local binary pattern histogram , average gradient amplitude and Fourier amplitude spectrum .

[0081] Compute average gradient magnitude absolute difference: Calculate the average gradient magnitude absolute difference between two images at each scale .

[0082] Compute KL divergence: Calculate the KL divergence between local binary pattern histograms of two images at each scale , which measures the difference in texture distribution; the larger the KL divergence, the greater the difference in texture distribution.

[0083] Compute mutual information: Calculate the mutual information between Fourier amplitude spectra of two images at each scale , which measures the similarity in frequency domain structure; the larger the mutual information, the more similar the frequency domain structure.

[0084] Synthesize correlation score: Synthesize all scale calculation indicators, substitute into the multi-scale structure similarity correlation function, and solve the final correlation score between two images , which is expressed as:

[0085]

[0086] Where: represents the total number of scales; is the scale index; is the scale related adjustable parameters, which are optimized by training data or experience setting; is the average gradient magnitude absolute difference between two images at scale ; is a very small positive number to prevent the denominator from being zero; is the KL divergence between local binary pattern histograms of two images at scale ; is a nonlinear adjustment index to adjust the influence of KL divergence on correlation; is the mutual information between Fourier amplitude spectra of two images at scale ; represents the correlation score between sub-slice images and ; this function considers gradient difference, texture distribution difference, and frequency domain structure similarity, and through multi-scale weighting, the correlation score can more accurately reflect the real adjacency relationship and matching potential between sub-slices.

[0087] Step four: Global stitching model construction and iterative optimization:

[0088] A global stitching graph model is constructed with sub-slices as nodes and correlation scores as edge weights, and a global energy function is established including a data fidelity term and a priori smoothing term, and an iterative optimization algorithm is used to solve the optimal stitching transformation parameters; specifically, this step includes:

[0089] Initialize the global stitching graph model: take all sub-slices as nodes, and based on the correlation scores calculated in step three Initialize the edge weights of a fully connected graph model; the higher the correlation score, the more likely there is an adjacent relationship between the two sub-slices and the higher the matching quality.

[0090] Define the global energy function: define the global energy function , which includes a data fidelity term and a priori smoothing term .

[0091] The data fidelity term is obtained by weighted summation of the re-projection errors between all image pairs with determined adjacent relationships, and the weight is the corresponding correlation; the re-projection error measures the difference in feature point positions after projecting one sub-slice to another sub-slice through the current transformation parameters; the data fidelity term aims to ensure the accuracy of local matching.

[0092] The priori smoothing term is used to penalize the discontinuous jump of texture flow field in adjacent regions of the panoramic image after stitching; this term ensures the visual smoothness and continuity of global stitching by constraining the consistency of texture flow field in the overlapping region of adjacent sub-slices.

[0093] The global energy function is represented as , where is the weight coefficient balancing the data fidelity term and the priori smoothing term.

[0094] Initialize the iteration state: set the current successfully stitched local region set and the remaining un-stitched sub-slice set; initially, the stitched set is empty or contains a seed sub-slice, and the remaining un-stitched set contains all other sub-slices.

[0095] Enter the iteration loop: in each iteration, the following operations are performed:

[0096] Select the optimal match and solve the local transformation parameters: in the remaining un-stitched set, select the optimal sub-slice or matching pair for stitching alignment according to the correlation, and solve the local transformation parameters; this can be achieved by finding the un-stitched sub-slice with the highest correlation with the stitched region, or finding the matching pair with the highest correlation among un-stitched sub-slices; the local transformation parameters can be affine transformation, perspective transformation, etc.

[0097] Update Sets: Update the set of spliced ​​regions and the set of remaining unspliced ​​sub-slices; move successfully spliced ​​sub-slices from the remaining set to the spliced ​​set;

[0098] Global energy function optimization: Calculate the global energy function based on all currently solved transformation parameters. The value of the function is determined, and optimization methods such as the Levenberg-Marquardt algorithm and graph optimization are used to fine-tune the transformation parameters of all sub-slices to minimize the function. This step is the core of global optimization. It minimizes the overall stitching error and ensures the smoothness of the texture flow field by adjusting the relative position and orientation of all sub-slices.

[0099] Calculate the global iterative stitching convergence criterion: Calculate the global iterative stitching convergence criterion under the current iteration. This criterion comprehensively considers splicing stability, remaining matching potential, and overall splicing progress; specifically, in the first... In the next iteration, the global iterative splicing convergence criterion is used to evaluate the convergence state. The calculation formula is:

[0100]

[0101] in: This represents the current iteration number; This represents the number of currently assembled local region blocks; and The first The first block in the... Second and third The image after the next iteration; This is the function for calculating the structural similarity index. These are the mean squared error calculation functions, which together measure the degree of change of an image patch between two iterations; and This is used to prevent extremely small positive numbers with a denominator of zero; For the first The set of remaining unjoined sub-slice indices after the next iteration; For sub-slices and The correlation score between them This reflects the maximum matching potential of the remaining unspliced ​​sub-slices; For set The total pixel area of ​​all sub-slices in the middle. The ratio of the two is the total pixel area of ​​all the sub-slices to be stitched together, which reflects the overall stitching progress. This is the attenuation coefficient, used to adjust the impact of the splicing progress on the convergence criterion; For the first The global convergence criterion index at the sub-iteration time; the criterion comprehensively reflects the convergence state of the global stitching by combining the stitching stability, the residual matching potential and the stitching progress.

[0102] The iteration termination judgment logic: preset convergence threshold and variation threshold; after calculating the convergence criterion at each iteration , it is judged whether the preset convergence condition is met; the convergence condition can include: the convergence criterion is less than a certain preset convergence threshold;

[0103] In a plurality of consecutive iterations, the variation of the convergence criterion is less than a certain preset variation threshold;

[0104] The number of local area blocks that have been stitched reaches a preset value or all sub-slices have been stitched.

[0105] If any of the convergence conditions is met, it is determined that the global stitching optimization process has converged, the iteration is terminated and step five is entered; otherwise, the next round of iteration optimization is continued.

[0106] Step five: panoramic image generation and output:

[0107] When the iteration meets the convergence condition corresponding to the convergence criterion, project all sub-slice images to a unified coordinate system, and perform fusion processing on the overlapping area to generate a seamless panoramic pathology image, output and store the final image and related stitching metadata; specifically, this step also includes a quality verification step:

[0108] Image projection and fusion: according to the optimal stitching transformation parameters solved in step four, project all sub-slice images into a unified coordinate system; for overlapping areas, use multi-band fusion, weighted average or Poisson fusion technology for processing to eliminate stitching gaps and brightness differences, and generate a visually seamless panoramic pathology image;

[0109] Quality verification: after generating the preliminary fused panoramic image, calculate the smoothness index of the overall texture flow field; the smoothness index can be calculated based on the divergence, vorticity or local consistency of the texture flow field; at the same time, detect and identify areas that may have stitching misplacement or texture discontinuity; this can be achieved by analyzing the texture flow field difference, local image gradient discontinuity or structural similarity index of the overlapping area;

[0110] Output and storage: The quality verification results are archived together with the stitching metadata, including parameters such as stitching parameters, sub-slice original information, stitching time, quality evaluation report, etc. If serious quality problems are found, such as smoothness index lower than the preset threshold or large area of discontinuous region, an alarm is triggered and the relevant steps, such as parameter adjustment in step four or correlation re-evaluation in step three, can be traced back to adjust the parameters and recalculate to ensure the quality of the final output panoramic image.

[0111] Embodiment two: pathological image stitching device based on pathological section correlation

[0112] The embodiment of the present application also provides a pathological image stitching device based on pathological section correlation, which is used to realize the pathological image stitching method of the above-mentioned embodiment one. The device can be an independent hardware device, or a software module integrated in a computer system or a combination of software and hardware system.

[0113] The device comprises the following modules:

[0114] Image acquisition and preprocessing module:

[0115] The module is used to control the digital scanning device to acquire pathological sub-slice sequence images, and complete color correction, illumination homogenization and noise filtering standardization preprocessing. It receives original sub-slice image data and performs preliminary processing to provide high-quality input images to subsequent modules. The module is also responsible for extracting the effective tissue area in the image and removing background interference.

[0116] Texture flow field analysis module:

[0117] The module is used to receive preprocessed sub-slice images and calculate the texture flow field of each sub-slice image. It uses a gradient structure tensor-based method to calculate the local principal direction and performs regularization filtering to smooth the flow direction. In addition, the module also constructs a multi-scale image pyramid and extracts local binary pattern histogram, average gradient amplitude and Fourier amplitude spectrum and other texture feature descriptors at each scale. These descriptors will serve as the basis for sub-slice correlation evaluation.

[0118] Multi-scale correlation calculation module:

[0119] The module is used to receive feature descriptors from the texture flow field analysis module. It calculates the correlation score between any two sub-slice images according to the multi-scale structural similarity correlation function. The module calculates the average gradient amplitude absolute difference, KL divergence and mutual information, and generates a correlation matrix reflecting the matching potential between sub-slices by integrating these indicators.

[0120] Global stitching optimization module:

[0121] The module takes the correlation matrix output by the multi-scale correlation calculation module as input, and constructs a global stitching relationship graph and an energy function; it executes an iterative optimization algorithm to solve the optimal stitching transformation parameters; in each iteration, the module calculates a convergence index according to a global iterative stitching convergence criterion formula, and determines whether to terminate the iteration according to a preset iteration termination judgment logic; when the convergence condition is met, the module outputs the optimal stitching transformation parameter sequence; by minimizing the global energy function, the module ensures the global consistency and accuracy of the stitching.

[0122] Panoramic image synthesis and output module:

[0123] The module is used for receiving the final transformation parameters output by the global stitching optimization module; it projects, aligns and multi-band fuses all sub-slice images according to the parameters to generate a final seamless panoramic pathology image; in addition, the module also performs a quality verification step, calculates the smoothness index of the overall texture flow field of the panoramic image, and detects and identifies the areas where stitching misplacement or texture discontinuity may exist; finally, the module packs the generated panoramic image and related metadata into a designated location, and can trigger an alarm or a backtracking mechanism according to the quality verification result.

[0124] The above device modules can be implemented by one or more processors executing corresponding computer programs, or by dedicated hardware circuits or a combination of software and hardware.

[0125] The technical scope of the present application is not limited to the above description, and those skilled in the art can make various modifications and changes to the above embodiments without departing from the technical idea of the present application, and these modifications and changes should be within the protection scope of the present application.

Claims

1. A method for stitching pathological images based on pathological slide association, characterized in that, Includes the following steps: Step 1: Acquisition and preprocessing of pathological subsection images: Acquire serialized digital images of pathological subsections, perform standardization preprocessing on the images, filter out interference information and extract effective tissue regions; Step 2: Texture flow field calculation and multi-scale analysis: Calculate the texture flow field that characterizes the arrangement features of the microstructure of the tissue within the effective tissue area, construct a multi-scale image pyramid, analyze the characteristics of the texture flow field at different scales, and generate corresponding multi-scale texture feature descriptors. Step 3: Adjacency evaluation based on multi-scale texture correlation: Based on the multi-scale texture feature descriptor, the texture distribution difference, structural consistency and frequency domain correlation between sub-slices are quantitatively calculated, and the adjacency correlation score is obtained through the multi-scale structural similarity correlation function. Step 4: Global stitching model construction and iterative optimization: Construct a global stitching relationship graph model with sub-slices as nodes and correlation scores as edge weights, establish a global energy function that includes data fidelity terms and prior smoothing terms, and use an iterative optimization algorithm to solve for the optimal stitching transformation parameters. During the iteration process, combine the current stitching state, the matching potential of the remaining sub-slices, and the overall stitching progress to calculate the global iterative convergence criterion. Step 5: Panoramic image generation and output: When the iteration satisfies the convergence condition corresponding to the convergence criterion, all sub-slice images are projected onto a unified coordinate system, and the overlapping areas are fused to generate a seamless panoramic pathological image. The final image and related stitching metadata are output and stored. The texture flow field calculation and multi-scale analysis in step two specifically include: A gradient structure tensor-based method is used to calculate the local principal direction of each pixel in the sub-slice image to form the initial flow field. The initial flow field is subjected to regularization filtering to smooth noise and enhance the consistency of flow direction, resulting in the final texture flow field; Gaussian pyramid downsampling is performed on the sub-slice images and their corresponding texture flow fields to generate multiple image sequences and flow field sequences at different scales; at each scale, the following operations are performed: From the image at the current scale, extract the local binary pattern histogram of its adjacent boundary region to describe the texture pattern distribution; Calculate the average gradient magnitude from the flow field at the current scale; Perform Fourier transform on the adjacent boundary regions of the image at the current scale, and extract their Fourier amplitude spectrum for frequency domain structure analysis; The adjacency evaluation based on multi-scale texture correlation in step three specifically includes: For the two sub-slice images to be evaluated, obtain the texture feature descriptors generated at each scale in step two, including: local binary pattern histogram, average gradient magnitude, and Fourier magnitude spectrum; Calculate the absolute difference of the average gradient magnitude between the two images at each scale; calculate the KL divergence between the local binary pattern histograms of the two images at each scale to measure the difference in texture distribution. Calculate the mutual information between the Fourier amplitude spectra of two images at each scale to measure frequency domain structural similarity; By combining the calculation indicators at all scales and substituting them into the multi-scale structural similarity correlation function, the final correlation score between the two images is obtained. Step four, the construction and iterative optimization of the global stitching model, specifically includes: Using all sub-slices as nodes, initialize the edge weights of a fully connected graph model based on the correlation scores calculated in step three; Define a global energy function, which includes a data fidelity term and a prior smoothing term. The data fidelity term is obtained by weighted summation of the reprojection errors between all image pairs with known adjacency relationships, and the weights are the corresponding correlation degrees. The prior smoothing term is used to penalize discontinuous jumps in the texture flow field of adjacent regions in the stitched panoramic image. Initialize the iteration state, setting the set of currently successfully stitched local regions and the set of remaining unstitched sub-slices; enter the iteration loop, and perform the following operations in each iteration: In the remaining unstitched set, the optimal sub-slice or matching pair is selected for stitching and alignment based on the degree of correlation, and the local transformation parameters are solved. Update the set of spliced ​​regions and the set of remaining unspliced ​​sub-slices; Based on all the currently solved transformation parameters, calculate the value of the global energy function, and use optimization methods to fine-tune the parameters to minimize the function; Calculate the convergence criterion for global iterative splicing in the current iteration.

2. The pathological image stitching method based on pathological slide association according to claim 1, characterized in that, In the iterative loop of step four, the data required to calculate the global iterative splicing convergence criterion includes: The number of local region blocks that have been stitched together, and the image state of each region block after the current iteration and after the previous iteration, are used to evaluate the stitching stability. The correlation scores between all pairs of sub-slices in the remaining unjoined sub-slice set are used to evaluate the remaining matching potential; The total pixel area of ​​all sub-slices in the remaining unstitched sub-slice set, and the total pixel area of ​​all sub-slices, are used to evaluate the overall stitching progress.

3. The pathological image stitching method based on pathological slide association according to claim 2, characterized in that, Step four also includes iteration termination judgment logic: Preset convergence threshold and change threshold; After calculating the convergence criterion in each iteration, determine whether the preset convergence condition is met. If the convergence condition is met, the global stitching optimization process is determined to have converged, the iteration is terminated and the process proceeds to step five; otherwise, the next round of iteration optimization continues.

4. The pathological image stitching method based on pathological slide association according to claim 3, characterized in that, Step five, the generation and output of the panoramic image, specifically includes a quality verification step: After generating the preliminary fused panoramic image, the smoothness index of its overall texture flow field is calculated; Detect and identify areas that may have splicing misalignment or texture discontinuity; The quality verification results are archived together with the spliced ​​metadata. If a serious quality problem is found, an alarm is triggered and the relevant steps can be traced back to adjust the parameters and recalculate.

5. The pathological image stitching method based on pathological slide association according to claim 4, characterized in that, The multi-scale structural similarity correlation function used in step three to calculate the correlation score between the two sub-slice images is: ； in: Indicates the total number of scales; For scale indexing; For scale Related adjustable parameters; For two images at different scales The absolute difference of the average gradient magnitude; This is used to prevent extremely small positive numbers with a denominator of zero; The KL divergence between the local binary pattern histograms of the two images at different scales; It is a non-linear adjustment index; For two images at different scales Mutual information between lower Fourier amplitude spectra; Represents sub-slice image and The correlation score between them.

6. The pathological image stitching method based on pathological slide association according to claim 5, characterized in that, The formula for calculating the global iterative splicing convergence criterion used to evaluate the convergence state in the th iteration of step four is as follows: ； in: This represents the current iteration number; This represents the number of currently assembled local region blocks; and The first The first block in the... Second and third The image after the next iteration; This is the function for calculating the structural similarity index. This is the function for calculating the mean square error; and This is used to prevent extremely small positive numbers with a denominator of zero; For the first The set of remaining unjoined sub-slice indices after the next iteration; For sub-slices and The correlation score between them; For set The total pixel area of ​​all sub-slices in the middle; The total pixel area of ​​all sub-slices to be stitched together; The attenuation coefficient; For the first Global convergence criterion for each iteration.

7. A pathological image stitching device based on pathological slide association, used to implement the pathological image stitching method based on pathological slide association as described in any one of claims 1 to 6, characterized in that, The device includes: Image acquisition and preprocessing module: Used to control the digital scanning equipment to acquire pathological subsection sequence images and to perform standardized preprocessing such as color correction, illumination homogenization and noise filtering; Texture flow field analysis module: used to calculate the texture flow field of each sub-slice image, construct a multi-scale image pyramid, and extract local binary pattern histograms, average gradient magnitudes, and Fourier amplitude spectrum texture feature descriptors at each scale; Multi-scale correlation calculation module: It receives feature descriptors from the texture flow field analysis module, calculates the correlation score between any two sub-slice images through the multi-scale structural similarity correlation function, and forms a global correlation matrix; Global stitching optimization module: It is used to construct a global stitching relationship graph and energy function with the correlation degree matrix as input, and execute an iterative optimization algorithm. In each iteration, it calculates the convergence index through the global iterative stitching convergence criterion formula, and determines whether to terminate the iteration according to the preset logic, and outputs the optimal stitching transformation parameter sequence. Panoramic image synthesis and output module: Based on the final transformation parameters output by the global stitching optimization module, it projects, aligns, and performs multi-band fusion of all sub-slice images to generate the final seamless panoramic pathological image, and packages and stores the image and related metadata in a specified location.

Images