High-resolution large-view-field imaging system based on multi-image splicing and fusion

Abstract

A high-resolution large-view-field imaging system based on multi-image splicing and fusion is characterized in that firstly, a plurality of high-resolution local pathological section images are acquired, and an illumination compensation technology is introduced to improve the image quality and ensure that details are clear and visible; secondly, performing semantic feature coding on the pathological section image subjected to illumination compensation by adopting a dynamic convolution marking model, capturing fine local features such as cell textures and edges, and introducing a selection method based on feature neighborhood activeness to highlight important pathological features; and finally, cascading the sparse pathological section image semantic features and inputting the sparse pathological section image semantic features into an image fusion engine based on an AIGC model to generate a global pathological section fusion image which covers a larger tissue range and retains high-resolution details. Therefore, the double advantages of high resolution and large view field are realized, clearer and more targeted image support is provided for pathological diagnosis, and the accuracy and efficiency of diagnosis are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent image processing technology, and more specifically, to a high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion. Background Art

[0002] Pathological biopsy analysis is an integral part of medical diagnosis, especially for the early detection and treatment of serious diseases such as cancer.

[0003] In traditional pathological imaging technology, microscopes are the most commonly used tools. However, traditional microscope imaging faces the dilemma of balancing resolution and field of view. Although high-resolution imaging can clearly present the microscopic structure of cells, such as the morphology of the nucleus and the details of organelles, it can often only observe very small local areas. If the field of view is expanded to observe a larger range of pathological sections, the resolution will decrease, resulting in blurred images and many key pathological features difficult to distinguish. However, in tumor pathology diagnosis, doctors need to understand not only the microscopic characteristics of tumor cells, but also the distribution range of the tumor in the entire tissue, the boundary with the surrounding normal tissue, and other information. Traditional imaging technology is difficult to meet both of these needs at the same time.

[0004] Therefore, a high-resolution and large-field-of-view imaging solution based on multi-image stitching and fusion is desired. Summary of the invention

[0005] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] In a first aspect, a high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion is provided, which comprises: A pathological slice image acquisition module, used for acquiring a plurality of pathological slice images; A pathological slice image compensation and extraction module, used for performing illumination compensation and feature extraction on each of the plurality of pathological slice images to obtain a plurality of pathological slice image semantic feature coding maps; A pathological slice image semantic feature selection module is used to perform pathological slice image semantic feature selection on each pathological slice image semantic feature coding map in the multiple pathological slice image semantic feature coding maps to obtain multiple sparse pathological slice image semantic feature coding maps, wherein the pathological slice image semantic feature selection module includes: an image semantic feature local decomposition unit, which is used to perform feature decoupling and feature flattening on the features of the pathological slice image semantic feature coding map to obtain a set of pathological slice image semantic local feature vectors; an image semantic feature local selection aggregation unit, which is used to perform feature neighborhood activity selection and shape reshaping on the set of pathological slice image semantic local feature vectors to obtain the sparse pathological slice image semantic feature coding map; The global pathology slice image generation module is used to cascade the multiple sparse pathology slice image semantic feature coding maps into a global pathology slice image semantic feature splicing coding map, and obtain a global pathology slice fusion image based on the global pathology slice image semantic feature splicing coding map.

[0007] Optionally, the pathological slice image compensation and extraction module includes: a pathological slice compensation unit, used to perform illumination compensation on each pathological slice image among the multiple pathological slice images to obtain multiple illumination-compensated pathological slice images; and a pathological slice feature extraction unit, used to input each illumination-compensated pathological slice image among the multiple illumination-compensated pathological slice images into a pathological slice image feature extractor respectively to obtain semantic feature coding maps of the multiple pathological slice images.

[0008] Optionally, the pathological slice image feature extractor is a pathological slice image feature extractor based on a dynamic convolutional labeling model.

[0009] Optionally, the image semantic feature local decomposition unit is used to: perform feature decoupling of the pathological slice image semantic feature coding map along the channel dimension to obtain a set of pathological slice image semantic feature coding matrices; and perform feature flattening on each pathological slice image semantic feature coding matrix in the set of pathological slice image semantic feature coding matrices to obtain a set of pathological slice image semantic local feature vectors.

[0010] Optionally, the image semantic feature local selection aggregation unit includes: an image semantic local feature descending sorting subunit, used to arrange the set of pathological slice image semantic local feature vectors in descending order to obtain a descending sequence of pathological slice image semantic local feature vectors; an image semantic local feature selection subunit, used to perform feature selection on the descending sequence of pathological slice image semantic local feature vectors to obtain a descending sequence of selected pathological slice image semantic local feature vectors; an image semantic feature reshaping subunit, used to perform feature reshaping on the descending sequence of selected pathological slice image semantic local feature vectors to obtain the sparse pathological slice image semantic feature coding map.

[0011] Optionally, the image semantic local feature descending order sorting subunit is used to: input each pathological slice image semantic local feature vector in the set of pathological slice image semantic local feature vectors into an importance measurement module to obtain a set of pathological slice image semantic local feature importance score values; based on the set of pathological slice image semantic local feature importance score values, arrange the set of pathological slice image semantic local feature vectors in descending order to obtain a descending sequence of the pathological slice image semantic local feature vectors.

[0012] Optionally, the image semantic local feature selection subunit is used to: calculate the feature neighborhood activity of each pathological slice image semantic local feature vector based on the neighborhood features of each pathological slice image semantic local feature vector in the descending sequence of the pathological slice image semantic local feature vector to obtain a sequence of pathological slice image semantic local feature neighborhood activities; and perform feature selection on the descending sequence of pathological slice image semantic local feature vectors based on the sequence of pathological slice image semantic local feature neighborhood activities to obtain a descending sequence of pathological slice image semantic local feature vectors after the selection.

[0013] Optionally, the global pathology slice image generation module is used to: after cascading the multiple sparse pathology slice image semantic feature coding maps into the global pathology slice image semantic feature splicing coding map, input it into an image fusion engine based on the AIGC model to obtain the global pathology slice fusion image.

[0014] Optionally, the image fusion engine is an image fusion engine based on an AIGC model.

[0015] The above technical solution first obtains multiple high-resolution local pathological slice images, and introduces illumination compensation technology to improve image quality and ensure that details are clearly visible; then, a dynamic convolutional labeling model is used to encode semantic features of the illumination-compensated pathological slice images to capture fine local features such as cell texture and edges, and a selection method based on feature neighborhood activity is introduced to highlight important pathological features; finally, the semantic features of the sparse pathological slice images are cascaded and input into the image fusion engine based on the AIGC model to generate a global pathological slice fusion image that covers a larger tissue range and retains high-resolution details. In this way, not only the dual advantages of high resolution and large field of view are achieved, but also clearer and more targeted image support is provided for pathological diagnosis, significantly improving the accuracy and efficiency of diagnosis.

[0016] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a block diagram of a high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to an embodiment of the present application.

[0019] Figure 2 It is a block diagram of the pathological slice image compensation and extraction module in the high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to an embodiment of the present application.

[0020] Figure 3 It is a block diagram of the pathological section image semantic feature selection module in the high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to an embodiment of the present application.

[0021] Figure 4 It is a block diagram of the image semantic feature local selection aggregation unit in the high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to an embodiment of the present application.

[0022] Figure 5 The present invention is a flowchart of a high-resolution, large-field-of-view imaging method based on multi-image stitching and fusion according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0024] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0025] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0026] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0027] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0028] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0029] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0030] It should be understood that resolution refers to the ability of an imaging system to distinguish the smallest details. High-resolution imaging means that the system can obtain images with richer details and more accuracy. In the context of pathological section imaging, high resolution allows pathologists to observe the subtle structure of cells, such as the morphology of the cell nucleus, the details of organelles in the cytoplasm, the integrity of the cell membrane, etc. The field of view refers to the range that the imaging system can observe. Wide field of view imaging means that the system can obtain images over a wider range. For pathological section examination, a large field of view means that more tissue areas can be seen in a fused image, which can provide more comprehensive tissue information for pathological analysis.

[0031] Based on this, the present application proposes a high-resolution, large-field-of-view imaging system 100 based on multi-image stitching and fusion, the technical concept of which is to obtain multiple pathological slice images, use machine learning-based image processing and imaging technology to perform illumination compensation on each of the pathological slice images, and then perform semantic feature encoding on each of the compensated illumination-compensated pathological slice images, and then perform neighborhood activity semantic feature selection on the semantic features of each pathological slice image, so as to intelligently generate a global pathological slice fusion image based on the representation of the global stitching of the semantic features of each sparse pathological slice image. This method of the present application not only retains the high-resolution details of each local image, but also covers a larger tissue range, achieving the dual advantages of high resolution and large field of view.

[0032] In this application, Figure 1 is a block diagram of a high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to an embodiment of the present application, such as Figure 1 As shown, a high-resolution and large-field-of-view imaging system 100 based on multi-image stitching and fusion includes: A pathological slice image acquisition module 110 is used to acquire a plurality of pathological slice images; A pathological slice image compensation and extraction module 120, configured to perform illumination compensation and feature extraction on each of the plurality of pathological slice images to obtain a plurality of pathological slice image semantic feature coding maps; A pathological slice image semantic feature selection module 130, configured to perform pathological slice image semantic feature selection on each of the plurality of pathological slice image semantic feature coding maps to obtain a plurality of sparse pathological slice image semantic feature coding maps; The global pathology slice image generation module 140 is used to cascade the multiple sparse pathology slice image semantic feature coding maps into a global pathology slice image semantic feature splicing coding map, and obtain a global pathology slice fusion image based on the global pathology slice image semantic feature splicing coding map.

[0033] In the above-mentioned high-resolution, large-field-of-view imaging system 100 based on multi-image stitching and fusion, the pathological slice image acquisition module 110 is used to acquire multiple pathological slice images. Specifically, first of all, in the process of acquiring pathological slice images, sample preparation is the first step and a crucial link. This stage includes sampling from the patient, usually obtaining tissue samples during surgical resection or biopsy, which are then processed through a series of steps, such as fixation (for example, using formalin), dehydration, transparency, wax immersion, etc., and then cut into extremely thin slices with a slicer and attached to a slide. In order to enhance the visibility of cell structures, pathological sections are also stained, and the most commonly used method is hematoxylin and eosin staining (H&E staining). In addition, depending on research needs, special immunohistochemical staining or other techniques may also be used to mark specific proteins or molecules in order to observe the target structure more clearly.

[0034] Next, we enter the microscopy imaging stage. Here, it is crucial to choose the right microscopy equipment. Based on the required resolution and field of view size, choose an appropriate microscope, such as a fully automated high-resolution scanning microscope, which can automatically scan the entire slide and generate a high-resolution digital image. After placing the prepared slide in the digital scanner, the instrument automatically adjusts the focus and translation platform according to the preset program to cover the entire slide surface. Each movement captures a high-definition image of a small area, the so-called "tile". For some specially stained or fluorescently labeled samples, it may also be necessary to use a multi-wavelength light source and corresponding filters to capture information from different color channels separately, which helps to more fully understand the tissue structure and function.

[0035] As the scanning process progresses, each tile is captured continuously, and the software stitches these individual tiles into a complete panoramic image in real time. This process relies on precise coordinate mapping and image registration algorithms to ensure seamless connection and no distortion. At the same time, important information about each image is recorded, such as magnification, exposure time, scanning date and other metadata, which is very important for subsequent analysis. After the image acquisition is completed, there is a quality check step to ensure that all images are clear and legible without obvious artifacts or defects. If there is a problem, the corresponding area needs to be rescanned to ensure the consistency and reliability of image quality.

[0036] After high-quality pathology slide images are successfully acquired, the next task is data management. All pathology slide images and their associated metadata will be properly stored for long-term archiving and future retrieval. Pathology slide images acquired in this way not only provide pathologists with more detailed and intuitive diagnostic evidence, but also provide strong support for medical research and development, promoting the understanding of disease mechanisms and the development of treatment methods.

[0037] In the above-mentioned high-resolution large-field-of-view imaging system 100 based on multi-image stitching and fusion, the pathological slice image compensation and extraction module 120 is used to perform illumination compensation and feature extraction on each pathological slice image in the multiple pathological slice images to obtain multiple pathological slice image semantic feature coding maps. That is, in the processing process, illumination compensation is first performed on each pathological slice image, which helps to improve image quality and retain more detail information. Then semantic feature encoding and selection are performed. One of the purposes of this series of operations is to maintain high-resolution characteristics after stitching and fusion. By accurately processing the features of each slice image, the final generated global image can inherit the high-resolution details of each slice, thereby providing clearer image information for pathological diagnosis. And by seamlessly stitching these "puzzle pieces" together, a global pathological slice fusion image is generated. This method realizes large-field-of-view imaging because it successfully combines multiple separate, high-resolution slice images so that the final image can cover a larger tissue range and provide a more complete view of the tissue structure.

[0038] In a specific embodiment of the present application, Figure 2 : is a block diagram of the pathological slice image compensation and extraction module in the high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to an embodiment of the present application, such as Figure 2 As shown, the pathological slice image compensation and extraction module 120 includes: a pathological slice compensation unit 121, which is used to perform illumination compensation on each pathological slice image among the multiple pathological slice images to obtain multiple illumination-compensated pathological slice images; a pathological slice feature extraction unit 122, which is used to input each illumination-compensated pathological slice image among the multiple illumination-compensated pathological slice images into a pathological slice image feature extractor respectively to obtain semantic feature coding maps of the multiple pathological slice images.

[0039] Wherein, the pathological slice image feature extractor is a pathological slice image feature extractor based on a dynamic convolutional labeling model.

[0040] Specifically, considering that in the process of acquiring pathological slice images, due to factors such as uneven distribution of light sources of imaging equipment, differences in slice placement angles, or interference from ambient light, it is easy to cause inconsistent illumination intensities in different areas. In addition, undesirable illumination conditions such as uneven illumination, too bright or too dark illumination will reduce the overall quality of the image, making the image look blurred, details lost, or noise generated. For example, too dark areas may make some subtle structures of cells hidden in the dark and difficult to distinguish, while too bright areas may cause color distortion and blurred cell boundaries. Therefore, in order to correct this uneven illumination phenomenon caused by external factors and to restore the true appearance of the pathological slices, the present application performs illumination compensation on each of the multiple pathological slice images to enhance the clarity and contrast of the image, so that the image quality is improved, and multiple pathological slice images after illumination compensation are obtained.

[0041] In the process of implementing illumination compensation, the original acquired pathological slice images need to be carefully analyzed first to identify the specific manifestations of uneven illumination. This can be achieved by calculating the brightness value of each pixel in the image and drawing a brightness histogram. By observing the brightness distribution, it can be determined which areas are too bright or too dark. In addition, methods such as spatial frequency analysis can be used to detect whether there are periodic brightness changes in the image, which may be caused by stripes or other non-uniformities of the light source itself. For example, if the problem is due to gradual uneven illumination caused by the strong center of the microscope light source and weak edges, a radially symmetric illumination model can be selected for fitting.

[0042] Once the specific manifestations of uneven illumination are determined, the next step is to build an illumination model. An illumination model is a mathematical description of actual illumination conditions that helps understand and predict how illumination affects images. For pathological slide images, a model that is suitable for a specific application scenario is usually selected. For example, in the case of radial uneven illumination mentioned above, a radially symmetric illumination model can better simulate the impact of the light source. Subsequently, a preprocessing algorithm is applied to estimate and remove the effects of illumination. Commonly used methods include global grayscale stretching, which is simple but may amplify noise and cannot solve the problem of local uneven illumination; local adaptive equalization (CLAHE), which dynamically adjusts the brightness according to the characteristics of the local area of ​​the image. It can improve the visibility of local details while maintaining the overall contrast of the image, and is particularly suitable for pathological sections with complex illumination patterns; multi-scale decomposition and reconstruction, which decomposes the image into multiple sub-bands at different scales, and then performs illumination correction on each sub-band separately, and finally recombine them into a complete image, effectively removing the uneven illumination caused by different frequency bands while retaining important texture information; and deep learning methods based on convolutional neural networks (CNNs). These models automatically capture illumination characteristics by learning from a large amount of labeled data to generate more natural and realistic correction results.

[0043] After selecting the appropriate preprocessing algorithm, you can start correcting uneven illumination. The specific approach is to calculate the ideal brightness value that each pixel should have based on the illumination model, and then adjust the brightness of the corresponding position in the original image so that the final output image is as close to the ideal state as possible. In this process, some boundary conditions must also be considered, such as avoiding excessive amplification of noise or introducing new artifacts. After completing illumination compensation, the results must be verified to ensure that the image quality has indeed been improved. This can be done through visual inspection and quantitative evaluation. Visual inspection relies on the professional's experience to see whether the image has become clearer, the contrast is appropriate, and the details are more obvious; while quantitative evaluation involves the use of objective indicators such as signal-to-noise ratio (SNR), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), etc., to measure the change in image quality before and after correction. Only when all tests show that illumination compensation has achieved the expected goals can the process be considered successful.

[0044] In a specific embodiment of the present application, it is assumed that there is a pathological slice image, the central part of which appears abnormally bright due to the concentration of the microscope light source, while the surrounding part is relatively dark. In this case, the pathological slice image is first loaded using image processing software and preliminarily analyzed, and it is found that the brightness distribution presents an obvious radial gradient. Based on this observation, it is decided to use a simple radially symmetric illumination model to simulate this uneven illumination phenomenon, and design a corresponding correction scheme accordingly. Next, the local adaptive equalization (CLAHE) algorithm is applied to preprocess the image to optimize the local contrast without changing the global brightness distribution. Then, the ideal brightness value of each pixel is calculated according to the selected illumination model, and the original image is adjusted according to this standard to ensure that the brightness of the center and edge areas tends to be consistent. Finally, the processed image is checked again to confirm that it is not only visually smoother and has no obvious transition marks, but also has significant improvements in quantitative indicators, such as higher SNR and better SSIM scores. Through the above steps, the pathological slice image that originally had serious uneven illumination problems was successfully restored to its proper details and contrast after illumination compensation processing, providing high-quality basic data for subsequent pathological research.

[0045] Then, considering that each pathological slice image contains many local details that are of great value to diagnosis, such as cell texture, edge and other detail information, and there is a global semantic association relationship between each local texture and edge detail information. Based on this, the present application uses a pathological slice image feature extractor based on a dynamic convolutional labeling model to extract features of each of the multiple illumination-compensated pathological slice images to obtain multiple pathological slice image semantic feature encoding maps. It should be understood that the dynamic convolutional labeling model is a model that combines the advantages of convolutional neural networks (CNNs) and Transformer architectures. Specifically, first, in the local feature extraction stage, the model divides the input image into multiple local blocks. For complex and detailed images such as illumination-compensated pathological slice images, CNN can efficiently process each local block. CNN slides on local blocks through convolution kernels to detect the change pattern between pixels, so that fine local features such as textures and edges can be accurately captured. For example, in pathological sections, CNN can keenly detect changes in cell texture, such as the difference in texture between cancer cells and normal cells, and the edge features of tissue boundaries, which are crucial for pathological diagnosis. Then, the global feature integration stage is entered. The model splices the feature maps obtained after CNN processing of each local block and uses it as the input of the Transformer model. The Transformer model uses its self-attention mechanism to calculate the relevance and importance of each element in the feature map in the global environment. In this way, the model can have a comprehensive semantic understanding of the entire image and grasp the relationship between various local features in the pathological section image as a whole, such as the spatial relationship between different cell groups, the mutual influence between the lesion area and the normal tissue area. In particular, the "dynamic" in the dynamic convolution labeling model refers to the model's ability to flexibly adjust its own processing method according to the characteristics of the input data. This includes dynamically adjusting the weight distribution of attention through the self-attention mechanism, or adaptively determining which local features need to be analyzed more deeply. This ability enables the model to adapt to scenes of different complexities more flexibly, thereby improving its adaptability and processing efficiency.

[0046] In the above-mentioned high-resolution and large-field-of-view imaging system 100 based on multi-image stitching and fusion, the pathological slice image semantic feature selection module 130 is used to perform pathological slice image semantic feature selection on each pathological slice image semantic feature coding map in the multiple pathological slice image semantic feature coding maps to obtain multiple sparse pathological slice image semantic feature coding maps. Figure 3 : is a block diagram of the pathological slice image semantic feature selection module in the high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to an embodiment of the present application, such as Figure 3As shown, the pathological slice image semantic feature selection module 130 includes: an image semantic feature local decomposition unit 131, which is used to perform feature decoupling and feature flattening on the features of the pathological slice image semantic feature coding map to obtain a set of pathological slice image semantic local feature vectors; an image semantic feature local selection aggregation unit 132, which is used to perform feature neighborhood activity selection and shape reshaping on the set of pathological slice image semantic local feature vectors to obtain the sparse pathological slice image semantic feature coding map.

[0047] Furthermore, it is considered that different pathological features have different values ​​in diagnosis. Some features may have a stronger direct correlation with the lesion, such as specific morphological features of diseased cells or inflammatory response features around diseased tissues, and although each pathological slice image semantic feature coding map contains rich semantic information, there may be a lot of redundancy. For example, cell feature coding in different regions may have similar parts, or some local features may be less important in the overall pathological judgment. Therefore, in order to screen out the most representative and key features and reduce these redundant information, the present application introduces a pathological slice image semantic feature selection method to process each pathological slice image semantic feature coding map in the multiple pathological slice image semantic feature coding maps separately to obtain multiple sparse pathological slice image semantic feature coding maps. That is, this method dynamically measures the importance and related interactions of a feature in its surrounding area (semantic neighborhood) through the activity of the feature neighborhood, thereby highlighting these key features and making subsequent processing more efficient and accurate.

[0048] In a specific embodiment of the present application, the image semantic feature local decomposition unit 131 is used to: perform feature decoupling on the pathological slice image semantic feature coding map along the channel dimension to obtain a set of pathological slice image semantic feature coding matrices; and perform feature flattening on each pathological slice image semantic feature coding matrix in the set of pathological slice image semantic feature coding matrices to obtain a set of pathological slice image semantic local feature vectors.

[0049] First, the pathological slice image semantic feature coding map is feature decoupled and feature flattened to obtain a set of pathological slice image semantic local feature vectors. Specifically, feature decoupling means decomposing the pathological slice image semantic feature coding map to obtain a set of pathological slice image semantic feature coding matrices.

[0050] The process can be expressed as follows:

[0051] in, is the semantic feature coding map of the pathological slice image, is a feature decoupling operation, , , and is the first, second, and third in the set of semantic feature encoding matrices of pathological slice images. and A semantic feature encoding matrix of pathological slice images.

[0052] Then, each pathological slice image semantic feature encoding matrix in the set of pathological slice image semantic feature encoding matrices is feature flattened to obtain a set of pathological slice image semantic local feature vectors, thereby ensuring that the elements in each pathological slice image semantic local feature vector represent different but related feature dimensions. It should be understood that the generation of independent components is guaranteed by feature decoupling and feature flattening, thereby providing materials for subsequent feature selection.

[0053] The process can be expressed as follows:

[0054] in, is the feature flattening operation, , , and They are the first, second, and third semantic local feature vectors in the set of pathological slice images. and The semantic local feature vector of pathological slice images, , , and is the first, second, and third in the set of semantic feature encoding matrices of pathological slice images. and A semantic feature encoding matrix of pathological slice images.

[0055] In a specific embodiment of the present application, Figure 4 : is a block diagram of the image semantic feature local selection aggregation unit in the high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to an embodiment of the present application, such as Figure 4As shown, the image semantic feature local selection aggregation unit 132 includes: an image semantic local feature descending sorting subunit 1321, which is used to arrange the set of pathological slice image semantic local feature vectors in descending order to obtain a descending sequence of pathological slice image semantic local feature vectors; an image semantic local feature selection subunit 1322, which is used to perform feature selection on the descending sequence of pathological slice image semantic local feature vectors to obtain a descending sequence of selected pathological slice image semantic local feature vectors; an image semantic feature shape reshaping subunit 1323, which is used to perform feature shape reshaping on the descending sequence of selected pathological slice image semantic local feature vectors to obtain the sparse pathological slice image semantic feature coding map.

[0056] Specifically, the image semantic local feature descending order sorting subunit 1321 is used to: input each pathological slice image semantic local feature vector in the set of pathological slice image semantic local feature vectors into an importance measurement module to obtain a set of pathological slice image semantic local feature importance score values; based on the set of pathological slice image semantic local feature importance score values, arrange the set of pathological slice image semantic local feature vectors in descending order to obtain a descending sequence of the pathological slice image semantic local feature vectors.

[0057] That is, each pathological slice image semantic local feature vector in the set of pathological slice image semantic local feature vectors is input into the importance measurement module to obtain a set of pathological slice image semantic local feature importance score values. The importance measurement module calculates the importance score of each pathological slice image semantic local feature according to a specific algorithm, and these scores reflect the degree of contribution of each pathological slice image semantic local feature importance score value to the overall task (such as classification or regression).

[0058] The process can be expressed as follows:

[0059] in, and They are The corresponding weight matrix and bias vector, is matrix multiplication, is the weight score vector, is a collection of importance scores of semantic local features of pathological slice images, , , and They are the first, second, and third in the set of importance scores of semantic local features of pathological slice images. and The importance score of the semantic local features of the pathological slice image, It is the first in the set of semantic local feature vectors of pathological slice images. Semantic local feature vectors of pathological slice images.

[0060] Furthermore, based on the set of importance scores of the semantic local features of the pathological slice images, the set of semantic local feature vectors of the pathological slice images is arranged in descending order to obtain a descending sequence of the semantic local feature vectors of the pathological slice images. That is, according to the obtained importance scores of the semantic local features of the pathological slice images, all the importance scores of the semantic local features of the pathological slice images are sorted from high to low in importance to form a descending sequence. This sorting mechanism can not only identify the semantic local features of the pathological slice images that are most critical to the task objectives, but also retain a certain proportion or number of high-scoring semantic local features of the pathological slice images according to actual needs, thereby achieving the purpose of sparseness. Sorting itself does not change the content of the semantic local features of the pathological slice images, but provides a clear view of the relative importance of the semantic local features of the pathological slice images. This step not only simplifies the selection process of the semantic local features of the pathological slice images, but also enables subsequent computing resources to be concentrated on the most important semantic local features of the pathological slice images.

[0061] The process can be expressed as follows:

[0062] in, Based on Arrange the set of semantic local feature vectors of the pathological slice image in descending order, , , and They are the first, second, and third in the descending sequence of the semantic local feature vectors of the pathological slice image. and The semantic local feature vector of pathological slice images, , , and They are the first, second, and third semantic local feature vectors in the set of pathological slice images. and Semantic local feature vectors of pathological slice images.

[0063] Specifically, the image semantic local feature selection subunit 1322 is used to: calculate the feature neighborhood activity of each pathological slice image semantic local feature vector in the descending sequence of the pathological slice image semantic local feature vector based on the neighborhood features of each pathological slice image semantic local feature vector to obtain a sequence of pathological slice image semantic local feature neighborhood activities; and perform feature selection on the descending sequence of pathological slice image semantic local feature vectors based on the sequence of pathological slice image semantic local feature neighborhood activities to obtain a descending sequence of pathological slice image semantic local feature vectors after the selection.

[0064] On this basis, based on the neighborhood features of each pathological slice image semantic local feature vector in the descending sequence of the pathological slice image semantic local feature vector, the feature neighborhood activity of each pathological slice image semantic local feature vector is calculated to obtain a sequence of pathological slice image semantic local feature neighborhood activity. That is, considering the relationship between the local features of the pathological slice image semantic local feature vector and its neighboring features (i.e., feature neighborhood), the index of the pathological slice image semantic local feature neighborhood activity can be obtained by calculating the degree of interaction between the local features of each pathological slice image semantic local feature vector and the surrounding features. The pathological slice image semantic local feature neighborhood activity reflects the strength of association between the local features of each pathological slice image semantic local feature vector and the surrounding features, emphasizing the interaction between features rather than the importance of a single feature. By introducing the pathological slice image semantic local feature neighborhood activity, the spatial distribution characteristics of the neighborhood features of the pathological slice image semantic local feature vector can be further considered, which deepens the understanding of the feature space of the neighborhood features of the pathological slice image semantic local feature vector, and provides a more comprehensive perspective for selecting the most effective neighborhood features of the pathological slice image semantic local feature vector.

[0065] The process can be expressed as follows:

[0066] in, is the first in the descending sequence of the semantic local feature vector of the pathological slice image The semantic local feature vector of pathological slice images, yes Middle The eigenvalues ​​at the positions, yes The number of eigenvalues ​​in , and They are and The corresponding pathological slice image semantic weighted average feature value, yes The corresponding pathological slice image semantic local feature neighborhood activity.

[0067] Furthermore, based on the sequence of neighborhood activity of the semantic local features of the pathological slice image, feature selection is performed on the descending sequence of the semantic local feature vectors of the pathological slice image to obtain the descending sequence of the semantic local feature vectors of the pathological slice image after selection. That is, the descending sequence of the semantic local feature vectors of the pathological slice image with the most value is further screened out by using the sequence of neighborhood activity of the semantic local features of the pathological slice image. At this stage, the neighborhood feature selection of the semantic local feature vectors of the pathological slice image no longer depends solely on the importance score of the individual features, but comprehensively considers the relationship between the neighborhood features of the semantic local feature vectors of the pathological slice image and their neighbors. The neighborhood features of the semantic local feature vectors of the pathological slice image selected in this way can not only represent the original data well, but also better capture the patterns and regularities in the data. The descending sequence of the semantic local feature vectors of the pathological slice image after selection is expected to be more compact and more expressive, which is conducive to improving the performance and interpretability of the model. In a specific implementation, the neighborhood feature selection strategy of the semantic local feature vectors of the pathological slice image can be flexibly adjusted according to the specific application, such as setting a fixed activity threshold, dynamically selecting a certain percentage of the highest activity features, or adopting more complex combination rules.

[0068] The process can be expressed as follows:

[0069]

[0070] in, , , and They are the first, second, and third in the descending sequence of the semantic local feature vectors of the pathological slice image. and The semantic local feature vector of pathological slice images, , , and They are the first, second, and third in the descending sequence of the semantic local feature vectors of the selected pathological slice image. and The semantic local feature vector of the selected pathological slice image, yes The corresponding pathological slice image semantic local feature neighborhood activity, For Perform feature selection, is the preset threshold.

[0071] Finally, the descending sequence of the semantic local feature vectors of the selected pathological slice image is reshaped to obtain the sparse pathological slice image semantic feature coding map. That is, the semantic local feature vectors of the selected pathological slice image are reorganized back into the spatial structure of the pathological slice image semantic feature coding map to form the sparse pathological slice image semantic feature coding map. The reshaping process involves reorganizing the one-dimensional semantic local feature vectors of the selected pathological slice image into the multi-dimensional form of the sparse pathological slice image semantic feature coding map, ensuring consistency with the original input format, which is crucial to maintaining the consistency and compatibility of the model architecture. The reshaped sparse pathological slice image semantic feature coding map can also be used as an intermediate representation for downstream tasks, such as feature visualization or auxiliary diagnostic tools.

[0072] The process can be expressed as follows:

[0073] in, , , and They are the first, second, and third in the descending sequence of the semantic local feature vectors of the selected pathological slice image. and The semantic local feature vector of the selected pathological slice image, For the reshape operation, It is the semantic feature coding map of the sparse pathological slice image.

[0074] In the above-mentioned high-resolution, large-field-of-view imaging system 100 based on multi-image stitching and fusion, the global pathology slice image generation module 140 is used to: after cascading the multiple sparse pathology slice image semantic feature coding maps into the global pathology slice image semantic feature stitching coding map, input it into the image fusion engine based on the AIGC model to obtain the global pathology slice fusion image.

[0075] Wherein, the image fusion engine is an image fusion engine based on the AIGC model.

[0076] Then, after the multiple sparse pathological slice image semantic feature coding maps are cascaded into a global pathological slice image semantic feature splicing coding map, they are input into the image fusion engine based on the AIGC model to obtain a global pathological slice fusion image. It should be understood that each sparse pathological slice image semantic feature coding map contains the key semantic features of the corresponding pathological slice after screening. These features are local and represent the important information of each slice itself. Therefore, in order to integrate the scattered semantic features into a unified representation and form a more comprehensive semantic feature representation, the present application cascades the multiple sparse pathological slice image semantic feature coding maps into a global pathological slice image semantic feature splicing coding map. This helps to grasp the semantic information of all slices as a whole and form a "panoramic view" perspective covering all slice contents. For example, for a larger pathological tissue sample, different slices may contain different parts of the diseased tissue or normal tissue parts adjacent to the diseased tissue. After cascading, the distribution and spread of the lesion in the entire tissue can be better understood. Then it is input into the image fusion engine of the AIGC model, which uses the AIGC model's excellent ability in image generation and fusion to intelligently generate high-quality image content based on the input feature information and learn a large number of data patterns and semantic relationships to obtain a global pathology slice fusion image. In this way, the fused image can be constructed based on the most valuable features, thereby maintaining high-resolution details while better integrating the semantic information of each slice, providing clearer and more targeted images for pathology diagnosis.

[0077] Here, in the case where each of the multiple pathological slice image semantic feature coding maps respectively represents the dynamic local image semantic association features of the pathological slice image after illumination compensation in a single sample domain, when performing feature selection based on feature neighborhood activity, the complexity of the feature neighborhood activity mechanism in the global sample domain will lead to insufficient long-distance selective splicing representation of the global pathological slice image semantic feature splicing coding map obtained by cascading the multiple sparse pathological slice image semantic feature coding maps, thereby reducing the expression effect of the global pathological slice image semantic feature splicing coding map and affecting the image quality of the global pathological slice fusion image obtained by inputting it into the image fusion engine based on the AIGC model.

[0078] Preferably, in one example, when the global pathology slice image semantic feature splicing coding map is input into an image fusion engine based on an AIGC model to obtain a global pathology slice fusion image, the global pathology slice image semantic feature splicing coding map is optimized, including the following steps: Expanding the global pathology slice image semantic feature splicing coding map to obtain a global pathology slice image semantic feature splicing coding feature set composed of each feature value of the global pathology slice image semantic feature splicing coding map; In response to the first one of the global pathological slice image semantic feature splicing coding feature set The eigenvalue and The eigenvalues ​​between The distance value is less than or equal to the distance difference hyperparameter ,Right now:

[0079] in, and Respectively represent the first The eigenvalue and Eigenvalue; The said The eigenvalue is multiplied by the predetermined weight and then The eigenvalues ​​are added to get the optimized Eigenvalue , represents the predetermined weight; In response to the first one of the global pathological slice image semantic feature splicing coding feature set The eigenvalue and The absolute value of the difference between the eigenvalues ​​is greater than the distance difference hyperparameter ,Right now : After subtracting the eigenvalue from the square of each eigenvalue of the global pathological slice image semantic feature splicing encoding feature set, summing all eigenvalues ​​of the global pathological slice image semantic feature splicing encoding feature set to obtain a medical image shallow-deep joint perception encoding heterogeneous projection value;

[0080] Among them, the scale of the global pathological slice image semantic feature splicing coding map is is equal to the width of the feature matrix of the global pathological slice image semantic feature splicing coding map multiplied by the height and then multiplied by the number of channels of the global pathological slice image semantic feature splicing coding map, Represents the medical image shallow-deep joint perception encoding heterogeneous projection value; Each feature value of the medical image shallow-deep joint perception coding heterogeneous projection value and the global pathological slice image semantic feature splicing coding map is After adding the product of the scale of the global pathological slice image semantic feature splicing coding map, it is divided by the square of the scale of the global pathological slice image semantic feature splicing coding map to obtain each feature value of the global pathological slice image semantic feature splicing coding map. The corresponding medical image shallow-deep joint perception encoding multi-dimensional latent primitive values:

[0081] in, Represents each eigenvalue The corresponding medical image shallow-deep joint perception encodes multi-dimensional latent primitive values; The multi-dimensional latent primitive value of the medical image shallow-deep joint perception coding is multiplied by the first After the eigenvalue, calculate the product with the first The weighted sum between the eigenvalues ​​is the optimized Eigenvalue ,in, represents the first weighted hyperparameter, represents the second weighted hyperparameter; The optimization of combining the global pathological slice image semantic features and splicing encoding feature sets is Eigenvalue To obtain an optimized global pathology slice image semantic feature splicing coding map, wherein the minimum eigenvalue of the global pathology slice image semantic feature splicing coding feature set remains unchanged.

[0082] Finally, the optimized global pathology slice image semantic feature splicing coding map is input into the image fusion engine based on the AIGC model to obtain the global pathology slice fusion image.

[0083] That is, because the feature set of the global pathology slice image semantic feature splicing coding map obtained by cascading the multiple sparse pathology slice image semantic feature coding maps will produce a global feature collaboration efficiency attenuation phenomenon under the preset feature value arrangement rule when the non-local feature span breaks through the distribution constraint of adjacent elements. To solve this problem, a high-dimensional representation method of heterogeneous feature inner product projection of the global pathology slice image semantic feature splicing coding map is adopted to explicitly model the association network topology of the entire domain of feature values ​​of the global pathology slice image semantic feature splicing coding map.

[0084] Therefore, by constructing a dynamic generation mechanism of multi-scale potential primitives of the global pathological slice image semantic feature splicing coding map, a coupling conduction model between feature dimensions of the global pathological slice image semantic feature splicing coding map is established, and feature coding reconstruction of the global pathological slice image semantic feature splicing coding map is achieved under the condition of maintaining spatiotemporal continuity, ultimately improving the multi-level feature decoupling performance, improving the selective splicing expression effect of the global pathological slice image semantic feature splicing coding map, and improving the image quality of the global pathological slice fusion image obtained by inputting the image fusion engine based on the AIGC model.

[0085] Based on this, it first obtains multiple high-resolution local pathological slice images, and introduces illumination compensation technology to improve image quality and ensure that details are clearly visible; then, a dynamic convolutional labeling model is used to encode semantic features of the illumination-compensated pathological slice images to capture fine local features such as cell texture and edges, and at the same time introduces a selection method based on feature neighborhood activity to highlight important pathological features; finally, the semantic features of the sparse pathological slice images are cascaded and input into the image fusion engine based on the AIGC model to generate a global pathological slice fusion image that covers a larger tissue range and retains high-resolution details. In this way, not only the dual advantages of high resolution and large field of view are achieved, but also clearer and more targeted image support is provided for pathological diagnosis, significantly improving the accuracy and efficiency of diagnosis.

[0086] In one embodiment of the present application, Figure 5 Flow chart of a high-resolution and large-field-of-view imaging method based on multi-image stitching and fusion according to an embodiment of the present application, as shown in FIG. Figure 5As shown, according to the embodiment of the present application, the high-resolution large field of view imaging method based on multi-image stitching and fusion includes: S210, acquiring multiple pathological slice images; S220, performing illumination compensation and feature extraction on each of the multiple pathological slice images to obtain multiple pathological slice image semantic feature coding maps; S230, performing pathological slice image semantic feature selection on each of the pathological slice image semantic feature coding maps in the multiple pathological slice image semantic feature coding maps to obtain multiple sparse pathological slice image semantic feature coding maps, wherein S230, performing pathological slice image semantic feature selection on each of the pathological slice image semantic feature coding maps in the multiple pathological slice image semantic feature coding maps to obtain multiple sparse pathological slice image semantic feature coding maps. Slice image semantic feature selection to obtain multiple sparse pathology slice image semantic feature coding maps, including: S231, feature decoupling and feature flattening of the pathology slice image semantic feature coding map features to obtain a set of pathology slice image semantic local feature vectors; S232, feature neighborhood activity selection and shape reshaping of the set of pathology slice image semantic local feature vectors to obtain the sparse pathology slice image semantic feature coding map; S240, cascading the multiple sparse pathology slice image semantic feature coding maps into a global pathology slice image semantic feature splicing coding map, and obtaining a global pathology slice fusion image based on the global pathology slice image semantic feature splicing coding map.

[0087] Those skilled in the art will appreciate that the specific operations of each step in the above-mentioned high-resolution and large-field-of-view imaging method based on multi-image stitching and fusion have been described in detail above. Figures 1 to 4 It has been introduced in detail in the description of the high-resolution and large-field-of-view imaging system 100 based on multi-image stitching and fusion, and therefore, its repeated description will be omitted.

[0088] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

[0089] In addition, although each operation is described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or to be performed in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the application. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0090] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely exemplary forms of implementation. With respect to the apparatus in the above-mentioned embodiments, the specific manner in which each module performs the operation has been described in detail in the embodiments related to the method, and will not be described in detail here.

Claims

1. A high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion, characterized in that: include: A pathological slice image acquisition module, used for acquiring a plurality of pathological slice images; A pathological slice image compensation and extraction module, used for performing illumination compensation and feature extraction on each of the plurality of pathological slice images to obtain a plurality of pathological slice image semantic feature coding maps; A pathological slice image semantic feature selection module is used to perform pathological slice image semantic feature selection on each pathological slice image semantic feature coding map in the multiple pathological slice image semantic feature coding maps to obtain multiple sparse pathological slice image semantic feature coding maps, wherein the pathological slice image semantic feature selection module includes: an image semantic feature local decomposition unit, which is used to perform feature decoupling and feature flattening on the features of the pathological slice image semantic feature coding map to obtain a set of pathological slice image semantic local feature vectors; an image semantic feature local selection aggregation unit, which is used to perform feature neighborhood activity selection and shape reshaping on the set of pathological slice image semantic local feature vectors to obtain the sparse pathological slice image semantic feature coding map; The global pathology slice image generation module is used to cascade the multiple sparse pathology slice image semantic feature coding maps into a global pathology slice image semantic feature splicing coding map, and obtain a global pathology slice fusion image based on the global pathology slice image semantic feature splicing coding map.

2. The high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to claim 1, characterized in that: The pathological slice image compensation and extraction module comprises: A pathological slice compensation unit, used for performing illumination compensation on each of the plurality of pathological slice images to obtain a plurality of illumination-compensated pathological slice images; The pathological slice feature extraction unit is used to input each of the illumination compensated pathological slice images in the multiple illumination compensated pathological slice images into the pathological slice image feature extractor to obtain the semantic feature coding images of the multiple pathological slice images.

3. The high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to claim 2, characterized in that: The pathological slice image feature extractor is a pathological slice image feature extractor based on a dynamic convolutional labeling model.

4. The high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to claim 3, characterized in that: The image semantic feature local decomposition unit is used to: Decoupling the pathological slice image semantic feature coding map along the channel dimension to obtain a set of pathological slice image semantic feature coding matrices; Feature flattening is performed on each pathological slice image semantic feature coding matrix in the set of pathological slice image semantic feature coding matrices to obtain a set of pathological slice image semantic local feature vectors.

5. The high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to claim 4, characterized in that: The image semantic feature local selection aggregation unit includes: An image semantic local feature descending order sorting subunit is used to sort the set of pathological slice image semantic local feature vectors in descending order to obtain a descending sequence of pathological slice image semantic local feature vectors; An image semantic local feature selection subunit, used for performing feature selection on the descending sequence of the semantic local feature vectors of the pathological slice image to obtain a descending sequence of the semantic local feature vectors of the pathological slice image after selection; The image semantic feature shape reshaping subunit is used to reshape the descending sequence of the semantic local feature vectors of the selected pathological slice image to obtain the sparse pathological slice image semantic feature coding map.

6. The high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to claim 5, characterized in that: The image semantic local feature descending order sorting subunit is used to: Inputting each pathological slice image semantic local feature vector in the set of pathological slice image semantic local feature vectors into an importance measurement module to obtain a set of pathological slice image semantic local feature importance score values; Based on the set of importance score values ​​of the semantic local features of the pathological slice images, the set of semantic local feature vectors of the pathological slice images is arranged in descending order to obtain a descending sequence of the semantic local feature vectors of the pathological slice images.

7. The high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to claim 6, characterized in that: The image semantic local feature selection subunit is used to: Based on the neighborhood features of each pathological slice image semantic local feature vector in the descending sequence of the pathological slice image semantic local feature vectors, calculating the feature neighborhood activity of each pathological slice image semantic local feature vector to obtain a sequence of pathological slice image semantic local feature neighborhood activity; Based on the sequence of neighborhood activities of the semantic local features of the pathological slice images, feature selection is performed on the descending sequence of the semantic local feature vectors of the pathological slice images to obtain the descending sequence of the semantic local feature vectors of the selected pathological slice images.

8. The high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to claim 7, characterized in that: The global pathology slice image generation module is used to: after cascading the multiple sparse pathology slice image semantic feature coding maps into the global pathology slice image semantic feature splicing coding map, input it into the image fusion engine based on the AIGC model to obtain the global pathology slice fusion image.

9. The high-resolution and large-field-of-view imaging system based on multi-image stitching and fusion according to claim 8, characterized in that: The image fusion engine is an image fusion engine based on the AIGC model.

Images