Pathological image quality AI correction system based on multi-modal image fusion

Abstract

The pathological image quality AI correction system based on multi-modal image fusion comprises a feature point marking unit, an image feature acquisition unit, a non-image image acquisition unit, a scale division unit, an integration unit and a fusion unit, then, the image feature acquisition unit and the non-image image acquisition unit respectively acquire a multi-modal image, hardness information and electrophysiological activity based on the positions of the feature points, and the hardness information and the electrophysiological activity can be converted to obtain a hardness information matrix and an electrophysiological signal feature spectrum; a multi-modal image is divided into a low-frequency structure part and a high-frequency structure part, the low-frequency structure part is integrated with a hardness information matrix, the high-frequency structure part is integrated with an electrophysiological signal characteristic spectrum, and the integrated low-frequency structure part and high-frequency structure part are finally subjected to weighted fusion. Comprehensive information can be reserved, the image quality is enhanced, and the data fusion effect is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image fusion correction, and in particular to an AI correction system for pathological image quality based on multimodal image fusion. Background Art

[0002] In today's medical diagnosis field, pathological images play a vital role and are the key basis for disease diagnosis, condition assessment and treatment plan formulation. With the rapid development of medical technology, multimodal imaging technology has emerged. Different types of microscopes such as optical microscopes, electron microscopes and fluorescence microscopes can collect different pathological image information, which brings richer and more comprehensive information to pathological research and clinical diagnosis. However, while multimodal imaging brings many opportunities for medical diagnosis, it is also accompanied by a series of severe challenges. First, due to the different imaging principles of different modal images, their image quality is disturbed by various factors, resulting in different types of artifacts and noise. Secondly, due to the different resolutions, contrasts, spatial scales and coordinate systems of each modality image, the difficulty of multimodal image fusion is relatively high. Therefore, in the medical field, single-modal images are often used to assist in the diagnosis of diseases. In addition, although there are some multimodal pathological image fusion methods, only image features are considered during fusion, and non-image features are not considered. Accurate evaluation data cannot be obtained, which easily leads to inconsistency in diagnostic results and cannot meet the precise and efficient development needs of modern medicine. Summary of the invention

[0003] In view of this, the present invention proposes a pathological image quality AI correction system based on multimodal image fusion, which can not only realize the fusion of multimodal images, but also add non-imaging features to improve the accuracy of diagnostic results.

[0004] The technical solution of the present invention is achieved in this way:

[0005] Pathological image quality AI correction system based on multimodal image fusion, including:

[0006] A feature point marking unit, used to mark a number of feature points on the pathological tissue;

[0007] An image feature acquisition unit, used to acquire multimodal images of pathological tissues containing feature points using an optical microscope, an electron microscope, and a fluorescence microscope;

[0008] A non-imaging image acquisition unit, used to generate a hardness information matrix according to the hardness information of the feature point and its surrounding area, and to generate an electrophysiological signal characteristic map according to the electrophysiological activities of cells in the area;

[0009] A scale division unit, used for dividing the multimodal image into a low-frequency structure part and a high-frequency structure part;

[0010] An integration unit, used for integrating the hardness information matrix into the low-frequency structure part and integrating the electrophysiological signal characteristic map into the high-frequency structure part;

[0011] A fusion unit, used for weighted fusion of the integrated low-frequency structure part and the high-frequency structure part;

[0012] The image feature acquisition unit is data-connected with the scale division unit, and the integration unit is data-connected with the non-image image acquisition unit, the scale division unit and the fusion unit respectively.

[0013] Preferably, the execution steps of the feature point marking unit include:

[0014] Step S11, using a high-precision microscope to search the pathological tissue, and obtain the boundary intersection points, the center of the specific cell group and the blood vessel branch points in the pathological tissue;

[0015] Step S12: Grid point marking and slicing are performed at boundary intersections, centers of specific cell groups, and blood vessel branch points to form a number of feature points.

[0016] Preferably, a preprocessing unit is further included, and the preprocessing unit is used to preprocess and extract features of the multimodal images, and the execution steps include:

[0017] Step S21, performing format conversion, noise removal, grayscale normalization and image enhancement on the multimodal images;

[0018] Step S22, inputting the preprocessed multimodal image into the trained convolutional neural network, and obtaining a feature representation of the multimodal image by the convolutional neural network;

[0019] The preprocessing unit is data-connected with the image feature acquisition unit and the scale division unit respectively.

[0020] Preferably, the execution steps of the non-image acquisition unit include:

[0021] Step S31, using atomic force microscopy and ultrasonic elastic imaging technology to locate the feature points, measure the hardness values ​​of the feature points and their surrounding areas, and construct a hardness information matrix associated with the feature points;

[0022] Step S32, using a microelectrode array to be placed close to the marked feature point, recording the electrophysiological activity of cells in the area, extracting the firing frequency of action potentials and the amplitude change of local field potentials, and constructing a characteristic map of electrophysiological signals;

[0023] Preferably, it also includes a spatial registration unit for registering the hardness information matrix and the electrophysiological signal characteristic map to correspond to the multimodal image using an image registration algorithm, and the spatial registration unit is respectively connected to the preprocessing unit, the non-image image acquisition unit and the integration unit data.

[0024] Preferably, the execution steps of the scale division unit include:

[0025] Step S41, performing Gaussian filtering on the multimodal image, and then downsampling to obtain the first layer of pyramid;

[0026] Step S42, repeating step S41 to generate a multi-level pyramid layer by layer;

[0027] Step S43: output the top layer of the pyramid as a low-frequency structure part, and output the bottom layer of the pyramid as a high-frequency structure part.

[0028] Preferably, the execution steps of the integration unit include:

[0029] Step S51, respectively extracting the low-frequency part in the hardness information matrix and the high-frequency part in the electrophysiological signal characteristic spectrum;

[0030] Step S52, adding the low frequency part and the low frequency structure part;

[0031] Step S53: multiply the high frequency part by the high frequency structure part, and normalize the product.

[0032] Preferably, the execution steps of the fusion unit include:

[0033] Step S61, assigning different weights to the low-frequency structure part and the high-frequency structure part respectively;

[0034] Step S62, summing the weighted low-frequency structure part and the high-frequency structure part;

[0035] Step S63: smooth the summation result by using a smoothing filtering method, and perform feature enhancement at the same time.

[0036] Preferably, the specific steps of step S61 include:

[0037] Step S611, querying whether the pathological tissue has a clinical diagnosis requirement, and when there is a clinical diagnosis requirement, allocating weights to the low-frequency structure part and the high-frequency structure part based on the clinical diagnosis requirement;

[0038] Step S612: If there is no clinical diagnosis requirement, a deep learning network is used to identify the pathological tissue image, obtain the tissue type, and assign weights to the low-frequency structure part and the high-frequency structure part based on the tissue type;

[0039] Step S613: If the deep learning network cannot accurately identify the tissue type, weights are assigned to the low-frequency structure part and the high-frequency structure part based on the source organ of the pathological tissue.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The pathological image quality AI correction system based on multimodal image fusion of the present invention can not only collect multimodal images at the feature points, but also collect non-imaging features at the feature points after marking the feature points on the pathological tissue, wherein the non-imaging features include hardness information and electrophysiological activity information around the feature points. The collected multimodal images will be divided into a low-frequency structure part and a high-frequency structure part, and then the hardness information matrix and the electrophysiological signal feature map will be integrated into the low-frequency structure part and the high-frequency structure part respectively. Due to the setting of the feature points, the integration process can be carried out accurately, and the addition of hardness information and electrophysiological activity information can provide more comprehensive physiological and pathological information, thereby improving the accuracy of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 It is a schematic diagram of the pathological image quality AI correction system based on multimodal image fusion of the present invention.

[0044] Figure 2 This is a diagram of the execution steps of the feature point marking unit of the pathological image quality AI correction system based on multimodal image fusion of the present invention.

[0045] Figure 3 This is a diagram of the execution steps of the preprocessing unit of the pathological image quality AI correction system based on multimodal image fusion of the present invention.

[0046] Figure 4 This is a diagram of the execution steps of the non-image image acquisition unit of the pathological image quality AI correction system based on multimodal image fusion of the present invention.

[0047] Figure 5 This is a diagram of the execution steps of the scale division unit of the pathological image quality AI correction system based on multimodal image fusion of the present invention.

[0048] Figure 6This is a diagram of the execution steps of the integration unit of the pathological image quality AI correction system based on multimodal image fusion of the present invention.

[0049] Figure 7 This is a diagram of the execution steps of the fusion unit of the pathological image quality AI correction system based on multimodal image fusion of the present invention.

[0050] Figure 8 This is a specific step diagram of step S61 of the pathological image quality AI correction system based on multimodal image fusion of the present invention.

[0051] In the figure, 1. feature point marking unit; 2. image feature acquisition unit; 3. non-image acquisition unit; 4. scale division unit; 5. integration unit; 6. fusion unit; 7. preprocessing unit; 8. spatial registration unit. DETAILED DESCRIPTION

[0052] In order to better understand the technical content of the present invention, a specific embodiment is provided below, and the present invention is further described in conjunction with the accompanying drawings.

[0053] See also Figures 1 to 8 The pathological image quality AI correction system based on multimodal image fusion provided by the present invention includes:

[0054] A feature point marking unit 1, used for marking a number of feature points on the pathological tissue;

[0055] An image feature acquisition unit 2 is used to acquire multimodal images of pathological tissues containing feature points using an optical microscope, an electron microscope, and a fluorescence microscope;

[0056] The non-image acquisition unit 3 is used to generate a hardness information matrix according to the hardness information of the feature point and its surrounding area and to generate an electrophysiological signal characteristic map according to the electrophysiological activities of the cells in the area;

[0057] A scale division unit 4, used for dividing the multimodal image into a low-frequency structure part and a high-frequency structure part;

[0058] An integration unit 5, used for integrating the hardness information matrix into the low-frequency structure part and integrating the electrophysiological signal characteristic map into the high-frequency structure part;

[0059] The fusion unit 6 is used to perform weighted fusion on the integrated low-frequency structure part and the high-frequency structure part.

[0060] The image feature acquisition unit 2 is data-connected with the scale division unit 4 , and the integration unit 5 is data-connected with the non-image image acquisition unit 3 , the scale division unit 4 and the fusion unit 6 , respectively.

[0061] The pathological image quality AI correction system based on multimodal image fusion of the present invention is applied to the fusion and correction of pathological images for disease diagnosis, wherein the multimodal images collected for pathological tissues include pathological images collected by optical microscopes, electron microscopes and fluorescence microscopes. These pathological images have obvious differences in resolution, imaging principles, image characteristics, etc., and cannot be directly fused, requiring special processing. In order to improve the accuracy of disease diagnosis, the present invention also introduces non-imaging features, including hardness information and electrophysiological activity. According to the hardness information and electrophysiological activity, the hardness information matrix and the electrophysiological signal characteristic map can be obtained respectively, and then the multimodal image is scaled and divided into a low-frequency structure part and a high-frequency structure part. The hardness information matrix and the electrophysiological signal characteristic map can be integrated into the low-frequency structure part and the high-frequency structure part respectively, and finally after fusion through the fusion unit 6, a pathological image combining multimodal images and non-imaging features can be obtained, so as to provide medical staff with auxiliary judgment of the disease, thereby improving the accuracy of the diagnosis result.

[0062] Before collecting multimodal images and non-imaging features, several feature points are first marked on the pathological tissue. The selected feature points can be used as a reference for subsequent data collection and fusion. Then, when the image feature collection unit 2 uses different microscopes to collect pathological tissue, it can accurately locate the same tissue area to ensure that the cell and tissue information covered by different modal images are spatially corresponding. Similarly, when the non-imaging image collection unit 3 generates the hardness information matrix and the electrophysiological signal feature map, it mainly collects data from the feature points and their surrounding areas to ensure that the collected non-imaging features can correspond to the multimodal images. By using the feature points as anchor points, data fusion can be effectively guided to reduce feature mismatch problems caused by inaccurate registration, so that the physical and physiological characteristics reflected by the tissue hardness and electrophysiological signals can be accurately superimposed on the corresponding image area.

[0063] Preferably, the execution steps of the feature point marking unit 1 include:

[0064] Step S11, using a high-precision microscope to search the pathological tissue, and obtain the boundary intersection points, the center of the specific cell group and the blood vessel branch points in the pathological tissue;

[0065] Step S12, marking and slicing grid points at boundary intersections, centers of specific cell groups, and vascular branch points to form a number of feature points;

[0066] After the pathological tissue is processed, it is observed under a high-precision microscope, and the relatively stable and highly recognizable boundary intersection points, specific cell group centers and vascular branch points in the pathological tissue are selected as feature points. Then, for easy identification, physical markings are made at the feature points, and notches are made at the marked points with an operating needle, leaving marking traces that are invisible to the naked eye but recognizable under a microscope, so as to facilitate the subsequent acquisition of multimodal images and non-imaging features.

[0067] Preferably, a preprocessing unit 7 is further included, and the preprocessing unit 7 is used to preprocess and extract features of the multimodal images, and the execution steps thereof include:

[0068] Step S21, performing format conversion, noise removal, grayscale normalization and image enhancement on the multimodal images;

[0069] Step S22, inputting the preprocessed multimodal image into the trained convolutional neural network, and obtaining a feature representation of the multimodal image by the convolutional neural network;

[0070] The preprocessing unit 7 is data-connected to the image feature acquisition unit 2 and the scale division unit 4 respectively.

[0071] After collecting pathological images through different microscopes, the formats of these pathological images are different. After format conversion, they are read into a unified digital image format, and then median filtering, Gaussian filtering and other methods are used to remove salt and pepper noise, Gaussian noise, etc. in the image, and the grayscale value of the image is normalized to a specific range. Finally, histogram equalization and contrast-limited adaptive histogram equalization methods are used to enhance the details and contrast of the image to ensure the accuracy of subsequent analysis and fusion results.

[0072] After preprocessing, it is necessary to extract features from the multimodal images. The present invention constructs a convolutional neural network and performs training, and inputs the preprocessed multimodal images into the convolutional neural network. The convolutional neural network automatically learns the features in the image through convolution layers, pooling layers, etc., and extracts higher-level and more abstract feature representations. These features are usually more representative and discriminatory, and can be better used for subsequent classification, diagnosis and other tasks.

[0073] Preferably, the execution steps of the non-image acquisition unit 3 include:

[0074] Step S31, using atomic force microscopy and ultrasonic elastic imaging technology to locate the feature points, measure the hardness values ​​of the feature points and their surrounding areas, and construct a hardness information matrix associated with the feature points;

[0075] Step S32, using a microelectrode array to be placed close to the marked feature point, recording the electrophysiological activity of cells in the area, extracting the firing frequency of action potentials and the amplitude change of local field potentials, and constructing a characteristic map of electrophysiological signals;

[0076] Atomic force microscopy can infer tissue hardness by measuring the force between the probe and the sample surface, while ultrasound elastography reflects tissue elasticity based on ultrasound echo signals. The collected hardness data needs to be quantified and converted into a hardness value matrix corresponding to image pixels to facilitate subsequent fusion;

[0077] After recording the action potential, local field potential and other electrophysiological signals of pathological tissue with the help of a microelectrode array, the collected electrophysiological signals are filtered, amplified and processed to remove noise interference and extract key features, such as the frequency, amplitude, and phase of the signal. These features are then associated with the spatial position of the pathological tissue to construct an electrophysiological signal feature map.

[0078] Preferably, it also includes a spatial registration unit 8, which is used to use an image registration algorithm to register the hardness information matrix and the electrophysiological signal characteristic map to correspond to the multimodal image. The spatial registration unit 8 is respectively connected to the preprocessing unit 7, the non-image image acquisition unit 3 and the integration unit 5.

[0079] By using registration algorithms based on feature points, mutual information or deformation models, accurate spatial correspondence between multimodal images and non-imaging features can be achieved, and hardness information and electrophysiological signals can be mapped into the multimodal image space. This ensures that the imaging features and non-imaging features of the same pathological tissue can be accurately associated, achieving consistency in data space and laying the foundation for fusion.

[0080] Preferably, the execution steps of the scale division unit 4 include:

[0081] Step S41, performing Gaussian filtering on the multimodal image, and then downsampling to obtain the first layer of pyramid;

[0082] Step S42, repeating step S41 to generate a multi-level pyramid layer by layer;

[0083] Step S43, outputting the top layer of the pyramid as a low-frequency structure part, and outputting the bottom layer of the pyramid as a high-frequency structure part;

[0084] The Gaussian pyramid decomposition method is used to divide the scale of multimodal images. The low-frequency structure part can outline the overall contour and general structure of the tissue or organ, assisting doctors to quickly understand the macroscopic morphology and provide direction for disease screening. The high-frequency structure part can show the fine texture and local changes of cells, tiny blood vessels and other microscopic levels in detail, helping to capture early hidden lesions. After the multimodal images are divided into low-frequency structure parts and high-frequency structure parts, further data processing can be performed separately to obtain accurate pathological images.

[0085] Preferably, the execution steps of the integration unit 5 include:

[0086] Step S51, respectively extracting the low-frequency part in the hardness information matrix and the high-frequency part in the electrophysiological signal characteristic spectrum;

[0087] Step S52, adding the low frequency part and the low frequency structure part;

[0088] Step S53, multiplying the high frequency part and the high frequency structure part, and normalizing the product;

[0089] Since the hardness of pathological tissue reflects the macroscopic physical properties of pathological tissue, its changes are relatively slow and have a strong correlation with the low-frequency components of multimodal images. The active areas of electrophysiological signals are usually closely related to the active areas of cell functions, and the active areas of cell functions are mostly reflected in the images as high-frequency detail information. Therefore, the hardness information matrix and the electrophysiological signal feature map can be integrated with the low-frequency structure part and the high-frequency structure part respectively. Before integration, the hardness information matrix and the electrophysiological signal feature map are processed respectively using the Gaussian pyramid decomposition method that is the same as the multimodal image scale division, and the low-frequency part and the high-frequency part can be obtained respectively. Then the low-frequency part can be directly added to the low-frequency structure part to obtain the integrated low-frequency structure part, while the high-frequency part and the high-frequency structure part are multiplied and normalized after multiplication, so that the integrated data is within a reasonable range, which is convenient for subsequent analysis and processing.

[0090] Preferably, the execution steps of the fusion unit 6 include:

[0091] Step S61, assigning different weights to the low-frequency structure part and the high-frequency structure part respectively;

[0092] Step S62, summing the weighted low-frequency structure part and the high-frequency structure part;

[0093] Step S63, smoothing the summation result by using a smoothing filtering method, and simultaneously performing feature enhancement;

[0094] When performing fusion, a weighted fusion method is adopted, in which the low-frequency structure part and the high-frequency structure part will be assigned weights of different sizes, and the sum is taken after weighting. The sum result is the fusion result. In order to ensure that the fusion result is smoother and more stable, smoothing filtering methods such as Gaussian filtering and mean filtering are used to process the fused matrix to remove possible noise or discontinuous fluctuations. In addition, the fusion result will be feature enhanced to highlight certain specific features to facilitate medical staff to diagnose diseases.

[0095] Preferably, the specific steps of step S61 include:

[0096] Step S611, querying whether the pathological tissue has a clinical diagnosis requirement, and when there is a clinical diagnosis requirement, allocating weights to the low-frequency structure part and the high-frequency structure part based on the clinical diagnosis requirement;

[0097] Step S612: If there is no clinical diagnosis requirement, a deep learning network is used to identify the pathological tissue image, obtain the tissue type, and assign weights to the low-frequency structure part and the high-frequency structure part based on the tissue type;

[0098] Step S613: if the deep learning network cannot accurately identify the tissue type, weights are assigned to the low-frequency structure part and the high-frequency structure part based on the source organ of the pathological tissue;

[0099] As for the determination of the weights assigned to the low-frequency structure part and the high-frequency structure part, it needs to be determined according to different situations. In the present invention, it will be determined based on three factors, namely clinical diagnosis needs, tissue type and the source organ of the pathological tissue, among which the priorities are from high to low: clinical diagnosis needs, tissue type and the source organ of the pathological tissue. Because multimodal image fusion is ultimately to assist clinical diagnosis and treatment, the primary goal is to meet the actual clinical needs. Clinicians often have different emphases on the diagnosis of different diseases. For example, when diagnosing cardiovascular diseases, they may pay more attention to the overall morphology of the blood vessels and the low-frequency information of hemodynamics to understand whether the blood vessels have structural lesions such as stenosis and dilation. At this time, the weight of the low-frequency structure part needs to be increased; but for some arrhythmia diseases, the high-frequency characteristics of the electrophysiological signal are more critical for judging abnormal electrical activity of the heart. At this time, the weight of the high-frequency structure part will be increased.

[0100] The second is the tissue type. The tissue type directly determines the nature and characteristics of the disease. Different tissue types, such as tumor tissue, inflammatory tissue, degenerative tissue, etc., have fundamental differences in their cell structure, physiological function and pathological changes. These differences are directly and obviously reflected in the low-frequency and high-frequency characteristics of multimodal images. Therefore, this application uses a deep learning network to identify tissue types. If the tissue type can be accurately obtained, different weights can be assigned to the low-frequency structure part and the high-frequency structure part according to the different emphasis of the tissue type.

[0101] Finally, the source organ of the pathological tissue. The characteristics of the source organ provide a basic background for the physiological function and structure of the tissue. Different organs have different physiological functions, and their tissues will also appear differently in multimodal images. For example, the electrophysiological activity of cardiac tissue has a unique expression in the high-frequency part of the electrophysiological signal feature map. Due to its metabolic function, liver tissue has metabolism-related characteristics in the low-frequency and high-frequency parts of the image. Therefore, different weights can be determined based on the different source organs.

[0102] By fusing the low-frequency structure part and the high-frequency structure part in a weighted fusion manner, comprehensive information can be retained, image quality can be enhanced, feature stability can be improved, and data fusion effect can be enhanced, providing valuable basis for subsequent disease diagnosis and analysis.

[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. Pathological image quality AI correction system based on multimodal image fusion, characterized by: include: A feature point marking unit, used to mark a number of feature points on the pathological tissue; An image feature acquisition unit, used to acquire multimodal images of pathological tissues containing feature points using an optical microscope, an electron microscope, and a fluorescence microscope; A non-imaging image acquisition unit, used to generate a hardness information matrix according to the hardness information of the feature point and its surrounding area, and to generate an electrophysiological signal characteristic map according to the electrophysiological activities of cells in the area; A scale division unit, used for dividing the multimodal image into a low-frequency structure part and a high-frequency structure part; An integration unit, used for integrating the hardness information matrix into the low-frequency structure part and integrating the electrophysiological signal characteristic map into the high-frequency structure part; A fusion unit, used for weighted fusion of the integrated low-frequency structure part and the high-frequency structure part; The image feature acquisition unit is data-connected with the scale division unit, and the integration unit is data-connected with the non-image image acquisition unit, the scale division unit and the fusion unit respectively.

2. The pathological image quality AI correction system based on multimodal image fusion according to claim 1 is characterized in that: The execution steps of the feature point marking unit include: Step S11, using a high-precision microscope to search the pathological tissue, and obtain the boundary intersection points, the center of the specific cell group and the blood vessel branch points in the pathological tissue; Step S12: Grid point marking and slicing are performed at boundary intersections, centers of specific cell groups, and blood vessel branch points to form a number of feature points.

3. The pathological image quality AI correction system based on multimodal image fusion according to claim 1, characterized in that: The system also includes a preprocessing unit, which is used to preprocess and extract features of the multimodal images, and the execution steps include: Step S21, performing format conversion, noise removal, grayscale normalization and image enhancement on the multimodal images; Step S22, inputting the preprocessed multimodal image into the trained convolutional neural network, and obtaining a feature representation of the multimodal image by the convolutional neural network; The preprocessing unit is data-connected with the image feature acquisition unit and the scale division unit respectively.

4. The pathological image quality AI correction system based on multimodal image fusion according to claim 3 is characterized in that: The execution steps of the non-image acquisition unit include: Step S31, using atomic force microscopy and ultrasonic elastic imaging technology to locate the feature points, measure the hardness values ​​of the feature points and their surrounding areas, and construct a hardness information matrix associated with the feature points; Step S32: Place a microelectrode array close to the marked feature points to record the electrophysiological activity of cells in the area, extract the firing frequency of action potentials and the amplitude changes of local field potentials, and construct a characteristic map of electrophysiological signals.

5. The pathological image quality AI correction system based on multimodal image fusion according to claim 4 is characterized in that: It also includes a spatial registration unit for registering the hardness information matrix and the electrophysiological signal characteristic map to correspond to the multimodal image using an image registration algorithm. The spatial registration unit is respectively connected to the preprocessing unit, the non-image image acquisition unit and the integration unit data.

6. The pathological image quality AI correction system based on multimodal image fusion according to claim 3 is characterized in that: The execution steps of the scale division unit include: Step S41, performing Gaussian filtering on the multimodal image, and then downsampling to obtain the first layer of pyramid; Step S42, repeating step S41 to generate a multi-level pyramid layer by layer; Step S43: output the top layer of the pyramid as a low-frequency structure part, and output the bottom layer of the pyramid as a high-frequency structure part.

7. The pathological image quality AI correction system based on multimodal image fusion according to claim 1, characterized in that: The execution steps of the integration unit include: Step S51, respectively extracting the low-frequency part in the hardness information matrix and the high-frequency part in the electrophysiological signal characteristic spectrum; Step S52, adding the low frequency part and the low frequency structure part; Step S53: multiply the high frequency part by the high frequency structure part, and normalize the product.

8. The pathological image quality AI correction system based on multimodal image fusion according to claim 1, characterized in that: The execution steps of the fusion unit include: Step S61, assigning different weights to the low-frequency structure part and the high-frequency structure part respectively; Step S62, summing the weighted low-frequency structure part and the high-frequency structure part; Step S63: smooth the summation result by using a smoothing filtering method, and perform feature enhancement at the same time.

9. The pathological image quality AI correction system based on multimodal image fusion according to claim 8, characterized in that: The specific steps of step S61 include: Step S611, querying whether the pathological tissue has a clinical diagnosis requirement, and when there is a clinical diagnosis requirement, allocating weights to the low-frequency structure part and the high-frequency structure part based on the clinical diagnosis requirement; Step S612: If there is no clinical diagnosis requirement, a deep learning network is used to identify the pathological tissue image, obtain the tissue type, and assign weights to the low-frequency structure part and the high-frequency structure part based on the tissue type; Step S613: If the deep learning network cannot accurately identify the tissue type, weights are assigned to the low-frequency structure part and the high-frequency structure part based on the source organ of the pathological tissue.

Images