Method and application based on multi-layer fusion of fluorescent glass slides and AI multi-layer fusion processing

Abstract

The present invention proposes a method and application based on multi-layer fusion of fluorescent glass slides and AI multi-layer fusion processing, including determining the camera height, tissue thickness and lens depth of field to calculate the shooting parameters; extracting effective information of multi-layer images through AI models; and generating an image with clear layers using feature fusion technology. The device includes fluorescence scanning, image processing and fusion modules. The present invention improves the spatial resolution and multiple labeling accuracy of fluorescence microscopic images.

Description

Technical Field

[0001] The present invention relates to the field of fluorescence microscopy imaging technology, and in particular to a method and application thereof based on fluorescent slide multi-layer fusion and AI multi-layer fusion processing. Background Art

[0002] Fluorescence microscopy is an important tool in biological research. It uses fluorescent markers to mark specific structures, thereby achieving high-resolution observation of the internal structure of cells or tissue samples. In recent years, with the development of imaging technology and fluorescent labeling technology, fluorescent glass slide multilayer fusion technology has been widely used as a method to enhance the depth and details of microscopic images. This technology usually involves stacking multiple glass slides with different fluorescent markers and performing image processing through computer software to obtain the final composite image.

[0003] However, in practical applications, traditional multi-layer fusion technology faces some challenges due to the different thickness of tissue samples and the limitation of camera depth of field. In particular, when the thickness of the sample exceeds the depth of field of the camera, cells at different levels will appear blurred to varying degrees in the image, which not only reduces the quality of the image but may also lead to errors in the recognition of cell features. In addition, since cells at different levels may have different optical properties, how to effectively extract clear and effective cell information from these levels has become an urgent problem to be solved.

[0004] The current technical means for solving the above problems are not mature yet, especially in terms of how to ensure that cells at different levels can be clearly presented and how to accurately integrate effective image information at different levels. The existing technical solutions often fail to achieve satisfactory results. Therefore, it is urgent to develop new technical means to overcome the shortcomings of the existing technology. Summary of the invention

[0005] The embodiment of the present invention provides a method and application based on fluorescent slide multi-layer fusion and AI multi-layer fusion processing, which aims to solve the problems of immature technical solutions in current technology.

[0006] The core technology of the present invention is mainly a method to improve the quality of multi-layer fluorescent slide microscopic images through automatic scanning, AI image processing and multi-layer fusion technology, which solves the problem of cell blurring caused by tissue thickness exceeding the camera depth of field.

[0007] In a first aspect, the present invention provides a method based on fluorescent glass multi-layer fusion and AI multi-layer fusion processing, the method comprising the following steps:

[0008] S00, fluorescence scanning logic:

[0009] Determine the camera height H, tissue thickness W, and camera lens depth of field f for a clear field of view, and use this to calculate the number of photos N, offset S, and starting height C;

[0010] Move the Z axis of the camera to the starting height C, trigger a photo every offset distance S, until N pictures are taken, then move to the next field of view to repeat the photo shooting, and after the current slide is photographed, obtain the multi-layer fluorescence microscope image data;

[0011] Among them, multi-layer fluorescence microscopy image data covers different depths and planes;

[0012] S10, AI image processing to obtain effective information:

[0013] The effective information of different layers of multi-layer image data is extracted through the AI ​​fluorescence image multi-layer model, so as to extract the clear part of the image from each layer of the image. The AI ​​fluorescence image multi-layer model is trained by collecting multi-layer fluorescence microscope image data of historical data;

[0014] S20, AI fluorescence image multi-layer fusion:

[0015] Extract the features of each image in the multi-layer fluorescence microscope image data and merge the features;

[0016] The feature maps from different fluorescence channels are superimposed or weighted summed to obtain the integrated comprehensive feature expression;

[0017] Using spatial pyramid pooling techniques of different scales, feature maps of different resolutions are aggregated and integrated to capture multi-scale spatial information;

[0018] Multi-scale feature fusion is achieved by combining feature representations from different levels, scales or fields of view. The fusion strategy is used to improve the comprehensive understanding and description capabilities of the complex structure of the sample, thereby fusing the images of all clear parts together to generate an image with clear layers.

[0019] Furthermore, in step S10, the training steps of the AI ​​fluorescence image multi-layer model include:

[0020] Collect historical multi-layer fluorescence microscopy image data to ensure that the dataset covers images at different depths and planes, including samples labeled with multiple fluorescent probes;

[0021] Preprocess the image data;

[0022] Select a deep learning model suitable for processing multi-level information and train the model using fluorescent slide image data with multi-layer structures;

[0023] Evaluate the trained model, perform performance testing using a validation dataset, and optimize the model based on the evaluation results to improve spatial resolution and multiple labeling accuracy;

[0024] The trained model is applied to the actual fluorescence scanning logic to extract effective information at different levels.

[0025] Furthermore, a convolutional neural network was selected and the model was trained using fluorescent slide image data with a multi-layer structure.

[0026] Furthermore, preprocessing includes contrast enhancement and normalization.

[0027] Furthermore, in step S20, after extracting the features of each image in the multi-layer fluorescence microscope image data, wavelet transform or convolutional neural network is used to merge the features.

[0028] Furthermore, in step S20, the information labeled by different fluorescent probes is comprehensively analyzed through the integrated comprehensive feature expression.

[0029] Furthermore, the calculation formula for the number of photos N, the offset S and the starting height C is:

[0030] N = W / (f / 2);

[0031] S=W / N;

[0032] C=HN / 2*S.

[0033] In a second aspect, the present invention provides a multi-layer fusion process based on a fluorescent glass slide and an AI multi-layer fusion processing device, comprising:

[0034] The fluorescence scanning module is used to input the camera height H of the clear field of view, the tissue thickness W and the depth of field f of the camera lens, so as to calculate the number of photos N, the offset S and the starting height C; the Z axis of the camera is moved to the starting height C, and a photo is triggered every distance of the offset S until N photos are taken, and then the photo is moved to the next field of view and repeated. After the current slide is photographed, multi-layer fluorescence microscope image data is obtained;

[0035] Among them, the multi-layer fluorescence microscope image data covers different depths and planes; the calculation formulas for the number of photos N, the offset S, and the starting height C are:

[0036] N=W / (f / 2); S=W / N; C=HN / 2*S;

[0037] The AI ​​image processing module extracts effective information at different levels of multi-layer image data through an AI fluorescence image multi-layer model, thereby extracting a clear part of the image from each layer. The AI ​​fluorescence image multi-layer model is trained by collecting multi-layer fluorescence microscope image data from historical data;

[0038] AI fluorescence image multi-layer fusion module, which extracts the features of each image in the multi-layer fluorescence microscope image data and merges the features;

[0039] The feature maps from different fluorescence channels are superimposed or weighted summed to obtain the integrated comprehensive feature expression;

[0040] Utilize multi-scale convolution kernels and pooling operations to extract features from multiple levels, and then cascade or process these features in parallel to enhance the understanding of cell structure and tissue distribution;

[0041] Multi-scale feature fusion is achieved by combining feature representations from different levels, scales or fields of view. The fusion strategy is used to improve the comprehensive understanding and description capabilities of the complex structure of the sample, thereby fusing the images of all clear parts together to generate an image with clear layers.

[0042] In a third aspect, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned method based on the fluorescent glass slide multi-layer fusion process and AI multi-layer fusion processing.

[0043] In a fourth aspect, the present invention provides a readable storage medium storing a computer program, wherein the computer program includes a program code for controlling a process to execute the process, and the process includes the method based on the above-mentioned fluorescent glass slide multi-layer fusion process and AI multi-layer fusion processing.

[0044] The main contributions and innovations of the present invention are as follows:

[0045] 1. Solved the problem of blurred cells at different levels caused by tissue thickness exceeding the camera's depth of field: The appropriate number of photos, offset and starting height are determined through automatic scanning logic to ensure that cells at each level can be clearly imaged.

[0046] 2. Effectively extract effective information at different levels: Use AI image processing technology to extract clear parts from each layer of the image, improving the usability and analysis value of the image.

[0047] 3. Improved the quality of image fusion: Through multiple fusion strategies such as feature-level fusion, channel-level fusion, spatial pyramid pooling, and attention mechanism, information at different levels is effectively integrated to obtain clearer and more accurate cell and tissue structure images.

[0048] 4. Enhanced ability to understand complex sample structures: The use of multi-scale convolution kernels and pooling operations enhances the ability to understand cell structure and tissue distribution, which helps to more comprehensively understand the complex structure of samples.

[0049] 5. Improved work efficiency and accuracy: Automated processing reduces the need for manual operations and improves processing speed and image analysis accuracy.

[0050] 6. Optimized spatial resolution and accuracy of multiple labeling: Through the training and optimization of the AI ​​model, the spatial resolution and accuracy of multiple labeling are improved, which helps to more accurately identify and analyze cell characteristics.

[0051] 7. Improved the overall efficiency of image processing: Through integrated fluorescence scanning, image processing and fusion technology, an efficient image processing process is achieved, which speeds up research and diagnosis.

[0052] In summary, the present invention significantly improves the quality and analysis efficiency of multi-layer fluorescent slide microscopic images through automated scanning logic, advanced AI image processing technology and multi-layer fusion methods, providing strong support for biological research, medical diagnosis and other fields.

[0053] The details of one or more embodiments of the invention are set forth in the following drawings and description so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0055] Figure 1 It is a process of a method based on a fluorescent glass multi-layer fusion process and an AI multi-layer fusion processing according to an embodiment of the present invention;

[0056] Figure 2 is a layer 1 image captured according to an embodiment of the present invention;

[0057] Figure 3 is a layer 2 image captured according to an embodiment of the present invention;

[0058] Figure 4 is a layer 3 image taken according to an embodiment of the present invention;

[0059] Figure 5 is a 4th layer image taken according to an embodiment of the present invention;

[0060] Figure 6is a layer 5 image taken according to an embodiment of the present invention;

[0061] Figure 7 is a 6th layer image taken according to an embodiment of the present invention;

[0062] Figure 8 is a first layer image extracted according to an embodiment of the present invention;

[0063] Fig. 9 is a second layer image extracted according to an embodiment of the present invention;

[0064] Fig.10 is a layer 3 image extracted according to an embodiment of the present invention;

[0065] Fig.11 is a 4th layer image extracted according to an embodiment of the present invention;

[0066] Fig.12 is the fifth layer image extracted according to an embodiment of the present invention;

[0067] Fig.13 is the 6th layer image extracted according to the embodiment of the present invention;

[0068] Fig.14 is an image fused according to an embodiment of the present invention;

[0069] Fig.15 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0070] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0071] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0072] Since the lens has a depth of field, the image is clear within a certain height range. However, the product on the fluorescent slide also has thickness. When the thickness of the product is greater than the depth of field of the lens, the image will be blurred in some areas. Therefore, due to the different thickness of the product, the depth of field of the camera in the existing technology cannot cover cells at different levels, resulting in blurry cells in some layers. The traditional multi-layer fusion has recognition errors, resulting in deviations between the fused image and the actual image.

[0073] Based on this, the present invention solves the problems existing in the prior art based on the fluorescent glass slide multi-layer fusion process and AI multi-layer fusion processing.

[0074] Embodiment 1

[0075] The present invention aims to propose a method based on the fluorescent glass slide multi-layer fusion process and AI multi-layer fusion processing. Through automated scanning logic, advanced AI image processing technology and multi-layer fusion method, the quality and analysis efficiency of multi-layer fluorescent glass slide microscopic images are significantly improved, providing strong support for biological research, medical diagnosis and other fields.

[0076] Specifically, the embodiment of the present invention provides a method based on fluorescent glass multi-layer fusion and AI multi-layer fusion processing, specifically, referring to Figure 1 , the method comprising:

[0077] Scanning process 1.1. Offset calculation S, number of photos N, calculation of starting height C: input the height of clear field of view H, input tissue thickness W, input lens depth of field f, the calculation formula is as follows:

[0078] N=W / (f / 2), S=W / N, C=HN / 2*S.

[0079] 1.2. Taking photos at different layers: Move the Z axis to the starting height C, trigger taking photos every S distance, and move to the next field of view after taking N photos.

[0080] like Figure 2-7 As shown, they are images from the first layer to the sixth layer respectively.

[0081] AI image processing to obtain effective information: AI image processing is used to obtain effective information in the image. The training steps of the AI ​​fluorescence image multi-layer training model are as follows: 2.1 Data acquisition and preparation: Collect high-quality multi-layer fluorescence microscope image data covering different depths and planes. Ensure that the image data contains samples labeled with multiple fluorescent probes so that the training model can learn multiple labeling information.

[0082] 2.2 Data preprocessing: Preprocess the image data to enhance contrast, normalize, etc. to optimize the training effect. The preprocessing here is a routine operation for general model training, and its specific steps and principles will not be repeated here.

[0083] 2.3 Model selection and training: Select a suitable deep learning model, convolutional neural network (CNN), to process multi-level information. The model is trained using fluorescent slide image data with multi-layer structures to learn the cell structure or tissue distribution at different depths and planes.

[0084] 2.4 Evaluate the trained model and use the validation dataset for performance testing. Optimize the model based on the evaluation results to improve spatial resolution and multi-label accuracy.

[0085] 2.5 Apply the trained model to the actual fluorescence scanning logic to extract effective information at different levels.

[0086] That is, after taking photos in each field of view, use the trained model to extract information. After extracting information from all layers, switch to the next field of view to take photos. After taking photos, call the model again to extract, and so on.

[0087] 3. Multi-layer fusion of fluorescence images

[0088] 3.1 Feature-level fusion: Extract features from each image using wavelet transform or convolutional neural network (CNN) techniques, and then merge these features. For example:

[0089] 3.1.1 Feature extraction

[0090] Use convolutional neural network (CNN) or other appropriate feature extraction algorithms to extract features from each image. CNN can extract local and global features in images through a series of convolutional layers, activation functions, pooling layers, etc.

[0091] 3.1.2 Feature Merging

[0092] Wavelet Transform:

[0093] Apply wavelet transform to perform multi-resolution analysis on the image to obtain feature representations at different scales. Select key coefficients from the results of wavelet transform as feature representations. Merge wavelet coefficients at different layers, which can be simple weighted average, maximum selection or other strategies.

[0094] Convolutional Neural Networks (CNN):

[0095] Use the output of the intermediate layer of CNN as the feature representation. CNN feature maps of different layers can be cascaded or concatenated. Features can also be merged through fully connected layers or other fusion layers.

[0096] 3.1.3 Fusion Strategy

[0097] Choose the appropriate fusion strategy based on the specific situation:

[0098] Weighted average: Assign different weights to the features of each layer and then sum them up.

[0099] Maximum Selection: Select the maximum value at each position.

[0100] Concatenation / concatenation: concatenate feature maps from different layers together to form a larger feature vector.

[0101] Attention mechanism: Dynamically adjust the importance of different features through attention weights to achieve intelligent fusion.

[0102] 3.1.4 Post-processing

[0103] The fused features are further processed, such as normalization and dimensionality reduction, to improve the efficiency and effect of subsequent processing.

[0104] Through the above steps, features can be effectively extracted from images of different layers, and feature fusion can be achieved by using wavelet transform or convolutional neural network. This fusion method can better preserve the details of different layers and improve the quality and information content of the final image.

[0105] 3.2 Channel-level fusion: Superimpose or weighted sum feature maps from different fluorescence channels to obtain integrated comprehensive feature expression. Comprehensively analyze the information marked by different fluorescent probes. For example:

[0106] 3.2.1 Feature map superposition

[0107] For feature maps of different fluorescence channels, pixel-level superposition can be performed directly. Assume that there are two feature maps F1 and F2, which have the same size H*W*C (where H and W are the height and width, and C is the number of channels), and directly add these two feature maps to get a new feature map F stacked :

[0108]

[0109] Where x and y are pixel coordinates and c is the channel index.

[0110] 3.2.2 Weighted Sum

[0111] If you need to consider the different importance of different channels, you can use the weighted summation method. Assume that each feature map F i There is a weight w i , then the comprehensive feature map F weighted for:

[0112]

[0113] Here, n is the number of feature maps.

[0114] 3.2.3 Weight calculation

[0115] Weight w i This can be determined in a number of ways:

[0116] Manually specify: Weights are manually assigned to each channel based on experience or prior knowledge.

[0117] Attention mechanism: Dynamically adjust the weight of each channel by training an attention model. The attention model can be another small CNN or a simple fully connected layer that dynamically assigns weights based on the content of the feature map.

[0118] Adaptive weights: Use adaptive weighting methods, such as automatically calculating weights based on the correlation or difference between feature maps.

[0119] 3.2.4 Post-processing

[0120] The fused feature maps are further processed, such as normalization and dimensionality reduction, to improve the efficiency and effect of subsequent processing.

[0121] Through the above steps, features can be effectively extracted from images of different fluorescence channels, and feature maps can be superimposed or weighted summed to achieve channel-level fusion. This fusion method can better preserve the information of different fluorescent probe labels and improve the quality and information content of the final image.

[0122] 3.3 Spatial Pyramid Pooling: Use spatial pyramid pooling techniques of different scales to aggregate and integrate feature maps of different resolutions to capture multi-scale spatial information. For example:

[0123] 3.3.1 Pyramid level division

[0124] The extracted feature map is divided into multiple scale levels to form a pyramid structure. Each pyramid level corresponds to a different pooling window size. For example, pooling windows of different sizes such as 1X1, 2X2, and 4X4 can be used.

[0125] 3.3.2 Pooling Operation

[0126] Pooling is performed on the feature map of each pyramid level. The pooling operation can be maximum pooling or average pooling, which is used to reduce the resolution of the feature map while maintaining the invariance of the features. The result of each pooling operation will produce a feature vector of fixed size.

[0127] 3.3.3 Feature Merging

[0128] The feature vectors generated by pooling operations at different levels are merged to form a unified fixed-size feature vector. The merging of feature vectors can be done by simple concatenation or cascading.

[0129] 3.3.4 Post-processing

[0130] The merged feature vectors are further processed, such as normalization and dimensionality reduction, to improve the efficiency and effect of subsequent processing.

[0131] Through the above steps, multi-scale spatial information can be effectively extracted from feature maps of different resolutions and summarized and integrated through spatial pyramid pooling technology. This fusion method can better preserve information of different scales and improve the quality and information content of the final image.

[0132] 3.4 Multi-scale feature fusion: Combine feature representations from different levels, scales or fields of view, and use appropriate fusion strategies to improve the comprehensive understanding and description capabilities of the complex structure of the sample.

[0133] Among them, this process emphasizes the ability to use different receptive fields to capture various types of information in the image. Different levels: refers to the different layers in the network; different scales: generally refers to using convolution kernels of different sizes to process input data; different fields of view: usually refers to observing data from different perspectives or positions.

[0134] In this way, the overall workflow is: get the starting height C of the Z axis movement, trigger the camera every S distance, and get an image corresponding to the current height, such as Figure 2 (First floor) - Figure 7 (Sixth layer), extract the clear part of the image from the current image, such as Figure 8 (First floor) - Fig.13 (Sixth layer), loop N images at a time, after all the loops are completed, merge the clear parts of all images together to finally generate a composite image with clear layers. Fig.14 (Fusion image), this image covers images at different heights, allowing it to display and obtain more effective information.

[0135] It can be seen that the present invention realizes digital pathology fluorescence multi-layer processing, and particularly relates to a new processing flow and logic of a digital pathology fluorescence device, which uses a deep learning model, a convolutional neural network (CNN), to extract high-level features from each image. These features can capture more complex image patterns, such as texture, shape, and semantic information. The AI ​​model is able to learn the relationship and trade-off between different images, so as to automatically adjust the weights or operations during the fusion process to achieve the best fusion effect. This adaptability can improve the quality and accuracy of the fusion results. AI can be applied to pixel-level operations such as denoising, super-resolution, and color correction to improve the visual quality and information clarity of the image. Efficient deep learning models can process a large amount of image data in a shorter time, and are suitable for applications that require real-time processing and response. The use of image fusion based on the AI ​​large model method can not only improve the fusion quality and effect, but also realize automatic and efficient processing, which is suitable for a variety of complex application requirements.

[0136] Embodiment 2

[0137] Based on the same concept, the present invention also proposes a multi-layer fusion process based on fluorescent glass and an AI multi-layer fusion processing device, comprising:

[0138] The fluorescence scanning module is used to input the camera height H of the clear field of view, the tissue thickness W and the depth of field f of the camera lens, so as to calculate the number of photos N, the offset S and the starting height C; the Z axis of the camera is moved to the starting height C, and a photo is triggered every distance of the offset S until N photos are taken, and then the photo is moved to the next field of view and repeated. After the current slide is photographed, multi-layer fluorescence microscope image data is obtained;

[0139] Among them, the multi-layer fluorescence microscope image data covers different depths and planes; the calculation formulas for the number of photos N, the offset S, and the starting height C are:

[0140] N=W / (f / 2); S=W / N; C=HN / 2*S;

[0141] The AI ​​image processing module extracts effective information at different levels of multi-layer image data through an AI fluorescence image multi-layer model, thereby extracting a clear part of the image from each layer. The AI ​​fluorescence image multi-layer model is trained by collecting multi-layer fluorescence microscope image data from historical data;

[0142] AI fluorescence image multi-layer fusion module, which extracts the features of each image in the multi-layer fluorescence microscope image data and merges the features;

[0143] The feature maps from different fluorescence channels are superimposed or weighted summed to obtain the integrated comprehensive feature expression;

[0144] Using spatial pyramid pooling techniques of different scales, feature maps of different resolutions are aggregated and integrated to capture multi-scale spatial information;

[0145] Feature-level fusion based on the attention mechanism dynamically adjusts the weight of features according to the importance of different channels or spatial positions to enhance the representation ability of specific information;

[0146] Utilize multi-scale convolution kernels and pooling operations to extract features from multiple levels, and then cascade or process these features in parallel to enhance the understanding of cell structure and tissue distribution;

[0147] Multi-scale feature fusion is achieved by combining feature representations from different levels, scales or fields of view. The fusion strategy is used to improve the comprehensive understanding and description capabilities of the complex structure of the sample, thereby fusing the images of all clear parts together to generate an image with clear layers.

[0148] Embodiment 3

[0149] This embodiment also provides an electronic device, referring to Fig.15 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0150] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0151] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0152] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0153] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any of the methods based on fluorescent glass slide multi-layer fusion and AI multi-layer fusion processing in the above embodiments.

[0154] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0155] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0156] Input / output devices 408 are used to input or output information.

[0157] Embodiment 4

[0158] This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process. The process includes a method based on a fluorescent glass slide multi-layer fusion process and AI multi-layer fusion processing according to the first embodiment.

[0159] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0160] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0161] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 1 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within the processor, magnetic media such as hard disk or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0162] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A method based on fluorescent glass multi-layer fusion and AI multi-layer fusion processing, characterized in that: The following steps are involved: S00, fluorescence scanning logic: Determine the camera height H, tissue thickness W, and camera lens depth of field f for a clear field of view, and use this to calculate the number of photos N, offset S, and starting height C; Move the Z axis of the camera to the starting height C, trigger a photo every offset distance S, and move to the next field of view to repeat the photo shooting after taking N pictures. After completing the photo shooting of the current slide, obtain the multi-layer fluorescence microscope image data; Among them, multi-layer fluorescence microscopy image data covers different depths and planes; S10, AI image processing to obtain effective information: The effective information of different layers of multi-layer fluorescence microscope image data is extracted through the AI ​​fluorescence image multi-layer model, so as to extract the clear part of the image from each layer of the image. The AI ​​fluorescence image multi-layer model is trained by collecting multi-layer fluorescence microscope image data of historical data; S20, AI fluorescence image multi-layer fusion: Extract the features of each clear part of the image in the multi-layer fluorescence microscope image data and merge the features; The feature maps from different fluorescence channels are superimposed or weighted summed to obtain the integrated comprehensive feature expression; Using spatial pyramid pooling techniques of different scales, feature maps of different resolutions are aggregated and integrated to capture multi-scale spatial information; Multi-scale feature fusion is achieved by combining feature representations from different levels, scales or fields of view. The fusion strategy is used to improve the comprehensive understanding and description capabilities of the complex structure of the sample, thereby fusing the images of all clear parts together to generate an image with clear layers.

2. The method based on fluorescent glass multi-layer fusion and AI multi-layer fusion processing as claimed in claim 1, characterized in that: In step S10, the training steps of the AI ​​fluorescence image multi-layer model include: Collect historical multi-layer fluorescence microscopy image data to ensure that the dataset covers images at different depths and planes, including samples labeled with multiple fluorescent probes; Preprocess the image data; Select a deep learning model suitable for processing multi-level information and train the model using fluorescent slide image data with multi-layer structures; Evaluate the trained model, perform performance testing using a validation dataset, and optimize the model based on the evaluation results to improve spatial resolution and multiple labeling accuracy; The trained model is applied to the actual fluorescence scanning logic to extract effective information at different levels.

3. The method based on fluorescent glass multi-layer fusion and AI multi-layer fusion processing as claimed in claim 2, characterized in that: A convolutional neural network was selected and the model was trained using fluorescent slide image data with a multi-layer structure.

4. The method based on fluorescent glass multi-layer fusion and AI multi-layer fusion processing as claimed in claim 2, characterized in that: Preprocessing includes contrast enhancement and normalization.

5. The method based on fluorescent glass multi-layer fusion and AI multi-layer fusion processing as claimed in claim 1, characterized in that: In step S20, after extracting the features of each image in the multi-layer fluorescence microscope image data, wavelet transform or convolutional neural network is used to merge the features.

6. The method based on fluorescent glass multi-layer fusion and AI multi-layer fusion processing as claimed in claim 1, characterized in that: In step S20, the information labeled by different fluorescent probes is comprehensively analyzed through the integrated comprehensive feature expression.

7. The method based on fluorescent glass multi-layer fusion and AI multi-layer fusion processing according to any one of claims 1 to 6, characterized in that: The calculation formula for the number of photos N, offset S and starting height C is: N = W / (f / 2); S=W / N; C=HN / 2*S.

8. A multi-layer fusion process based on fluorescent glass and an AI multi-layer fusion processing device, characterized in that: include: The fluorescence scanning module is used to input the camera height H of the clear field of view, the tissue thickness W and the depth of field f of the camera lens, so as to calculate the number of photos N, the offset S and the starting height C; the Z axis of the camera is moved to the starting height C, and a photo is triggered every distance of the offset S until N photos are taken, and then the photo is moved to the next field of view to repeat the photo shooting, and after the current slide is shot, the multi-layer fluorescence microscope image data is obtained; Among them, the multi-layer fluorescence microscope image data covers different depths and planes; the calculation formulas for the number of photos N, the offset S, and the starting height C are: N=W / (f / 2); S=W / N; C=HN / 2*S; The AI ​​image processing module extracts effective information at different levels of multi-layer fluorescence microscope image data through an AI fluorescence image multi-layer model, thereby extracting a clear part of the image from each layer of the image. The AI ​​fluorescence image multi-layer model is trained by collecting multi-layer fluorescence microscope image data from historical data; AI fluorescence image multi-layer fusion module extracts the features of each clear part of the multi-layer fluorescence microscope image data and merges the features; The feature maps from different fluorescence channels are superimposed or weighted summed to obtain the integrated comprehensive feature expression; Using spatial pyramid pooling techniques of different scales, feature maps of different resolutions are aggregated and integrated to capture multi-scale spatial information; Multi-scale feature fusion is achieved by combining feature representations from different levels, scales or fields of view. The fusion strategy is used to improve the comprehensive understanding and description capabilities of the complex structure of the sample, thereby fusing the images of all clear parts together to generate an image with clear layers.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method based on fluorescent glass slide multi-layer fusion and AI multi-layer fusion processing according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute the process, and the process includes the method based on fluorescent glass multi-layer fusion and AI multi-layer fusion processing according to any one of claims 1 to 7.

Images