Image processing device and method based on fluorescence microscope acquisition

Abstract

The invention discloses an image processing device and method based on fluorescence microscope collection, and the method comprises the steps: obtaining a to-be-processed image collected by a fluorescence microscope, and generating a collection environment distribution feature vector of the to-be-processed image; inputting the to-be-processed image and the collection environment distribution feature vector thereof into a pre-trained multi-class image processing model, and outputting a target image corresponding to the to-be-processed image; wherein the pre-trained multi-class image processing model comprises an image attention distribution module, an image distributed processing module and an image feature fusion output module, and the image distributed processing module comprises a plurality of classes of image processing models. Therefore, differential image processing is carried out on the to-be-processed images under different acquisition conditions, and the adaptability and the processing effect of the image processing model are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and bioengineering technology, and in particular to an image processing device and method based on fluorescence microscope acquisition. Background Art

[0002] Deoxyribonucleic acid (DNA) microarray, also known as gene chip, is one of the applications of microarray technology, which is used for high-throughput qualitative or quantitative measurement of nucleic acids present in organisms. Fluorescence microscopy is the main method for gene chip data acquisition. However, the image quality of the gene chip images currently collected is low, and the main reasons include: The type and dose of fluorescence need to be strictly controlled to maintain the biological activity of DNA, but the fluorescence interference excited by the adjacent area on the same focal plane will reduce the contrast of the collected image; the errors accompanying the optical collection and imaging process, such as the working error of electronic devices and the loss of signal conversion, lead to image noise; the defects of the LED light source in the upright fluorescence microscope and its susceptibility to axial and lateral interference lead to uneven illumination and reduce the proportion of effective information. At present, although the laser confocal system can obtain high-quality and high-resolution gene chip images, the equipment cost is high.

[0003] The Chinese invention patent with patent application number 202310631275.4 discloses a model training method and an image processing method and device, which uses gene chip images collected through a fluorescence microscope as sample images, and uses the loss value determined by the signal-to-noise ratio of the sample image and the processed image output by the model to train the image processing model, and applies the trained image processing model in practice to obtain high-quality, high-resolution gene chip images.

[0004] However, in actual application scenarios, due to the complex differences in the acquisition conditions of fluorescence microscopy (fluorescent dye type, light intensity, experimental conditions, etc.), different types of sample images may have different characteristics and processing requirements. A fixed and unified image processing model is difficult to adapt to sample images of all acquisition conditions. Under the condition of focusing on global training parameter optimization of all acquisition conditions, it may perform poorly for certain local single acquisition conditions and lacks targeted image processing processes. Summary of the invention

[0005] The present application provides an image processing method based on fluorescence microscopy acquisition, thereby achieving differentiated image processing for images to be processed under different acquisition conditions, thereby improving the adaptability and processing effect of the image processing model.

[0006] The present application provides an image processing method based on fluorescence microscopy acquisition, comprising: S101, obtaining an image to be processed acquired by a fluorescence microscope, and generating an acquisition environment distribution feature vector of the image to be processed; S102, inputting the image to be processed and its acquisition environment distribution feature vector into a pre-trained multi-class image processing model, and outputting a target image corresponding to the image to be processed; The pre-trained multi-class image processing models include: an image attention distribution module, an image distributed processing module, and an image feature fusion output module. The image distributed processing module includes several types of image processing models. The multi-class image processing models are used to: A1. The received image to be processed and its acquisition environment distribution feature vector are transmitted to the image attention distribution module, a distributed processing value sequence corresponding to the sample image is generated, and transmitted to the image distributed processing module; A2, the image distributed processing module inputs the sample images into the corresponding class image processing models according to the received distributed processing value sequence, outputs several classes of target images, and transmits them to the image feature fusion output module; A3. The image feature fusion output module performs weighted fusion on several types of target images received according to the distributed processing value sequence and outputs the final target image.

[0007] Preferably, the image distributed processing module includes several types of image processing models, each type of image processing model is assigned a unique image class label, and the image class label is determined according to the center point of the acquisition environment distribution feature vector of the training image set in the corresponding type of image processing model.

[0008] Preferably, the training process of the several types of image processing models specifically includes: C1. Collect a large number of sample images collected by fluorescence microscope and generate the collection environment distribution feature vector of the sample images; C2. Input all collected acquisition environment distribution feature vectors into a preset clustering algorithm to obtain k clusters, and form a training image set with sample images corresponding to all acquisition environment distribution feature vectors in each cluster, that is, each cluster corresponds to a training image set; C3, constructing a k-type image processing model having a model structure consisting of a plurality of feature extraction layers and a plurality of convolutional transposition layers as the class image processing model to be trained; C4, based on each training image set, input the sample image therein into the corresponding class image processing model to be trained, and output the class target image; C5, based on the class target image output by each class image processing model and the sample image input, the algorithm is determined by using the preset model optimization index to generate the target evaluation index; C6. Use the target evaluation index to train the image processing model. Repeat steps C4 to C6 in a loop, continuously iterate the training, and gradually reach the target state of the target evaluation index.

[0009] Preferably, the image attention distribution module is specifically used for: B1. Obtain image class labels of all image processing models in the image distributed processing module; B2. Calculate the similarity values ​​of the acquisition environment distribution feature vector of the image to be processed and all image class labels respectively; B3. All the calculated similarity values ​​are sequentially organized into a distributed processing value sequence, and each similarity value in the distributed processing value sequence corresponds to a corresponding class image processing model.

[0010] Preferably, the preset model optimization index determination algorithm specifically includes: S201, the target image and the input sample image are divided into n sub-image blocks, and the feature vector of each sub-image block is calculated. The sub-image block feature vector of the sample image is , the sub-image block feature vector of the target image is and One to one correspondence; S202, calculating the mutual information between the sample image and the corresponding sub-image block in the class target image on the feature vector, and calculating the visual information fidelity of each sub-image block in the class target image relative to the sample image according to the following formula:

[0011] in, For visual information fidelity, is the feature vector of the i-th sub-image block of the sample image, is the feature vector of the i-th sub-image block of the target image, n is the number of sub-image blocks divided into the sample image and the target image, for and The mutual information between for Mutual information with itself; S203, based on each sub-image block in the class target image, calculating the pixel difference between it and the corresponding sub-image block in the sample image; S204, determining the ratio of the visual information fidelity to the pixel difference as the covariance direction value, and obtaining the target evaluation index by combining the visual information fidelity.

[0012] Preferably, the target evaluation index is calculated according to the following formula:

[0013]

[0014] Among them, L is the target evaluation index, n is the number of sub-image blocks divided into target images, is the image quality index of the i-th sub-image block in the target image, is the visual information fidelity of the i-th sub-image block in the target image, is the covariance direction value of the i-th sub-image block in the target image, is a pre-set weight parameter used to adjust the influence of the covariance direction value on the image quality index. is the influence degree of the image quality index of the i-th sub-image block on the target evaluation index.

[0015] Preferably, in S201, the sub-image block division method is generated according to a preset adaptive segmentation mechanism, and the adaptive segmentation mechanism specifically includes: S301, identifying connected domains on the sample image to obtain a number of connected domains; S302, based on each connected domain in the sample image, taking the pixel coordinates of the central pixel point in the sample image as the position coordinates of the connected domain, respectively calculating the distance values ​​between any two connected domains; S303, based on the distance values ​​between the two connected domains, a preset clustering algorithm is used to obtain a plurality of clusters, each cluster including at least one connected domain, and the maximum circumscribed rectangle of each cluster is used as a key area; S304, based on all the key areas identified in the sample image, divide the sample image into a number of sub-image blocks using a non-uniform division strategy, ensuring that each sub-image block includes at most one key area; S305 , based on the positions of all sub-image blocks of the sample image, the sub-image blocks are divided in the class target image to be consistent with the sample image.

[0016] Preferably, the influence degree value of the image quality index of the i-th sub-image block on the target evaluation index is calculated according to the following formula:

[0017] in, is the influence of the image quality index of the i-th sub-image block on the target evaluation index, is the area value of the i-th sub-image block, S is the area of ​​the sample image, is the number of connected domains in the i-th sub-image block, N is the total number of connected domains in the sample image, and is a pre-set weight factor used to determine the importance of the area value and the number of connected domains to the impact value. .

[0018] Preferably, after S102, the method further includes: S103, using the adaptive segmentation mechanism to divide the image to be processed into a number of sub-image blocks, and assigning a unique position label to each sub-image block; S104, based on the position tag of the sub-image block of the image to be processed, determining the sub-image block with the same position tag in the target image, and calculating the maximum area value of the connected domain in the sub-image block with the position tag in the image to be processed and the target image, respectively, to obtain a first area value and a second area value; S105, based on each sub-image block in the target image, calculating its image quality index and maximum area domain difference, and determining whether there is an abnormal sub-image block; S106, if it exists, segment the abnormal sub-image block from the image to be processed to obtain a sub-sample, execute step S102, input it into the multi-class image processing model, output the sub-target image corresponding to the sub-sample, and replace the sub-image block at the position label in the target image according to its position label, obtain a new target image, and replace the original target image.

[0019] The present application also provides an image processing device based on fluorescence microscope acquisition, comprising: an acquisition module, an image processing module, the image processing module comprising multiple types of image processing models, the multiple types of image processing models comprising: an image attention distribution module, an image distributed processing module, and an image feature fusion output module; the acquisition module is used to: obtain an image to be processed acquired by a fluorescence microscope, generate a collection environment distribution feature vector of the image to be processed, and input it into the image processing module; the image processing module is used to: input the image to be processed and its collection environment distribution feature vector into the pre-trained multiple types of image processing model, and output a target image corresponding to the image to be processed; The multi-class image processing model is used for: A1, transmitting the received image to be processed and its acquisition environment distribution feature vector to the image attention distribution module, generating a distributed processing value sequence corresponding to the sample image, and transmitting it to the image distributed processing module; A2, the image distributed processing module inputs the sample image into the corresponding class image processing model according to the received distributed processing value sequence, outputs several classes of target images, and transmits them to the image feature fusion output module; A3, the image feature fusion output module performs weighted fusion on the received several classes of target images according to the distributed processing value sequence, and outputs the final target image.

[0020] One or more technical solutions provided in this application have at least the following technical effects or advantages: By introducing the acquisition environment distribution feature vector and multi-class image processing model, differentiated image processing is realized for the images to be processed under different acquisition conditions based on the distributed processing value sequence, which improves the adaptability and effect of image processing; solves the problem that a fixed and unified image processing model is difficult to adapt to sample images under all acquisition conditions, and improves the flexibility and accuracy of image processing; The image quality index combines two important indicators, VIF and pixel difference, to more comprehensively reflect the quality of image processing. The image quality index introduces the covariation direction value to clearly quantify the change direction between the visual information fidelity and the pixel difference, making the evaluation result more instructive. By adjusting the weight parameter α, the evaluation focus of CQI can be customized according to specific application scenarios and needs. Based on the joint evaluation of visual information fidelity and covariation direction value, the contingency of single indicator judgment can be avoided, and the change direction can be used to constrain the visual information fidelity, making the evaluation result more reliable. Through the adaptive segmentation mechanism, the key information features in the fluorescence microscope images can be captured more accurately. During the image processing model training process, the key areas of the sample images are differentially analyzed and corresponded to the target images. The target evaluation indicators can be more accurately and finely analyzed from local to global, reflecting the processing effect of the model on the image, and improving the flexibility and accuracy of image processing, especially for images containing complex biological structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of an image processing method based on fluorescence microscope acquisition according to an embodiment of the present invention; Figure 2 4 is a structural block diagram of an image processing device based on fluorescence microscope acquisition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] To facilitate the understanding of the present invention, the present application will be described more comprehensively below with reference to the relevant drawings; the drawings show preferred embodiments of the present invention, but the present invention can be implemented in many different forms and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to enable a more thorough and comprehensive understanding of the disclosed content of the present invention.

[0023] It should be noted that the terms “vertical”, “horizontal”, “up”, “down”, “left”, “right” and similar expressions used in this document are only for illustrative purposes and do not represent the only implementation method.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs; the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the term "and / or" used herein includes any and all combinations of one or more related listed items.

[0025] Embodiment 1: Figure 1 It is a flowchart of an image processing method based on fluorescence microscope acquisition according to an embodiment of the present invention.

[0026] like Figure 1 As shown, an image processing method based on fluorescence microscopy acquisition includes the following steps: S101, obtaining an image to be processed acquired by a fluorescence microscope, and generating an acquisition environment distribution feature vector of the image to be processed.

[0027] The collection environment distribution feature vector includes lighting conditions, fluorescent dye type, and ambient temperature.

[0028] In fluorescence microscopy, the wavelength, intensity, and indoor lighting conditions of the excitation light will have a direct impact on the excitation effect of fluorescent dyes. Too strong light may lead to fluorescence quenching, while too weak light may not effectively excite the fluorescent dye, affecting image quality. The choice of fluorescent dyes is crucial for specific biological samples or cell structures. Different dyes may have different affinities for specific cell components. The use of inappropriate dyes may lead to weak signals, high background noise, or poor specificity. The ambient temperature has a certain impact on the performance of the fluorescence microscope and the stability of the sample. The appropriate ambient temperature can maintain the stability of the microscope and the activity of the sample. Too high or too low a temperature may cause the microscope performance to deteriorate or the sample to deform, thereby affecting the image quality.

[0029] Specifically, in the field of fluorescence microscopy, due to the complex differences in acquisition conditions, different types of sample images may have different characteristics and processing requirements. Sample images acquired by a fluorescence microscope are collected, and the corresponding acquisition environment distribution feature vectors are generated according to the acquisition conditions.

[0030] S102, inputting the image to be processed and its acquisition environment distribution feature vector into a pre-trained multi-class image processing model, and outputting a target image corresponding to the image to be processed.

[0031] Therefore, with the support of multiple types of image processing models, it is possible to output the most optimized processing results for different types of images to be processed and acquisition conditions, thereby improving the usability and accuracy of the images.

[0032] In some embodiments, the pre-trained multi-category image processing model includes: an image attention distribution module, an image distributed processing module, and an image feature fusion output module, and connections are established between the modules in sequence.

[0033] Specifically, multi-class image processing models are used to: A1. The received image to be processed and its acquisition environment distribution feature vector are transmitted to the image attention distribution module, a distributed processing value sequence corresponding to the sample image is generated, and transmitted to the image distributed processing module; Among them, the image attention distribution module is specifically used for: B1. Obtain image class labels of all image processing models in the image distributed processing module; B2. Calculate the similarity values ​​of the acquisition environment distribution feature vector of the image to be processed and all image class labels respectively (the calculation can be performed using the Euclidean distance algorithm, which will not be described in detail in the present invention); B3. All the calculated similarity values ​​are sequentially organized into a distributed processing value sequence, and each similarity value in the distributed processing value sequence corresponds to a corresponding class image processing model.

[0034] A2, the image distributed processing module inputs the sample images into the corresponding class image processing models according to the received distributed processing value sequence, outputs several classes of target images, and transmits them to the image feature fusion output module; The image distributed processing module includes several types of image processing models, each type of image processing model is assigned a unique image class label, and the image class label is determined according to the center point of the acquisition environment distribution feature vector of the training image set in the corresponding type of image processing model; Among them, each image processing model includes a feature extraction layer and a transposed convolution layer. Each feature extraction layer is used to perform convolution processing on the image features output by the previous network layer to obtain the convolved image features and perform feature filling to restore the image feature size; each transposed convolution layer is used to process the image features output by the previous network layer and expand the feature size.

[0035] It should be noted that for the specific processing principles of the feature extraction layer and the transposed convolution layer in the model, please refer to the relevant existing technology, and the present invention will not elaborate on this.

[0036] Among them, the training process of several types of image processing models specifically includes: C1. Collect a large number of sample images collected by fluorescence microscope and generate the collection environment distribution feature vector of the sample images; C2. Input all collected acquisition environment distribution feature vectors into a preset clustering algorithm (K-means clustering) to obtain k clusters, and form a training image set with sample images corresponding to all acquisition environment distribution feature vectors in each cluster, that is, each cluster corresponds to a training image set; please refer to the relevant prior art for the detailed clustering process principle, which will not be elaborated in the present invention; the center point of each cluster is used as the image class label of the corresponding training image set; C3, constructing a k-type image processing model having a model structure consisting of a plurality of feature extraction layers and a plurality of convolutional transposition layers as the class image processing model to be trained; C4, based on each training image set, input the sample image therein into the corresponding class image processing model to be trained, and output the class target image; C5, based on the class target image output by each class image processing model and the sample image input, the algorithm is determined by using the preset model optimization index to generate the target evaluation index; C6, using the target evaluation index to train the image processing model (gradient descent and other optimization algorithms can be used, refer to the relevant prior art, the present invention will not elaborate on this), repeat the steps C4 to C6 in a loop, continuously iterate the training, and gradually reach the target state of the target evaluation index (the larger the target evaluation index, the better the model performance), so that the model output is closer to the expected output; Therefore, a large number of sample images are collected to generate a collection environment distribution feature vector; a clustering algorithm is used to obtain k clusters, each cluster corresponds to a training image set; a k-class image processing model is constructed; the model is trained based on the training image set, and the model parameters are continuously optimized until the target state of the target evaluation index is reached; a well-trained class image processing model is provided for subsequent image processing; and a k-class trained image processing model is obtained, which can process sample images under different collection conditions.

[0037] A3. The image feature fusion output module performs weighted fusion on several types of target images received according to the distributed processing value sequence and outputs the final target image.

[0038] Specifically, each pixel coordinate in the target image corresponds to a pixel grayscale value, and the grayscale values ​​of each pixel coordinate in several target images of the same type are weighted and summed to obtain the target grayscale value of the target image at the pixel coordinate; wherein the weight value corresponding to each target image of the same type corresponds to the similarity value of the target image of the same type in the distributed processing value sequence.

[0039] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: By introducing the acquisition environment distribution feature vector and multi-category image processing models, differentiated image processing is achieved for images to be processed under different acquisition conditions based on distributed processing value sequences, thereby improving the adaptability and effect of image processing; solving the problem that a fixed and unified image processing model is difficult to adapt to sample images of all acquisition conditions, thereby improving the flexibility and accuracy of image processing.

[0040] Embodiment 2: The model optimization index determination algorithm in Embodiment 1 is improved and limited so as to optimize the performance of the image processing model.

[0041] Therefore, the embodiments of the present application are optimized to a certain extent based on the above embodiments.

[0042] In some embodiments, the pre-set model optimization index determination algorithm specifically includes: S201, the target image and its input sample image are divided into n sub-image blocks (the sub-image blocks with the same position coordinates in the target image and its sample image are in a one-to-one relationship), and the feature vector of each sub-image block is calculated, specifically: the sample image and class target images and One to one correspondence; Specifically, the feature vector of the sub-image block is obtained as follows: Through the pre-set convolutional neural network (CNN) extraction, in CNN, the sub-image block is input into the network, and after multiple layers of convolution and pooling operations, a fixed-length feature vector is finally obtained. These feature vectors capture the high-level visual features of the sub-image block, specifically including: preprocessing the sub-image block, such as normalization, denoising, etc.; performing convolution operations on the image block through the convolution layer to extract local features; downsampling the convolved features through the pooling layer to reduce the feature dimension; inputting the pooled features into the fully connected layer to obtain a fixed-length feature vector; in CNN, parameter indicators include convolution kernel size, pooling window size, number of network layers, activation function, etc. These parameters will affect the extraction effect and calculation efficiency of the feature vector, and are set according to the actual scene requirements during actual application, and the present invention will not elaborate on this.

[0043] S202, calculating the mutual information between the sample image and the corresponding sub-image block in the class target image on the feature vector, and calculating the visual information fidelity of each sub-image block in the class target image relative to the sample image according to the following formula:

[0044] in, For visual information fidelity, is the feature vector of the i-th sub-image block of the sample image, is the feature vector of the i-th sub-image block of the target image, n is the number of sub-image blocks divided into the sample image and the target image, for and The mutual information between for The mutual information with itself (as a normalization factor).

[0045] VIF is used to evaluate image quality. It measures the quality loss of an image by comparing the fidelity of visual information between a sample image and a target image. Mutual information is a measure of the degree of mutual dependence between two random variables. In VIF, mutual information is used to measure the similarity between a sub-image block Ri of a sample image and a sub-image block Di of a target image. It should be noted that the calculation of mutual information usually involves a similarity measure between feature vectors of sub-image blocks. Feature vectors can be extracted through deep learning methods such as convolutional neural networks (CNNs). The specific calculation formula for mutual information may vary depending on the implementation method, but the general form is: MI( , ) = H( )+H( )-H( , ), where H( ) and H( ) are and The entropy, H( , )yes and The calculation of entropy: The calculation of entropy usually involves the estimation of probability distribution. In image processing, entropy can be calculated by estimating the probability distribution of the feature vector of the image block, which is usually achieved by methods such as histogram statistics or kernel density estimation; The calculation of joint entropy: The calculation of joint entropy is similar, but it is necessary to consider the joint probability distribution of the feature vectors of the two image blocks. The calculation principle of entropy can be understood by referring to the relevant prior art, and the present invention will not elaborate on this.

[0046] S203, based on each sub-image block in the class target image, calculating the pixel difference between it and the corresponding sub-image block in the sample image (taking the variance value of the pixel difference values ​​corresponding to all pixel grids in the sub-image block); S204, determining the ratio of the visual information fidelity to the pixel difference as the covariance direction value, and calculating the target evaluation index according to the following formula:

[0047]

[0048] Among them, L is the target evaluation index, n is the number of sub-image blocks divided into target images, is the image quality index of the i-th sub-image block in the target image, is the visual information fidelity of the i-th sub-image block in the target image, is the covariance direction value of the i-th sub-image block in the target image, is a pre-set weight parameter used to adjust the influence of the covariance direction value on the image quality index. is the influence of the image quality index of the ith sub-image block on the target evaluation index. Therefore, the greater the visual information fidelity, the greater the covariance direction value, and the higher the image quality. The covariance direction value is used to quantify the negative correlation between the visual information fidelity and the pixel difference, which more directly reflects the directionality of "the greater the pixel error, the smaller the VIF".

[0049] In some embodiments, in step C6, the training model is continuously iterated to gradually reach the target state of the target evaluation indicator. The target state is pre-set and can be set according to actual conditions to indicate that the model's image processing capability has reached the target state. When the target evaluation indicator stably reaches the target state during the training process, the model training is completed.

[0050] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: The image quality index combines two important indicators, VIF and pixel difference, to more comprehensively reflect the quality of image processing. The image quality index introduces the covariation direction value to clearly quantify the change direction between the visual information fidelity and the pixel difference, making the evaluation result more instructive. By adjusting the weight parameter α, the evaluation focus of CQI can be customized according to specific application scenarios and needs. Based on the joint evaluation of visual information fidelity and covariation direction value, the contingency of single indicator judgment can be avoided, and the change direction can be used to constrain the visual information fidelity, making the evaluation result more reliable. Generate a target evaluation index based on the CQI of each sub-image block in the target image. The larger the target evaluation index, the better the image processing capability of the image processing model. It can more accurately reflect the performance of the model in the image processing process, especially in the balance between processing effect and pixel-level accuracy. It can also more clearly guide the training optimization direction of the image processing model, so that the image quality processing can reach a more ideal state. When optimizing parameters during model training, the before and after analysis of the training image is converted from local sub-image block analysis to global image quality analysis, combined with local differentiated fusion, so that the final global evaluation index can better reflect the overall quality of the image, improve the model image processing effect, and avoid the omission of analysis results of some pixel areas due to global image analysis.

[0051] Example 3: In Example 2, the sub-image blocks of the model optimization index determination algorithm are usually obtained based on uniform division. However, images collected by fluorescence microscopes often contain specific biological structures or other contents, such as DNA, which have high brightness or fluorescence intensity in the image. Uniform division may not be able to effectively capture the characteristics of these key information, resulting in ignoring the special processing requirements of these important areas in model training when evaluating image quality.

[0052] Therefore, the embodiments of the present application are optimized to a certain extent based on the above embodiments.

[0053] In some embodiments, in step S201, the sub-image block division method may be generated according to a preset adaptive segmentation mechanism, and the adaptive segmentation mechanism specifically includes: S301, performing connected domain identification on a sample image to obtain a plurality of connected domains.

[0054] Specifically, based on the grayscale values ​​of all pixels in the sample image, a threshold segmentation or edge detection method is used to identify connected domains in the sample image. The connected domains usually correspond to areas with higher fluorescence intensity, such as DNA distribution areas.

[0055] S302, based on each connected domain in the sample image, taking the pixel coordinates of the central pixel point in the sample image as the position coordinates of the connected domain, respectively calculating the distance values ​​between any two connected domains.

[0056] S303, based on the distance values ​​between the two connected domains, a preset clustering algorithm (for example, K-means clustering) is used to obtain a number of clusters (assuming that there are r clusters), each cluster includes at least one connected domain, and the maximum circumscribed rectangle of each cluster is used as the key area.

[0057] S304: Based on all the key areas identified in the sample image, the sample image is divided into n sub-image blocks using a non-uniform division strategy, ensuring that each sub-image block includes at most one key area.

[0058] The non-uniform partitioning strategy is: divide the sample image excluding the key area into the largest rectangle to obtain m (m is less than n) rectangular areas, and combine the m rectangular areas and r key areas into n sub-image blocks, m+r=n.

[0059] Therefore, each sub-image block in the sample image includes at least one connected domain, and the key area is determined based on the degree of aggregation of all connected domains. The key area includes a DNA tissue image that can reflect a certain biological structure. The distribution of key areas of different sample images is different. Therefore, differential analysis of the key areas of the sample images to analyze the quality indicators of the image before and after processing can more accurately and finely reflect the processing effect of the model on the image.

[0060] S305 , based on the positions of all sub-image blocks of the sample image, the sub-image blocks are divided in the class target image to be consistent with the sample image.

[0061] In some embodiments, the influence degree value of the image quality index of the i-th sub-image block on the target evaluation index is calculated according to the following formula:

[0062] in, is the influence of the image quality index of the i-th sub-image block on the target evaluation index, is the area value (number of pixels) of the i-th sub-image block, S is the area of ​​the sample image (total number of pixels), is the number of connected domains in the i-th sub-image block, N is the total number of connected domains in the sample image, and It is a pre-set weight factor, which is used to determine the importance of the area value and the number of connected domains to the influence value. It is set according to the actual situation. .

[0063] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: Through the adaptive segmentation mechanism, the key information features in the fluorescence microscope images can be captured more accurately. During the image processing model training process, the key areas of the sample images are differentially analyzed and corresponded to the target images. The target evaluation indicators can be more accurately and finely analyzed from local to global, reflecting the processing effect of the model on the image, and improving the flexibility and accuracy of image processing, especially for images containing complex biological structures.

[0064] Example 4: In fluorescence microscopy image processing, due to the complexity and diversity of sample images, the global image processing model may not be able to fully adapt to the characteristics of all local areas, resulting in local image loss or abnormalities, especially when there are many connected domains with a small proportion. It is difficult to avoid the possibility of these connected domains being merged when the new image to be processed is input into the trained image processing model for global processing, resulting in the loss of local image details. The output corresponds to a larger connected domain, and there is a probability of image loss in abnormal areas based only on the pixel-level judgment of the previous and next images.

[0065] Therefore, the embodiments of the present application are optimized to a certain extent based on the above embodiments.

[0066] In some embodiments, after step S102, the method further includes: S103, using an adaptive segmentation mechanism to divide the image to be processed into a number of sub-image blocks, and assigning a unique position label to each sub-image block (which can be set according to the central pixel coordinates of the sub-image block).

[0067] S104, based on the position label of the sub-image block of the image to be processed, determine the sub-image block with the same position label in its target image, calculate the maximum area value of the connected domain in the sub-image block with the position label in the image to be processed and the target image respectively, and obtain the first area value and the second area value.

[0068] S105, based on each sub-image block in the target image, calculate its image quality index and the maximum area domain difference (the difference between the second area value and the first area value), and determine whether there is an abnormal sub-image block.

[0069] The step of judging whether there is an abnormal sub-image block specifically includes: If the image quality index is less than the preset quality threshold, and the maximum area domain difference is greater than the preset difference threshold, the position label of the corresponding sub-image block is marked as abnormal. The quality threshold and difference threshold are set according to expert experience and actual conditions, and the difference threshold is greater than 0.

[0070] S106, if it exists, segment the abnormal sub-image block from the image to be processed to obtain a sub-sample, execute step S102, input it into the multi-class image processing model, output the sub-target image corresponding to the sub-sample, and replace the sub-image block at the position label in the target image according to its position label to obtain a new target image, replace the original target image, and effectively repair the problem of local or detail missing.

[0071] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: Through the adaptive segmentation mechanism, local differential anomaly analysis is performed on the target image of the processed image output, which can process and analyze the image more finely, improving the accuracy of image processing and the ability to capture details; by comparing the image quality index and the maximum connected domain area value of the sub-image block, the abnormal sub-image block caused by the global processing of the image to be processed by the model is identified, and the local anomaly is singled out and input into the pre-trained multi-category image processing module, and the image processing of the local area is carried out in a targeted manner, which effectively enhances the robustness of the model, retains more image details, and makes the output image more stable and reliable.

[0072] Furthermore, an embodiment of the present invention also provides an image processing device based on fluorescence microscopy acquisition.

[0073] Figure 2 It is a structural block diagram of an image processing device based on fluorescence microscope acquisition according to an embodiment of the present invention.

[0074] like Figure 2 As shown, the image processing device based on fluorescence microscope acquisition includes: an acquisition module, an image processing module, the image processing module includes multiple types of image processing models that have been pre-trained, and the multiple types of image processing models include: an image attention distribution module, an image distributed processing module, and an image feature fusion output module.

[0075] Specifically, the acquisition module is used to: obtain the image to be processed acquired by the fluorescence microscope, generate the acquisition environment distribution feature vector of the image to be processed, and input it into the image processing module; The image processing module is used to: input the image to be processed and its acquisition environment distribution feature vector into a pre-trained multi-class image processing model, and output a target image corresponding to the image to be processed; The multi-class image processing model is used for: A1. Transmitting the received image to be processed and its acquisition environment distribution feature vector to the image attention distribution module, generating a distributed processing value sequence corresponding to the sample image, and transmitting it to the image distributed processing module; A2. The image distributed processing module inputs the sample image into the corresponding class image processing model according to the received distributed processing value sequence, outputs several classes of target images, and transmits them to the image feature fusion output module; A3. The image feature fusion output module performs weighted fusion on the received several classes of target images according to the distributed processing value sequence, and outputs the final target image.

[0076] It should be noted that other specific implementation contents of the embodiment of the present invention may refer to the above-mentioned image processing method based on fluorescence microscopy acquisition.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image processing method based on fluorescence microscopy acquisition, characterized in that: include: S101, obtaining an image to be processed acquired by a fluorescence microscope, and generating an acquisition environment distribution feature vector of the image to be processed; S102, inputting the image to be processed and its acquisition environment distribution feature vector into a pre-trained multi-class image processing model, and outputting a target image corresponding to the image to be processed; The pre-trained multi-class image processing models include: an image attention distribution module, an image distributed processing module, and an image feature fusion output module. The image distributed processing module includes several types of image processing models. The multi-class image processing models are used to: A1. The received image to be processed and its acquisition environment distribution feature vector are transmitted to the image attention distribution module, a distributed processing value sequence corresponding to the sample image is generated, and transmitted to the image distributed processing module; A2, the image distributed processing module inputs the sample images into the corresponding class image processing models according to the received distributed processing value sequence, outputs several classes of target images, and transmits them to the image feature fusion output module; A3. The image feature fusion output module performs weighted fusion on several types of target images received according to the distributed processing value sequence and outputs the final target image.

2. The image processing method based on fluorescence microscopy acquisition according to claim 1, characterized in that: The image distributed processing module includes several types of image processing models, each type of image processing model is assigned a unique image class label, and the image class label is determined according to the center point of the acquisition environment distribution feature vector of the training image set in the corresponding type of image processing model.

3. The image processing method based on fluorescence microscopy acquisition according to claim 2, characterized in that: The training process of the several types of image processing models specifically includes: C1. Collect a large number of sample images collected by fluorescence microscope and generate the collection environment distribution feature vector of the sample images; C2. Input all collected acquisition environment distribution feature vectors into a preset clustering algorithm to obtain k clusters, and form a training image set with sample images corresponding to all acquisition environment distribution feature vectors in each cluster, that is, each cluster corresponds to a training image set; C3, constructing a k-type image processing model having a model structure consisting of a plurality of feature extraction layers and a plurality of convolutional transposition layers as the class image processing model to be trained; C4, based on each training image set, input the sample image therein into the corresponding class image processing model to be trained, and output the class target image; C5, based on the class target image output by each class image processing model and the sample image input, the algorithm is determined by using the preset model optimization index to generate the target evaluation index; C6. Use the target evaluation index to train the image processing model. Repeat steps C4 to C6 in a loop, continuously iterate the training, and gradually reach the target state of the target evaluation index.

4. The image processing method based on fluorescence microscopy acquisition according to claim 2, characterized in that: The image attention distribution module is specifically used for: B1. Obtain image class labels of all image processing models in the image distributed processing module; B2. Calculate the similarity values ​​of the acquisition environment distribution feature vector of the image to be processed and all image class labels respectively; B3. All the calculated similarity values ​​are sequentially organized into a distributed processing value sequence, and each similarity value in the distributed processing value sequence corresponds to a corresponding class image processing model.

5. The image processing method based on fluorescence microscopy acquisition according to claim 3, characterized in that: The preset model optimization index determination algorithm specifically includes: S201, the target image and the input sample image are divided into n sub-image blocks, and the feature vector of each sub-image block is calculated. The sub-image block feature vector of the sample image is , the sub-image block feature vector of the target image is and One to one correspondence; S202, calculating the mutual information between the sample image and the corresponding sub-image block in the class target image on the feature vector, and calculating the visual information fidelity of each sub-image block in the class target image relative to the sample image according to the following formula: ； in, For visual information fidelity, is the feature vector of the i-th sub-image block of the sample image, is the feature vector of the i-th sub-image block of the target image, n is the number of sub-image blocks divided into the sample image and the target image, for and The mutual information between for Mutual information with itself; S203, based on each sub-image block in the class target image, calculating the pixel difference between it and the corresponding sub-image block in the sample image; S204, determining the ratio of the visual information fidelity to the pixel difference as the covariance direction value, and obtaining the target evaluation index by combining the visual information fidelity.

6. The image processing method based on fluorescence microscopy acquisition according to claim 5, characterized in that: The target evaluation index is calculated according to the following formula: ； ； Among them, L is the target evaluation index, n is the number of sub-image blocks divided into target images, is the image quality index of the i-th sub-image block in the target image, is the visual information fidelity of the i-th sub-image block in the target image, is the covariance direction value of the i-th sub-image block in the target image, is a pre-set weight parameter used to adjust the influence of the covariance direction value on the image quality index. is the influence degree of the image quality index of the i-th sub-image block on the target evaluation index.

7. The image processing method based on fluorescence microscopy acquisition according to claim 6, characterized in that: In S201, the sub-image block division method is generated according to a preset adaptive segmentation mechanism, and the adaptive segmentation mechanism specifically includes: S301, identifying connected domains on the sample image to obtain a number of connected domains; S302, based on each connected domain in the sample image, taking the pixel coordinates of the central pixel point in the sample image as the position coordinates of the connected domain, respectively calculating the distance values ​​between any two connected domains; S303, based on the distance values ​​between the two connected domains, a preset clustering algorithm is used to obtain a plurality of clusters, each cluster including at least one connected domain, and the maximum circumscribed rectangle of each cluster is used as a key area; S304, based on all the key areas identified in the sample image, divide the sample image into a number of sub-image blocks using a non-uniform division strategy, ensuring that each sub-image block includes at most one key area; S305 , based on the positions of all sub-image blocks of the sample image, the sub-image blocks are divided in the class target image to be consistent with the sample image.

8. The image processing method based on fluorescence microscopy acquisition according to claim 6, characterized in that: The influence degree of the image quality index of the i-th sub-image block on the target evaluation index is calculated according to the following formula: ； in, is the influence of the image quality index of the i-th sub-image block on the target evaluation index, is the area value of the i-th sub-image block, S is the area of ​​the sample image, is the number of connected domains in the i-th sub-image block, N is the total number of connected domains in the sample image, and is a pre-set weight factor used to determine the importance of the area value and the number of connected domains to the impact value. .

9. The image processing method based on fluorescence microscopy acquisition according to claim 7, characterized in that: After S102, the method further includes: S103, using the adaptive segmentation mechanism to divide the image to be processed into a number of sub-image blocks, and assigning a unique position label to each sub-image block; S104, based on the position tag of the sub-image block of the image to be processed, determining the sub-image block with the same position tag in the target image, and calculating the maximum area value of the connected domain in the sub-image block with the position tag in the image to be processed and the target image, respectively, to obtain a first area value and a second area value; S105, based on each sub-image block in the target image, calculating its image quality index and maximum area domain difference, and determining whether there is an abnormal sub-image block; S106, if it exists, segment the abnormal sub-image block from the image to be processed to obtain a sub-sample, execute step S102, input it into the multi-class image processing model, output the sub-target image corresponding to the sub-sample, and replace the sub-image block at the position label in the target image according to its position label, obtain a new target image, and replace the original target image.

10. An image processing device based on fluorescence microscopy acquisition, applied to an image processing method based on fluorescence microscopy acquisition as claimed in any one of claims 1 to 9, characterized in that: include: An acquisition module and an image processing module, wherein the image processing module includes a multi-class image processing model, and the multi-class image processing model includes: an image attention distribution module, an image distributed processing module, and an image feature fusion output module; the acquisition module is used to: obtain the image to be processed acquired by a fluorescence microscope, generate a collection environment distribution feature vector of the image to be processed, and input it into the image processing module; the image processing module is used to: input the image to be processed and its collection environment distribution feature vector into the pre-trained multi-class image processing model, and output a target image corresponding to the image to be processed; The multi-class image processing model is used for: A1, transmitting the received image to be processed and its acquisition environment distribution feature vector to the image attention distribution module, generating a distributed processing value sequence corresponding to the sample image, and transmitting it to the image distributed processing module; A2, the image distributed processing module inputs the sample image into the corresponding class image processing model according to the received distributed processing value sequence, outputs several classes of target images, and transmits them to the image feature fusion output module; A3, the image feature fusion output module performs weighted fusion on the received several classes of target images according to the distributed processing value sequence, and outputs the final target image.

Images